SEVEN QUESTION SETS · ORBITA, EXA, LINKUP
Is the answer in what
the agent gets back?
We asked Orbita, Exa and Linkup the same questions from seven sets and checked one thing: does the correct answer appear in the text each search API returns to the agent. Every question, every answer and every service's result is on this page.
Results by set
Share of questions, in %, where any returned result contains the answer. Each service as it is used.
| Set | Orbita Fast5 results | Orbita Standard10 results | Orbita Deep25 results | Exa Auto10 results | Linkup Standard10 results |
|---|---|---|---|---|---|
| SimpleQA300 questions | 60.3 | 63.0 | 66.0 | 90.0 | 86.7 |
| Natural Questions300 questions | 76.0 | 78.7 | 83.0 | 89.0 | 87.3 |
| HotpotQA250 questions | 59.2 | 64.4 | 72.4 | 74.8 | 62.0 |
| AA-Omniscience200 questions | 39.5 | 48.0 | 54.5 | 81.5 | 70.5 |
| BrowseComp100 questions | 5.0 | 9.0 | 13.0 | 27.0 | 11.0 |
| TechQA144 questions | 95.1 | 95.8 | 96.5 | 91.0 | 92.4 |
| TechQA-open118 questions | 93.2 | 95.8 | 96.6 | 100.0 | 97.5 |
TechQA-open before Orbita crawled the full documentation of its 21 sites (the same questions on the index as it was): Orbita Fast 53.4 % anywhere, 48.3 % first; Orbita Standard 57.6 % anywhere, 50.0 % first; Orbita Deep 72.9 % anywhere, 62.7 % first. 24 of its 118 source pages were in the index then, 97 after the crawl.
Orbita Fast returns up to 5 results, Standard up to 10, Deep up to 25; Exa Auto and Linkup Standard were asked for 10. "Anywhere in the results" compares each service as it is used; "In the first five" compares them at the same depth; "In the first 1,000 tokens" gives every service the same reading budget.
The same questions, Orbita against Exa
Percentage points on the questions both answered, with a 95 % interval from 2 000 bootstrap resamples. An interval that crosses zero is not a difference this test can show.
| Set | Fast, first five | Standard, first ten | Deep, all 25 against Exa’s 10 | Fast, first 1,000 tokens |
|---|---|---|---|---|
| SimpleQA300 questions | -28.0-33.7 to -22.3 | -27.0-32.7 to -21.7 | -24.0-29.3 to -18.7 | -19.0-26.0 to -12.0 |
| Natural Questions300 questions | -6.3-11.3 to -1.0 | -10.3-14.7 to -6.0 | -6.0-10.3 to -2.0 | -5.0-11.3 to 1.7 |
| HotpotQA250 questions | -8.8-14.8 to -2.8 | -10.4-16.4 to -4.8 | -2.4-8.0 to 3.6 | -6.0-12.4 to 0.0 |
| AA-Omniscience200 questions | -35.5-42.5 to -28.0 | -33.5-41.0 to -26.0 | -27.0-34.5 to -19.5 | -34.5-43.0 to -26.5 |
| BrowseComp100 questions | -19.0-27.0 to -11.0 | — | -14.0-24.0 to -4.0 | -18.0-26.0 to -11.0 |
| TechQA144 questions | +6.20.7 to 11.8 | +4.90.0 to 9.7 | +5.60.7 to 10.4 | +20.113.2 to 27.8 |
| TechQA-open118 questions | -5.1-10.2 to 0.0 | -4.2-8.5 to -0.8 | -3.4-6.8 to -0.8 | 0.0-7.6 to 8.5 |
Seconds per query
How long each search took, over all questions. Orbita's "16 at once" is the test itself, where 16 questions were asked at the same time on one machine; "one at a time" is a separate pass of 60 questions in each mode. Exa and Linkup were called from our laptop over the internet, 8 at a time, so their times include the network.
| Service | Median | Mean | p90 | p95 | Max | Queries |
|---|---|---|---|---|---|---|
| Orbita Fast, 16 at once | 2.68 s | 4.68 s | 8.75 s | 11.89 s | 28.79 s | 1,409 |
| Orbita Fast, one at a time | 2.27 s | 2.29 s | 2.33 s | 2.42 s | 2.75 s | 60 |
| Orbita Standard, 16 at once | 3.61 s | 5.34 s | 9.98 s | 10.73 s | 28.78 s | 1,409 |
| Orbita Standard, one at a time | 2.73 s | 2.76 s | 2.97 s | 3.08 s | 3.35 s | 60 |
| Orbita Deep, 16 at once | 5.03 s | 6.31 s | 10.27 s | 11.76 s | 24.88 s | 1,409 |
| Orbita Deep, one at a time | 3.81 s | 3.95 s | 4.37 s | 4.72 s | 5.90 s | 60 |
| Exa Auto, 8 at once over the internet | 1.63 s | 1.82 s | 2.62 s | 3.28 s | 6.10 s | 1,412 |
| Linkup Standard, 8 at once over the internet | 1.40 s | 1.44 s | 1.64 s | 1.77 s | 3.17 s | 1,412 |
These times describe this test, not the production service: Orbita ran on one test machine, not on the production setup.
Every question
The number is the position of the first result whose text contains the answer, and under it how many tokens an agent reads, in order, until the answer; a dash means no returned result contained it. BrowseComp's questions are not shown: its authors ask that they are not published in plain text.
How it was measured
- The index. Orbita searched a test index of 5,853,868 pages built for this run: 5,428,668 English Wikipedia articles from the Wikimedia dump of 1 November 2023, plus 425,200 pages our crawler collected on 24 October 2026 from 1,927 developer documentation, AI, reference and news sites, cleaned by Gravity, with text read from images. It is not the production index, which is smaller.
- Orbita's search. The same search as the hosted API, on the test index. The three modes differ only in depth, as described in the documentation; no language model rewrites the question in any of them.
- The others. Exa (type auto) and Linkup (depth standard) searched the whole web on 24 October 2026 through their public APIs, 10 results with page text.
- The check. An answer counts as found when a correct answer string (any listed alias), normalised for case, accents, punctuation, number words and date formats, appears in a result's title or text. The same code scores every service.
- Sampling. Fixed random samples (seed 42) of each public set, drawn before any search ran.
- TechQA-open. Our technical set made without looking at any index: 30 documentation sites fixed in advance, 9 pages from each site's sitemap at random (23 sites had usable sitemaps, 21 gave questions), one question per page with a short answer copied from the page, written before any search ran.
- TechQA-open, then the docs crawl. We first asked Orbita on its index as it was: few of the 118 source pages were in it. After that we crawled the whole documentation of the 21 sites from their sitemaps (every docs page, not the sampled ones) and asked again in the main run. Both results are on this page; the crawl came after the questions were written.
What this does not show
- Not answer quality. It checks that the answer is in the returned text, not whether an agent would use it well. It is not the Artificial Analysis Search Index, which grades a model's final answers.
- Index size decides a lot. Exa and Linkup search billions of pages; Orbita's test index is a few million. A question whose page Orbita never collected cannot be found by any ranking.
- Text length differs. Exa returned up to 4 000 characters a result, Linkup shorter passages, Orbita the passage around each match. More text gives a string more chances to appear; the tokens column shows how much each one sends.
- String matching misses paraphrases ("the 1990s" against "1994"), the same way for everyone.
- TechQA favours Orbita. Its questions were written from pages in Orbita's own crawl. TechQA-open does not: its pages were drawn from the sites' sitemaps, and Orbita's crawl took at most 3,000 pages a site, so many of them are not in its test index.
- Three BrowseComp questions are longer than Orbita's 1,000-character query limit. The API refused them, so they count as not found in every Orbita mode.
- The seconds are this test's, not the service's. Orbita ran on one test machine, 16 questions at once in the main run; Exa and Linkup were timed from our laptop over the internet. Compare the times with that in mind.
Sets and sources
| Set | What it asks | Questions | Source | Licence |
|---|---|---|---|---|
| SimpleQA | Short fact questions written to be hard for language models. | 300 | OpenAI simple-evals | MIT |
| Natural Questions | Real Google search questions with answers from Wikipedia. | 300 | Google Research, nq_open (validation) | CC BY-SA 3.0 |
| HotpotQA | Questions that need two Wikipedia articles together. | 250 | HotpotQA (distractor, validation) | CC BY-SA 4.0 |
| AA-Omniscience | Knowledge questions across many fields, public subset. | 200 | Artificial Analysis, AA-Omniscience-Public | Apache-2.0 |
| BrowseComp | Hard questions built to need long browsing; the hardest set here. | 100 | OpenAI simple-evals | MIT |
| TechQA | Facts from developer and AI documentation. Written by us from pages Orbita collected, so it favours Orbita. | 144 | Orbita, written for this test on 24.10.2026 | — |
| TechQA-open | Facts from official developer and AI documentation, pages drawn at random from 21 docs sites' sitemaps. Orbita crawled those sites in full after the questions were written; its result before that crawl is under the results table. | 118 | Orbita, written for this test on 24.10.2026 | — |
All data behind this page, every question with each service's result, is in benchmarks-2026-10-24.json.