Cito
Hybrid academic search over 236M papers, built for agents
102 followers
Hybrid academic search over 236M papers, built for agents
102 followers
Cito is a hybrid search engine over the Semantic Scholar corpus: 236M papers in the keyword index, 146M with SPECTER2 dense vectors, fused with RRF and reranked by a cross-encoder. Free web search with no signup, a plain JSON API, and a native MCP endpoint so agents like Claude Code can run deep literature research without upstream rate limits. Built because every academic API throttled my agents to death.
Cito
The MCP endpoint is what sold me. I keep hitting the Semantic Scholar 1 req/sec wall when Claude runs a deep lit-research pass and it just stalls. Does the free 100/min hold up for a full run, or does it throttle once an agent goes wide?
Cito
@hung_tran_from_notebook_os
Short answer: it holds, and going wide is specifically the case it's built for.
The 100/min is a fixed-window counter per API key, not a pacer. There's nothing forcing a gap between calls the way S2's 1 rps does, so you can fire all 100 inside the first second and none of them stall. The window just resets.
Two things worth knowing when an agent fans out:
The limit is per key, not per connection. Ten parallel workers on one key share the same 100; they don't get 100 each.
Search is the metered part. Paper lookups aren't: /paper/batch takes up to 1000 IDs in a single call. In a real literature research pass, most of your volume is resolving and hydrating references, not issuing new queries, so the 100 tends to go a lot further than the raw number suggests. There's also /search/batch (50 queries per request), which cuts round trips, though each query still counts against the 100.
If you do have a run that genuinely needs more, ping me. The limit is a per-key value, so it can be raised for your key.
the batch endpoints angle is what would actually get me to switch. most academic search wrappers I've tried just proxy the same rate limit upstream with extra steps, so an agent doing real literature review still stalls after a dozen calls. curious how fresh the index stays though - is there a lag between a paper landing in Semantic Scholar and it showing up searchable here, or is it close to real time?
Cito
@omri_ben_shoham1 Honest answer: not real time, and it's a deliberate trade-off. The index is built from Semantic Scholar's bulk dataset releases, which ship roughly weekly, and I apply the diffs on a biweekly cadence. So worst case, a paper that just landed in S2 takes two to three weeks to become searchable here; the typical case is more like one to two weeks. Note that this lag is relative to S2's dataset exports, which themselves trail the S2 live API by up to a week.
The reason it works this way is the same reason the batch endpoints don't stall: nothing in the query path ever calls upstream. The whole corpus (236M papers, BM25 + SPECTER2 vectors) is served locally, so there's no rate limit to proxy and no per-call upstream dependency. Freshness is the price of that. For literature review, the trade-off has been fine in practice, since you're rarely chasing papers published this week; for "what came out yesterday," arXiv itself is still the right tool.
Tightening the sync to weekly (matching S2's release cadence) is mostly an ops cost question, not an architecture one. So if freshness turns out to be the blocker for users, it's an easy knob to turn.
@Tao An that's a genuinely clear answer, the "freshness is the price of no upstream dependency" tradeoff makes sense architecturally. one follow-up: is the corpus's last-sync date surfaced anywhere in the UI or API response, so a user querying something time-sensitive can tell at a glance whether they should also go check arXiv directly, or is that on the user to just know based on your docs
Cito
@omri_ben_shoham1 Good question, and checking it turned up a bug worth admitting.
The short answer was “partly”. Every response already carried corpus_release, and the footer showed it. But it was reporting a constant baked into the build rather than the live sync pointer, so it was already understating the corpus by two weeks: the API said 2026-06-24 while the index was actually serving 2026-07-07.
Fixed. GET /status and every search response now carry corpus_release (which S2 release is being served) plus corpus_synced_at (when it finished applying, ISO-8601 UTC), so you can judge a time-sensitive query without a second call. The footer shows both, and /docs/data#freshness explains them.
Your question also made me re-check the sync cadence, and that turned out to be the bigger finding. I had it gated to biweekly. But the diffs API returns a stepwise chain and the sync applies every step, so batching two releases downloads and writes exactly what two single-release syncs do. The gate was buying nothing except staleness. Meanwhile a release from 2026-07-14 had been sitting unused for five days because the gate wanted 14 days of drift before firing. It now follows S2’s own cadence: each release is applied within a day of appearing.
So the freshness story is better than what I told you yesterday. The remaining lag is upstream, not mine: S2’s bulk releases trail their own live API by up to a week. That part I can’t fix without calling upstream per query, which is the rate limit this project exists to escape.
One thing more useful than the display itself: since nothing in the query path calls upstream, corpus_synced_at is a hard ceiling rather than a hint. No paper that appeared upstream after that timestamp can be in the results at all. So an empty result on a very recent topic is a coverage boundary, not evidence the work doesn’t exist. That’s in the MCP tool description now too, so an agent reports it that way instead of telling you the literature is empty.
@Tao An that's a genuinely good answer - most people would've just said "yeah it's fresh" and moved on, not gone and found a baked-in constant plus a gate that was actively hurting freshness. the hard-ceiling framing for corpus_synced_at is the right call, an empty result reading as "nothing indexed yet" instead of "nothing exists" is a real difference for anyone trusting the output. is that distinction shown anywhere in a human-facing UI too, or is it mainly baked into the API/MCP tool description for agents right now?
Cito
@omri_ben_shoham1 Mainly the API and MCP description. The footer carries corpus_release and corpus_synced_at on every page and /docs/data explains them, but that is the data, not the inference: the empty state just said "No results." So the agent got the hard-ceiling reading spelled out and the human was left to infer it. That is backwards, and it is now fixed. Zero hits states the boundary, the sync date and the release being served.
Though writing that fix made me notice I solved the easier half. Empty results at least make someone suspicious. The riskier case is a time-sensitive query in hybrid mode, which always returns nearest neighbours and so comes back with a full page of plausible, older work and no hint that the last few weeks are missing. That is the case your original question was really about, and I do not have a good answer for it yet. Probably the ceiling needs to sit near the results themselves, not only in the footer, and probably only when the query looks time-sensitive. Thinking about it.
Really glad someone finally built a search engine that doesn't choke an agent after three requests. If you could expose a streaming endpoint or webhook for long queries, agents could kick off a deep search and poll instead of holding a connection open the whole time. That would make it way easier to chain searches across Claude Code sessions without timing out.
Cito
@elifnurs7bw
Thanks! Good news is you mostly don't need it: a normal query returns in well under a second even cold, so there's no long connection to hold. The slow path is really when an agent wants 50 things, and for that we just shipped batch endpoints (/search/batch and /paper/batch, plus direct DOI/arXiv lookup) so you can do one round trip instead of fifty. If we ever add something genuinely long-running, for example LLM query expansion over large result sets, an async job plus poll pattern is how I would approach it. What's your use case where a single query runs long enough to time out? I would love a concrete example.
finally a search tool that doesn't choke when my agents hammer it. the MCP integration just worked out of the box with claude code, which is more than i can say for most academic apis.
Cito
@hafizeldgy Glad it just worked, that was the whole point of building it. Upstream APIs rate-limiting agents into the ground is exactly why this project exists. If your agents are doing bulk work, check out the new batch endpoints (/search/batch and /paper/batch, plus direct DOI and arXiv lookup). They landed this week and significantly reduce round trips.