I Mapped 8.5 Million Research Papers. The Map Isn’t Even the Best Part.

You know the drill. You find a paper. You open the PDF. Then you spend the next 45 minutes playing detective — hunting for the code repo, chasing down citations, checking if anyone replicated it, figuring out which genes or drugs it touches. By the time you’ve assembled the full picture, you’ve forgotten why you started reading in the first place.

I spent months trying to fix this. Not with another search engine. Not with another PDF summarizer. I wanted to rebuild how research papers connect to each other — and then let AI agents walk that web of connections like they own it.

Reading a paper in 2025 shouldn’t require six browser tabs, three GitHub searches, and a prayer that someone linked the dataset.

Here’s what I built: an interactive atlas of 8.5 million papers from arXiv, PubMed Central, bioRxiv, and medRxiv. Every paper is a dot. Every dot sits in a cluster labeled by topic. You zoom in, you see fields. You click a dot, you get an LLM-generated TLDR, key findings, citations, peer reviews where they exist, and similar work. It’s a knowledge graph rendered as a galaxy you can fly through.

But here’s the twist — and this is the part most people miss.

The map is cool. The map is not the point.

The real unlock isn’t helping humans read papers faster. It’s letting machines read them at all.

Each paper in the atlas is rebuilt with linked entities — genes, proteins, diseases, drugs — normalized to real IDs via PubTator, not regex garbage. Clinical trials, 3D structures, code repos, citation graphs. All of it. And it’s all exposed as an MCP server, which means you can point Claude or any AI agent at 8.5 million papers and say “go find me every paper on this protein that has open code and was replicated.” The agent queries the corpus. You stop pasting PDFs into chat windows like it’s 2023.

This is the shift nobody’s talking about. We’ve been so focused on making AI read single documents that we forgot the real bottleneck was never reading speed — it was access. A human reads one paper at a time. An agent with an MCP server reads 8.5 million. That’s not a productivity improvement. That’s a different species of research.

The future of literature review isn’t a better search bar. It’s an agent that already read everything.

Now, the hard part — because this wasn’t a weekend project. The ingestion pipeline was brutal. Over 45 heterogeneous sources, each with its own schema, its own naming conventions, its own ideas about what a “paper” even is. Normalizing all of that into one coherent structure was the real work. Entity linking was the real nightmare. Papers use gene names, not accession IDs. They reference drugs by brand names and common names interchangeably. Regex gets you maybe 60% of the way there and confidently wrong the other 40%. Curated link tables plus PubTator beat regex every single time, but building that pipeline took months of iteration.

The stack: Next.js up front, Postgres on Neon, Cloudflare R2 for content storage, embeddings plus UMAP and HDBSCAN for the map, and a custom WebGL renderer to push millions of dots at 60fps. The visualization layer was the fun part. The data engineering was the part that almost killed it.

Nobody sees the pipeline. Everybody judges the map. That’s the curse of building infrastructure.

There are limits, and I’ll be honest about them. Google indexing is still slow, so title search is rough right now. Some older arXiv figures are missing. Peer-review coverage is partial — because peer review data itself is fragmented and siloed, which is its own indictment of the system. But the core works. The entity links hold up. The TLDRs are useful. The map flies.

It’s free, no signup, no paywall. Go explore it. Click into papers you know well and tell me if the TLDRs and entity links hold up. That’s the test that matters — not whether it looks pretty, but whether it’s accurate on the papers where you already know the ground truth.

Because here’s what I believe: research papers are not documents. They’re nodes in a graph that we’ve been pretending are standalone files for thirty years. The PDF is a lie. The knowledge has always been connected. We just never had the tools — or the models — to see the connections.

Now we do. And the agents are going to see connections we never could.

FAQ

Q: Is this just another pretty visualization with shallow data underneath?

A: No. The map is the entry point, but each paper is rebuilt with normalized entity links — genes, proteins, diseases, drugs mapped to real IDs via PubTator, not regex. The ingestion pipeline normalizing 45+ sources into one schema was the actual hard part, and it's what makes the entity links accurate rather than confidently wrong.

Q: What does the MCP server actually let me do that I can't do now?

A: Instead of pasting a PDF into Claude and asking about one paper, you point an agent at 8.5 million papers and ask cross-cutting questions: 'Find every paper on this protein with open code that was replicated.' The agent queries the structured corpus. You stop being a manual search engine.

Q: Won't AI agents hallucinate connections if the entity linking isn't perfect?

A: They might — which is exactly why the entity linking uses curated tables and PubTator instead of regex. The contrarian take: imperfect but navigable is still a massive upgrade over perfect but siloed. Researchers already tolerate far worse signal-to-noise ratios scrolling through Google Scholar.

📎 Source: View Source