How Many PDFs Can Gemini 3.1 Pro Analyze Without Hallucinating?
Google markets a 1M token context window. Reliable multi-fact recall ends around 128K. Here's what the MRCR v2 benchmarks actually show — and how to work within those limits.
Key Takeaways
- Gemini 3.1 Pro's official context window is 1M tokens, but multi-fact recall starts degrading past 128K tokens on the MRCR v2 benchmark — long before you hit the marketing limit.
- Google's single-needle recall scores are strong even at 1M tokens. The problem is multi-needle retrieval — the kind that matters in real legal and academic research.
- Claude Opus 4.6 holds 91.9% on MRCR v2 multi-needle at 256K tokens — currently the strongest model for citation-critical research. Opus 4.7 regressed significantly on this benchmark.
- CONTENT GAP — Most published guides still recommend Claude Opus 4.7 for long-context research without noting its MRCR v2 regression. This article corrects that.
📤 Save & Share: The 3 key findings from this article at a glance.
Why Do Standard AI Models Fail at Multi-Document Research?
The context window is basically the model's working memory. Everything you upload has to fit inside it simultaneously — and older models with 4K or 8K token limits would literally drop earlier documents to make room for new ones. If you uploaded ten 100-page PDFs, the model might retain seven while the first three just disappeared from its awareness entirely.
Gemini 3.1 Pro's 1 million token context window changes that equation significantly. But here's what Google's marketing copy skips over: holding information in context and accurately retrieving specific facts from deep within that context are two completely different problems. The window is the container. Recall accuracy is whether the model can actually find what you put inside it.
And that second problem — recall — is where things get messy. Fast. The model doesn't tell you it's struggling. It just answers with something that sounds right.
What Is the Actual Context Window Limit of Gemini 3.1 Pro?
Google DeepMind's official model card puts the input context at up to 1 million tokens with 65,000 tokens of output capacity. That translates to roughly 500-700 standard 100-page PDFs — or about 40 hours of video content. Impressive numbers on paper.
Reality is messier. The gap between single-needle and multi-needle retrieval is where Gemini's marketing falls apart. On the simple NIAH test — one fact hidden in a large document — almost every frontier model scores above 90% even at 1M tokens. That's the number Google quotes. But real research doesn't have one needle. It has dozens. The MRCR v2 benchmark, which tests multi-needle retrieval, tells a completely different story.
According to Google DeepMind's own published MRCR v2 scores and independent analysis from LLM Context Windows 2026, the real picture across token ranges looks like this:
- Up to 128K tokens: Gemini 3.1 Pro scores 84.9% on MRCR v2 multi-needle — strong and reliable for most research workloads
- 128K–256K tokens: Performance starts to degrade; this is where careful prompting becomes essential
- 256K–1M tokens: Google's own MRCR pointwise score drops to 26.3% at 1M — well below the headline claim. Use for summarization only, never for specific fact retrieval
Late April 2026. I had a legal research project — 127 court documents, roughly 380,000 tokens total, loaded into Gemini 3.1 Pro Advanced. Well past the 256K threshold where MRCR scores start degrading. The high-level summaries were actually excellent. Clear, well-organized, genuinely useful.
Then I asked it to cross-reference a specific precedent from document 47 against a claim in document 89. It gave me two case citations. Both sounded completely plausible. Neither existed. I verified this against primary legal databases — fabricated, start to finish.
That's the "lost in the middle" problem in practice. Not a glitch. Not a one-off. It's what the MRCR benchmark predicts will happen at that context size — and it did. My rule now: anything past 128K tokens on fact-critical work gets a secondary verification pass. That threshold is where the benchmark data says to stop trusting and start checking.
Does Gemini 3.1 Pro Suffer from "Lost in the Middle" Hallucinations?
The "needle in a haystack" (NIAH) benchmark tests exactly this. A specific fact gets hidden somewhere in a large body of text, and the model has to find it. Gemini 3.1 Pro scores well in standard NIAH tests. But standard NIAH is a single needle. Real research involves dozens of needles scattered across hundreds of documents — and that's where the MRCR v2 benchmark tells a different story.
The reason is architectural. Transformer attention mechanisms naturally weight information at the beginning and end of the context more heavily than content in the middle. When you load 500 documents, a specific claim buried in document 47, page 400 of that document, becomes statistically unlikely to get the attention weight it needs during generation. So the model fills the gap. Confidently. With something plausible.
Comparing Gemini 3.1 Pro to the Claude line on this specific problem — and this is where the data gets counterintuitive:
- Claude Opus 4.6 leads multi-needle retrieval. On MRCR v2 8-needle at 256K tokens, Opus 4.6 scores 91.9%. At 1M tokens, it holds at 78.3%. For legal research, Opus 4.6 is currently the strongest option for this task.
- Claude Opus 4.7 is not the right choice for long-context retrieval. It launched with improvements to coding and reasoning, but MRCR v2 scores dropped significantly — from 91.9% to 59.2% at 256K, and from 78.3% to 32.2% at 1M. Anthropic disputes the benchmark, but the practical advice from independent testing is consistent: for retrieval-heavy work over 128K tokens, Opus 4.6 outperforms Opus 4.7.
- Output window: Both Claude models offer larger output capacity than Gemini 3.1 Pro's 65K tokens — useful for long synthesis tasks.
Even within the reliable token range, Gemini 3.1 Pro has fabricated case references that sound entirely plausible. In legal contexts, a non-existent precedent isn't just an error — it can mean professional misconduct. Cross-reference every specific case name, citation number, and statute reference against a primary legal database before using it.
How Should Researchers Prompt for Maximum Recall Accuracy?
The difference between a generic prompt and a structured extraction prompt is not subtle. It's about breaking the task into sequential steps that match how the model's attention actually works — and that sequencing is what keeps it honest at higher token counts.
Instead of "Summarize these PDFs," use a structured sequence: "First, extract every quote related to [specific topic]. Second, arrange those quotes into a chronological timeline. Third, flag any direct contradictions between documents." Each step forces the model to complete a retrieval task before synthesizing — and that sequencing is what keeps it honest.
For research specifically, I've settled on a three-phase approach based on our testing methodology:
- Phase 1 — Targeted Extraction (0–128K tokens): Ask the model to pull only factual statements directly related to your research question. Nothing else. This keeps the first pass clean and stays within the reliable recall zone.
- Phase 2 — Cross-Reference Synthesis (128K–256K tokens): Once you have extracted facts, ask the model to find relationships between documents. Not summaries — relationships. Contradictions, agreements, gaps.
- Phase 3 — Batch Verification (256K+ tokens): If your context exceeds 256K tokens, work in batches of 50K tokens. First pass extracts. Second pass synthesizes across batches. Slower. But accurate.
This approach pairs well with our guide on safe AI research and citation practices — especially when the output is going into something with professional or legal consequences.
Which Model Should You Use for Multi-Document Research?
| Model | Max Context Window | MRCR v2 Multi-Needle at 256K | Best Use Case for Research |
|---|---|---|---|
| Claude Opus 4.6 | 200,000 tokens | 91.9% ✅ Best for citations | Legal and academic research where citation accuracy is non-negotiable — the current benchmark leader |
| Gemini 3.1 Pro | 1,000,000 tokens | 84.9% at 128K, degrades sharply past 256K | Multimodal research, video analysis, large document sets needing summary — not citation |
| Claude Opus 4.7 | 1,000,000 tokens | 59.2% ⚠️ Regressed from 4.6 | Coding and agentic tasks — not recommended for long-context multi-fact retrieval |
| GPT-5.5 | 2,000,000 tokens | Strong single-needle; multi-needle data limited | Massive document sets, enterprise workflows — verify on your own corpus first |
The honest takeaway: the model most people expect to recommend here — Claude Opus 4.7, the latest release — is actually the wrong choice for this specific task. Opus 4.7 regressed on multi-needle retrieval. Opus 4.6 is the current benchmark leader for citation-critical research at 91.9% on MRCR v2 at 256K tokens. That's 32 percentage points ahead of Opus 4.7 at the same context size.
Gemini 3.1 Pro holds well at 128K but degrades past 256K on multi-fact tasks. For document sets pushing into that range, it works for summarization. It doesn't work for specific citation retrieval. And if your research involves depositions or recorded hearings, Gemini's multimodal support is a real advantage that no Claude model currently matches. You can also check our full model comparison across use cases if you're still deciding.
What Should You Do Next? (30-Second Decision Tool) 🎯
Methodology & Sources
All tools mentioned in this article were evaluated using our standardised testing methodology. The PDF limit testing was conducted in late April and early May 2026 using Gemini 3.1 Pro Advanced with real legal document sets. All recall accuracy figures in this article are sourced from published benchmark data — specifically Google DeepMind's MRCR v2 results and independent long-context analysis — not from internally generated estimates. Where benchmarks conflict with vendor claims, the article notes the discrepancy explicitly.
The topic for this article was identified using the Search Gap Method: community demand was validated on Reddit, and Google's top 5 results were assessed for content gap classification (CONTENT GAP) before writing began. A key finding during research: the Claude Opus 4.7 vs 4.6 long-context regression is underreported — most published comparisons still recommend 4.7 without noting the MRCR v2 score drop.
- Google DeepMind Gemini 3.1 Pro Model Card — Official context window, output specs, and MRCR benchmark methodology
- MyClaw — Gemini 3.1 Pro MRCR v2 pointwise scores — Including the 26.3% score at 1M tokens from Google's own data
- LLM Context Windows 2026 — RULER, MRCR v2, and NoLiMa Analysis — Independent multi-model benchmark comparison past 200K tokens
- Anthropic Claude Opus 4.7 Release Notes — Official capabilities and known benchmark results
- Artificial Analysis — Head-to-head model comparison data
- Our guide on AI hallucinations in search and research contexts — Related reading on fabrication patterns across AI tools
Frequently Asked Questions
How many PDFs can Gemini 3.1 Pro actually analyze?
Gemini 3.1 Pro officially supports a 1 million token context window, which theoretically allows processing 500-700 standard 100-page PDFs. But reliable multi-fact recall only holds to around 128K–256K tokens — roughly 100-200 PDFs. Google's own MRCR v2 pointwise score at 1M tokens is 26.3%. For general summarization you can push further, but for specific fact retrieval or citation work, stay under 128K tokens and verify every citation against a primary source.
Why does Gemini 3.1 Pro hallucinate more at higher token counts?
Even with a massive context window, models hit what researchers call the "lost in the middle" problem. Transformer-based models weight information at the beginning and end of the context more heavily than facts buried in the middle. When you load 500+ documents, a specific claim buried on page 400 of document 47 becomes statistically unlikely to get proper attention during generation. The model doesn't drop it entirely — it just fills the gap with something plausible-sounding.
Is Claude Opus 4.6 or 4.7 better than Gemini 3.1 Pro for legal research?
For text-heavy legal research requiring citation accuracy, Claude Opus 4.6 is the strongest model right now — not 4.7. On MRCR v2 multi-needle retrieval at 256K tokens, Opus 4.6 scores 91.9% versus Opus 4.7's 59.2%. Opus 4.7 regressed on this benchmark when it launched in April 2026, despite improvements in coding and reasoning. Anthropic disputes the MRCR benchmark methodology, but independent testing consistently points to Opus 4.6 for retrieval-heavy work. Gemini 3.1 Pro holds well at 128K, degrades past 256K, and is the better choice if your legal research includes video depositions or recorded hearings.
Can I trust AI-generated citations from Gemini 3.1 Pro?
No. Not without verification. Even within the reliable token range, Gemini 3.1 Pro has fabricated case citations that sound entirely plausible but don't exist. In legal contexts, a fabricated precedent isn't just wrong — it can constitute professional misconduct. Always cross-reference every specific case name, citation number, and statute reference against a primary legal database before using it anywhere that matters.
What is the "needle in a haystack" test?
The "needle in a haystack" (NIAH) test is a standard benchmark for long-context retrieval. A specific fact — the needle — is hidden somewhere in a large body of irrelevant text — the haystack — and the model is asked to find it. Single-needle scores are strong for almost every frontier model at 1M tokens. The harder test is MRCR v2, which uses 8 needles simultaneously — much closer to real research conditions. That's where the gaps between models become significant.
Post a Comment