OpenAI's API Captured Less Than One-Third of ChatGPT's Cited Pages, Study Reveals

A preprint paper published October 11, 2026 found that Google search results and OpenAI's Responses API together matched only 26.2% of the pages ChatGPT cited — raising serious questions about how accurately marketers and SEOs can track AI citation behavior without direct access to ChatGPT's web app.

The findings carry significant weight for digital marketers and SEO professionals who rely on API-based tools to monitor how AI models source and cite content. If the proxies being used to simulate ChatGPT's behavior are missing nearly three-quarters of the actual cited pages, visibility strategies built on that data may be fundamentally incomplete. To understand why this gap exists, it helps to first understand what an application programming interface (API) is and how it works — because the distinction between API-based access and direct application access sits at the heart of this problem.

---

How the Study Was Designed and What It Measured

Researchers at the National University of Singapore and Xi'an Jiaotong University designed their study to answer a straightforward but consequential question: can Google search results or OpenAI's own API serve as reliable stand-ins for what ChatGPT actually cites?

To find out, they recorded the data flow within their own signed-in ChatGPT sessions — capturing the searches ChatGPT performed, which pages surfaced, and which pages were ultimately selected as sources. This direct observation method gave them ground-truth citation data that API queries alone cannot replicate.

The study ran 80 questions drawn from Search Arena, a public database of real user interactions with AI-enabled search tools. The questions were evenly split — 40 in English and 40 in Chinese — and covered four request types:

• Recommendation
• Fact lookup
• Information synthesis
• Analysis

Each question was run 20 times on GPT-5.3 Instant and GPT-5.5 Instant. Data collection spanned March 29 through July 29, 2026.

To build the proxy datasets, researchers pulled up to five pages of Google organic results per question using SerpAPI, averaging around 43 results per question. They also queried OpenAI's Responses API 20 times per question with web search enabled — all within three days of the original ChatGPT runs.

The paper is an arXiv preprint and has not undergone peer review.

---

The Gap Between Proxies and Reality

The headline numbers

The headline numbers tell a clear story. For GPT-5.5 Instant, the combination of Google results and the OpenAI API matched just 26.2% of cited pages. For GPT-5.3 Instant, the figure was slightly better at 28.3%. At the site level — treating subdomains as distinct entities — the combined proxies matched 48.7% and 50.3% of cited sites respectively.

Breaking it down further:

• Google's organic results alone accounted for only 6.4% of GPT-5.5 Instant cited pages and 9.4% for GPT-5.3 Instant
• The API performed better in raw source volume, covering 22.5% and 22.9% of cited pages respectively
• Overlap between API-cited pages and ChatGPT-cited pages remained limited — GPT-5.5 Instant cited 23.3% of the pages the API cited, compared to just 16.4% for GPT-5.3 Instant

Why the gap is so large

One reason for the gap: ChatGPT never queried using the exact wording of a user's original question in either GPT-5.3 or GPT-5.5 runs where searches were visible. The model consistently rewrites prompts before searching — a behavior OpenAI acknowledges on its help pages. In GPT-5.2 Instant, exact-wording queries appeared in only 5.5% of all runs.

Language transformation added another layer of complexity. Among runs involving Chinese-language questions, GPT-5.3 Instant conducted searches entirely in English 67% of the time. GPT-5.5 Instant did so 69.6% of the time. This aligns with findings from Peec AI reported earlier this year, which found that 43% of fan-out queries for non-English prompts ran in English.

Citation variability between runs

Result variability between individual runs compounded the tracking challenge further. For GPT-5.5 Instant, 44.5% of distinct pages returned for a given question appeared in only one of the 20 runs. For GPT-5.3 Instant, that figure was 40.8%. Like the AI equivalent of Schrödinger's cat, ChatGPT's citations appear to exist in a state of flux — shifting depending on when and how many times you ask.

This level of instability is not merely a technical inconvenience. It reflects a broader challenge that businesses face when building strategies around the risks and limitations of artificial intelligence in business decision-making — outputs that appear authoritative can vary substantially across identical inputs.

---

What SEOs and Marketers Should Do With This Information

Monitoring recommendations from the researchers

The researchers offered concrete monitoring recommendations that practitioners can act on now:

1. Track citations at both the individual webpage level and the site level simultaneously — since proxies match sites far more reliably than specific pages
2. Run the same prompt multiple times rather than relying on a single query result
3. Account for the fact that ChatGPT rewrites prompts before executing searches

Practitioners using commercial tracking tools should verify whether those tools collect data via the API or directly through the ChatGPT application — a distinction the paper identifies as critical. The study did not test any commercial tracking tool directly.

What model updates mean for your strategy

Looking ahead, the paper's findings apply specifically to GPT-5.2 through GPT-5.5 Instant. GPT-6 is now rolling out to eligible Plus, Pro, Business, and Enterprise users while GPT-5.6 Luna becomes the default for Free and Go users. The authors explicitly state that monitoring strategies should be re-evaluated with each model change.

Sistrix data from around the GPT-5.5 transition in May showed day-to-day citation variation in German-language ChatGPT results spike to 47% — against a normal baseline of 1% to 2% — underscoring how dramatically model updates can shift citation patterns.

Until researchers repeat this comparison using GPT-6, the figures in this paper represent the most reliable available picture of ChatGPT citation behavior from March through July 2026.

The practical takeaway for digital marketers

For digital marketers, the practical takeaway is direct: API-based monitoring captures a real but partial picture of AI citation behavior. Strategies that depend solely on proxy data risk overestimating or misidentifying which content earns visibility inside ChatGPT.

Understanding how artificial intelligence works at a foundational level becomes increasingly relevant here — the way large language models search, rewrite, and select sources is fundamentally different from how traditional search engines index and rank content, and treating them as equivalent leads to measurement gaps exactly like those this study identifies.

Combining site-level and page-level tracking, repeating prompts across multiple runs, and revisiting methodology after model updates are no longer optional best practices — they are baseline requirements for accurate AI search visibility measurement.