Does Google Search Visibility Carry Over to AI Overviews?

Abstract

Do websites appearing more often in Google's organic results also appear more often among Google AI Overview sources? In this historical benchmark, the observed website-level Spearman association was moderately-strong: 0.533, across 7,468 questions and 2,053 websites appearing on at least five eligible questions. Pearson was 0.863. The central 95% question-resampling range for Spearman was 0.541 to 0.578. This range describes benchmark-composition sensitivity, excludes repeated-run and temporal variability, and is shifted above the observed coefficient by sparse-count effects. It is not a calibrated population-confidence bound.

The website-frequency rule changes the conclusion. Including all 30,653 observed websites makes Spearman slightly negative, −0.083. Removing ten dominant websites leaves the recurring-site coefficient close to its primary value, at 0.526. On individual questions, an average 41.8% of Google-listed websites appear among AI Overview sources, while 49.6% of AI Overview websites appear in Google's saved results. The hypothesis is therefore supported, conditional on recurring benchmark websites.

1. Research question and contribution

The hypothesis is a positive association because AI Overviews use web search. This is a mechanism to investigate, not a conclusion to assume. We examine how strongly saved Google organic presence and AI Overview source presence associate, how that relationship varies across query groups, and whether rare or dominant websites change its interpretation.

The original researchers already measured source overlap and domain prevalence. Pinnacle adds an explicit website-level association analysis, with documented availability cohorts, deduplication, website identity and sensitivity checks. We make no first-ever comparison claim. Related work examines optimization interventions and search-source composition. GEO, C-SEO Bench, Chen et al., Kirsten et al..

Version 1.1.0 narrows the active study to Google versus AI Overviews, at the user's request after the version 1.0 results were known. The metrics and relevant sensitivity rules were specified before the original correlations; this scope amendment is not a blinded or pre-result specification. The methodological history and resampling interpretation correction are retained in methods/.

2. Dataset and evidence audit

The pinned repository revision is 58f57d7db8c932bc67dcd9b94474db3223d65452; the paper version is arXiv 2604.27790v1. Original bytes, retrieval times, download URLs and SHA-256 checksums are preserved locally. The actual source archive is benchmark_dataset_links.zip, resolving the singular filename in the README. All 11,500 source-link IDs agree with the main query metadata on query text, source group, label and AIO flag. Pinned repository.

The paper describes December 7–8, 2025 collection through SerpAPI, simulating a mobile device in Newark, New Jersey. links_serp represents saved organic results; links_aio represents saved AI Overview sources. These are historical API observations. The final raw-response-to-release transformation and complete status logs are unavailable, limiting verification of extraction details and organic rank. We call the outcome saved source presence and do not interpret reference order as endorsement. Collection protocol.

Six Google lists are empty for unknown reasons. The AIO flag is positive on 7,540 records: 56 have no saved sources and 7,484 have usable AIO sources. Another 3,960 have no recorded AIO. Five AIO URL tokens are null; they are excluded individually while other sources and original positions are retained. Empty, failed and absent observations are not interchangeable website zeros. Long AIO lists include Google Maps review/place links; a Google-hosted review counts as a Google-hosted source, not as the discussed business's own website.

Eligibility precedes deduplication. For identical original query strings, the lowest eligible numeric ID is retained; distinct wording and localized needs remain separate. The full release has 11,471 literal distinct strings and 11,467 after lowercase-and-trim normalization. The latter is a sensitivity. Primary eligibility requires only usable Google and AIO lists, and necessarily excludes many searches without recorded AIO sources.

The nine groups describe query provenance, not industries or demand: ORCAS; Amazon Retail, Retail Q and Retail Comp; NQ and NQ keywords; ELI5; Debate; and Localized. Retail questions/comparisons and NQ keywords are synthetic reformulations. Verified parent IDs are unavailable, so row offsets are not used to invent query families. ORCAS labels are automated classifier outputs; Abstain remains unclassified. Deterministic samples cover every group, unusual list lengths, null tokens and hosted websites, with parsed records checked against original bytes. Maximum list lengths are 11 Google and 107 AIO; no arbitrary ten-source truncation is applied. Intent-classification reference.

3. Measures, arithmetic and resampling

Website identity and visibility

A checksum-pinned offline Public Suffix List, including private suffixes, identifies publishing websites. Ordinary subdomains collapse to registrable domains; hosted tenants remain distinct. Explicit rules preserve WordPress/Substack tenants, Google Sites paths, Medium author/publication paths and subdomains, and Wix site paths. Address styles are not merged through unsupported alias inference, separate companies are not combined by ownership, and redirects are not revisited. Registrable-domain-only normalization is a sensitivity. Public Suffix List.

For website $d$ across the same $N$ eligible questions:

Here, $S_G$ and $S_A$ are the question's Google and AI Overview website sets. Each score is the fraction of eligible questions containing the website. Several pages from one website on one question count once. Questions receive equal weight; scores do not measure search demand, traffic or conversions. Absence from a usable saved list means absence from that list, not from the wider web.

The primary universe retains websites present on at least five distinct questions in the union of the two systems. A website present in both on one question counts once toward this rule. Google-only and AIO-only websites are retained. The rule keeps 2,053 of 30,653 observed websites and excludes 28,600. This defines a recurring-site population rather than an inconsequential cleaning step.

Association and a worked example

The primary association is Spearman correlation:

Spearman asks whether higher Google visibility accompanies higher AIO visibility; ties receive average ranks. Pearson separately measures association in score values and is more sensitive to very frequent websites.

This artificial four-question example uses a one-question threshold for teaching:

The vectors $G=(4,3,1,0)/4$ and $A=(1,2,0,4)/4$ give $\rho_s=-0.4$. One-sided websites c and d remain included; repeated pages cannot inflate their counts.

What the resampling ranges mean

Each analysis resamples eligible questions 5,000 times with replacement, using fixed seeds and NumPy PCG64, recomputes scores and preserves its original website universe. Reported ranges span the 2.5th–97.5th percentiles. They concern question-composition sensitivity, not repeated provider runs, temporal change, personalization or market uncertainty. Unknown query-family dependence remains; excluding known synthetic groups tests part of that concern.

Verification exposed sparse-rank bias: a question is omitted from approximately 37% of ordinary bootstrap samples. A website observed only on that question can then become zero in both systems, creating joint tied zeros and shifting rank correlation upward. The primary range is above the observed point; the all-site range is positive despite a negative point. We preserve these computed values but withdraw the original mechanical interval-sign hypothesis decision. This documented interpretive correction addresses validity. The ranges are not calibrated population-confidence bounds or sufficient evidence of significance.

4. Main finding and sensitivity checks

The observed rank relationship is moderately-strong among recurring websites. That magnitude label is an interpretation, not a universal disciplinary cutoff. The adjacent range describes benchmark composition only, excludes repeated-run and temporal variability, and has the sparse-count bias explained above. Pearson's larger value does not imply equally close agreement throughout the website ranking.

Including all observed websites reverses the rank association. The all-site population contains 9,598 Google-only and 11,307 AIO-only websites. These rare, one-sided appearances matter. The positive bootstrap range cannot override the negative observed coefficient because joint-zero bias is severe in this population. A universal positive relationship across observed websites is unsupported.

The ten websites with greatest distinct-question union frequency are removed under a predetermined rule. Question denominators stay fixed. Spearman changes little and Pearson remains approximately 0.867, so a few dominant sites do not solely explain the recurring-site relationship. Normalization and duplicate-key changes are also small. Excluding synthetic groups raises Spearman to 0.576 but does not prove independence among remaining questions. None of these checks resolves the rare-site reversal.

5. Query groups and available intent labels

Each group uses its own eligible questions and five-question website universe. Correlations require at least 100 questions and 20 websites. Differences are descriptive and are not effects of an optimization intervention. Ranges retain the sparse-count qualifications above.

Eligible group coefficients range from 0.293 for Retail Comp to 0.637 for NQ. Amazon Retail is too small under the declared rule. Localized searches have a lower coefficient than NQ in this benchmark; that contrast does not establish which search type will behave similarly elsewhere.

Navigational and Instrumental ranges are broad. Transactional is omitted because only 72 questions and 13 qualifying sites are available. Abstain remains unclassified despite adequate counts. These distinctions come from a classifier, not manual intent assessment or industry classification.

6. Individual searches and website differences

Website-level association and same-question overlap answer different questions. For one question:

The first share asks how much of Google's website list also appears in AIO; the second asks how much of AIO's list appears in Google. For $S_G=\{a,b,c\}$ and $S_A=\{a,d\}$, they are one-third and one-half. Jaccard instead divides the intersection by the union. Question-level averages use all eligible questions and do not impose the website-frequency threshold.

Substantial same-question divergence remains. Exact URLs retain text fragments; stripping only the original authors' #:~:text= suffix raises the Google-direction share from 9.0% to 29.8%. Domain identity, exact strings and fragment-adjusted URLs are distinct outcomes, not interchangeable estimates. None establishes that a retrieved page supports a claim.

Examples are nearest the minimum, median and maximum website Jaccard, with lowest numeric ID breaking ties. They illustrate divergence and agreement, not representative proof. Query wording is omitted because redistribution permission is uncertain; pinned acquisition and IDs enable original inspection. Verified organic rank is unavailable, so rank-by-rank analysis is omitted.

These signed percentage-point differences use the same questions. YouTube has greater saved AIO presence; Reddit and Facebook have less. Google-hosted reviews still require distinguishing the source website from the discussed business. The figure shows up to ten sites in each direction with absolute gaps of at least one point; complete differences remain in results. No gap measures traffic or recommendation strength.

Conclusion

Google visibility and AI Overview source visibility have a moderately strong positive rank relationship among recurring benchmark websites. The pattern survives dominant-site removal and several routine sensitivities, but slightly reverses when all rare websites are included.