Our research standard • Transparent methodology

Manual AI Visibility Audit Methodology and Evidence Standards

If an audit cannot explain what it tested, what it observed and what remains uncertain, its conclusions are difficult to trust. This is the framework we use to design evidence-backed client investigations.

What a research standard should prove

What is a manual AI visibility audit methodology?

A manual AI visibility audit methodology defines how real buyer questions are selected, how comparable AI answers are captured, how cited sources and competitors are reviewed, and how observed facts are separated from hypotheses. A defensible report explains the question sample, test conditions, evidence and limitations rather than claiming a universal AI ranking.

The method behind our manual AI visibility audit service is human-led: an analyst investigates what a response actually said, opens supporting links and identifies website issues that can be independently checked. It does not require pretending to know an AI provider's private search strategy or model weights.

Specific research scope—markets, languages, platforms, question counts, repeat runs and reporting depth—is agreed for a project. The process below describes standards and possible deliverables, not completed client results. For a DIY walkthrough visit How to Check AI Visibility; for the mathematical interpretation of responses visit How to Measure AI Visibility.

Question selection and bias control

How are representative AI visibility audit prompts chosen?

Start with real customer needs, not queries that already mention the brand. A reliable panel groups prompts by buyer intent and documents why each was included.

Illustrative prompt groups; not real tested responses.
Buyer intentExample research questionWhat to observe
Unbranded discoveryWhich providers offer [relevant service] to [buyer]?Are businesses recommended without prompting with their names?
ComparisonHow should I compare [service] agencies?Which criteria, references and competing providers appear?
Problem-led purchaseWho can solve [specific buyer problem]?Does the brand fit the practical purchase question?
InformationalHow does [specific service] work?Are explanations and citations relevant, regardless of shortlist presence?
Branded controlWhat does [real company] do?Is the business described accurately? Do not pool with unbranded discovery.

Choose prompts using actual buyer conversations, service descriptions, relevant keyword research and search evidence where available. Store an exact prompt ID and version so future tests remain comparable. Three repetitions per prompt can reveal obvious variation but cannot make a small panel representative of every user. Avoid artificially inflating a measured rate by inserting many branded prompts.

A transparent record, not a magic score

What exactly should a manual AI visibility audit record?

Each counted response needs enough context to be reviewed later. Planned runs and inaccessible answers are not zero mentions; they are missing observations and must be disclosed.

Record groupRequired fieldsReview check
PromptID, exact wording, buyer intent, panel version.Was the question actually the same between periods?
TestPlatform, visible mode, run number, date/time, language, locale.Are non-comparable conditions being blended?
AnswerCompleted/failed, response text or permitted export, reviewer.Was this genuinely returned by the stated platform?
EntitiesBrand mention, provider shortlist, competitors, factual errors.Is a mention incorrectly called a recommendation?
SourcesActual linked URL, source type, checked claim, access result.Does the linked page really support the claim?
DecisionEvidence class, confidence, affected URL, next action.Would a second reviewer agree with the interpretation?

Maintain separate fields for own-domain citations, independent reporting, directories, professional profiles and competitor sources. This prevents one source type from being mistaken for another. Record full context where allowed, since a single cropped sentence can materially change an interpretation.

Our process

Eight steps from buyer context to action.

The precise number of prompts, platforms and repeat tests is defined in the agreed scope—there is no one-size-fits-all score.

  1. Define the buyer journey

    Agree market, services, buyer intent, geography, decision criteria, competitors and success conditions.

  2. Research the question panel

    Develop representative discovery, comparison, service, objection and branded prompts. Record why each prompt matters.

  3. Capture AI observations

    Sample selected platforms and runs, preserving exact prompts, date/time, location or language where exposed, interface and answers.

  4. Verify linked sources

    Open citations and investigate relevance, freshness, accessibility and the claims the sources actually support.

  5. Investigate competitors

    Review co-mentioned businesses and repeatedly cited pages; distinguish direct observations from causal theories.

  6. Audit site foundations

    Inspect crawlability, canonical links, architecture, accessible service content, entity consistency and appropriate metadata.

  7. Write evidence-backed recommendations

    Classify confirmed defects, credible opportunities and experiments. Prioritize by potential impact, effort and confidence.

  8. Define repeat tests

    Preserve a stable panel for follow-up and pair visibility observations with business metrics where attributable.

What we record

Every observation needs context.

Observation fields

  • Exact question and intent category
  • Platform, interface and run identifier
  • Date/time and locale where available
  • Brand mention vs shortlist/recommendation
  • Named competitors and answer excerpts

Source investigation fields

  • Displayed citations and actual target URLs
  • Whether links were opened and checked
  • Source type and claim relevance
  • Inaccessible or mismatched evidence
  • Screenshot/export reference and researcher notes
Limits and ethics

What the data cannot tell us.

These boundaries are part of the service, not fine print.

A cited website is not necessarily the cause of an AI model's answer. A recommendation appearing in one test does not prove it will recur for all users. Brand mentions, citations, recommendations, referral clicks and qualified leads are different metrics.

We do not use fabricated testimonials, buy fake mentions, invent competitor results, or claim access to the private algorithms of AI providers. We protect unnecessary client information and document where a hypothesis needs further testing.

For a hands-on starting point, use our guide to checking AI visibility. For an agency-led scope and deliverables, see the manual AI visibility audit.

Why this matters

Clear evidence and transparent uncertainty make optimization decisions more useful. They also make it possible to compare observations responsibly over time.

Evidence behind the answer

How we verify AI citations and competitor sources

When a response shows citations, the visible links need examination. Counting the URLs alone does not reveal whether an organization was recommended, whether the material is current or which link caused a model to answer as it did.

Review the actual claim

Open each material citation and inspect the specific claim it appears to support: definition, geography, eligibility, comparison, pricing, service facts or recommendation. An accurate general definition is not evidence that the publisher offers a particular commercial service.

Check source accessibility and accuracy

Note redirects, HTTP errors, inaccessible content, outdated service statements or missing conditions. A broken or unrelated citation should be recorded as an evidence limitation rather than supplemented with invented source details.

Distinguish source relationships

Separate a business's own website from independent editorial coverage, directory claims, user posts and competing providers. A first-party service description is useful but does not constitute independent third-party endorsement.

Avoid attributing secret logic

A URL displayed in an answer, or a crawler request visible in logs, does not expose the model's hidden search fan-out or prove why a recommendation was generated. Causal explanations require stronger, controlled evidence and remain hypotheses otherwise.

Transparent inference

Confirmed defects, observed patterns and hypotheses must stay separate

These categories keep an audit useful without turning sample descriptions into unsupported ranking explanations. All examples below are hypothetical.

Evidence classificationExampleWhat can responsibly be concluded?
Verified website issueAn important service page has a canonical tag pointing elsewhere.Fix and retest the actual HTML; Google index changes are a separate outcome.
Sampled AI-answer factA competitor appears in 2 of 10 comparable completed answers.Describe only that panel and its test conditions, not population-wide share.
Verified source discrepancyA cited directory lists an obsolete service category.Document the mismatch and any authorized correction path.
Explanation needing a testUnclear service eligibility might contribute to missing shortlist mentions.Label this as an inference; a source gap is not proof of hidden ranking cause.
Observed business outcomeAn enquiry has a reliable documented attribution path.Report what the analytics actually establish, including attribution limits.

Our reporting distinguishes direct observations from inferred causes. A strong explanation is not a substitute for verifiable data; the same caveat applies when presenting apparently positive outcomes.

Integrity and repeatability

What quality controls apply to manual AI visibility research?

Human review should add scrutiny, not just more screenshots. The controls below are part of a responsible methodology and must be adapted to the agreement and available authorizations.

Review ambiguous classifications

Where mention or recommendation coding is disputable, inspect the full answer context, document how the decision was made and preserve any changed rule. Do not silently redefine the numerator between reporting periods.

Keep question and site versions

Save exact questions, source links, observation dates and the relevant site's change history. New prompts should be reported as new cohorts. Later comparisons should disclose model or interface changes when observed.

Protect client information

Use permissioned access and platform-compliant collection. Avoid inserting confidential client data into unapproved systems; agree storage, redaction, report recipients and retention before sharing captures.

Disclose missing evidence

Label unavailable platforms, failed tests and uncited answers explicitly. Never fabricate completed runs, customer results, public endorsements, author profiles or model recommendation guarantees.

Our approach follows established website requirements rather than a secret “AI schema.” Google states that its AI Search features do not require special Schema.org markup: official AI features guidance. Optional proprietary retrieval mapping remains an editorial research idea, not a proven ranking protocol.

A report clients can interrogate

What should an evidence-led manual audit report deliver?

Specific deliverables depend on the signed scope. A useful report can be broken into components that a client can read, challenge and use to plan verified improvements.

Scope and question appendix

Priority services and markets, selected buyer intents, actual prompt panel, test dates, accessible modes, repetitions, exclusions and any sample limitations.

Observation and source inventory

Completed answers, separate mention and recommendation coding, competing providers, cited URLs, link checks and evidence notes. Missing observations remain visible.

Verified findings and hypotheses

Technical defects, inaccurate external descriptions and repeatable answer patterns recorded separately from ideas that still require tests. No unexplained proprietary visibility score.

Prioritized changes and follow-up

Affected pages, proposed content or technical fixes, owner decisions, acceptance tests and a stable question set for repeat monitoring where included in the engagement.

Need the commercial audit scope?

See which parts of an investigation can be included and what information we would need to begin.

Explore manual audit services
Research standards, explained

Manual audit methodology: frequently asked questions

How many prompts make a manual AI visibility audit reliable?

No single number establishes reliability. Sampling should reflect buyer needs, market, accessible platforms and available research effort. Repeated tests help reveal volatility; results remain estimates for a defined panel, not all AI users.

Why test unbranded and branded questions separately?

Branded questions test whether a business is described correctly when its name is already known. Unbranded questions investigate whether a buyer could discover it without prior awareness. Mixing the two can inflate an apparently commercial appearance rate.

Can a cited URL prove why ChatGPT recommended a company?

No. The link shows a source presented in an observed answer, not every hidden retrieval or ranking step. We verify the visible claim and label causal explanations as hypotheses unless stronger direct evidence exists.

How do reviewers know whether something is a recommendation?

They inspect the exact answer and the user's intent. A business in a list of examples is not necessarily an endorsed provider. Ambiguous classifications need a consistent coding rule and evidence notes, not a guessed “AI rank.”

Does a manual audit require my Search Console login?

No for every initial observation. Public source and answer research may be possible without private access. Detailed indexing, property-level reports, logs and private analytics require authorized access and an agreed scope.

Does the method promise placement in ChatGPT, Gemini or Google AI results?

No. It documents what can be observed and improved. Eligibility and content usefulness may matter, but no audit methodology can guarantee a particular platform's citations, shortlists, rankings or qualified enquiries.

Start with the evidence

Know what AI answers actually show.

Request a scoped manual audit to understand how your business appears across relevant AI-assisted search experiences and what can be improved.