Reliable means repeatable. You need a fixed list of 20 to 50 real questions from your audience, asked under the same conditions every time, each question several times, and you record only three states per run: mentioned, cited, absent. A single chat is not a measurement, because the same question can return different sources on every run. The numbers become dependable only through repetition over weeks.
- The core metric is the mention rate: the share of runs in which your brand or domain appears in the answer.
- Ask each question several times. Repeated answers to the same question name different brands in different orders.
- Fix the conditions: account, memory, location, model, search on or off. Each of these shifts the result.
- With 30 questions and three runs each, a rate around 20 percent has an uncertainty of roughly plus or minus 8 percentage points if you treat the runs as independent.
- A second, independent signal is the "AI assistant" channel in Google Analytics 4.
- The finding is the trend over weeks, not the value of a single week.
Why does a single query prove nothing?
Ask ChatGPT the same thing three times and you will often get three different source lists. That is not a bug, it is how the system works. A research group around Edward Malthouse notes in a paper on evaluating brand recommendations in language models (September 2026) that repeated responses to the same query can produce different brands and rankings. It therefore estimates how often and how prominently a brand appears through repeated sampling. OpenAI's help article on searching the web with ChatGPT puts it briefly: placement is not guaranteed.
The platform also moves. Our analysis of PromptWatch data on the collapse of Reddit citations in ChatGPT shows how quickly shares can shift: around 3.8 percent of citations for weeks, then an average of 0.52 percent. If you check once a quarter, you see such breaks weeks later and cannot separate them from the normal spread of your own measurement.
Usage shows why the effort pays off. According to a Bitkom survey from 20 November 2025 of 1,156 people aged 16 and over, half of German internet users at least occasionally use AI chats instead of classic search engines. Among 16 to 29 year olds, it is two thirds.
Which three metrics matter?
Mention rate, citation rate and context show whether and how ChatGPT names you. We cover the general framework for all AI systems in How do I measure my visibility in AI answers? For ChatGPT, calculate across runs, not questions: 30 questions times three runs gives you 90 observations. You only need the question level for your list of gaps.
| Metric | What you count | Made-up example |
|---|---|---|
| Mention rate | Share of runs in which your brand or domain appears in the answer text | 27 of 90 runs, so 30 percent |
| Citation rate | Share of runs in which your page also appears as a linked source | 12 of 90 runs, so 13 percent |
| Context and tone | Whether the mention is factually correct | Named as a "monitoring dashboard" although you produce content: counts as a mention, flagged as a misattribution |
Keep mentions and citations separate. A mention creates brand contact, a citation can also bring clicks and can be verified.
Which conditions shift the result?
Account, location, search mode and model all change the outcome. If you measure signed out from Hamburg today and signed in with memory next week, you are comparing two different experiments.
| Condition | Why it shifts the result | How to fix it |
|---|---|---|
| Account and memory | With memory on, ChatGPT can use saved memories when it rewrites your question into search queries | Measure signed out, since web search is available without an account, or use a test account without memory |
| Location | ChatGPT estimates an approximate location from your IP address and may share it with search providers | Always measure from the same country, without a VPN. For local questions, put the city or postal code in the question |
| Search on or off | ChatGPT searches automatically when a question benefits from current information, so not for every answer | Turn on search manually via "View all tools" and count answers without sources separately |
| Model and plan | The mode in use can affect where results come from, as we describe for Labrador, ChatGPT's own index | One model per series, note any change in the log |
The details on memory, location and automatic search come from OpenAI's help article on web search.
How do you set up the measurement?
Seven steps turn individual queries into a series of measurements:
- Fix the question list. 20 to 50 real questions from your audience, relevant commercially or in substance. Phrase them neutrally: "Which software helps you get cited in AI answers?" measures visibility. "Why is flize the best GEO tool?" only measures how strongly the question steers the answer.
- Set up a log. One row per question and run, with the fields from the template below this list.
- Ask each question three times, in a new chat each time. No follow-ups, no refinement, otherwise you measure your follow-up instead of the answer.
- Assign a status. "Cited" requires a linked source. A bare mention of the name is a mention.
- Calculate rates and note the uncertainty. With 90 observations, a rate around 20 percent has roughly plus or minus 8 percentage points of room if you treat the runs as independent. A jump from 20 to 25 percent is not yet a success.
- Repeat weekly, with identical questions and identical conditions. If you change the list, a new time series begins.
- Cross-check with visitor data. Compare the trend in your citation rate with visits from ChatGPT.
Date | Question | Model | Run 1-3 | Search on (y/n) | Status (mentioned/cited/no) | Sources | Context noteWhat does Google Analytics add?
Google Analytics 4 shows how many people actually reach your website from ChatGPT. According to Google's help page on the default channel group, the "AI assistant" channel covers users who arrive through sources such as ChatGPT, Gemini, Deepseek, Copilot or Grok. AI Overviews and Google's AI Mode are excluded. How to analyze the channel is covered in How can I tell in Google Analytics which visitors come from ChatGPT or Perplexity?
That is a good second signal, not a replacement. You only see visits when someone clicks. If your mention rate rises while the channel stays at zero, you probably have mentions without citations, so without a link.
Can I automate the measurement?
Scripting it through the ChatGPT interface is risky. OpenAI's Terms of Use for Europe prohibit automatically or programmatically extracting data or output. Business customers fall under the OpenAI Services Agreement, effective 1 January 2026. Its section 3.3 (f) prohibits extracting data from the services except as the services permit. In practice: measure manually, or use a provider whose data collection relies on an access route intended for that purpose. This is our reading of the terms, not legal advice.
How does a number become a task?
Start with the questions where three runs in a row come back "absent". Your website either does not answer these questions or does not answer them in a citable way. The flize Answer Check compares the questions it finds with your website and shows the status and the location down to the section. What ChatGPT favors when choosing sources is covered in How do I get cited in ChatGPT?
How many runs per question make sense?
+
Three per measurement day is a good compromise between effort and insight. More runs reduce the spread, but take time that is often better spent on more questions. What matters more than the number of runs is that it stays the same every week.
Do I have to measure signed out?
+
It is the simpler option, because web search works without an account and it rules out memory and chat history as sources of noise. A dedicated test account with memory turned off works too, as long as you use it for nothing else.
Can I add questions along the way?
+
Yes, but separately. New questions get their own start date while the original core list continues unchanged. Otherwise any change in your rate is explained by the changed list rather than by your work.
Does a mention without a link count at all?
+
Yes, as a mention. It creates brand contact, even if it produces no visit and therefore shows up in no analytics report. That is exactly why traffic from AI sources is not enough as your only metric.
