Study methodology
How we measure GEO ranking for Dénia restaurants
This article describes the full design of the measurement, published before we close a ranking. If the final ranking departs from what's described here, that's our mistake, and we'll correct and flag it.
By the ArribaIA team · Updated
Summary
We measure which Dénia restaurants AI assistants mention (via API, with a web-interface phase planned later) by continuously sending automated queries that cross cuisine, intent, language, budget, zone, and time slot. A second model acts as a referee to extract restaurant, position, attributes, and tone from each response. It's not a survey or an editorial opinion: it's a measurement of visibility and recommendation frequency, with limits we detail below.
Objective
What we want to answer
The objective is to document, with a reproducible method, which Dénia restaurants show up in several AI assistants' responses when asked where to eat, and under what conditions that response changes.
We are not trying to determine why a model cites one restaurant and not another: no provider publishes how its response system actually works internally, and we won't present as a finding something the experiment can't demonstrate.
Research questions
- 01Which Dénia restaurants do the AI assistants analyzed mention most often, and at what position?
- 02Does the response change based on the query's cuisine, intent, language, budget, or zone?
- 03Are there meaningful differences between the AI assistants analyzed?
- 04Does the response change depending on the time slot a query is sent in?
- 05What difference, if any, is there between querying via API and querying with the assistant's real web interface?
Experimental design
How the experiment is built
An in-house system generates and sends automated queries to each assistant, saves the full response, and passes it to a second model acting as a referee to extract structured data. No step in the process involves a human opinion about which restaurant is better.
Collection cadence
Measurement has no closing date: we send queries continuously, spread across several time slots a day, Madrid time, and periodically analyze what's accumulated up to that point. Every figure we publish is a snapshot of one specific cut, never a final total.
Query volume
We've already sent thousands of queries, distributed across every combination of the variables described below, and the count keeps growing every day. We don't publish an exact number here because it would be stale the moment we wrote it: the confirmed count is kept current on the study landing page's figures section and updated at every data cut we close for the ranking.
Variables
Each query crosses several of these variables at once, to get closer to how people actually ask rather than repeating a single script.
Cuisine
- Restaurants in general
- Japanese and sushi
- Asian
- Mexican
- Italian and pizzeria
- Valencian rice dishes
- Seafood
- Tapas
- Fine dining
"Restaurants in general" works as a reference query with no specific cuisine, not as one more cuisine category: we treat it separately in analysis, not as a direct comparison to the other eight.
Search intent
- Immediate booking
- Future booking
- Discovery
- Situational query
Language
- Spanish
- English
- Valencian
The design calls for an approximate split of 65% Spanish, 25% English, and 10% Valencian. We'll confirm the dataset's real split and correct this if it doesn't match the plan.
Budget
- Query with an explicit budget
- Query with no budget (the AI decides what to assume)
Dénia zone
- Dénia in general
- Les Marines
- Les Rotes
- The port
- The old town
- The beach
Queries that don't specify a zone are treated as "Dénia in general": we don't force a zone into questions where a real person wouldn't mention one.
Models
Which AI assistants we analyze, and how
The current phase of the experiment sends queries against providers' APIs, not their consumer interfaces. Using the API gives automation, control over each query's conditions, repeatability, and an exact record of what was sent and what came back, conditions that are hard to guarantee by hand at this volume with a web interface.
That also means these results describe API behavior, not necessarily the consumer interface a regular person uses. We cover that difference in detail in GEO in practice: API vs interface.
Providers queried via API in this phase
- OpenAI, with and without the web search tool enabled
- DeepSeek
User interface phase
A second phase is planned to repeat a representative sample of queries using each assistant's real interface. It's slower and harder to scale than the API, but it can better represent the product a real person uses, and it lets us check whether the API and the interface agree. We'll publish which providers and exact versions are included in that phase once it starts, not before.
API vs interface
Why this isn't the same as opening ChatGPT and asking
An API and an assistant's interface can share the same underlying model and still not give the same answer: the interface can apply personalization, conversation history, default-enabled tools, or product settings the API call doesn't include unless explicitly turned on.
This isn't our own assumption: the official documentation for this kind of tool describes API-based search as a configurable capability with controls (filters, sources, live access) that aren't always present the same way in the consumer product.
Extraction
How we turn a text response into structured data
Every response is saved in full, exactly as the assistant returns it. A second model, acting as a referee, reads that response and extracts which restaurants are mentioned, at what position in the text, with what attributes, and with what tone.
This automated extraction can get it wrong: it can miss an ambiguous mention, misclassify an attribute, or misread an indirect reference to a restaurant. It's a known limitation of the design, not a detail we hide, and we cover it further in the limitations section.
Definitions
What exactly we mean by position and by mention
- Mention
- Any identifiable reference to a specific Dénia restaurant within the assistant's response, whether as the main recommendation, an alternative, or part of a list.
- Position
- The order in which each mentioned restaurant appears within the response, start to finish. This is not a search-engine ranking position: it's the order of appearance within a prose or list-style answer.
- Normalized name
- Each restaurant's name is normalized with a deterministic process (same input, same output every time) to avoid counting the same place twice over a spelling or capitalization variation.
- Mention source
- Whether the restaurant appears in the response's text, in a cited URL, or both.
Attributes
What the system records per query
This is the data schema the system records for every query, both the ones already sent and the ones still to come. We publish it so it can be checked against the explorable dataset once it's available.
- The exact question sent to the assistant
- The full response returned
- Model and provider queried
- Whether the web search tool was enabled
- URLs cited in the response, if any
- Restaurants mentioned and their normalized name
- Position of each mention
- Sentiment expressed about each restaurant mentioned
- Cuisine, price, zone, terrace, and views, when the response states them
- Mention source (text, cited URL, or both)
Analysis criteria
How we tell a pattern from a cause
When an analysis derived from the dataset (by cuisine, by zone, by web presence) shows that an attribute appears more often alongside restaurants with more mentions, we call that a descriptive association or a correlation, not a cause.
We don't claim that having a terrace, having a website, or belonging to a particular cuisine makes a restaurant show up better in AI responses, unless the experiment's own design lets us isolate that variable, which this experiment isn't built to do in general.
Limitations
What this experiment cannot claim
- Any figure published here is a snapshot of one specific data cut: collection keeps moving after that snapshot, so the full dataset is always bigger than whatever was last published.
- Name normalization is deterministic but not infallible: an unanticipated spelling variant can slip through the process.
- Automated extraction with a referee model can misclassify data or miss an ambiguous mention.
- Sampling is designed to cover variable combinations, not to represent the real proportion of searches people in Dénia actually make.
- Results via API may not exactly match what a person would see using that same assistant's consumer interface.
- Each provider and model has its own behavior: we don't generalize a finding from one model to "AI" in general.
- The same model can answer the same question differently at a different time: we don't treat any single response as stable over time.
Reproducibility
What it takes to repeat this experiment
The design (variables, collection cadence, extraction process, and definitions) is the one described on this page, and it stays that way once results are published: if anything changes, we flag it with a modification date.
The same architecture is built to be repeated for another sector or another city, not only Dénia restaurants: it's the editorial and technical foundation ArribaIA will use for future local GEO visibility studies.
Sources and references
What documentation we relied on
We only list sources we checked directly against the provider's own documentation, not a third party's summary of one.
- Official documentationGoogle Search CentralAI Features and Your Website
Google's official documentation on how AI Overviews and AI Mode work within Search: there's no separate AI index, and no additional technical requirements beyond standard SEO best practices.
- Official documentationGoogle Search CentralSpam Policies for Google Web Search
Google's official policy on scaled content abuse, including using generative AI to create many pages without adding real value for the reader.
- Official documentationOpenAIWeb search (Responses API)
Official documentation for OpenAI's API web search tool: describes the citations and sources the API returns when search is enabled, and the available controls that aren't always present the same way in the consumer product.
- Official documentationDeepSeekDeepSeek API Docs
DeepSeek's official API documentation, used to confirm the endpoint format and the models available at the time the experiment was designed.
Frequently asked questions
Common questions about the method
Why use an AI model to extract data from another AI model's responses?
Because at this query volume, reviewing every response by hand isn't viable, and the alternative (a fixed text pattern) doesn't understand natural language reliably enough. It's a fast method, but with a real margin of error, which we acknowledge in the limitations section instead of presenting it as flawless.
Why not wait for the ranking before publishing the methodology?
Because publishing the method before the result is what makes the result genuinely auditable. If we published the methodology after the ranking, anyone could suspect we tuned it to fit what came out.
Will you update this page with real figures?
Yes. Every time we close a data cut to publish or revise the ranking, we update this page's figures (query volume, language split) and flag it with the modification date.
Want us to analyze your sector?
This measurement architecture is built to be repeated. If you run a local business and want to understand its AI visibility, or you work in research and want to propose a collaboration, write to us.