Why AI Gives Confident, Wrong Answers About Your Data

AI gives confident, wrong answers about your data because a model knows how business works in general but not how yours does: your definitions, your terminology, and which of your tables actually count. That is why a smarter model is not the fix. The fix is a semantic layer: a map between the model and your data that curates which data counts, puts the meaning next to each field, enforces who may see what, and stays current as the business changes.

The size of the gap is measurable. On a benchmark of real, messy databases (BIRD), a leading model answered only about 40 to 55 percent of questions correctly, against 92.96 percent for humans (Li et al., BIRD, 2023). That gap is missing context. Context is something you supply.

What follows draws on a talk by the CTO of Omni, an AI analytics platform, on how they built their data-analyst agent (named Blobby) on Claude. The figures attributed to that talk are the speaker's; the architectural argument holds regardless of any single number. I'm writing about it because it matches exactly what I see when I build these systems for clients, and what I learned running our own agency on AI employees.

Why does AI get my business data wrong if it is so smart?

Because intelligence and context are different things. A frontier model can reason about a P&L, explain cohort retention, and draft a pricing analysis better than most people you could hire. What no amount of raw capability gives it is knowledge of how your company defines its terms and stores its data.

The clearest example comes straight from the talk. Inside one company, the phrase "last quarter" meant two different things: in product and engineering it referred to the calendar quarter, while on the sales team it meant the fiscal quarter. Same words, same company, two correct answers depending on who is asking. No model resolves that from intelligence alone, because the information is not in the question and not in the model. It lives in your organization, in the heads of the people who would say "oh, you mean fiscal."

When a model has no briefing on your company, it behaves like a brilliant analyst who started this morning, has never met your team, cannot ask a clarifying question, and has been told to answer anyway. A smarter analyst with the same non-existent briefing makes the same confident mistakes faster. The failure is not gibberish you would catch in a second. It is a wrong answer wearing a right answer's clothes, with the same chart and confident summary as a correct one. I come at this from a mechanical-engineering background, where inputs have to lead to predictable outputs. A confident wrong answer is the worst kind of failure, because nothing about it signals the input was incomplete.

What is a semantic layer, and how does it fix this?

A semantic layer is a translation layer that sits on top of your data warehouse and gives the model a map for turning a plain-English question into the right query against the right data. The model never reasons directly over raw tables. It reasons over the map, and the map turns its intent into something safe to run.

This is the half of AI readiness most teams skip. Getting the agent access (connections, credentials, the tools to reach the data) is necessary, and I cover what an agent actually needs to do real work in what an AI employee is. But access is not understanding. An agent that can reach every table and does not know which one counts is not more ready; it is more dangerous, because now it can be confidently wrong across all of your data instead of some of it. The research bears this out: the BIRD benchmark was built specifically because earlier tests ignored real database contents, and models stumble on "dirty" values and the domain knowledge needed to connect a question to the data (Li et al., 2023).

Will a bigger or smarter model fix it?

Not on its own. The accuracy gap on real databases has closed mainly through multi-step pipelines that supply structure and verification around the model. Raw model size barely moved it. And the stakes of skipping that work show up in the failure rate of AI projects: Gartner forecasts that more than 40 percent of agentic AI projects will be canceled by the end of 2027, largely from cost and unclear value (Gartner, 2025). A confidently-wrong analyst is precisely the kind of project that gets quietly canceled. The cure is an architecture that grounds the model, which is what the next four jobs describe.

What are the four jobs of a semantic layer?

A semantic layer earns its name by doing four jobs. Get all four and a capable model becomes a reliable analyst of your business. Skip any one and the confident-wrong answers leak back in through the gap.

Curate. Real warehouses hold tens of thousands of datasets. In the talk's words, "there's 100 revenue tables, there's 100 opportunity tables," accreted over years of exports, staging copies, and abandoned pipelines. A human survives this on tribal memory; the model has none. Curation names the canonical table for each concept, defines how tables join, and marks the rest not-for-analysis, so the model stops choosing blindly among look-alikes.
Localize the context. Put the meaning next to the field it describes. A distant glossary nobody opens does not help the model. As the speaker put it, "context localized to the actual definition that the context refers to makes it all the better," the same reason a context file beside the code beats one giant document at the root. In practice you attach a few kinds of metadata to each field: a note written for the model, one or two sample queries, and sample values.
Permission. The layer is a boundary as well as a dictionary. A dashboard exposes the views you built; a question box exposes everything the agent can reach to anyone who can phrase a question, unless the layer stops it. Whether "what is the highest salary in the company?" returns an answer must depend on who is asking, enforced on every query before it runs, not on the model's good intentions.
Close the loop. Definitions change constantly, so a map drawn once drifts out of true. Every question is evidence: a fumble points at a missing definition, a repeated phrasing points at a synonym to add, a hand correction is a rule to write down. The agent or your data team proposes the refinement; a person who owns the definition approves it. I go deeper on running this without a team babysitting the agent in how to stop babysitting your AI agents.

Sample values deserve a closer look, because they are the subtle one. Show the model that a region field holds EMEA, NAM, and APAC, and two things happen: it infers these are world-region abbreviations, and when a user asks about "the United States" it knows to look for the stored value rather than the literal phrase typed, even surviving a typo. People type the way people talk; the data is stored the way the data is stored. Sample values bridge the two.

What does the semantic layer look like on a real question?

Here is the layer doing all four jobs at once on a single ordinary question: "How many pull requests were merged in the Omni repository last month?"

The agent reads the intent, recognizing that "PRs" means GitHub pull requests, which general capability handles for free. Everything after that does not. It finds the right data because you curated, landing on the one canonical pull-request dataset instead of a stale mirror. It resolves "the Omni repository" to the value actually stored, fuzzy-matching a typo, because you localized context with sample values. It stays scoped to what this user may see because you permissioned. Only then does it generate the query, run it, draw a chart, and summarize the result in a sentence.

Pull any one job out and that step fails quietly: no curation and it grabs the wrong table; no sample values and the filter returns nothing because the typed phrase never literally appears; no permission and it leaks. The agent feels smart. The layer is what makes it right. And when an answer still looks surprising, the reliable pattern is to let a person open and validate it: AI to build, a human to refine.

This is the same discipline I put around every AI employee I build: deterministic rails for what has to be exact, the model for judgment, and a named owner for keeping the map current. It is also why I tell people you don't need another AI subscription. You need AI that works on your business, not businesses in general.

Frequently asked questions

Why does AI give wrong answers about my own business data?

Because a model knows business in general but not your definitions, terminology, or which of your tables count. The missing piece is context that lives in your organization. Intelligence was never the bottleneck. Without a semantic layer to supply it, the model guesses, and a confident wrong answer looks identical to a correct one.

What is a semantic layer in AI analytics?

A semantic layer is a map between the model and your data warehouse that defines which tables count, what each field means, how tables join, and who may see what. The model queries through this layer rather than over raw tables, so plain-English questions translate into correct, governed queries.

Will a smarter or bigger AI model fix wrong answers about my data?

No. On real databases the accuracy gap has closed mainly through pipelines that add structure and verification around the model. Model size barely moved it. A bigger model with no map of your business produces a more fluent wrong answer, not a more correct one. Grounding is the fix.

How accurate is "talk to your data" on real databases?

Far below human accuracy without grounding. On the BIRD benchmark of real, messy databases, leading models scored roughly 40 to 55 percent execution accuracy against 92.96 percent for humans. Production systems close the gap with a semantic layer, curated metadata, and verification, not with raw model capability alone.

How do you stop an AI from showing data a user should not see?

Enforce permissions in the semantic layer, applied to every query before it runs, so the agent inherits the asker's access rights. A question box exposes everything the agent can reach unless the layer scopes it. The control belongs in the system, not in the model's instructions, which can be talked around.

Sources and further reading

Li et al., "Can LLM Already Serve as a Database Interface? A Big Bench for Large-Scale Database Grounded Text-to-SQLs" (BIRD) (2023): on real, dirty databases (95 DBs, 33.4 GB, 37 domains), ChatGPT reached 40.08% execution accuracy and GPT-4 with external knowledge about 54.89%, versus 92.96% for humans. Current leaderboard figures move; verify before reuse.
Gartner (2025): more than 40% of agentic AI projects forecast to be canceled by the end of 2027, largely from cost and unclear value.
Anthropic: Building Effective AI Agents: why the scaffolding and context around a model should be deliberate, and how grounding shapes results.
"Building the best agentic analytics harness," a talk by the CTO of Omni at Code with Claude, 2026: the source for "last quarter" meaning two things, the curate / localize / permission / feedback-loop jobs of a semantic layer, and the Blobby walkthrough. Stated figures are the speaker's and should be verified before reuse.

The pattern behind all of this (deterministic rails, AI for judgment, and a clear owner for keeping the map current) is how I build at Brixon AI, and we have run it on our own agency's stack since early 2026. If you want a foundation that makes your AI reliable about your own business rather than businesses in general, talk to me about where to start or read more about how we work.