You spend weeks fine-tuning an AI chatbot. The answers are precise. Stakeholders approve it and you submit it. Three months later, the system reliably gets about a third of what users ask wrong. Nobody changed the model and nobody touched the instructions. The world moved, prices changed, a policy was updated, a product specification shipped a new version, and the underlying knowledge store didn’t move with it.
This is not hypothetical. It’s one of the most common production failure modes in enterprise AI right now, and most data engineering teams don’t have the right tools to detect it, regardless of how the AI system retrieves the data.
The failure that doesn’t seem like a failure
An AI application doesn’t care whether the retrieval is from a vector store, a document index, or an API call. Whatever the mechanism, nothing in a standard recovery process checks whether what it is serving is still correct. An obsolete pricing document is retrieved as reliably as a current one, because the system rates relevance or availability, not accuracy. A record with a missing field silently passes as cleanly as a complete one, for the same reason.
So the fault is invisible by design. Outdated or incomplete data still scores high in terms of relevance or passes all checks for which a data pipeline was created. The model responds with complete confidence because the retrieved context appears authoritative. All the panels you are viewing remain green. The system appears to be working. It’s just wrong.
I’ve seen a similar version of this happen outside of the AI context, on a fintech channel. An upstream system changed a field without notifying downstream users. The pipeline did not fail; it simply propagated incorrect values across dashboards because the system was only checking to see if the job had completed, not if the data was still correct. The problem arose only when a customer noticed something inconsistent. By then, the bad data had already moved downward.
Whether it is an obsolete document or a field that has quietly disappeared, the form of failure is the same: the absence of an error is not the presence of correctness, and without creating appropriate validation layers, nothing in the process could identify the problem.
Why this is a data engineering problem
Computers that suffer from this fault tend to misdiagnose it and tend to do it twice.
Blame the model: The first instinct is to blame the model, try a different LLM, and adjust the message. The real problem lies higher up, in the data engineering layer, the same instinct behind the fintech failure mentioned above: monitoring built for the pipeline, not the data.
Blame the recovery layer: Once the model is discarded, the next instinct is to blame the recovery or context layer and buy a better one. The timing is no coincidence: as companies introduce these systems into the world of real production, this gap is exactly what is starting to emerge, and the response from suppliers has been all over the place.
-
AWS only entered the "context layer" career with a knowledge graph that learns from the use of agents.
-
Snowflake’s new Horizon Context and Cortex Sense target the exact symptom This piece opened with: Agents giving wrong, safe answers because nothing governs the underlying business logic.
Both are real answers to a real problem, but they are a layer above it; a knowledge graph still depends on what feeds it.
The real problem lies higher up, at the data engineering layer. Teams check whether a job ran, not whether the data it moved is still true, an instinct that predates AI by years. Monitoring is designed for the process, not the data.
What’s really missing: data observability
Data observability is a well-known concept that does not receive enough attention in terms of how it is actually implemented. The relevant metric is not a percentage, but coverage: what fraction of critical data sets have a lineage that is actually queryable, rather than just living in someone’s head.
Uber built a platform dedicated to data quality and observability long before augmented recovery generation existed. Its Unified Data Quality platform supports more than 2,000 critical data sets and detects around 90% of data quality incidents before they reach downstream consumers.
Netflix solved a different part of the same problem, build an enterprise-wide data lineage system so anyone could answer where a data set came from and what touched it along the way. Map dependencies between Kafka topics, machine learning models, and experimentation, not just warehouse tables. Like Uber, the platform was built for humans and has now become more important with the rise of AI/LLM applications.
Between them, Uber and Netflix cover two of the four things worth building for. In practice, I think of it as four dimensions, each of which can be measured on its own terms.
Accuracy: Does each record conform to the shape and rules it is supposed to, correct field types, no unexpected null values, values within range? Tools like Great expectations and Soda handle this well: automated validation at the row and column level instead of manual checks after something breaks. Track the percentage of records that pass validation per run.
Freshness: Is the data still up to date relative to its source, not just up to date since your last check? Track time since last successful update per source, with one SLA per data set rather than an overall threshold, as some sources need an hourly update and others don’t.
Consistency: Does the same fact read the same everywhere it is stored or indexed? This fails silently, only appearing when two systems powered by the same source start to disagree. A periodic cross-check between downstream targets, flagging the mismatch rate above a threshold, is sufficient to detect it early.
Lineage: Can you trace any output back to its source and every transformation it went through? The same question Netflix built its system to answer.
None of this requires infrastructure that most data teams don’t already have. I know this because I have built it, not just defended it.
In surecustomer data arrived in whatever form the customer wanted to send it, and sometimes quietly wrong. The challenge was to build a system where bad data could be identified before it propagated downward. The same principles applied: validate what came in, understand where it came from, and prevent bad data from becoming someone else’s problem.
Great Expectations became part of that foundation: validation of schemas and ranges at ingestion, per-source SLAs for updating, cross-system checks to ensure file-level consistency and lineage. All of this was behind a write-audit-publish The pattern, where the data reached the staging stage, was validated and only moved down if it passed the required checks.
The result showed in the future: greater precision across the board, in reporting, in machine learning models, and in AI retrieval based on that same data.
What to do on Monday morning
If you are running recovery-based AI systems in production, the diagnostic question is not which model to test next or which recovery architecture to migrate to. There are four more specific questions:
-
Is the underlying data validated to the standards required by your consumers?
-
What is the oldest content currently offered with high trust?
-
Would two fragments from the same source ever disagree with each other on the same recovery result?
-
Could you trace where it came from if it turned out to be incorrect?
If you can’t answer those questions, then the gap is in the pipeline between your source systems and whatever your agent reads. This is a data engineering solution, not a model change or vendor migration.
Whether you’re building reporting pipelines, machine learning systems, or AI agents, correctness, recency, consistency, and lineage are what make data trustworthy. AI simply exposes weaknesses that have existed in data engineering forever.
