Zillow, the real estate technology company, doesn’t have a single conversation with its customers. They jump from a phone screen to a loan officer to a real estate agent, sometimes for months or years, and expect context to follow. A single chatbot could never carry that thread.
In VB Transformation 2026Zillow senior vice president of engineering Toby Roberts and Glean co-founder and CEO Arvind Jain described how they built an AI architecture intended to convey context throughout that journey, and why context, not raw data, turned out to be the most difficult problem to solve. Zillow’s products affect approximately 80% of U.S. real estate transactions each year, and the company has been using AI long before ChatGPT existed.
"We quickly identified that we were going to need a persistent context layer that would meet our clients and professionals wherever they were." Roberts said.
Data was never the hard part
Roberts said Zillow’s AI effort began where most enterprise AI efforts begin: with the data itself.
"We started with a big effort to make sure our data was on the right basis." Roberts said. That meant a data mesh approach, clear data lineage, and a governance structure with permissions and identity attached to the data itself.
None of that turned out to be the difficult problem. The hard problem was building something that remembered where a customer was in their journey and carried them forward, no matter what surface they appeared on next.
"This layer of context has to live in order to support you where you are at any point in your journey." Roberts said. Zillow chose to own that layer rather than rely on a single external chat interface, a decision Roberts said the team made quickly once it looked at the form of an actual transaction rather than a single conversation.
Why Zillow built its own architecture and where Glean fits into it
Zillow built its own harness instead of routing customers through a single-model API. The team built on 20 years of machine learning history behind products like Zestimate, relying on smaller, fine-tuned models for specific tasks rather than a general-purpose model.
Internally, that harness runs alongside Glean. Roberts said Zillow now has thousands of Glean agents in production, handling repetitive tasks with tens of thousands of executions across the company. Glean’s argument, according to Jain, is to centralize that integration work once, through the Glean MCP gateway, rather than allowing finance, legal and marketing to rebuild their own connections to the same systems.
This centralization is also a cost lever. Jain pointed out two mechanisms: model routing, which sends most tasks to smaller, cheaper models instead of defaulting to frontier models, and precomputed context, which prevents an agent from burning tokens to assemble its own context from scratch.
"Claude is also very slow because the first part of putting together that context actually takes forever." Jain said. Instead, submitting that request through Glean, he said, can reduce token consumption by up to half.
What Zillow and Glean’s Approach Means for Businesses
Regarding data, costs and permissions, the session offered some practical takeaways for companies building agent AI in their own systems.
Build the measurement baseline before the AI push, not after. Roberts said Zillow’s ability to credibly attribute a 40% increase in shipped code to AI adoption is based on a foundation of DORA metrics the team implemented years earlier, not the AI launch itself.
Centralize context once instead of letting each team rebuild it. Jain’s central argument in favor of the Glean platform is that duplicate integration work between finance, legal and marketing teams is a hidden cost that most companies have not taken into account.
Don’t assume that permission inheritance is sufficient for regulated data. Even with a contextual, permission-aware platform, Zillow overlaid strict rules and ongoing compliance monitoring for its most sensitive categories, rather than trusting the architecture to handle them automatically.
Treat context as a cost lever, not just a capability. Model routing and precomputed context were the two mechanisms Jain pointed out to reduce AI spend, both aimed at reducing wasted token consumption rather than adding new capabilities.
"Models alone are not enough to bring AI automation within your company." Jain said. "You have to connect it with the context of your company."
