There is an important distinction between AI that simply works today and AI that endures at scale. Many companies heavily optimize the first without even asking themselves if they are building the second.
Speed without discipline and strategic direction is a liability, not an asset. The hardest part of building AI at scale is not getting a model to work once. It’s about building systems that continue to work, scale beyond individual teams and use cases, and constantly improve over time.
Today’s AI systems do more than just predict and optimize. They talk, reason and act more and more. An autonomous system that makes decisions on behalf of a traveler creates a very different set of expectations around reliability, governance, and accountability. As AI takes on more of those roles, the principles behind how these systems work matter more than ever.
We’ve been applying AI and machine learning (ML) to the entire traveler journey for years, from personalization, classification and recommendations to fraud prevention, customer service and, most recently, generative and agentic AI experiences. That deep experience is what led us to develop a set of machine learning and AI principles to guide how we build, deploy, and evolve AI systems across our company.
The goal is simple: make sure the systems we build create real business value, scale, and operate securely. These principles define how we measure, design, govern and operate our systems.
From principles to practice
Publishing principles are the easy part. The hardest and most important work is turning them into operational mechanisms: recommendations, requirements, tools, and release processes that teams actually use.
We have started using ‘agent release’ tollgates – a set of recommended and, in some cases, required checks before releasing agent AI features. These tolls translate principles like clear ownership, risk-based governance, assessment, secure deployment, and monitoring into concrete expectations for teams.
Some of these recommendations and requirements are already being automated and integrated into the software development life cycle (SDLC). Over time, the goal is for these expectations to be integrated into the way we design, evaluate, approve, launch, and monitor AI systems from the beginning.
Results: measure what really matters
The first test for any model is whether it improves a business outcome and ultimately the traveler experience, not just whether it improves a technical metric.
-
Align models with metrics with business impact: Each ML effort should tie directly to a key business outcome or traveler experience metric. Technical optimizations are useful midpoints, not end goals..
-
Optimize cost return: The value a model creates has to justify what it costs to develop, train and monitor it, in addition to the operational complexity it adds. Favor solutions that generate a lasting impact in relation to what it costs to execute.
-
Justify complexity against solid foundations: Complexity must be earned, not assumed. Start with a solid foundation: an existing general model, a simple heuristic, a ready-to-use solution. Look for specialized models or more complex architectures only when simpler options really can’t meet the standard.
-
Require both offline and online assessment: No model reaches wide deployment solely with offline validation or jumps directly to A/B testing. Each model must work in both online and offline assessments. Over time, our offline assessments should reliably predict what we see online.
Design: building systems that go beyond the teams that build them
Making a model work is a challenge. Making its value extend beyond a single piece of equipment or use case is the hardest.
-
Build on shared foundations; specialize only when justified: Encourage platform-wide shared foundations for core capabilities, data representations, and model building blocks. Specialization should be based on those foundations, not isolated accumulations, so that when the foundation improves, gains flow throughout the organization.
-
Treat data like a first-class product: The quality of a model is limited by the quality of its data. We need to maintain strong pipelines, clear lineage, reproducibility, and reusable features built with documented ownership, clear schemas, and service level agreements that other teams can rely on.
-
Prioritize generality over local optimization: When two approaches work similarly, prefer the one whose learnings, assets, and operating patterns can be reused across teams, brands, and use cases. We must optimize not only local performance, but also how quickly improvements can spread throughout the company and compound over time.
-
Minimize and end manual business rules: Manual rules are sometimes necessary for policy, security, or compliance, but they should be explicit and reviewed periodically, never silent patches for weak models or a source of ongoing maintenance debt.
-
Reproducibility and traceability by default: Training data, features, configurations, evaluation results, deployment versions, and key decisions must be documented and retrievable. That’s what allows you to debug a production issue months later and transfer ownership without losing institutional knowledge.
Trust: ownership, governance and responsible operation at scale
The bar for implementing AI is not only "works?" Is "Can we support it?" Trust is not something that is added at the end; It is earned over time and maintained throughout the life cycle of each model we ship.
-
Assign clear ownership and responsibility: Each model needs a defined ownership throughout its lifecycle: a business owner, a product owner, an AI owner, and an operational owner. It doesn’t have to be four people, but the responsibilities should be explicit. Who is responsible for the results? Who is responsible if the model deviates? Who responds to the incident at 2 am? Without this, models become orphans and problems arise with no one to own them.
-
Adhere to standards and governance: AI and ML models must use approved platforms and comply with standards, release gates, and governance processes established by the company. Operating outside these barriers requires a clear and defined path to correction or disapproval, rather than an open exception.
-
Govern proportionally to risk: The level of review, rigor of evaluation, and human oversight should scale with a model’s impact. A customer-facing model that impacts pricing or availability for millions of travelers requires a much higher bar than an internal tool used by a small team. For high impact, security sensitive or highly autonomous systems, human presence checkpoints are integrated from the beginning.
-
Design for fairness, privacy and transparency: We actively test for unwanted bias, have strong data guardrails, and support explainability when decisions significantly impact users. These are built in from the beginning, not added.
-
Design for safe deployment, rollback, and control: Deployments are progressive, with rollback routes, rollback mechanisms, and circuit breakers ready before launch. The ability to safely undo a deployment is as important as the ability to send it.
-
Continuously monitor and adapt: Once up and running, teams must actively monitor quality, drift, latency, cost, and business performance and retrain or recalibrate when data changes. A team should always be able to explain how their model is performing now, not just how it performed when it was released.
These principles do more than define how we build. They define what we are willing to ship and how we support it. In a world where AI systems are increasingly important and make real decisions for real travelers and partners, these standards are important. Applied consistently, they create responsible and lasting AI.
Xavi Amatriain is Director of AI and Data at Expedia Group
Xavier will share more details about Expedia’s architecture during his session at VB Transformation on July 14 at 11:10 a.m. PT. He will discuss: "Expedia’s plan to create autonomous agents for high-risk transactional systems."
Interested in attending VB Transform 2026? Record here. A select number of free passes are also available for senior technology leaders. Contact us to get yours.
