Legacy infrastructure, not the models themselves, is what’s really holding AI agents back. That was the shared conclusion of three infrastructure leaders: from LinkedIn, Walmart and Zendesk. in VB Transformation 2026.
The panel brought together Animesh Singh, senior director of AI platform and infrastructure at LinkedIn, Desiree Gosby, senior vice president of corporate technology services and technology strategy at Walmart, and Sami Ghoche, vice president of applied AI at Zendesk, each of whom described what really went wrong when they moved agents from pilot to production. Each came to the same conclusion from a different starting point: none of the bottlenecks they encountered were model problems.
What tied their answers together was a shared premise: most enterprise infrastructure was built for how humans work, not how agents work. The gap between those two speeds is where the real engineering happened.
Gosby put it clearly when asked what he’s learned about how to scale agents within Walmart’s own workforce. The goal, he said, is to ensure that "Don’t let engineering become the bottleneck of what we’re trying to do again."
Where was the bottleneck really?
Each company ran into a different version of the same wall: Infrastructure designed for the way people work doesn’t hold up once agents do the work.
At LinkedIn, the first bottleneck wasn’t a model, it was Kubernetes, which assumes containers spin up on demand, a process that takes seconds. Singh said that’s too slow for officers. The solution was to move from on-demand provisioning to pre-provisioned container pools that swap agent workloads in and out in real time.
A second, more complicated problem arose once LinkedIn allowed agents to control their own orchestration. A five-point grading system seemed clean, but the hallucinations kept coming anyway. Singh said the problem was structural: an LLM evaluating the output of another LLM shares the same failure mode as the object it is evaluating.
"We built our own harness, our own control flow, and pushed the LLMs to the leaf instead of them orchestrating the loop." Singh said. About 80% of the workflow is now deterministic and programmed code, with LLM used only when reasoning is required, and evidence of each step is saved to disk before the system moves forward.
Walmart’s bottleneck came from success. An agent harness placed directly in the hands of employees went viral internally, and what Gosby called "citizen developers" They began creating their own agents to solve problems that once required a formal engineering roadmap. The advantage was a real innovation. The downside was duplication, dozens of overlapping agents with no coordination. The solution was not to control the harness, but to develop governance to detect duplication, promote the best version of an agent, and put it into production without engineering becoming a bottleneck.
Zendesk found its bottleneck on the data side. Ghoche, who joined through Zendesk’s acquisition of Forethoughtwhich closed in March 2026, described what it called a public figure of 20 billion customer conversations in the Zendesk repository. The instinct is to hand that story to a large language model with a large context window and let it generate the agents a company needs. Ghoche said that doesn’t work. "You can’t really do that, so you have to really invest in the underlying data pipelines and all the data infrastructure that comes with that." said.
The role of open source
When it came to open source, all three leaders had a similar instinct: own what you can and lean on cutting-edge labs only where they still have a clear advantage.
Ghoche said his view is that most companies would prefer to own their models and infrastructure whenever possible, and that reasoning is what drives Zendesk’s own approach. The exception is frontier reasoning work, where labs still lead, although he said that portion of use cases is shrinking relative to everything companies do now with AI.
LinkedIn’s response was to build two subsystems specifically for independence. The first is what the company calls an AI gateway, a single interface through which each outgoing call passes to a model, regardless of the provider. The second component is a memory subsystem created to maintain the context independent of any model provider.
"Every outbound call that goes to an LLM, whether in a public cloud or on-premises in our own data centers, follows the same semantics, the same API calls. We can quickly switch between different providers," Singh said.
Walmart built its own internal gateway to remain vendor independent across three types of workloads: fully deterministic workflows, planning and reasoning workflows for open tasks, and a hybrid of the two. Compliance-intensive work remains deterministic by design; Governance, security, and evaluation pass through the gateway, regardless of which model is on the other end. Gosby said the choice between a frontier model and an open weight model comes down to what is most effective for the specific workload, not a fixed policy.
Tips for the modernization journey
Three councils emerged directly, each of them linked to the wall that a leader had already hit.
Invest in evaluations before anything else. Ghoche called it something common to every use case, internal or customer-facing.
"What is common to all of these are the evaluations. It will force you to analyze the problem, and once you have a solid set of assessments, you will be able to move forward much faster." he said,
Own your agent harness from day one. Gosby’s advice was to put the AI harness directly in the hands of employees from the beginning, along with the infrastructure to monitor what it produces.
"It will unlock a huge amount of innovation," she said.
Build for model and context independence. Ensuring flexibility is critical to success.
"Build for independence, whether it’s a frontier model of today versus an open source model of tomorrow," Singh said. "Keep that context within your company so you can reuse it when you ship the model or harness tomorrow." Singh said.
