As AI agents become the backbone of modern software development lifecycles, the challenge for large-scale engineering teams has shifted from simple integration to sustainable operation. Uber has navigated this transition by treating its AI infrastructure as a managed software factory, moving away from ad-hoc developer sessions toward a centralized, high-efficiency model routing architecture.
Scaling Operations and Cost Structures
Between February 2026 and mid-August, Uber saw its weekly active AI users grow by 7x, with total agent requests surging by 9.4x. Despite this exponential increase in volume, the company successfully stabilized its total AI expenditure after April. By implementing rigorous optimization protocols, Uber reduced the cost per 1,000 requests by approximately 34% and the cost per session by 52% compared to their respective peaks.
To manage this, Uber decomposes agent session costs into six distinct categories, ranging from initial adoption and engagement to planning, exploration, and error handling. The company tracks unit economics by measuring cost per user, input/output tokens per request, and prompt cache hit rates. For managed agents, performance is evaluated through outcome-based metrics such as F1 scores and Mean Time to Repair (MTTR), ensuring that any model replacement maintains or improves the value-to-cost ratio.
Pareto-Optimal Model Routing and Context Optimization
Uber employs a benchmark-driven approach to model selection, running both frontier models and open-weight alternatives through a unified interface to determine the Pareto-optimal balance of cost, quality, and reliability. The 'uReview' system, for instance, uses a benchmark of actual pull requests categorized by difficulty to validate model transitions. Similarly, the 'Uber SWE Benchmark' provides a standardized framework for evaluating how different models perform within the company's massive monorepo environment.
To combat token waste, Uber aggressively utilizes prompt caching and context compression. The company automatically compresses 1M-token context windows down to 400K and has extended prompt cache Time-To-Live (TTL) from 5 minutes to 1 hour to better suit conversational sessions. Furthermore, to mitigate context bloat when using the Model Context Protocol (MCP), Uber implemented dynamic tool loading—which only loads necessary tools for CLI interpretation—and a 'code mode' that bundles multiple tool operations into a single script. In SQL-related tasks, this 'code mode' approach has reduced token consumption by 50% to over 90% by minimizing the loading of intermediate results.
Shifting from Conversational to Managed Agent Environments
Operational efficiency at this scale is supported by an 'AI Context Graph' consisting of 24 million nodes. By integrating over 30 internal systems into this graph, Uber has significantly reduced the 'exploration cost'—the tendency for agents to spend more turns searching for information than actually generating code. Engineers are provided with real-time cost counters and dashboards that detect 16 distinct patterns of cost waste, allowing teams to assess the return on investment for their AI-driven workflows.
Moving forward, the priority for practitioners is to transition from individual, terminal-based sessions to managed agent environments. By centralizing model routing and execution harnesses, teams can avoid the inefficiency of assigning high-performance frontier models to simple tasks, instead routing complex reasoning to primary models while delegating routine operations to specialized, cost-effective sub-agents.
By decoupling task complexity from model selection, engineering organizations can maintain high-velocity development without the linear cost scaling that typically plagues AI-integrated pipelines.
