How Uber Saved 140,000 Hours Monthly Using Generative AI Agents
Video
The Problem at Hand
Uber's data platform processes approximately 1.2 million interactive queries monthly, with 36% of these coming from the operations organization. This group—comprising engineers, data scientists, and operations professionals—analyzes data from hundreds of thousands of tables across various domains to derive actionable insights.
However, the process of composing and executing queries was a bottleneck:
10 minutes per query: Each actor spent an average of 10 minutes composing a query.
Inefficiency Loop: Users would sift through datasets, run queries, and validate results in a repetitive cycle.
Wasted Time: The cumulative effect of this inefficiency led to significant lost productivity.
This challenge is not unique to Uber. It resonates across industries, from e-commerce to customer support, where operations teams grapple with similar inefficiencies.
Enter QueryGPT: The Hackathon Solution
In 2023, a team at Uber's hackathon introduced QueryGPT, a prototype designed to streamline the query-generation process. Here's how it worked:
Metadata-Driven Query Generation:
Stored 20 SQL queries with metadata (table and schema information).
Mapped natural language prompts to these queries.
Few-Shot Prompting:
Used a Retrieval-Augmented Generation (RAG) technique to fetch relevant queries.
Generated SQL queries in response to user prompts.
Initial Results:
Reduced query composition time from 10 minutes to 3 minutes.
Achieved an 18% productivity gain.
While this was a promising start, the prototype faced scalability and technical challenges, necessitating further iterations.
Challenges and Iterative Solutions
Key Challenges
Prompt-to-Schema Mismatch:
The system struggled to align user prompts with relevant schemas.
Token Limitations:
Some schemas had over 200 columns, leading to token counts exceeding GPT-4's 32k limit.
The Final Architecture
The refined system, powered by Azure OpenAI and GPT-4, demonstrated remarkable efficiency:
Context Optimization: Leveraged a context window of 128k tokens to handle large schemas.
Human Validation: Ensured precision through user acknowledgment of suggested tables.
Scalable Design: Addressed the challenge of querying across hundreds of thousands of datasets.
Uber's engineering team implemented a robust architecture combining SQL, RAG, agents, and custom configurations. Here's a breakdown:
Domain-Specific Curation:
Decomposed datasets into business domains/workflows (e.g., mobility, trips, support).
Allowed the system to focus on smaller, relevant subsets of data.
Intent Agent:
Classified user prompts to map them to the appropriate domain or workspace.
Likely employed a vector-store-based intent classifier for high accuracy.
Table Agent:
Retrieved domain-specific tables and displayed them in a user-friendly interface.
Enabled human-in-the-loop validation to ensure table relevance.
Enhanced RAG Pipeline:
Generated few-shot prompts tailored to the specific domain.
Sent refined prompts to GPT-4 for SQL query generation.
Real-World Impact
By the 20th iteration, Uber's Query GPT achieved a staggering 140,000 hours saved monthly across its operations organization. This success underscores the value of combining AI, domain-specific curation, and user-centric design.
Key Takeaways for Your Business
Uber's solution offers valuable insights for tackling similar challenges in other industries:
Break Down Data Silos:
Organize datasets by business domains to streamline data retrieval.
Implement Intent Detection:
Use AI-driven agents to map user queries to relevant datasets or workspaces.
Leverage Human-in-the-Loop Systems:
Involve users in validating AI-generated outputs for enhanced accuracy.
Iterate for Scalability:
Start small, learn from challenges, and scale iteratively.
The Future of AI in Operations
Uber's journey with QueryGPT exemplifies the transformative potential of generative AI in operational analytics. By reducing manual effort and empowering teams with intelligent tools, businesses can unlock unprecedented productivity gains.
Whether you're in e-commerce, customer support, or any data-intensive field, the principles behind Uber's success can guide your own AI-driven innovations.
Want to delve deeper into the technical details? Check out Uber's engineering blog here for the full story.