Why Your Brand Needs to Show Up on Answer Engines (and How to Monitor It)

Over the past twenty years, we’ve all been conditioned to think in search-first workflows.

You type a query into Google, skim ten blue links, and decide whom to trust.

That mental model is disappearing—quietly, but fast.

Today, more users are skipping the “search → compare → decide” loop entirely and going straight to answer engines like ChatGPT, Perplexity, and Claude. These systems don’t present links; they present conclusions.

And that shift has enormous implications for brands.

The Rise of Answer-Led Discovery

According to data from multiple generative-AI usage studies, nearly 45–60% of new users now start with an AI assistant when exploring a product or service category for the first time. In fact, Perplexity itself claims a 20–30% month-over-month rise in “shopping and services queries.”

That means the customer’s first impression is no longer shaped by your homepage or your Google ranking. It’s shaped by whatever an LLM decides to say about you.

If your brand shows up in that answer, fantastic—you’ve just inherited pre-trained trust.
If it doesn’t, you’ve silently lost a high-intent customer without ever knowing they existed.

A Simple Visual Thought Experiment

Imagine this scenario:

A user asks:

“Who are the top curtain repair services near me?”

The LLM responds confidently, naming three providers.
Your brand is either:

• In the answer → instant credibility

• Not mentioned → invisible

• Misrepresented → losing trust without knowing why

This is the new discovery funnel.
And right now, most brands aren’t even aware it exists.

Why Monitoring Your Brand on ChatGPT & Perplexity Is No Longer Optional

Here’s the uncomfortable truth:
LLMs don’t pull answers from a single database—they synthesize information from various sources: review sites, listicles, Reddit threads, local directories, news mentions, and your own content (if it exists and is structured well enough to be recognized).

If they aren’t citing you, it’s usually because:

• Your content isn’t tuned for answer-engine style queries

• Your competitors have better-structured or more recent content

• Third-party sites describe your category better than you do

• Your brand simply isn’t “on the radar” of the model yet

In every case, the fix starts with monitoring.

What Monitoring Actually Gives You

By actively tracking your brand visibility across multiple queries, you begin to see:

• How often ChatGPT or Perplexity cite you

• Which sources they rely on

• Which competitors consistently show up

• What real customer queries in your category look like

• Which themes or topics you’re missing in your content

In Today’s Episode

In the video, I’ll walk you through a LangGraph agent that:

• Scrapes your website

• Identifies your domain, location, and service themes

• Generates likely customer queries

• Tests those queries against answer engines

• Monitors citations, competitor presence, and content gaps

So instead of guessing how answer engines perceive your brand, you’ll have a real-time dashboard telling you where you stand and how to improve.

Code

https://github.com/zahere-dev/agentexpert-labs/tree/main/business_use_cases/brand_monitoring

Hey everyone!

Have you ever wondered how AI agents like Claude, Anthropic’s Computer Use agent, Gemini’s CLI tools, or LangChain agents handle complex tasks so effectively?

There’s a powerful design pattern that all these top agents share - and today, I’m going to break it down for you.

It’s called write_todos or Todo Tool, and once you understand it, you’ll never build AI agents the same way again.

Full walkthrough of the custom write_todo implementation in Customer Service Agent.

In many AI agents, write_todos is used to create and manage TODO lists that help the agent stay organized and focused during long or complex tasks.

For example, Claude Code uses a feature called plan mode to generate a structured list of steps before starting any work. This is done using a tool called TodoWrite, which is based on a special prompt.

Each TODO item includes:

• A short, clear description of the task

• A status: pending, in progress, or completed

As tasks grow in size—sometimes involving 50 or more tool calls—it becomes harder for the agent to remember everything. This can lead to context rot, where the agent forgets earlier goals or drifts off-topic.

To prevent this, agents like Manus regularly rewrite and update their TODO lists. This helps them stay on track by constantly reminding themselves of their goals, reducing the risk of losing focus or deviating from the task.

There are 2 parts

Phase 1: Planning The agent receives a complex task and uses an LLM to decompose it into smaller, actionable subtasks. Think of it like a project manager breaking down a big project.

[Show example: “Handle customer complaint about delayed order”] The agent might plan:

• Todo 1: Search knowledge base for refund policy

• Todo 2: Look up similar past tickets

• Todo 3: Check shipping status

• Todo 4: Draft response with options

Phase 2: Writing (The Critical Part) [Highlight this]

Here’s the key difference: The agent writes these todos down - typically to a file, database, or persistent storage.

Why use ‘write_todos’?

Reason #1: Transparency and Debuggability

When your agent writes its plan to a file, you as a developer - or even the user - can see exactly what it’s planning to do BEFORE it does it.

Reason #2: Fault Tolerance and Recovery

Here’s where it gets really powerful. If your agent crashes halfway through a task, what happens?

Without write-todos: Everything is lost. Start over. With write-todos: Read the file, see what’s completed, continue where you left off.

Reason #3: Human-in-the-Loop Workflows

Sometimes you want human approval before execution. With write-todos:

1. Agent creates plan and writes it

2. Human reviews and approves (or modifies)

3. Agent reads the (potentially modified) plan and executes

When to use ‘write_todos’?

Multi-step tasks (3+ steps)

If your task naturally breaks down into multiple sequential or parallel steps, write-todos will help.

• Research and analysis

• Customer support workflows

• Data processing pipelines

• Code generation and deployment

• Content creation workflows

Long-running operations (minutes to hours)

Anything that takes significant time benefits from fault tolerance and observability.

High-stakes operations

Financial transactions, data deletion, production deployments - anything where you want a review before action.

Collaborative agent-human workflows

When humans need to be in the loop at any

❌ Don’t use write-todos when:

1. Simple, single-step tasks “What’s the weather?” - No need for todos.

2. Real-time, low-latency requirements File I/O adds overhead. If you need sub-second responses, keep it in memory.

3. Stateless, idempotent operations If the operation can be safely retried from scratch, the crash recovery benefit is less valuable.

The ToDo Methods in our Agent

Below is a simple implementation of the Todo tools for our Customer Service Agent.

Full source code here.

The generated Todo file

{

“created_at”: “2025-11-11T01:26:18.171559”,

“todos”: [

{

“id”: 1,

“task”: “Search knowledge base for \”refund policy\” to understand the guidelines for processing refunds.”,

“status”: “completed”,

“result”: “No relevant article found in knowledge base.”

},

{

“id”: 2,

“task”: “Look up the customer’s order status in the system to determine if the product has shipped or if there are any delays.”,

“status”: “completed”,

“result”: “Found 2 similar past tickets:\n- Ticket TICKET-2024-089: Order delayed - carrier issue\n Customer reported order stuck in transit for 8 days. Root cause: Holiday shipping backlog. Resolution: Expedited replacement sent, original order refu...\n Resolved in: 2 days\n- Ticket TICKET-2024-112: Package not delivered after 10 days\n Order placed 10 days ago, tracking shows ‘in transit’ with no updates. Customer requested refund. Resolution: Refund processed, new order sent with ex...\n Resolved in: 1 day\n”

},

{

“id”: 3,

“task”: “Check recent similar tickets or issues to see if other customers are experiencing similar delays with the same product.”,

“status”: “completed”,

“result”: “Found 2 similar past tickets:\n- Ticket TICKET-2024-089: Order delayed - carrier issue\n Customer reported order stuck in transit for 8 days. Root cause: Holiday shipping backlog. Resolution: Expedited replacement sent, original order refu...\n Resolved in: 2 days\n- Ticket TICKET-2024-112: Package not delivered after 10 days\n Order placed 10 days ago, tracking shows ‘in transit’ with no updates. Customer requested refund. Resolution: Refund processed, new order sent with ex...\n Resolved in: 1 day\n”

},

{

“id”: 4,

“task”: “Review current refund processing times to inform the customer about how long it may take to receive their refund if approved.”,

“status”: “completed”,

“result”: “Completed: Review current refund processing times to inform the customer about how long it may take to receive their refund if approved.”

}

]

}

I strongly recommend you go through the video or the code to full understand this pattern.

So there you have it - the write-todos pattern that powers some of the most sophisticated AI agents in production today.

To recap:

• What it is: Externalize the agent’s plan by writing it to persistent storage

• Why use it: Transparency, fault tolerance, human-in-the-loop, better planning, and testability

• When to use it: Multi-step tasks, long-running operations, high-stakes work, collaborative workflows

The complete code for this example is in the description below. I encourage you to:

1. Clone it and run it yourself

2. Modify it to handle different support scenarios

3. Try adding a “human approval” step before execution

4. Build your own agent using this pattern

Alright. So at work, I’ve been consulting a few teams to design MCP servers.

I have a solution in prod and some teams were interested to learn from my experience.

All my experiments on my blog, YouTube channel, and the solution at work were based on SSE—meaning the transport layer of my MCP servers was Server-Sent Events.

However, recently when one of the devs I was consulting told me that they read SSE transport was deprecated and only STDIO and Streamable HTTP were the only choices, I strongly denied such updates and told them there’s no way I would miss such an update.

They shared the link with me and I just couldn’t believe I had missed that update. An update that had come in late March 2025.

I felt embarrassed as I was recommending SSE transport for all our projects.

Luckily, the change to streamable_http wasn’t big and we could easily get on with the changes.

But I realized staying current with rapidly evolving AI protocols isn’t optional—it’s critical.

Missing a single protocol update can mean building on deprecated foundations, giving outdated advice, and potentially creating scalability issues that won’t surface until production load hits.

I know I’m a bit late here—but let’s understand why SSE was deprecated and learn how to build MCP servers with Streamable HTTP transport.

Why SSE Was Deprecated

The SSE (Server-Sent Events) transport was part of the original MCP specification, but it had fundamental architectural problems that became apparent at scale:

1. Two-Endpoint Architecture

SSE required maintaining two separate endpoints:

• GET /sse for server-to-client streaming

• POST /messages for client-to-server requests

This split architecture created complexity in connection management and made deployment harder.

2. Connection Limit Hell

SSE connections are long-lived HTTP connections. Under load, servers quickly hit OS-level file descriptor limits (~1024 on most systems). Performance testing showed:

• 1000+ concurrent TCP connections with SSE

• Execution times 4x slower than Streamable HTTP

• Success rates dropping sharply after ~1024 concurrent users

3. No Resumability

If an SSE connection dropped, there was no way to resume from where you left off. The entire operation had to restart.

4. One-Way Communication

SSE only supports server-to-client streaming. Client-to-server communication required separate HTTP POST requests, creating an asymmetric architecture.

5. Cloud Deployment Nightmare

Long-lived connections don’t play well with:

• Serverless functions (AWS Lambda, Cloud Run)

• Load balancers with connection timeouts

• Auto-scaling infrastructure

• Container orchestration platforms

Enter Streamable HTTP

Streamable HTTP was introduced in protocol version 2025-03-26 and addresses all of SSE’s limitations. Here’s what makes it better:

Single Endpoint Architecture

Everything goes through one endpoint: /mcp

Connection Reuse

Instead of 1000 concurrent SSE connections, Streamable HTTP reuses TCP connections, keeping the count to just ~50 even under high load.

Flexible Response Format

The server chooses the response type based on the operation:

• Quick operations → Instant JSON response

• Long operations → SSE stream with progress updates

Stateless Mode

Perfect for cloud deployments—no session state to maintain, works beautifully with serverless and auto-scaling.

Migration from SSE to Streamable HTTP

The migration is surprisingly simple:

Python

# Old (SSE)
mcp.run(transport=”sse”)

# New (Streamable HTTP)
mcp.run(transport=”streamable-http”)

Client Code

# Old
from mcp.client.sse import sse_client
async with sse_client(url) as (read, write):
    # ...

# New
from mcp.client.streamable_http import streamablehttp_client
async with streamablehttp_client(url) as (read, write, _):
    # ...

That’s it. The tool definitions, resources, and prompts remain unchanged.

Key Takeaways

1. SSE is deprecated since protocol 2025-03-26 (March 26, 2025)

2. Streamable HTTP is the standard for production deployments

3. Migration is straightforward - mostly just changing transport configuration

4. Performance is significantly better - 4x faster with better scalability

5. Cloud-friendly - Works seamlessly with serverless and auto-scaling

Here’s the video of the week

If you manage a website (or several), you know the pain of alt tags.

They’re essential for accessibility and SEO, but writing hundreds of them manually?

Soul-crushing work.

And let’s be honest: when we DO write them, half the time they end up as generic descriptions like “image-123.jpg” that help exactly no one.

How It Actually Works

The workflow is surprisingly elegant. You feed it any URL, and it:

1. Scrapes the page and finds every image missing an alt tag

2. Extracts rich context – page titles, nearby text, headings, captions, everything around that image

3. Sends both the image URL and context to GPT-4o-mini – this is key: the AI actually LOOKS at the image while understanding where it appears on the page

4. Generates perfect alt tags following Google’s accessibility guidelines

5. Stores everything in a database ready for your CMS

The magic is in the context. Most alt tag generators just describe what they see. This workflow understands PURPOSE. A photo of a laptop gets described differently on a tech news site versus an e-commerce product page versus a workspace inspiration blog.

Who This Is For

This is perfect if you’re:

• A developer managing multiple client sites

• An SEO specialist optimizing for image search

• A content manager drowning in accessibility backlog

• An agency that needs to scale accessibility services

• Anyone who cares about making the web more accessible

The Technical Bits

Don’t worry if you’re not an n8n expert – I walk through every single node in detail. You’ll learn:

• How to scrape websites properly

• JavaScript for HTML parsing (regex-based, no external libraries)

• How to structure prompts for vision AI

• Loop automation techniques

• Database integration basics

Even if you’ve never used n8n before, you’ll have a working automation by the end.

Watch It Here

Today, I decided to take it for a spin.

Here’s my experience.

n8n is everything the “influencers” claim it to be.

It’s intuitive, powerful, and packed with over 400 integrations, a drag-and-drop interface, and custom code options that make complex automations feel effortless.

From the ease of setting it up on my 12-year-old Mac Mini to developing a solution in just 30 minutes (thanks to its impressive documentation), I must say, I’m a fan

As someone who spent a decade building enterprise automation, I can see how n8n solves so many process-related challenges, like streamlining data flows and reducing manual errors.

Is it RPA?

Tools like Power Automate, Automation Anywhere, UiPath, and others do similar things—but deploying and scaling them at an enterprise level requires dedicated teams, hefty infrastructure, and ongoing vendor support.

In contrast, n8n stands out as open-source and self-hostable, with no vendor lock-in or per-bot licensing fees.

It’s API-focused for seamless app integrations rather than UI-based bot mimicry, making it lighter, more flexible, and cost-effective for modern workflows without sacrificing power.

So what did I build with this?

A production-ready n8n workflow that automatically analyzes thousands of customer reviews using AI and generates actionable business insights, such as sentiment trends, key pain points, and improvement recommendations.

My verdict

n8n is a game-changer for efficient, scalable automation. It’s developer-friendly yet accessible to non-coders, and its community-driven updates keep it evolving. If you’re tired of bloated tools, give n8n a try—you won’t regret it.

Why should you learn it?

In today’s fast-paced digital world, automation skills are essential for boosting productivity and staying competitive.

n8n empowers you to connect disparate tools—like APIs, databases, and AI services—without deep coding expertise, saving hours on repetitive tasks.

It’s future-proof: as no-code/low-code platforms rise, mastering n8n opens doors to freelance gigs, startup roles, or internal efficiencies at big companies.

Plus, its active open-source community means constant improvements, free resources, and real-world applications in marketing, data analysis, e-commerce, and more.

Whether you’re a solo entrepreneur automating lead gen or a team lead optimizing operations, learning n8n builds versatile problem-solving skills that pay off immediately and scale with your career.

Your websites, landing pages, and digital content are critical revenue drivers.

But when mismanaged, they leak profits.

As content scales, manual audits fall short, inviting brand inconsistency and compliance risks.

Discover how AI Agents can streamline content audits across thousands of pages.

The Agent-Driven Content Audit System

Let me walk you through the high-level design.

The first step is content ingestion—gathering all URLs and asset metadata from various sources like crawler lists, CSVs, or CMS integrations. This ensures complete, real-time coverage across your content footprint.

Next, the orchestrator prioritizes assets using AI and historical data, focusing audits on high-impact pages first. This guarantees critical content gets the right attention every time.

Multiple audit engines then run in parallel: checking readability, SEO, accessibility, compliance, and post-launch monitoring. They use both third-party tools and custom code for thorough reviews.

All results feed into secure databases that connect content to revenue and risk metrics. Finally, the prioritization engine turns data into business impact scores, triggering automated tickets and alerts so stakeholders can make fast, informed decisions.

I’ve done full walk-through of the code in this video.

The Four Pillars of Intelligent Routing

After analyzing hundreds of production agent systems, four distinct routing approaches have emerged as the foundation of modern agentic architectures. Each represents a different trade-off between accuracy, speed, cost, and implementation complexity.

1. LLM-Based Routing: The Universal Translator

LLM-based routing leverages the same language understanding capabilities that make modern AI so impressive. Instead of trying to predict every possible user intent, you simply ask the language model to analyze the input and make the routing decision.

The core mechanism is elegantly simple: present the LLM with the user's input and a carefully crafted prompt that defines your routing categories. The LLM analyzes the semantic meaning, context, and intent, then outputs a structured decision about where to route the request.

Where it shines:

• Handling ambiguous inputs that don't fit neat patterns

• Understanding complex, multi-part requests that span multiple categories

• Adapting to new types of queries without retraining or rule updates

• Processing context-dependent routing that considers conversation history

The architectural trade-off: Maximum flexibility comes at the cost of latency and API expenses. Each routing decision requires a full LLM inference, which can add 1-3 seconds and $0.001-0.01 per request. For high-throughput systems, these costs compound quickly.

Production pattern: Many sophisticated systems use LLM routing as a fallback for edge cases or as the primary router for high-value, low-volume interactions where accuracy matters more than speed.

2. Embedding-Based Routing: The Semantic Compass

Embedding-based routing transforms the routing problem into a mathematical similarity search. By converting both user inputs and routing destinations into high-dimensional vectors, the system can make routing decisions based on semantic similarity rather than surface-level pattern matching.

The mechanism involves pre-computing embeddings for each possible route (or representative examples of each route), then computing the embedding for each incoming request and finding the closest match using cosine similarity or other distance metrics.

Where it excels:

• Semantic understanding that goes beyond keyword matching

• Multi-language support since embeddings capture meaning across languages

• Consistent performance with sub-second routing decisions

• Handling synonyms and paraphrasing naturally

The sweet spot: Embedding routing offers the best balance of semantic understanding and computational efficiency for most production systems. Once the embeddings are computed, routing decisions happen in milliseconds with no API calls.

Implementation insight: The quality of your route embeddings determines everything. Spend time crafting comprehensive descriptions of each route's purpose and scope—these become the foundation for accurate similarity matching.

3. Rule-Based Routing: The Deterministic Workhorse

Rule-based routing might seem primitive compared to AI-powered alternatives, but it remains the backbone of many production systems for good reason. Using predefined patterns, keyword matching, and logical conditions, rule-based routing provides predictable, lightning-fast decisions.

The approach relies on explicit logic: if the input contains certain keywords, matches specific patterns, or triggers particular conditions, route to the designated handler. Complex decision trees can be built by combining multiple rules and conditions.

Where it dominates:

• High-throughput systems where microsecond response times matter

• Compliance-sensitive applications requiring auditable, deterministic routing

• Well-defined use cases with predictable input patterns

• Resource-constrained environments with minimal computational overhead

The reliability factor: Rule-based routing never has an "off day." It makes identical decisions for identical inputs, making it perfect for systems that require absolute consistency and predictability.

Modern evolution: Today's rule-based systems often incorporate regular expressions, fuzzy matching, and hierarchical decision trees, making them far more sophisticated than simple keyword matching.

4. ML Model-Based Routing: The Learning Specialist

Machine learning model-based routing represents a middle ground between the flexibility of LLM routing and the efficiency of rule-based systems. A specialized classification model is trained on labeled examples of correct routing decisions, creating a dedicated routing function optimized for your specific use case.

The core difference from LLM routing is crucial: instead of using a general-purpose language model prompted to make routing decisions, you train a discriminative model specifically for routing. The routing logic becomes encoded in the model's learned weights rather than expressed through prompts.

Where it excels:

• Domain-specific accuracy that improves with more training data

• Cost-effective operation with no ongoing API expenses

• Fast inference comparable to rule-based systems

• Continuous improvement through retraining on new examples

The training advantage: Unlike embeddings, which rely on general-purpose representations, ML routing models learn patterns specific to your application. They can discover subtle correlations between input features and routing decisions that human-designed rules might miss.

Implementation strategy: The most effective ML routing implementations combine traditional ML features (TF-IDF, n-grams) with modern embedding representations, creating hybrid models that capture both statistical and semantic patterns.

code: https://github.com/zahere-dev/agentexpert-labs

Authentication Flow

Before looking at the code, let’s see how the authentication flow works:

Key points from the flow:

• Clients log in via Auth0 and receive a JWT access token.

• The server uses Auth0’s JWKS (JSON Web Key Set) to validate token signatures.

• Requests without a valid token are rejected with 401 Unauthorized.

Create Auth0 Service (Or any other OAuth provider)

• **We are using Auth0 as it is free**

• Use your github/google account to signup to Auth0 service

• Create a Machine-to-Machine app and generate your domain, client id and secret.

Imports and Setup

import datetime
import os
from zoneinfo import ZoneInfo
from fastapi import Depends, FastAPI, HTTPException, status, Request
from fastapi.responses import JSONResponse
from fastapi.security import HTTPBearer, HTTPAuthorizationCredentials
import jwt
import requests
from jwt.algorithms import RSAAlgorithm
from starlette.applications import Starlette
from starlette.routing import Route, Mount
import logging

from mcp.server.fastmcp import FastMCP
from mcp.server.sse import SseServerTransport
from dotenv import load_dotenv

load_dotenv()

• FastAPI / Starlette: Provide the ASGI framework and request/response tools.

• jwt & RSAAlgorithm: For decoding and verifying JWTs signed with RS256.

• HTTPBearer: A security utility to parse Authorization: Bearer <token> headers.

• dotenv: Loads environment variables (e.g., API keys).

• logging: For server-side logging.

• MCP modules: FastMCP and SseServerTransport enable MCP over SSE.

Auth0 Configuration

AUTH0_DOMAIN = "zahere-dev.us.auth0.com"
API_AUDIENCE = "https://zahere-dev.us.auth0.com/api/v2/"
ALGORITHMS = ["RS256"]

# Set up logging
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger("mcp-server")

# Fetch JWKS keys once at startup
jwks_url = f"https://{AUTH0_DOMAIN}/.well-known/jwks.json"
jwks = requests.get(jwks_url, verify=False).json()

• AUTH0_DOMAIN: Your Auth0 tenant domain.

• API_AUDIENCE: The intended audience claim of the JWT (ensures tokens are meant for your API).

• ALGORITHMS: We only accept RS256 (RSA signature with SHA-256).

• JWKS Fetch: At startup, the server downloads Auth0’s public signing keys. These are later used to validate incoming tokens.

JWT Verification Logic

# Auth setup
security = HTTPBearer()

def verify_jwt(token: str):
    try:
        header = jwt.get_unverified_header(token)
        kid = header["kid"]

        # Find matching JWK
        key = next((k for k in jwks["keys"] if k["kid"] == kid), None)
        if not key:
            raise HTTPException(status_code=401, detail="Invalid auth key")

        public_key = RSAAlgorithm.from_jwk(key)

        payload = jwt.decode(
            token,
            public_key,
            algorithms=ALGORITHMS,
            audience=API_AUDIENCE,
            issuer=f"https://{AUTH0_DOMAIN}/"
        )
        return payload
    except Exception as e:
        logger.error(f"JWT validation error: {str(e)}")
        raise HTTPException(status_code=401, detail="Invalid or expired token")

• Extract header & KID: Each JWT contains a kid (key ID). This tells us which public key was used to sign it.

• Find JWK: Match the kid against Auth0’s JWKS set.

• RSAAlgorithm.from_jwk: Convert the JWK to a usable RSA public key.

• jwt.decode: Validate the token signature, audience, issuer, and expiry.

• If anything fails, raise a 401 Unauthorized error.

MCP Server & Tools (Minimal)

# ======================
# MCP Server Setup
# ======================
mcp = FastMCP(name="Weather and Time SSE Server")

@mcp.tool()
def TimeTool(input_timezone: str = None):
    current_time = datetime.datetime.now()
    if input_timezone:
        current_time = current_time.astimezone(ZoneInfo(input_timezone))
    return f"The current time is {current_time}."

transport = SseServerTransport("/messages/")

@mcp.tool()
def weather_tool(location: str):
    api_key = os.getenv("OPENWEATHERMAP_API_KEY")
    url = f"http://api.openweathermap.org/data/2.5/weather?q={location}&appid={api_key}&units=metric"
    response = requests.get(url)
    data = response.json()
    if data.get("cod") == 200:
        temp = data["main"]["temp"]
        description = data["weather"][0]["description"]
        return f"The weather in {location} is {description} with {temp}°C."
    return f"Couldn't fetch weather for {location}."

• A simple implementation of the MCP server tools

FastAPI App & Endpoints

app = FastAPI()

@app.get("/sse")
async def handle_sse(request: Request):
    auth_hdr = request.headers.get("authorization")
    token = None
    if auth_hdr and auth_hdr.lower().startswith("bearer "):
        token = auth_hdr.split(" ", 1)[1]
    else:
        token = request.query_params.get("access_token")

    if not token:
        raise HTTPException(status_code=401, detail="Missing token")

    verify_jwt(token)

    async with transport.connect_sse(request.scope, request.receive, request._send) as (in_stream, out_stream):
        await mcp._mcp_server.run(in_stream, out_stream, mcp._mcp_server.create_initialization_options())

• Supports two auth methods:

  1. Standard Authorization: Bearer <token> header.

  2. Query parameter ?access_token=... (useful for browser EventSource).

• Token is verified, then SSE connection is established.

Protecting the ASGI Transport

def auth_asgi_wrapper(asgi_app):
    async def app(scope, receive, send):
        if scope["type"] == "http":
            headers = {k.decode().lower(): v.decode() for k, v in scope.get("headers", [])}
            auth_hdr = headers.get("authorization")
            if not auth_hdr or not auth_hdr.lower().startswith("bearer "):
                resp = JSONResponse({"detail": "Not authenticated"}, status_code=401)
                await resp(scope, receive, send)
                return
            token = auth_hdr.split(" ", 1)[1]
            try:
                verify_jwt(token)
            except HTTPException as he:
                resp = JSONResponse({"detail": he.detail}, status_code=he.status_code)
                await resp(scope, receive, send)
                return
            except Exception:
                resp = JSONResponse({"detail": "Invalid or expired token"}, status_code=401)
                await resp(scope, receive, send)
                return
        await asgi_app(scope, receive, send)
    return app

• This wrapper protects the /messages/ endpoint. It intercepts requests, checks for a Bearer token, and validates it before passing control to the MCP transport.

Mount Messages Route

app.mount("/messages/", auth_asgi_wrapper(transport.handle_post_message))

• This makes sure POST requests to /messages/ are also protected by our wrapper

Run the Server

if __name__ == "__main__":
    import uvicorn
    uvicorn.run(app, host="0.0.0.0", port=8100)

The Client

**Below is only part of the client code**. Full code is available in repo.

def get_token():
    import http.client

    conn = http.client.HTTPSConnection(os.getenv("AUTH0_DOMAIN"))
    
    payload = {"client_id":os.getenv("CLIENT_ID"), "client_secret":os.getenv("CLIENT_SECRET"),"audience":os.getenv("AUTH_AUDIENCE"),"grant_type":"client_credentials"}

    headers = { 'content-type': "application/json" }

    conn.request("POST", "/oauth/token", json.dumps(payload), headers)

    res = conn.getresponse()
    data = json.loads(res.read())
    return data["access_token"]


async def main(queries: list[str]):
    sse_url = "http://localhost:8100/sse"
    logger.info(f"Generating Bearer token for authentication")
    auth_token = get_token()
    #auth_token = "12124234"
    logger.info(f"Token generated {auth_token[:20]} ")
    headers = {"Authorization": f"Bearer {auth_token}"}    
    try:
        logger.info("Connecting to MCP Server with token")
        # 1) Open SSE → yields (in_stream, out_stream)
        async with sse_client(url=sse_url, headers=headers) as (in_stream, out_stream):
            # 2) Create an MCP session over those streams
            logger.info("Connected to MCP Server")
            async with ClientSession(in_stream, out_stream) as session:
                
                # 3) Initialize
                info = await session.initialize()
                logger.info(f"Connected to {info.serverInfo.name} v{info.serverInfo.version}")

                # 4) List tools
                tools = (await session.list_tools())
                logger.info(tools)            
                
                ### Run the queries
                
                for query in queries:                
                
                    prompt = get_prompt_to_identify_tool_and_arguements(query,tools)
                    logger.info(f"Printing Prompt \n {prompt}")
                    
                    response = llm_client(prompt)
                    print(response)
                    
                    tool_call = json.loads(response)
                                
                    result = await session.call_tool(tool_call["tool"], arguments=tool_call["arguments"])
                    logger.success(f"User query: {query}, Tool Response: {result.content[0].text}")
    except httpx.HTTPStatusError as http_e:
        logger.exception(f"HTTP Error: {http_e}")
    except Exception as e:
        logger.exception(f"Error: {e}")
            

if __name__ == "__main__":
    queries = ["What is the time in Bengaluru?", "What is the weather like right now in Dubai?"]    
    asyncio.run(main(queries))

• Use the client_id and secret to connect with Auth0 and request for access token

• Send the token to the MCP Server as a header with “Authorization” key.

• If connection is established, you will see the message below, else 401 Unauthorized error

TL;DR

LLM agents become robust, scalable, and debuggable only when you build, test, and orchestrate deterministic tools first—before prompt crafting. Tools, not prompts, should encode business logic, connect to APIs, and manage data. Start With Tools Not Tasks

Video

Context

Let’s say you were given a task to build an Email Processing Agent that is capable of parsing incoming emails, understanding the intent, and executing the relevant tasks (like routing, calling downstream tools, and/or responding to the sender).

Below is how you would commonly approach these use cases with agent frameworks like CrewAI/LangGraph?

But should you start with Flow Methods, Nodes and Edges or should you start with the tool?

Most frameworks for building AI agents advise:

“Effective task definition is often more important than model selection.”

This is a great start, but I challenge this orthodoxy. In my experience, when deploying LLM agents for enterprise automation, this advice still puts you at risk of invisible failures and brittle systems.

My premise:
Don’t start with the task. Start with your tool design. Only then define the task and choose the model.

Why? Suppose your tools, the deterministic, testable modules that wrap core business logic and external APIs, aren’t designed and hardened up front. In that case, both task definition and model selection become sources of confusion and fragility.

In the case of the Email Parser Agent, the following can happen

• Increased Risk of System-Wide Failures: Untested tools like the Email Classifier or Support Ticket API can cause cascading errors across workflows, disrupting tasks like invoicing or meeting scheduling.

• Unreliable Tool Behavior: Unvalidated tools, such as the Calendar API, may mishandle edge cases (e.g., invalid time zones), leading to incorrect workflow outcomes.

• Complex Debugging and Maintenance: Building workflows first obscures whether errors stem from tools, LLMs, or orchestration, complicating debugging in CrewAI or LangGraph.

• Poor Handling of Edge Cases: Untested tools like the Email Classifier may fail on ambiguous inputs, breaking workflows like Technical Support Queries.

• Over-Reliance on LLMs: Without tools like the Invite Parser, workflows rely on unpredictable LLMs for deterministic tasks, reducing reliability.

• Scalability Challenges: Unoptimized tools (e.g., Support Ticket API) may fail under high load, causing bottlenecks in high-volume email processing.

Prompts can impress, effective tasks may clarify, but only robust tools make agents truly reliable.

So:
Start by designing and hardening your tools. Make each unit deterministic, observable, and testable.
Then:
Define the agent’s task in terms of tool orchestration—not raw prompting. Only at this point does model selection add real value.

“Build your tools first. Then the task. Then the prompt. That’s where scalable, production-ready AI starts.”

This guide provides you with a practical blueprint:

• Why “tools-first”? How does it differ from standard “task-first” or “prompt-first” wisdom?

• What, exactly, is a tool—and why is it the most controllable, reliable artifact in your stack?

• How do you design, validate, and orchestrate tools for enterprise-ready agents?

Let’s Start with Why! Why a Tools-First Approach?

Tools are the deterministic components of an AI agent’s architecture.

Unlike LLMs, which generate responses based on probabilities, tools are explicitly coded to perform specific tasks, such as fetching data from APIs, performing calculations, or interacting with databases.

By prioritizing tool development, you create a solid foundation for your AI agent, ensuring that the most predictable parts of your system are reliable before introducing the variability of LLMs.

Benefits of the Tools-First Approach

1. Deterministic Behavior: Tools execute predefined logic, making their behavior predictable and testable.

2. Modularity: Well-designed tools are reusable across multiple agents or workflows.

3. Debugging Simplicity: Isolating tools from LLMs allows you to test and debug them independently, reducing complexity.

4. Reliability: Thoroughly tested tools minimize runtime errors when integrated with LLMs.

5. Scalability: Robust tools can handle edge cases and scale with increased usage.

6. Security: Keeps critical validation logic out of fragile prompts, reducing prompt injection risks.

7. Faster Iteration: Tools can be developed, tested, and deployed independently.

Common Pitfalls of Skipping Tool Development

• Over-Reliance on LLMs: Developers sometimes use LLMs to handle tasks better suited for deterministic tools, leading to inconsistent results.

• Untested Tools: Rushing to integrate untested tools with LLMs can cause cascading failures.

• Poor Error Handling: Without proper tool validation, errors propagate through the agent, making debugging difficult.

What Is the “Tools-First” Approach?

Definition

A tool in agent systems is a deterministic, testable unit of logic encapsulating business operations, integrations, or API calls. Think:

• API connectors (CRM, weather, finance)

• Business logic/calculators

• Document parsers

• Database queries

• Integration wrappers (Slack, Jira)

A tool is deterministic code. Same input, same output; wrapping real business logic or external actions, callable by the agent.

LLM-First vs Tools-First

Core Tool Properties

• Deterministic: Same input → same output

• Composable: Reusable across agents and workflows

• Testable: Unit tests and mocks are doable in isolation

Key Principle

Write, test, and harden your tools first. Only then orchestrate with LLMs.

Best Practices for Tool Development

To maximize the effectiveness of the Tools-First Approach, follow these best practices when designing, writing, and testing tools:

1. Define Clear Specifications: Document the tool’s purpose, inputs, outputs, and error conditions. Use type hints (e.g., Python’s typing module) to enforce input/output contracts.

2. Implement Comprehensive Logging: Use structured logging to capture tool execution details, inputs, outputs, and errors for debugging and monitoring.

3. Enable Tracing: Integrate tracing (e.g., OpenTelemetry, Langfuse) to track tool execution in distributed systems, especially when tools interact with external services.

4. Handle Errors Gracefully: Anticipate edge cases and implement robust error handling with meaningful error messages.

5. Write Unit Tests: Achieve near-100% test coverage to ensure the tool behaves correctly under all conditions.

6. Validate Inputs and Outputs: Use schemas (e.g., Pydantic) to validate data entering and leaving the tool.

7. Optimize Performance: Profile the tool to ensure it performs efficiently, especially for high-frequency tasks.

8. Document the Tool: Provide clear documentation for other developers or agents to understand and use the tool.

9. Version Control: Use semantic versioning for tools to manage updates and dependencies.

Implementation

Below is the complete implementation of the Weather API Tool, incorporating best practices.

import requests
from typing import Dict, Any
from pydantic import BaseModel, Field, ValidationError
import structlog
from opentelemetry import trace
from opentelemetry.trace import Span
import os
from requests.exceptions import RequestException
from http import HTTPStatus

# Initialize structured logger
logger = structlog.get_logger()

# Initialize OpenTelemetry tracer
tracer = trace.get_tracer(__name__)

# Input schema
class WeatherInput(BaseModel):
    city: str = Field(..., min_length=1, description="Name of the city")
    api_key: str = Field(..., min_length=1, description="OpenWeatherMap API key")

# Output schema
class WeatherOutput(BaseModel):
    temperature: float = Field(..., description="Temperature in Celsius")
    condition: str = Field(..., description="Weather condition (e.g., Clear)")
    humidity: int = Field(..., ge=0, le=100, description="Humidity percentage")

class WeatherAPIError(Exception):
    """Custom exception for Weather API errors."""
    pass

class WeatherTool:
    """A tool to fetch current weather data for a given city."""

    def __init__(self, base_url: str = "http://api.openweathermap.org/data/2.5/weather"):
        self.base_url = base_url

    @tracer.start_as_current_span("fetch_weather")
    def fetch_weather(self, input_data: WeatherInput) -> WeatherOutput:
        """
        Fetch weather data for the specified city.

        Args:
            input_data: WeatherInput object containing city and API key.

        Returns:
            WeatherOutput object with temperature, condition, and humidity.

        Raises:
            WeatherAPIError: If the API request fails or returns invalid data.
        """
        span: Span = trace.get_current_span()
        span.set_attribute("city", input_data.city)

        logger.info("Fetching weather data", city=input_data.city)

        try:
            # Make API request
            params = {
                "q": input_data.city,
                "appid": input_data.api_key,
                "units": "metric"  # Use Celsius
            }
            response = requests.get(self.base_url, params=params, timeout=5)
            span.set_attribute("http.status_code", response.status_code)

            # Check for HTTP errors
            if response.status_code != HTTPStatus.OK:
                error_msg = f"API request failed: {response.status_code} - {response.text}"
                logger.error(error_msg)
                raise WeatherAPIError(error_msg)

            # Parse response
            data = response.json()
            weather_data = self._parse_weather_data(data)

            logger.info("Weather data fetched successfully", city=input_data.city)
            return weather_data

        except RequestException as e:
            error_msg = f"Network error: {str(e)}"
            logger.error(error_msg, exc_info=True)
            span.record_exception(e)
            raise WeatherAPIError(error_msg)
        except ValidationError as e:
            error_msg = f"Invalid response data: {str(e)}"
            logger.error(error_msg, exc_info=True)
            span.record_exception(e)
            raise WeatherAPIError(error_msg)
        except Exception as e:
            error_msg = f"Unexpected error: {str(e)}"
            logger.error(error_msg, exc_info=True)
            span.record_exception(e)
            raise WeatherAPIError(error_msg)

    def _parse_weather_data(self, data: Dict[str, Any]) -> WeatherOutput:
        """
        Parse raw API response into WeatherOutput schema.

        Args:
            data: Raw JSON response from the API.

        Returns:
            WeatherOutput object.

        Raises:
            ValidationError: If the response data is invalid.
        """
        return WeatherOutput(
            temperature=data["main"]["temp"],
            condition=data["weather"][0]["main"],
            humidity=data["main"]["humidity"]
        )

import asyncio
from pydantic import BaseModel
from agents import Agent, Runner, function_tool
from tools import WeatherTool, WeatherInput, WeatherOutput
from dotenv import load_dotenv

load_dotenv()

@function_tool
def get_weather(weather_input: WeatherInput) -> WeatherOutput:
    print("[debug] get_weather called")
    weather_tool = WeatherTool()
    return weather_tool.fetch_weather(weather_input)

agent = Agent(
    name="Hello world",
    instructions="You are a helpful agent.",
    tools=[get_weather],
)

async def main():
    result = await Runner.run(agent, input="What's the weather in Tokyo?")
    print(result.final_output)
    # The weather in Tokyo is sunny.

if __name__ == "__main__":
    asyncio.run(main())

{
  "name": "fetch_weather",
  "trace_id": "9a3a1ee00e4151e3c9e49a2127f8baab",
  "span_id": "1b66ffcf7ab68586",
  "attributes": {
    "city": "Tokyo",
    "http.status_code": 401
  },
  "status": "ERROR",
  "start_time": "2025-07-27T12:46:40.463738Z",
  "end_time": "2025-07-27T12:46:40.527270Z",
  "events": [
    {
      "name": "exception",
      "timestamp": "2025-07-27T12:46:40.521267Z",
      "attributes": {
        "exception.type": "tools.WeatherAPIError",
        "exception.message": "API request failed: 401 - {\"cod\":401, \"message\": \"Invalid API key. Please see https://openweathermap.org/faq#error401 for more info.\"}",
        "exception.stacktrace": "Traceback (most recent call last):\n  File \"C:\\Users\\Zahiruddin_T\\Documents\\LocalDriveProjects\\ToolFirstApproachForAgents\\src\\tools.py\", line 85, in fetch_weather\n    raise WeatherAPIError(error_msg)\ntools.WeatherAPIError: API request failed: 401 - {\"cod\":401, \"message\": \"Invalid API key. Please see https://openweathermap.org/faq#error401 for more info.\"}\n",
        "exception.escaped": "False"
      }
    },
    {
      "name": "exception",
      "timestamp": "2025-07-27T12:46:40.527270Z",
      "attributes": {
        "exception.type": "tools.WeatherAPIError",
        "exception.message": "Unexpected error: API request failed: 401 - {\"cod\":401, \"message\": \"Invalid API key. Please see https://openweathermap.org/faq#error401 for more info.\"}",
        "exception.stacktrace": "Traceback (most recent call last):\n  File \"C:\\Users\\Zahiruddin_T\\Documents\\LocalDriveProjects\\ToolFirstApproachForAgents\\src\\tools.py\", line 85, in fetch_weather\n    raise WeatherAPIError(error_msg)\ntools.WeatherAPIError: API request failed: 401 - {\"cod\":401, \"message\": \"Invalid API key. Please see https://openweathermap.org/faq#error401 for more info.\"}\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n  File \"c:\\Users\\Zahiruddin_T\\Documents\\LocalDriveProjects\\ToolFirstApproachForAgents\\.venv\\Lib\\site-packages\\opentelemetry\\trace\\__init__.py\", line 589, in use_span\n    yield span\n  File \"c:\\Users\\Zahiruddin_T\\Documents\\LocalDriveProjects\\ToolFirstApproachForAgents\\.venv\\Lib\\site-packages\\opentelemetry\\sdk\\trace\\__init__.py\", line 1105, in start_as_current_span\n    yield span\n  File \"C:\\Python3127\\Lib\\contextlib.py\", line 81, in inner\n    return func(*args, **kwds)\n           ^^^^^^^^^^^^^^^^^^^\n  File \"C:\\Users\\Zahiruddin_T\\Documents\\LocalDriveProjects\\ToolFirstApproachForAgents\\src\\tools.py\", line 109, in fetch_weather\n    raise WeatherAPIError(error_msg)\ntools.WeatherAPIError: Unexpected error: API request failed: 401 - {\"cod\":401, \"message\": \"Invalid API key. Please see https://openweathermap.org/faq#error401 for more info.\"}\n",
        "exception.escaped": "False"
      }
    }
  ]
}

Full Code

https://github.com/zahere-dev/tool-first-approach-to-building-agents

Summary: Think Like a Systems Engineer

References & Further Reading

Beginner:

• LangChain Tools & Decorators

• OpenAPI & Microservices Basics

Intermediate:

• Tenacity Retry Library

• LangGraph Node Design

Advanced:

• Anthropic’s Model Context Protocol Whitepapers & SDK

Bottom Line

In modern AI agent systems, tools are code, LLMs are orchestration glue. Harden your tools first. Let LLMs coordinate. That’s how you build agents that scale, adapt, and never leave you debugging in the dark.

In modern AI applications, communication between clients and tools isn’t always as simple as calling an API. The Model Context Protocol (MCP) provides a standardized way for clients to exchange information, invoke tools, and maintain shared context over persistent connections — typically via Server-Sent Events (SSE).

In this post, I’ll walk you through:

• Building an MCP SSE server using FastAPI

• Securing it with JWT authentication

• Implementing a custom Python client to connect, authenticate, and use MCP tools
  We’ll also build a simple BMI Calculator tool to demo tool calling through MCP.

🎛️ Project Overview

We’ll build:

• A FastAPI server that exposes an MCP-compliant SSE endpoint

• A token-based auth system

• Simple tools to get weather and time for a given location

• A Python client that authenticates, connects via SSE, and invokes the tool dynamically

📦 Tech Stack

• Python 3.11+

• Python MCP SDK

• FastAPI

• aiohttp

• PyJWT

• Pydantic

• loguru (for clean logs)

Setting up the MCP SSE Server (server.py)

Let’s first import the relevant libraries and get credentials and keys from env variables

import datetime
import os
from zoneinfo import ZoneInfo
from fastapi import  FastAPI, HTTPException, Request
from pydantic import BaseModel
import requests
from starlette.applications import Starlette
from starlette.routing import Route, Mount
import jwt
from mcp.server.fastmcp import FastMCP
from mcp.server.sse import SseServerTransport
from loguru import logger

from dotenv import load_dotenv

load_dotenv()

Let’s initialize the MCP server and setup the tools

# Initialize the MCP server with your tools
mcp = FastMCP(
    name="Weather and Time SSE Server"
)


@mcp.tool()
def TimeTool(input_timezone):
    "Provides the current time for a given city's timezone like Asia/Kolkata, America/New_York etc. If no timezone is provided, it returns the local time."
    format = "%Y-%m-%d %H:%M:%S %Z%z"
    current_time = datetime.datetime.now()    
    if input_timezone:
        print("TimeZone", input_timezone)
        current_time =  current_time.astimezone(ZoneInfo(input_timezone))
    return f"The current time is {current_time}."

transport = SseServerTransport("/messages/")


@mcp.tool()
def weather_tool(location: str):
    """Provides weather information for a given location"""        
    api_key = os.getenv("OPENWEATHERMAP_API_KEY")
    url = f"http://api.openweathermap.org/data/2.5/weather?q={location}&appid={api_key}&units=metric"
    response = requests.get(url)
    data = response.json()
    if data["cod"] == 200:
        temp = data["main"]["temp"]
        description = data["weather"][0]["description"]
        return f"The weather in {location} is currently {description} with a temperature of {temp}°C."
    else:
        return f"Sorry, I couldn't find weather information for {location}."

We will now setup the JWT auth system and use the starlette routes to essentially expose the app to the clients.

SECRET_KEY = "my_super_secret_key"
ALGORITHM = "HS256"     
def check_auth(request: Request):
    auth = request.headers.get("authorization", "")        
    if auth.startswith("Bearer "):
        token = auth.split(" ", 1)[1]
        try:
            payload = jwt.decode(token, SECRET_KEY, algorithms=[ALGORITHM])
            return True
        except jwt.ExpiredSignatureError:
            raise HTTPException(status_code=401, detail="Token expired")
        except jwt.InvalidTokenError:
            raise HTTPException(status_code=401, detail="Invalid token")

    raise HTTPException(status_code=401, detail="Unauthorized")

async def handle_sse(request):
    check_auth(request=request)
    # Prepare bidirectional streams over SSE
    async with transport.connect_sse(
        request.scope,
        request.receive,
        request._send
    ) as (in_stream, out_stream):
        # Run the MCP server: read JSON-RPC from in_stream, write replies to out_stream
        await mcp._mcp_server.run(
            in_stream,
            out_stream,
            mcp._mcp_server.create_initialization_options()
        )


#Build a small Starlette app for the two MCP endpoints
sse_app = Starlette(
    routes=[
        Route("/sse", handle_sse, methods=["GET"]),
        # Note the trailing slash to avoid 307 redirects
        Mount("/messages/", app=transport.handle_post_message)
    ]
)

Now let’s setup a FastAPI app and add a route for the clients to create a token. We will create a mock client store to simulate credential manager.

app = FastAPI()

# Mock client store
CLIENTS = {
    "test_client": "secret_1234"
}


class TokenRequest(BaseModel):
    client_id: str
    client_secret: str


@app.post("/token")
def generate_token(request: TokenRequest):
    if request.client_id in CLIENTS and CLIENTS[request.client_id] == request.client_secret:
        payload = {
            "sub": request.client_id,
            "exp": datetime.datetime.now() + datetime.timedelta(minutes=60)
        }
        token = jwt.encode(payload, SECRET_KEY, algorithm=ALGORITHM)
        return {"access_token": token}
    else:
        raise HTTPException(status_code=401, detail="Invalid credentials")

Mount the starlette app to FastAPI and setup gunicorn server to run the app.

app.mount("/", sse_app)

if __name__ == "__main__":
    import uvicorn
    uvicorn.run(app, host="0.0.0.0", port=8100)

🔒 Why JWT-Based Auth?

Using client_id / client_secret via POST is scalable:

• You can rotate credentials

• Enforce expiration

• Integrate external OAuth2/OIDC providers

• Track session-level auth

Setting up the MCP SSE Client (client.py)

Setting up client is similar to what we have seen previously in this series. The only change being we generate the token and pass that as headers in the SSE Client.

import asyncio
import json
from typing import Optional
from mcp import ClientSession
from mcp.client.sse import sse_client
from openai import OpenAI
import mcp.client.sse as _sse_mod
from httpx import AsyncClient as _BaseAsyncClient
from loguru import logger
import aiohttp

from dotenv import load_dotenv

load_dotenv()  # load environment variables from .env

import httpx
_orig_request = httpx.AsyncClient.request

async def _patched_request(self, method, url, *args, **kwargs):
    # ensure follow_redirects is set so 307 → /messages/ works
    kwargs.setdefault("follow_redirects", True)
    return await _orig_request(self, method, url, *args, **kwargs)

httpx.AsyncClient.request = _patched_request
def llm_client(message: str):
    client = OpenAI()

    completion = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[
            {"role": "system", "content": "You are an intelligent Assistant. You will execute tasks as instructed"},
            {
                "role": "user",
                "content": message,
            },
        ],
    )

    result = completion.choices[0].message.content
    return result



def get_prompt_to_identify_tool_and_arguements(query, tools):
    tools_description = "\n".join([f"{tool.name}: {tool.description}, {tool.inputSchema}" for tool in tools.tools])
    return  ("You are a helpful assistant with access to these tools:\n\n"
                f"{tools_description}\n"
                "Choose the appropriate tool based on the user's question. \n"
                f"User's Question: {query}\n"                
                "If no tool is needed, reply directly.\n\n"
                "IMPORTANT: When you need to use a tool, you must ONLY respond with "                
                "the exact JSON object format below, nothing else:\n"
                "Keep the values in str "
                "{\n"
                '    "tool": "tool-name",\n'
                '    "arguments": {\n'
                '        "argument-name": "value"\n'
                "    }\n"
                "}\n\n")




TOKEN_URL = "http://localhost:8100/token"
SSE_URL = "http://localhost:8100/sse"

async def get_token():
    payload = {"client_id": "test_client", "client_secret": "secret_1234"}
    async with aiohttp.ClientSession() as session:
        async with session.post(TOKEN_URL, json=payload) as resp:
            if resp.status != 200:
                logger.error(f"Failed to get token: {resp.status}")
                raise Exception("Unable to authenticate. Ensure you are using valid credentials")
            data = await resp.json()
            logger.info("Successfully generated token")
            return data["access_token"]

async def main(query:str):        
    try:
        auth_token = await get_token()    
        headers = {"Authorization": f"Bearer {auth_token}"}
        async with sse_client(url=SSE_URL,headers=headers) as (in_stream, out_stream):
            # 2) Create an MCP session over those streams
            async with ClientSession(in_stream, out_stream) as session:
                # 3) Initialize
                info = await session.initialize()
                logger.info(f"Connected to {info.serverInfo.name} v{info.serverInfo.version}")

                # 4) List tools
                tools = (await session.list_tools())
                logger.info(tools)            

                prompt = get_prompt_to_identify_tool_and_arguements(query,tools)
                logger.info(f"Printing Prompt \n {prompt}")

                response = llm_client(prompt)
                print(response)

                tool_call = json.loads(response)

                result = await session.call_tool(tool_call["tool"], arguments=tool_call["arguments"])
                logger.success(f"User query: {query}, Tool Response: {result.content[0].text}")
    except Exception as e:
        print(f"Encountered error: {e}")



if __name__ == "__main__":

    queries = ["What is the time in Bengaluru?", "What is the weather like right now in Dubai?"]
    for query in queries:
        asyncio.run(main(query))

Response when the client is run.

2025-05-18 18:00:48.230 | SUCCESS  | __main__:main:103 - User query: What is the weather like right now in Dubai?, Tool Response: The weather in Dubai is currently clear sky with a temperature of 30.37°C.

GitHub - zahere-dev/mcp-labs: Repository of my experiments with MCP

Repository of my experiments with MCP. Contribute to zahere-dev/mcp-labs development by creating an account on GitHub.

github.com

In this post, I’ll walk you through how to:

• Create an MCP Server using FastMCP

• Expose a tool that calculates BMI

• Build a Client that communicates with this server via stdio

• Use OpenAI’s GPT model to decide which tool to call and how to call it

Let’s break this down line by line — code and concept.

What is MCP?

Before diving into code, let’s understand what MCP is.

MCP (Model Context Protocol) is an open protocol developed by Anthropic to standardize how LLMs interact with tools. Think of it as the USB-C of AI apps — a universal way to connect and interact with tools, APIs, and services without writing tons of custom glue code.

Here’s a simple analogy:

• USB-C Port: One port to rule them all — display, power, storage.

• MCP: One protocol to access tools — calculators, search, databases, or any custom service.

By adopting MCP, we avoid the pain of writing one-off integration code for every LLM interaction. Instead, we define tools once and let the LLM figure out how to use them.

Courtesy: norahsakal.com

🛠️ Project Overview

We’ll build two things:

1. An MCP Server that exposes a simple tool to calculate BMI.

2. An MCP Client that communicates with the server via an LLM (like OpenAI GPT) and invokes the tool.

Let’s get started.

MCP Server

There are just 2 dependencies we need to install.

pip install "mcp[cli]"
pip install openai

Let’s create a file called bmi_server.py.

from mcp.server.fastmcp import FastMCP

mcp = FastMCP("BMI Server")

print(f"Starting server {mcp.name}")

@mcp.tool()
def calculate_bmi(weight_kg:float, height_m:float) -> float:
    """
    Calculate BMI given weight in kg and height in meters.
    """
    if height_m <= 0:
        raise ValueError("Height must be greater than zero.")
    return weight_kg / (height_m ** 2)


if __name__ == "__main__":
    mcp.run(transport="stdio")

What’s happening here?

• We use the FastMCP class to create the MCP server

• We create a BMI tool using the @mcp.tool()decorator.

• It takes weight and height and returns BMI.

• The server exposes this tool using standard input/output transport.

• We want this file to be run independently and not as a module

Obviously, this is a fundamental MCP server, but this should get us started on building our own MCP client.

MCP Client

Now, let’s create the client in a file named bmi_client.py.

Import all the dependencies. The key ones here being ClientSession, StdioServerParameters and stdio_client from the mcp package and OpenAI class to communicate with the LLM.

import asyncio
from openai import OpenAI
from mcp import ClientSession, StdioServerParameters
from mcp.client.stdio import stdio_client
import os
import json

We need to now establish a way to communicate with the server we just wrote. Let’s use the StdioServerParameters class from the mcp package to do that.

server_params = StdioServerParameters(command="python", args=["bmi-server.py"])

Essentially, we are telling the client what command to run and how to use the server file.

Let’s write a generic method that to communicate with an LLM API. This should simply take a prompt and return a response.

def llm_client(message:str):
    """
    Send a message to the LLM and return the response.
    """
    # Initialize the OpenAI client
    openai_client = OpenAI(api_key=os.getenv("OPENAI_API_KEY"))

    # Send the message to the LLM
    response = openai_client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[{"role":"system",
                    "content":"You are an intelligent assistant. You will execute tasks as prompted",
                    "role": "user", "content": message}],
        max_tokens=250,
        temperature=0.2
    )

    # Extract and return the response content
    return response.choices[0].message.content.strip()

We need to write a prompt that does 2 things

1. Share the context with the LLM about the tools it has at its disposal
2. Instruct a structured output so that we can execute the tool the LLM selected

def get_prompt_to_identify_tool_and_arguments(query,tools):
    tools_description = "\n".join([f"- {tool.name}, {tool.description}, {tool.inputSchema} " for tool in tools])
    return  ("You are a helpful assistant with access to these tools:\n\n"
                f"{tools_description}\n"
                "Choose the appropriate tool based on the user's question. \n"
                f"User's Question: {query}\n"                
                "If no tool is needed, reply directly.\n\n"
                "IMPORTANT: When you need to use a tool, you must ONLY respond with "                
                "the exact JSON object format below, nothing else:\n"
                "Keep the values in str "
                "{\n"
                '    "tool": "tool-name",\n'
                '    "arguments": {\n'
                '        "argument-name": "value"\n'
                "    }\n"
                "}\n\n")

We are passing two arguments - the original query from the user and the list of tools from the server(s). Let’s now see how get the list of tools from the server.

async def run(query: str):
    async with stdio_client(server_params) as (read, write):
        async with ClientSession(read,write) as session:

            await session.initialize()

            # Get the list of available tools
            tools = await session.list_tools()

            print(f"Available tools: {tools}")

            prompt = get_prompt_to_identify_tool_and_arguments(query,tools.tools)

            llm_response = llm_client(prompt)
            print(f"LLM Response: {llm_response}")

            tool_call = json.loads(llm_response)

            result = await session.call_tool(tool_call["tool"], arguments=tool_call["arguments"])

            print(f"BMI for weight {tool_call["arguments"]["weight_kg"]}kg and height {tool_call["arguments"]["height_m"]}m is {result.content[0].text}")

Key things happening here:

• We start the BMI Server process using stdio_client, and get access to its read and write streams

• Next, we create a session and initialize it. This sets up the communication between our client and the server.

• We then ask the server for a list of all available tools. In our case, it will return the calculate_bmi function we exposed earlier.

• “We prepare a detailed prompt using the earlier function, passing in the user's query and available tools. This prompt helps the language model figure out what tool to call and what arguments to pass.

• We parse the JSON response, extract the tool name and arguments, and then call the tool using session.call_tool.

• Finally, we return the output from the tool call — in this case, the calculated BMI.

To make all of the above work, we will add an entry point. When this script is run, it sets the user query — here we ask to calculate BMI — and runs the run() function using asyncio. The final result is printed on screen!

if __name__ == "__main__":
    import asyncio
    query = "Calculate BMI for height 5ft 10inches and weight 80kg"
    print(f"Sending query: {query}")
    result = asyncio.run(run(query))
    print(f"Result: {result}")

GitHub - zahere-dev/mcp-labs: Repository of my experiments with MCP

Repository of my experiments with MCP. Contribute to zahere-dev/mcp-labs development by creating an account on GitHub.

github.com

And that’s how you build an intelligent MCP client that can call tools dynamically using OpenAI and a BMI server. The magic lies in combining tool discovery, LLM prompting, and tool invocation — all within a simple and elegant flow.

What Are Guardrails in AI Agents?

Guardrails act as a safety net for AI agents, ensuring they operate within predefined boundaries and preventing misuse.

They work alongside agents, validating user inputs and outputs to safeguard against errors and inappropriate responses.

There are two types of guardrails:

• Input Guardrails: Validate user inputs before processing.

• Output Guardrails: Ensure the final response is appropriate before delivering it to the user.

Let’s see them in action!

Input Guardrails

• These validate initial user inputs before passing them to expensive models.

• They operate in three steps: receiving input, running validation functions, and triggering errors if misuse is detected.

Input guardrails are mechanisms put in place to validate, sanitize, and preprocess user inputs before they reach an AI model. These safeguards help in preventing:

• Malicious injections (e.g., prompt injection attacks)

• Profanity, hate speech, and harmful language

• Unstructured or irrelevant input that reduces model efficiency

• Bias amplification

By implementing input guardrails, developers can ensure that AI models receive well-structured and appropriate input, leading to better and safer outputs.

Why Are Input Guardrails Important?

1. Security: Prevents prompt injections, SQL injections, and adversarial attacks.

2. Quality Assurance: Filters out irrelevant or poorly structured queries.

3. Bias Mitigation: Helps remove explicit bias in prompts.

4. User Experience: Ensures clear and understandable input for meaningful responses.

5. Compliance: Adheres to ethical AI principles and regulatory requirements.

How to Implement Input Guardrails

Create the guardrail agent as below. Use the @input_guardrail decorator for the guardrail method.

More here in the documentation.

Input guardrails run in 3 steps:

1. First, the guardrail receives the same input passed to the agent.

2. Next, the guardrail function runs to produce a GuardrailFunctionOutput, which is then wrapped in an InputGuardrailResult

3. Finally, we check if .tripwire_triggered is true. If true, an InputGuardrailTripwireTriggered exception is raised, so you can appropriately respond to the user or handle the exception.

Pass the guardrail as an argument to the triage agent.

Handle the InputGuardrailTripwireTriggered exception.

Exception raised when the input is “Why is my order delayed? You guys are pathetic“

Output Guardrails

• These validate the final outputs generated by agents before they are delivered to users.

• They operate similarly to input guardrails but focus on the output stage to ensure accuracy and safety.

Output guardrails run in 3 steps:

1. First, the guardrail receives the same input passed to the agent.

2. Next, the guardrail function runs to produce a GuardrailFunctionOutput, which is then wrapped in an OutputGuardrailResult

3. Finally, we check if .tripwire_triggered is true. If true, an OutputGuardrailTripwireTriggered exception is raised, so you can appropriately respond to the user or handle the exception.

In the example below - we want to check for the “card” in the response of the order_agent and raise an exception accordingly.

Add the output guardrail as the argument to the final agent in the workflow.

We simulated the response of the order_agent for order 12346 to contain the word “card” and this is how the exception is caught.

Code

https://github.com/zahere-dev/openai-agents-sdk-tutorial

Conclusion:

• Guardrails are vital components of AI agent systems, ensuring they operate safely and efficiently.

• By implementing guardrails, developers can enhance user trust and prevent misuse scenarios effectively.
  

In this tutorial, we'll build a Restaurant Customer Support system using multiple specialized agents that can handle different types of customer queries automatically.

What is OpenAI Agent SDK?

The OpenAI Agent SDK provides a framework for creating multi-agent systems where specialized agents handle specific tasks. The core concepts include:

• Agents: The fundamental building blocks powered by language models and configured with specific instructions and tools

• Handoffs: The ability for agents to delegate tasks to other specialized agents

• Guardrails: Concurrent checks and validations that ensure interactions adhere to predefined criteria

• Tools: Functions or methods that agents can use to retrieve or manipulate data

Building a Restaurant Support System

Let's create a customer support system for a restaurant that can:

• Check order status

• Answer FAQs

• Handle complaints

• Manage table reservations

Setting Up the Environment

First, install the required package - that is all you need:

pip install openai-agents

Creating Our Agents

The Classifier (Triage) Agent

This agent serves as the entry point, determining the intent of customer queries and routing them to specialized agents.

The handoffs parameter is the key here - it takes a list of specialised agents as arguments.

The rest of agents look like below. Instead of handoffs we pass the tool/method the agent can use to process the task/ user query.

For example - in the case of order_agent, I have oversimplified the check_order_status tool that it uses to check the status of an order.

In a real use case, this function could be making an API call, or fetching the order status from a database.

We use @function_tool decorator to denote a method as a tool for an agent. This function can be used with an agent. It just needs be passed as an argument to the tools parameter in the Agent class.

FAQAgent, ComplaintAgent and ReservationAgent are also setup as above with their own relevant functions.

Running the Multi-Agent System

To create an interactive chat interface, the most important entity is to establish context or conversation history.

We can simple add “response.to_input_list()” which has the context from previous query added.

This makes it a very efficient chat interface.

Testing the System

Let's test our multi-agent system with various queries:

You will see that the agent can easily identify the intents and route them to specialised agents for processing.

All agents are easily able to extract relevant arguments for their tools and execute the functions.

Benefits of This Approach

1. Modularity: Each agent handles a specific domain, making code organization cleaner

2. Specialized Knowledge: Agents can be optimized for particular tasks

3. Maintainability: Add or modify agents without affecting the entire system

4. Contextual Understanding: The system maintains conversation context between interactions

Conclusion

The OpenAI Agent SDK provides a streamlined way to build powerful multi-agent systems. In this tutorial, we built a restaurant customer support system using specialized agents that can handle different types of queries. This approach provides a clean, modular architecture that can be extended to handle more complex use cases.

In a future tutorial, we'll explore more advanced features like guardrails and tracing to enhance the robustness and debuggability of our multi-agent system.

As an engineer in the rapidly evolving field of AI, I don't just want to leverage GenAI APIs and build agents.

The Catalyst: DeepSeek's Innovation

Recently, the AI community has been abuzz with the groundbreaking innovations from DeepSeek.

This Chinese startup has developed a new AI model, R1, which matches or even surpasses the performance of leading models like ChatGPT, but at a fraction of the cost.

DeepSeek's approach has highlighted the importance of efficiency and deep understanding of AI fundamentals, particularly in mathematics.

This has inspired me to reset my learning and focus on building a strong mathematical foundation.

I believe having a solid math foundation will serve me well for a long and successful career.

So here’s what I am doing about it.

Quarterly Goals and Tracking

For the past year, I've been setting quarterly goals and tracking them using a tracker board that I have right above my monitor.

This board helps me visualize my progress and stay consistent.

Over the last six months, I've been able to post weekly videos and newsletters by setting clear goals. My previous quarterly goals included creating 12 videos, writing 12 news articles, losing 5 kilos, and working on specific content.

The 12-Week Plan

For this quarter, I've divided my 12-week plan into three tracks:

1. Work Track: This track takes up 80-90% of my time, roughly 40-50 hours per week. It includes my regular job responsibilities and tasks. The learning in these hours is immense as well, as I read code, documentation of the systems and tools I use, and of course the internet/Google to execute my day-to-day tasks.

2. Latest Trends Track: I dedicate 4-5 hours per week to stay updated with the latest trends in ML, DL, and GenAI. This involves reading new research papers, exploring new frameworks, and learning about new techniques. I spread this time throughout the week or focus on it during weekends.

3. Learning Track: This quarter, I'm focusing on resetting my learning by diving deep into the mathematics behind AI. After spending two years in this field, I've realized the importance of understanding the mathematical foundations of algorithms. This knowledge will help me optimize, fine-tune, and (hopefully) conduct deep research in ML and AI.

Learning Mathematics for AI: 12-Week Plan

To build a strong foundation in Math for AI, I've structured my learning track into a simple 12-week plan. Here's the simple breakdown:

Three core topics - Linear Algebra, Calculus, and Probability & Statistics are spread over 12 weeks - with 4 weeks/40 hours dedicated to each.

Will this make me an expert in Math?

Definitely not!

Will I be better at Math than I am today?

ABSOLUTELY!

I will share my learnings here every week. Subscribe to the newsletter, if you haven’t already, to tag along with me.

Conclusion

This 12-week plan is designed to help me balance my work, stay updated with the latest trends, and deepen my understanding of AI.

Thank you for joining me on this journey. Stay tuned for updates on my progress and feel free to share your own learning plans in the comments!

If you have any questions or need further assistance, feel free to ask!

The Problem at Hand

Uber's data platform processes approximately 1.2 million interactive queries monthly, with 36% of these coming from the operations organization. This group—comprising engineers, data scientists, and operations professionals—analyzes data from hundreds of thousands of tables across various domains to derive actionable insights.

However, the process of composing and executing queries was a bottleneck:

• 10 minutes per query: Each actor spent an average of 10 minutes composing a query.

• Inefficiency Loop: Users would sift through datasets, run queries, and validate results in a repetitive cycle.

• Wasted Time: The cumulative effect of this inefficiency led to significant lost productivity.

This challenge is not unique to Uber. It resonates across industries, from e-commerce to customer support, where operations teams grapple with similar inefficiencies.

Enter QueryGPT: The Hackathon Solution

In 2023, a team at Uber's hackathon introduced QueryGPT, a prototype designed to streamline the query-generation process. Here's how it worked:

1. Metadata-Driven Query Generation:

   • Stored 20 SQL queries with metadata (table and schema information).

   • Mapped natural language prompts to these queries.

2. Few-Shot Prompting:

   • Used a Retrieval-Augmented Generation (RAG) technique to fetch relevant queries.

   • Generated SQL queries in response to user prompts.

3. Initial Results:

   • Reduced query composition time from 10 minutes to 3 minutes.

   • Achieved an 18% productivity gain.

While this was a promising start, the prototype faced scalability and technical challenges, necessitating further iterations.

Challenges and Iterative Solutions

Key Challenges

1. Prompt-to-Schema Mismatch:

   • The system struggled to align user prompts with relevant schemas.

2. Token Limitations:

   • Some schemas had over 200 columns, leading to token counts exceeding GPT-4's 32k limit.

The Final Architecture

The refined system, powered by Azure OpenAI and GPT-4, demonstrated remarkable efficiency:

• Context Optimization: Leveraged a context window of 128k tokens to handle large schemas.

• Human Validation: Ensured precision through user acknowledgment of suggested tables.

• Scalable Design: Addressed the challenge of querying across hundreds of thousands of datasets.

Uber's engineering team implemented a robust architecture combining SQL, RAG, agents, and custom configurations. Here's a breakdown:

1. Domain-Specific Curation:

   • Decomposed datasets into business domains/workflows (e.g., mobility, trips, support).

   • Allowed the system to focus on smaller, relevant subsets of data.

2. Intent Agent:

   • Classified user prompts to map them to the appropriate domain or workspace.

   • Likely employed a vector-store-based intent classifier for high accuracy.

3. Table Agent:

   • Retrieved domain-specific tables and displayed them in a user-friendly interface.

   • Enabled human-in-the-loop validation to ensure table relevance.

4. Enhanced RAG Pipeline:

   • Generated few-shot prompts tailored to the specific domain.

   • Sent refined prompts to GPT-4 for SQL query generation.

Real-World Impact

By the 20th iteration, Uber's Query GPT achieved a staggering 140,000 hours saved monthly across its operations organization. This success underscores the value of combining AI, domain-specific curation, and user-centric design.

Key Takeaways for Your Business

Uber's solution offers valuable insights for tackling similar challenges in other industries:

1. Break Down Data Silos:

   • Organize datasets by business domains to streamline data retrieval.

2. Implement Intent Detection:

   • Use AI-driven agents to map user queries to relevant datasets or workspaces.

3. Leverage Human-in-the-Loop Systems:

   • Involve users in validating AI-generated outputs for enhanced accuracy.

4. Iterate for Scalability:

   • Start small, learn from challenges, and scale iteratively.

The Future of AI in Operations

Uber's journey with QueryGPT exemplifies the transformative potential of generative AI in operational analytics. By reducing manual effort and empowering teams with intelligent tools, businesses can unlock unprecedented productivity gains.

Whether you're in e-commerce, customer support, or any data-intensive field, the principles behind Uber's success can guide your own AI-driven innovations.

Want to delve deeper into the technical details? Check out Uber's engineering blog here for the full story.

Google's recent white paper on "Agents" has created quite a buzz.

The paper explores the concept of AI agents and delves into their architecture and potential. Let's break down what this white paper offers, its key takeaways, and some areas where it could improve.

The Marketing Angle: A Platform-Centric View?

At first glance, the white paper feels like a marketing tool for Google's Vertex AI. And that's perfectly fine—after all, companies often use such publications to showcase their platforms.

However, adopting a more platform-agnostic approach could have made the paper more universally applicable.

For instance, many examples in the white paper are tied to Vertex AI-specific features, which might be unfamiliar to those using other agentic frameworks.

Additionally, certain concepts, like extensions, are introduced but not elaborated on in sufficient detail, leaving room for better documentation and clarity.

Despite these limitations, the paper provides a solid starting point for understanding agents. Let’s dive into the key concepts.

What is an Agent?

Google defines an agent as:

"An application that attempts to achieve a goal by observing the world and acting upon it using the tools at its disposal."

This definition is both simple and powerful. It captures the essence of what an agent is without overcomplicating things. While many experts on platforms like LinkedIn and YouTube often layer terms like reasoning, context awareness, and more onto the definition, the core idea remains straightforward.

Interestingly, my personal favorite definition comes from Hugging Face, which describes AI agents as:

"Programs where LLM outputs control the workflows."

This succinctly highlights the operational dynamics of agents, especially when integrated with language models.

The Agentic Architecture: Core Components

The white paper also details the architecture of agents, a topic I’ve previously discussed on my channel. Here's a simplified breakdown of the three primary components that define an agentic system:

1. The Model

At the heart of any agent lies a language model. This serves as the foundation for the agent's intelligence and capabilities. Trained on extensive datasets, the model enables the agent to comprehend language, process instructions, and provide knowledge.

In an agentic framework, the model is not just a passive responder. Its capabilities drive the decision-making processes within the orchestration layer.

2. The Tools

Tools are what set agents apart from simple LLM calls. Since LLMs are inherently limited—they can’t interact with external systems or access real-time information—tools extend their capabilities.

Agents use tools to interact with the external world, making them more dynamic and useful. Frameworks like LangChain and LlamaIndex exemplify how tools can augment the performance of agentic systems, enabling them to achieve their goals effectively.

3. The Orchestration Layer

Often referred to as the reasoning loop, this layer governs the agent's ability to:

• Plan: Decide the next steps in a workflow.

• Reason: Analyze the gathered information.

• Execute: Take action based on the plan.

This iterative process is the backbone of an agent’s functionality, ensuring it can adapt and respond intelligently to various scenarios.

Tools (Key Takeaways)

1. Extensions

   • Definition: Extensions are interfaces that bridge the gap between APIs and agents. They allow for seamless API execution by teaching agents how to use APIs via examples.

   • Use Case: Extensions are ideal for scenarios where the agent needs to dynamically interact with APIs like booking flights or fetching weather data. They reduce ambiguity in API calls by guiding the agent with context and examples.

   • Your Example: Your custom tool implementation fits here because it defines tools with a name and description, guiding the LLM to invoke the correct tool and arguments based on context.

2. Functions

   • Definition: Functions are reusable logic modules that allow developers to define behavior and handle specific tasks.

   • Difference: Unlike extensions, functions offload API execution to client-side logic or middleware, especially in cases where security or authentication constraints prevent direct calls from the LLM.

   • Your Observations: Google's distinction clarifies that functions give developers fine-grained control and decouple execution from the agent, making iteration easier without redeploying infrastructure.

As we approach 2025, there are rapid advancements in technology that we, as engineers, cannot afford to ignore. Whether you’re looking to add value to your organization, grow your skillset, or simply future-proof your career, there are certain core skills that will set you apart.

Let’s dive right in.

Video

The New Baseline: AI and Business Context Understanding

Gone are the days when being a full-stack engineer meant just knowing how to develop and deploy applications. Today, understanding the "why" behind what you’re building is just as important as the "how."

If you find yourself working on user stories without knowing the business context or understanding the decisions made by your engineering or product managers, it’s time to rethink your approach.

Business understanding is what differentiates an average engineer from a great one.

And as AI continues to integrate into every facet of technology, this understanding will become even more critical.

Here’s my take on the essential skills every full-stack engineer should master to thrive in this new era.

1. Core AI/ML Engineering Skills

Large Language Models (LLMs)

By now, you’ve probably experimented with tools like ChatGPT or integrated APIs into your projects. But to truly excel, you need to:

• Understand how LLMs work at a deeper level.

• Learn about fine-tuning models for specific use cases.

• Master prompt engineering – arranging instructions in ways that yield the best results.

Retrieval-Augmented Generation (RAG)

RAG will remain indispensable for enterprise applications, even as LLMs handle increasingly large context windows. Why? Because RAG ensures that only the most relevant data is fed into the model, optimizing performance and cost. Gaining expertise in designing RAG pipelines and integrating them with business workflows is a must.

AI Agents

The buzzword for 2025 is "agents." Multi-agent frameworks and systems are set to revolutionize how we build solutions. Understanding how to design and orchestrate these systems will keep you ahead of the curve.

2. Technical Stack Mastery

Python: The Go-To Language

Python’s ecosystem for AI and ML is unparalleled. From frameworks like PyTorch and scikit-learn to tools like LangChain and LlamaIndex, Python should be in every engineer’s toolkit. If you’re new to the space, start experimenting with these frameworks.

Cloud Platforms and Vector Databases

• Familiarize yourself with cloud platforms like AWS, Azure, or GCP, and tools like VMware Tanzu.

• Learn about vector databases, such as PGVector, which allow efficient storage and retrieval of embeddings. Even spinning up a simple Docker instance can give you hands-on experience.

API Development

Frameworks like FastAPI and Flask make it easier to create and deploy AI-powered web applications. Combine this with an understanding of socket programming for real-time communication tools like chatbots, and you’ll be unstoppable.

3. Project and Professional Skills

Business Context Awareness

This is the game-changer. Knowing how your application aligns with business goals will not only make you a better engineer but also ensure your contributions are recognized. Always ask:

• Why is this feature important?

• How will this decision impact the user or business?

Experimentation Over Perfection

Start small. Use APIs to create prototypes and simulate workflows. For instance:

• Identify areas in your organization where generative AI can add value.

• Build mock solutions with sample data to prove concepts.

MLOps and Deployment

Understanding MLOps tools and practices is becoming essential for deploying AI solutions. Even if you’re not directly managing infrastructure, knowing how to streamline deployment pipelines will make you invaluable to your team.

My Journey: From Experimentation to Mastery

When I transitioned into AI engineering in early 2023, I didn’t start by mastering machine learning fundamentals. Instead, I began tinkering with LLM APIs and building prototypes. This hands-on experimentation allowed me to solve real-world problems while gradually deepening my knowledge of ML fundamentals over the next six months.

This approach worked wonders for me, and I recommend it to anyone looking to enter the field. Don’t get bogged down by theory. Instead, balance learning with application. Keep a 60/40 or even 50/50 ratio between theory and practice to maximize your growth.

Final Thoughts

The expectations for full-stack engineers are evolving, and AI skills are no longer optional.

By mastering core AI/ML engineering concepts, staying updated with the latest frameworks, and understanding business context, you’ll position yourself as a leader in your field.

So, what’s your game plan for 2025? Let me know in the comments below. And if you’re ready to start tinkering, I’ve got tons of tutorials on my channel to help you get started. Until next time, happy coding!

Dynamic prompt generation has become a cornerstone of modern AI workflows.

Whether you're building personalized email campaigns, travel itineraries, or AI-driven recommendations, the ability to generate structured content dynamically is invaluable.

In this blog, we'll explore how Jinja2, a powerful templating engine, stands out in this domain and compare it with tools like LangChain for crafting dynamic prompts.

Why Dynamic Prompting Matters

If you are building an AI assistant tasked with creating personalized travel itineraries or summarizing user activities, static prompts won't cut it here – you need templates that adapt to the data at hand. This is where tools like Jinja2 and LangChain's prompt templates shine.

Jinja2: The All-Rounder for Dynamic Templates

Jinja2 is a versatile templating engine widely known for its use in web development but equally adept at generating dynamic text for emails, reports, and prompts. Here's why Jinja2 should be in your toolkit:

1. Seamless Integration of Logic

Jinja2 allows you to embed loops, conditionals, and filters directly in your templates. For example, creating tailored recommendations becomes straightforward:

Dear {{ user_name }},

Here’s a summary of your recent activities:
{% for activity in activities %}
- On {{ activity.date }}: {{ activity.description }}
{% endfor %}

{% if status == "pass" %}
Congratulations on passing the test. Keep up the great work!
{% else %}
Keep trying, and you'll get there!
{% endif %}

2. Readable and Reusable

With its clean syntax, Jinja2 makes templates easy to maintain and reuse across projects. It's perfect for use cases like:

• Personalized emails

• Travel itineraries

• AI-driven content generation

3. Performance Efficiency

Jinja2 minimizes overhead, making it an excellent choice for applications requiring rapid dynamic rendering.

Comparing Jinja2 and LangChain for Prompt Templates

While Jinja2 excels in general-purpose dynamic content generation, LangChain's PromptTemplate is specifically designed for AI workflows, making it the go-to for LLM integrations.

LangChain Example:

from langchain.prompts import PromptTemplate

template = """
Dear {user_name},

Here’s a summary of your recent activities:
{activities}

Here are some tailored recommendations for you:
{recommendations}

{closing_note}
"""

prompt = PromptTemplate(
    input_variables=["user_name", "activities", "recommendations", "closing_note"],
    template=template,
)

email = prompt.format(
    user_name="Alice",
    activities="- Completed the Python course.\n- Joined the AI workshop.",
    recommendations="- Read 'Deep Learning for Beginners'.\n- Join the Advanced AI Projects Club.",
    closing_note="Congratulations on passing the test!",
)

print(email)

Key Differences:

• Flexibility: Jinja2 supports complex logic directly in the template, while LangChain separates logic and content.

• AI Integration: LangChain is optimized for workflows where prompts are fed into LLMs.

• Learning Curve: Jinja2 has a gentler curve for general developers, whereas LangChain is ideal for those already in the AI ecosystem.

Real-World Use Case: Personalized Travel Itineraries

Using Jinja2, you can craft luxurious travel experiences tailored to user preferences. Here's an example:

Input Data

codeinput_data = {
    "name": "John Doe",
    "destination": "Paris, France",
    "interests": ["art", "history", "fine dining"],
    "travel_type": "luxury",
    "suggested_activities": [
        {"name": "Private Louvre Tour", "description": "Explore iconic art pieces with a guide."},
        {"name": "Seine River Dinner Cruise", "description": "Enjoy a gourmet dinner on a Seine cruise."},
    ],
    "recommended_accommodations": [
        {"name": "Le Meurice", "description": "5-star luxury hotel with Michelin dining."},
    ],
}

Jinja2 Template

 """
Traveler Profile:
- Name: {{ name }}
- Age: {{ age }}
- Travel Dates: {{ travel_dates }}
- Travel Destination: {{ destination }}
- Interests: {{ interests|join(", ") }}

{% if travel_type == 'luxury' %}
The traveler prefers a luxury experience. Suggest the following premium activities and accommodations:
{% elif travel_type == 'adventure' %}
The traveler seeks adventure. Recommend these thrilling activities and adventurous destinations:
{% else %}
The traveler is interested in a balanced experience. Consider these activities and attractions:
{% endif %}

{% for activity in suggested_activities %}
- {{ activity.name }}: {{ activity.description }}
  {% if activity.requirements %}
  Requirements: {{ activity.requirements|join(", ") }}
  {% endif %}
{% endfor %}

{% if recommended_accommodations %}
Recommended Accommodations:
{% for accommodation in recommended_accommodations %}
- {{ accommodation.name }}: {{ accommodation.description }}
  Location: {{ accommodation.location }}
  Amenities: {{ accommodation.amenities|join(", ") }}
{% endfor %}
{% endif %}

Traveler's Notes:
{% if traveler_notes %}
{% for note in traveler_notes %}
- {{ note }}
{% endfor %}
{% endif %}

Based on the above information, create a 3-day itinerary tailored to the traveler’s preferences and needs. Ensure activities, meals, and downtime are appropriately balanced.
""

Output

Traveler Profile:
- Name: John Doe
- Age: 35
- Travel Dates: 2024-01-15 to 2024-01-18
- Travel Destination: Paris, France
- Interests: art, history, fine dining, luxury shopping


The traveler prefers a luxury experience. Suggest the following premium activities and accommodations:



- Private Louvre Tour: Enjoy a private, guided tour of the Louvre Museum, exploring its iconic art pieces.

  Requirements: Comfortable walking shoes, Museum pass


- Seine River Dinner Cruise: Experience a luxurious evening with a gourmet dinner on a Seine River cruise.

  Requirements: Formal attire


- Champs-Élysées Shopping Tour: Indulge in a day of shopping at high-end boutiques along the Champs-Élysées.




Recommended Accommodations:

- Le Meurice: A 5-star luxury hotel offering Michelin-star dining and exceptional service.
  Location: Rue de Rivoli, Paris
  Amenities: Spa, Fine dining, Concierge service

- Hôtel Plaza Athénée: Iconic Parisian hotel with stunning views of the Eiffel Tower.
  Location: Avenue Montaigne, Paris
  Amenities: Luxury suites, Haute couture stores, Gourmet restaurants



Traveler's Notes:


- Has dietary restrictions: no shellfish.

- Prefers private tours over group activities.



Based on the above information, create a 3-day itinerary tailored to the traveler’s preferences and needs. Ensure activities, meals, and downtime are appropriately balanced.

Price Monitoring Agent | Part 2 | Full Tutorial

I published a full video tutorial on building a Price Monitoring Agent using Pydantic AI. I accomplished the below goal.

Keeping track of fluctuating product prices across e-commerce platforms can be a daunting task.

Whether you're tracking a personal wishlist or monitoring competitors' pricing for your business, automating this process can save time and effort.

In this guide, we’ll explore how to build a price monitoring agent using the Pydantic AI framework—a robust agentic framework from the creators of Pantic, a popular data validation library.

This tutorial is part one of a series. Today, we’ll focus on building a scraper agent to extract key product details like title, description, price, and more.

In the next part, we’ll expand this agent to store data in a database and send notifications for price changes.

What is Pydantic AI?

Pydantic AI is revolutionizing the way developers build applications that leverage Generative AI. As a Python Agent Framework, it simplifies the creation of production-grade applications by integrating robust data validation with the power of LLMs. Here’s why Pydantic AI stands out:

• Built on Proven Foundations: Developed by the creators of Pydantic, which is widely used in various AI frameworks like OpenAI and LangChain, Pydantic AI inherits a strong legacy of type safety and structured data management.

• Model-Agnostic Flexibility: Currently supporting models like OpenAI, Gemini, and Groq, Pydantic AI allows developers to easily implement support for additional models through a simple interface. This flexibility ensures that your application can adapt to various AI technologies without significant overhead.

• Enhanced Developer Experience: With features like vanilla Python control flow and a novel dependency injection system, Pydantic AI empowers developers to apply familiar coding practices. This leads to more maintainable code and a smoother development process.

• Streamlined Response Validation: The framework not only validates incoming data but also ensures that responses from LLMs are structured and validated, enhancing reliability in application behavior.

Overview of the Price Monitoring Agent

Our agent will:

1. Scrape product details (title, description, price, currency, and image URL) from a given URL.

2. Parse the information into a structured format.

3. Prepare for database storage and notification handling (to be implemented in part two).

Here’s how the process works (diagram above)

1. Input: Product page URL

2. Scraper Tool: Extracts structured data using Beautiful Soup and Markdownify.

3. Agent: Processes the scraped data using Pydantic AI for type-safe responses.

Code

https://colab.research.google.com/drive/1wEFO0_W13J_DBtjd55vNK7vQa_UGo7Hm#scrollTo=3O2V5Jth2Q8F

To be continued in Part 2

Today, we’re diving into an exciting project: creating a Multi-Agent Orchestrator.

This post is an extension of my earlier guide, "Building an AI Agent from Scratch."

If you’re new here, I recommend revisiting that post to get up to speed, as we’ll build upon its concepts and code.

In this project, we’ll tackle orchestrating actions between multiple agents, enabling seamless execution of tasks such as fetching weather information and the current time. Let’s jump in!

What Is a Multi-Agent Orchestrator?

A Multi-Agent Orchestrator is a system that:

1. Identifies the intent of a user’s input.

2. Selects the appropriate agent to handle the request.

3. Executes tasks using tools associated with the agent.

Think of it as a manager assigning tasks to specialized team members. This orchestration ensures complex queries involving multiple tasks are handled efficiently.

What We’ll Build

We’ll create:

• Agents: Specialized entities to handle tasks like fetching weather or time.

• Tools: Functional utilities for the agents, such as APIs or database queries.

• Orchestrator: The central system managing task delegation and execution.

Key Components of an Agent

An agent has three main components:

1. Reasoning Loop: Decides the next action based on context.

2. Model: Uses a language model (LLM) for decision-making.

3. Tools: A list of utilities to perform specific tasks.

Our agents will dynamically decide which tool to use, making them highly adaptable.

The Agent Class

Here’s a high-level breakdown of the Agent class:

• Constructor: Initializes the agent with a name, description, tools, and an LLM model.

• Process Input: Takes user input, decides on a tool, and executes the task.

• Prompting: Constructs a prompt for the LLM to guide decision-making.

Agents also handle parsing JSON responses from the LLM to ensure smooth execution.

from abc import ABC, abstractmethod
import ast
import os
import requests
from llm.llm_ops import query_llm
from tools.base_tool import Tool
import json


class Agent:
    def __init__(self, Name: str, Description: str, Tools: list, Model: str):        
        self.memory = []
        self.name = Name
        self.description = Description
        self.tools = Tools
        self.model = Model
        self.max_memory = 10

    def json_parser(self, input_string):

      print(type(input_string))

      python_dict = ast.literal_eval(input_string)
      json_string = json.dumps(python_dict)
      json_dict = json.loads(json_string)

      if isinstance(json_dict, dict) or isinstance(json_dict,list):
        return json_dict

      raise "Invalid JSON response"

    def process_input(self, user_input):
        self.memory.append(f"User: {user_input}")
        12

        context = "\n".join(self.memory)
        tool_descriptions = "\n".join([f"- {tool.name()}: {tool.description()}" for tool in self.tools])
        response_format = {"action":"", "args":""}

        prompt = f"""Context:
        {context}

        Available tools:
        {tool_descriptions}

        Based on the user's input and context, decide if you should use a tool or respond directly.        
        If you identify a action, respond with the tool name and the arguments for the tool.        
        If you decide to respond directly to the user then make the action "respond_to_user" with args as your response in the following format.

        Response Format:
        {response_format}

        """

        response = query_llm(prompt)
        self.memory.append(f"Agent: {response}")

        response_dict = self.json_parser(response)

        # Check if any tool can handle the input
        for tool in self.tools:
            if tool.name().lower() == response_dict["action"].lower():
                return tool.use(response_dict["args"])

        return response_dict

The Orchestrator

The orchestrator coordinates multiple agents:

1. Accepts user input.

2. Selects the right agent based on the intent.

3. Manages task execution, including cases where multiple tasks are requested.

Core Features of the Orchestrator:

• Maintains context by storing user queries, agent responses, and intermediate results.

• Uses a reasoning loop to determine the next steps.

• Constructs prompts to guide the LLM in selecting the right agent and tools.

import ast
import json
from llm.llm_ops import query_llm
from agents.base_agent import Agent
from logger import log_message

class AgentOrchestrator:
    def __init__(self, agents: list[Agent]):
        self.agents = agents
        self.memory = []  # Stores the reasoning and action steps taken
        self.max_memory = 10

    def json_parser(self, input_string):

      print(type(input_string))

      python_dict = ast.literal_eval(input_string)
      json_string = json.dumps(python_dict)
      json_dict = json.loads(json_string)

      if isinstance(json_dict, dict) or isinstance(json_dict,list):
        return json_dict

      raise "Invalid JSON response"

    def orchestrate_task(self, user_input: str):        
        self.memory = self.memory[-self.max_memory:]

        context = "\n".join(self.memory)

        print(f"Context: {context}")

        response_format = {"action":"", "input":"", "next_action":""}

        def get_prompt(user_input):
            return f"""

                Use the context from memory to plan next steps.                
                Context:
                {context}

                You are an expert intent classifier.
                You need will use the context provided and the user's input to classify the intent select the appropriate agent.                
                You will rewrite the input for the agent so that the agent can efficiently execute the task.                                                

                Here are the available agents and their descriptions:
                {", ".join([f"- {agent.name}: {agent.description}" for agent in self.agents])}

                User Input:
                {user_input}              

                ###Guidelines###
                - Sometimes you might have to use multiple agent's to solve user's input. You have to do that in a loop.
                - The original userinput could have multiple tasks, you will use the context to understand the previous actions taken and the next steps you should take.
                - Read the context, take your time to understand, see if there were many tasks and if you executed them all
                - If there are no actions to be taken, then make the action "respond_to_user" with your final thoughts combining all previous responses as input.
                - Respond with "respond_to_user" only when there are no agents to select from or there is no next_action
                - You will return the agent name in the form of {response_format}
                - Always return valid JSON like {response_format} and nothing else.                

                """


        response = ""
        loop_count = 0
        self.memory = self.memory[-10:]        
        prompt = get_prompt(user_input)
        llm_response = query_llm(prompt)

        llm_response = self.json_parser(llm_response)
        print(f"LLM Response: {llm_response}")

        self.memory.append(f"Orchestrator: {llm_response}")


        action=  llm_response["action"]
        user_input = llm_response["input"]

        print(f"Action identified by LLM: {action}")


        if action == "respond_to_user":
            return llm_response
        for agent in self.agents:
            if agent.name == action:
                print("*******************Found Agent Name*******************************")
                agent_response = agent.process_input(user_input)
                print(f"{action} response: {agent_response}")
                self.memory.append(f"Agent Response for Task: {agent_response}")
                print(self.memory)
                return agent_response                


    def run(self):
        print("LLM Agent: Hello! How can I assist you today?")
        user_input = input("You: ")
        self.memory.append(f"User: {user_input}")

        while True:            
            if user_input.lower() in ["exit", "bye", "close"]:
                print("See you later!")
                break

            response = self.orchestrate_task(user_input)
            print(f"Final response of orchestrator {response}")
            if isinstance(response, dict) and response["action"] == "respond_to_user":                
                log_message(f"Reponse from Agent: {response["input"]}", "RESPONSE")
                user_input = input("You: ")
                self.memory.append(f"User: {user_input}")                
            elif response == "No action or agent needed":
                print("Reponse from Agent: ", response)
                user_input = input("You: ")
            else:
                user_input = response

Tools in Action

Agents use tools to perform tasks. For example:

• Weather Tool: Fetches real-time weather data from OpenWeatherMap.

• Time Tool: Determines the local time for a given city, even without a timezone.

Each tool includes:

• A name and description to guide the LLM.

• A use method to perform the task.

import os
import requests
from tools.base_tool import Tool

class WeatherTool(Tool):
    def name(self):
        return "Weather Tool"

    def description(self):
        return "Provides weather information for a given location. The payload is just the location. Example: New York"

    def use(self, location:str):        
        api_key = os.getenv("OPENWEATHERMAP_API_KEY")
        url = f"http://api.openweathermap.org/data/2.5/weather?q={location}&appid={api_key}&units=metric"
        response = requests.get(url)
        data = response.json()
        if data["cod"] == 200:
            temp = data["main"]["temp"]
            description = data["weather"][0]["description"]
            response = f"The weather in {location} is currently {description} with a temperature of {temp}°C."
            print(response)
            return response
        else:
            return f"Sorry, I couldn't find weather information for {location}."

Demo: Running the Orchestrator

Here’s a quick demonstration:

1. Query: “What’s the weather in Bangalore, and what’s the current time?”

2. Execution:

   • The orchestrator identifies the intent (weather and time).

   • Delegates tasks to the respective agents.

   • Combines responses to provide the final answer.

Example Output:

• "The weather in Bangalore is misty with a temperature of 22°C. The current time in Bangalore is 12:27 AM."

from agents.base_agent import Agent
from tools.weather_tool import WeatherTool
from tools.time_tool import TimeTool
from orchestrator import AgentOrchestrator

from dotenv import load_dotenv
import os

# Load environment variables from .env file
load_dotenv()

# Create Weather Agent
weather_agent = Agent(
    Name="Weather Agent",
    Description="Provides weather information for a given location",
    Tools=[WeatherTool()],
    Model="gpt-4o-mini"
)

# Create Time Agent
time_agent = Agent(
    Name="Time Agent",
    Description="Provides the current time for a given city",
    Tools=[TimeTool()],
    Model="gpt-4o-mini"
)

# Create AgentOrchestrator
agent_orchestrator = AgentOrchestrator([weather_agent, time_agent])

# Run the orchestrator
agent_orchestrator.run()

What’s Next?

This orchestrator is just the beginning. You can:

• Add more agents for tasks like translation, currency conversion, or database queries.

• Optimize prompts for better LLM responses.

• Extend the system for real-world applications like customer support or smart assistants.

Full Code

https://github.com/zahere-dev/augmate

Final Thoughts

Building a Multi-Agent Orchestrator showcases the power of combining LLMs with task-specific agents. By modularizing tasks and leveraging context effectively, you can create systems that are both scalable and intelligent.

Stay tuned for more updates, and feel free to share your thoughts or ask questions in the comments below. Don’t forget to check out the accompanying video for a detailed walkthrough of the code.

Happy coding! 🚀