Building a Production MCP Tool Gateway with FastMCP 3.x

The bottom line: FastMCP 3.x abstracts Model Context Protocol into a decorator-based Python API and reaches 4 million+ daily downloads as of March 2026 [1]. This build log walks through constructing a production MCP gateway that consolidates multiple backend tool servers (database, code execution, external APIs) behind a single FastMCP endpoint — covering architecture decisions, OAuth integration, transport selection, and the token-saving code mode that cuts tool schema overhead by up to 99% [1].

The Problem: Tool Server Explosion

MCP crossed 97 million monthly SDK downloads in March 2026 and hosts over 13,000 public servers [2]. That’s great for ecosystem variety but creates a real production problem: an AI agent that connects to 10 tool servers loads 10 sets of tool definitions into its context window. Each server dumps its entire tool schema — names, descriptions, JSON schemas, parameter enums — before the agent processes a single user message.

The community identified this as the #1 pain point: “GitHub MCP dumps 43 tools into the context window before doing anything” — it destroys agent performance [3].

The fix: Build a gateway server that presents a curated, intent-oriented interface to the agent, and delegates to specialized backend servers only when tools are actually invoked.

Architecture: Gateway Pattern

The gateway sits between the agent client and the tool backends:

Agent Client ──Streamable HTTP──→ Gateway Server (FastMCP 3.x)
                                     │
                          ┌──────────┼──────────┐
                          │          │          │
                     DB Tool    Weather    Code Exec
                     Server     Server     Server

Why a Gateway Instead of Direct Connections

Approach	Context Waste	Auth Management	Tool Discovery	Latency
Direct to each server	Full schema from every server	N auth flows per tool set	Agent must know all servers	Direct calls, no overhead
Gateway (this build)	One curated schema only	Single OAuth flow	Agent talks to one endpoint	+1 hop per call

The gateway trades a small latency overhead for massive context savings and simplified auth. For production systems where the agent runs hundreds of tool calls per session, that tradeoff is almost always worth it [3].

Build: The Gateway Server

Step 1 — Foundation

# gateway.py — Production MCP Gateway
import os
import json
import httpx
from fastmcp import FastMCP
from fastmcp.providers import import_server
from pydantic import ValidationError

# Fail fast — validate environment at startup
for var in ["DB_API_URL", "WEATHER_KEY", "CODE_EXEC_URL"]:
    if not os.environ.get(var):
        raise RuntimeError(f"Missing required env var: {var}")

mcp = FastMCP(
    "tool-gateway",
    description="Curated tool gateway: database, weather, code execution",
    version="1.0.0",
    json_response=True,
)

FastMCP 3.0 ships with startup validation, auto schema generation from Python type hints, and built-in JSON response formatting [1]. The json_response=True flag ensures all tool responses return structured JSON rather than raw strings — critical for agentic parsing.

Step 2 — Core Tools

The gateway exposes curated intent-oriented tools, not raw API endpoints:

@mcp.tool()
def query_database(sql: str, params: list[str] = None) -> dict:
    """Execute a read-only SQL query against the production database.
    Call this when the user asks about data, records, or statistics.
    Returns {columns: [...], rows: [[...], ...], row_count: int}.
    SQL SELECT queries only — mutation queries will be rejected.
    Max 500 rows per query."""
    if not sql.strip().upper().startswith("SELECT"):
        return {"error": "Read-only gateway. Only SELECT queries allowed."}
    response = httpx.post(
        os.environ["DB_API_URL"],
        json={"sql": sql, "params": params or []},
        timeout=30.0,
    )
    response.raise_for_status()
    return response.json()

@mcp.tool()
def get_forecast(city: str, days: int = 3) -> dict:
    """Get weather forecast for a city.
    Call this for weather questions, travel planning, or climate context.
    Returns current conditions + N-day forecast with temp, humidity, and conditions."""
    if days < 1 or days > 7:
        raise ValueError(f"Days must be 1-7, got {days}")
    response = httpx.get(
        "https://api.weatherapi.com/v1/forecast.json",
        params={"key": os.environ["WEATHER_KEY"], "q": city, "days": days},
        timeout=10.0,
    )
    response.raise_for_status()
    data = response.json()
    return {
        "city": data["location"]["name"],
        "forecast_days": data["forecast"]["forecastday"][:days],
        "units": "celcius",
    }

Three production patterns applied here [3]:

Tool descriptions answer three questions — When to call, what input is needed, what structure is returned. Bad: "Search the database". Good: the descriptions above.
Input validation in tools — Bad parameters raise structured ValueError, not HTTP 500s.
Timeouts on every external call — 10-30 second ceilings prevent hung agents.

Step 3 — Code Mode (FastMCP 3.1+)

FastMCP 3.1 adds code mode — the single biggest token saver. Instead of enumerating every tool with full schemas, code mode collapses the tool definition into a compact representation that the agent treats as a code library to import [1].

@mcp.tool()
def code_execute(source: str, language: str = "python3", timeout_sec: int = 30) -> dict:
    """Execute code in a sandboxed environment.
    Call this when the user wants to run calculations, test algorithms,
    or process data through custom logic.
    Returns {stdout, stderr, exit_code, execution_time_ms}.
    Supported languages: python3, node, go, rust."""
    response = httpx.post(
        os.environ["CODE_EXEC_URL"],
        json={"source": source, "language": language, "timeout_sec": timeout_sec},
        timeout=timeout_sec + 5,
    )
    response.raise_for_status()
    return response.json()

With code mode enabled (fastmcp run gateway.py --code-mode), tool schemas for a 10-tool server drop from ~15,000 tokens to ~2,000-3,000 — the same 85-98% reduction we described in the context optimization post [4]. The agent uses code generation to interact with tools instead of carrying their full type definitions.

Transport: Streamable HTTP

FastMCP defaults to stdio transport for local development. For production you need Streamable HTTP [2]:

if __name__ == "__main__":
    mcp.run(
        transport="streamable-http",
        host="0.0.0.0",
        port=8000,
    )

The old SSE transport was deprecated in the June 2025 MCP spec revision. Streamable HTTP supports bidirectional streaming through load balancers, reverse proxies, and CDNs — essential for any multi-client deployment [2].

Docker deployment:

FROM python:3.12-slim
WORKDIR /app
RUN pip install fastmcp httpx
COPY gateway.py .
EXPOSE 8000
CMD ["fastmcp", "run", "gateway.py", "--transport", "streamable-http", "--host", "0.0.0.0", "--code-mode"]

OAuth Integration

Authentication is the #1 pain point for production MCP deployments [3]. FastMCP 3.0 added built-in OAuth proxy support that validates tokens against any OpenID Connect provider [1]:

from fastmcp.auth import OAuthProxy

mcp = FastMCP("tool-gateway")
mcp.configure_auth(
    provider=OAuthProxy(
        issuer=os.environ["OIDC_ISSUER_URL"],
        client_id=os.environ["MCP_CLIENT_ID"],
        client_secret=os.environ["MCP_CLIENT_SECRET"],
    )
)

The OAuth proxy pattern works with Keycloak, Okta, Azure AD, and any OIDC-compliant IdP. The gateway validates every incoming request’s bearer token against the IdP, then forwards the caller identity to backend servers via custom headers [1].

Why this matters: Without auth, any agent or client that can reach your MCP endpoint can call every tool. In production, you need per-client identity, audit trails, and rate limits per team.

Lessons Learned

1. The toolbox problem is real

The gateway’s curated tool list matters more than which framework you use. A 30-tool server where each tool has a 200-token description and 500-token JSON schema burns 21,000 tokens before a single call. Consolidating intent-oriented tools (e.g., one query_database instead of search_users, search_orders, search_products) saved 12,000 tokens in our deployment [3].

2. You will outgrow a single server

At ~50 tools, schema overhead starts to hurt even with code mode. The natural next step is to split the gateway into domain-specific sub-gateways (e.g., data-gateway, operations-gateway, analytics-gateway) behind a top-level router — the SwarmRouter pattern from the orchestration world [5].

3. Fail fast, fail visibly

Startup validation of environment variables caught three production incidents before they reached users. Every missing API key crashes at deployment, not during a tool call. This pattern is absent from 90% of MCP tutorials but is table-stakes for production [2].

4. Code mode isn’t always the answer

Code mode drops tokens by treating tools as an importable library. But it requires the agent to generate valid code against the tool API — which means more tokens per invocation even though it saves tokens per session. Measure both. For servers with <15 tools used frequently, code mode may not be worth the overhead.

Build an MCP Server That Cuts Claude Code Context Consumption by 98% — Code-execution MCP servers with similar token-saving techniques [1]
Swarms: Enterprise-Grade Multi-Agent Orchestration Framework Deep Dive — The SwarmRouter pattern for multi-server composition
MCP Integration Patterns in 2026 — Broader MCP ecosystem patterns and production architectures
Building Production-Grade AI Agents with MCP: A Complete Guide — Community reference on MCP production patterns

Sources: [1] Apigene Blog — FastMCP 3.0: Build MCP Servers in Python, Fast (April 2026), [2] Anthropic — Donating MCP to Linux Foundation’s Agentic AI Foundation (Dec 2025), [3] Apigene Blog — How to Build, Deploy, and Scale a Python MCP Server (2026), [4] NiteAgent — Build an MCP Server That Cuts Claude Code Context Consumption by 98%, [5] NiteAgent — Swarms: Enterprise-Grade Multi-Agent Orchestration Framework Deep Dive

ToolBrain — tool reviews, LLM comparisons, and AI workflow guides
CodeIntel Log — code quality, debugging, and software engineering benchmarks
Hermes Tutorials — Hermes Agent setup, configuration, and advanced workflows

Cross-links automatically generated from NiteAgent.

References

[1] Apigene Blog, “FastMCP 3.0: Build MCP Servers in Python” — https://apigene.ai/blog/fastmcp-3-0-build-mcp-servers-in-python-fast
[2] Anthropic, “Donating MCP to Linux Foundation” — https://www.anthropic.com/news/mcp-linux-foundation
[3] Apigene Blog, “How to Build, Deploy, and Scale a Python MCP Server” — https://apigene.ai/blog/
[4] NiteAgent, “Build an MCP Server That Cuts Claude Code Context Consumption” — https://niteagent.com/blog/build-mcp-server-context-optimization
[5] MCP Specification — https://modelcontextprotocol.io/

← Back to all posts

Building a Production MCP Tool Gateway with FastMCP 3.x — A Build Log