MCP Runtime and Deployment

MCP Runtime and Deployment

The MCP specification standardizes how clients and servers talk. The runtime layer is a separate concern: how a server actually runs — on a laptop, a LAN, or production cloud — and stays available, secure, and observable while it does. Treat every MCP server as a microservice, and the operational playbook becomes familiar.

For the protocol roles and primitives assumed here, see MCP Foundations and Architecture.

The runtime, part by part

An MCP server listens for JSON-RPC requests, does work against a tool or resource, and returns a structured response. In production that breaks into five moving parts:

Component Job Typical implementation
Transport Moves data between client and server. stdio (local) · HTTP/Streamable HTTP (remote)
Execution core Hosts the tool logic and resource handlers. FastMCP, Node SDK, PHP SDK
Auth middleware Verifies connections and scopes. OAuth 2.1, or a local token map
Lifecycle manager Startup, shutdown, restarts, timeouts. Docker, PM2, Kubernetes
Instrumentation Metrics, logs, and traces. OpenTelemetry, Prometheus, Datadog

Runtime modes

Local (stdio). The server runs as a subprocess (npx, uvx, python, php) and exchanges JSON-RPC frames over standard I/O. Ideal for development and trusted desktop agents — zero network latency, easy to debug — but single-user, with no horizontal scaling.

{
  "mcpServers": {
    "chrome-devtools": {
      "command": "npx",
      "args": ["chrome-devtools-mcp@latest"]
    }
  }
}

Remote (HTTP / Streamable HTTP). The server exposes a persistent /mcp endpoint reachable by remote agents, with OAuth 2.1 and multi-tenant session handling. This is the path to scale, at the cost of a real attack surface: one exposed endpoint can leak every tool behind it.

Hybrid / on-prem. Trusted local I/O for sensitive data sources, remote HTTP for public APIs. Common in regulated settings where governance demands on-prem processing but agents still need cloud services.

Transports and networking

Transport Use case Security
stdio Local development, IDEs (Cursor, VS Code) OS trust model
Streamable HTTP Production remote agents TLS + OAuth 2.1 (required)
Legacy SSE Deprecated by the 2025 spec — migrate to Streamable HTTP Limited

Networking notes:

  • Avoid reverse-proxy compression that breaks chunked streams.
  • Use keep-alive for long-running tool calls.
  • Watch throughput and throttle as token limits approach.

Deployment strategies

  • Single host (prototype). Launch with npx <package> or python server.py; register in the client’s mcpServers config.
  • Containerized (Docker). The reproducible baseline for multi-agent access:

dockerfile
FROM python:3.12-slim
WORKDIR /app
COPY . .
RUN pip install -r requirements.txt
EXPOSE 8080
CMD ["fastmcp", "serve", "--port", "8080"]

Orchestrate with Docker Compose or Kubernetes.
– Cloud-native. Serverless containers (Fargate, ACI) for ephemeral tools; API Gateway plus functions for low-volume ad-hoc calls; a WAF in front for transport security.
– Edge / on-prem. For regulated industries, keep servers behind a VPC or VPN with scoped tokens. Air-gapped deployments run stdio only.

Lifecycle and configuration

Task Approach
Startup / shutdown Process manager (PM2) or container health checks
Auto-scaling Kubernetes HPA on latency or invocation metrics
Graceful termination Handle SIGTERM; send a close message before exit

Common environment variables: MCP_PORT (default 8080), OAUTH_CLIENT_ID / OAUTH_CLIENT_SECRET / OAUTH_REDIRECT_URI, LOG_LEVEL, and CACHE_TTL.

Scaling and load

Strategy What it does
Tool budgeting Rate-limit per tool or per client to cap runaway cost.
Request batching Aggregate tools/call requests where the client supports it.
State caching Cache read-only resource responses (in memory or Redis).
Horizontal scaling Stateless containers with sticky sessions for long reads.
Metric-driven scaling Scale on latency or invocation count, not generic CPU.

Observability

Instrumentation is what makes AI–tool interactions debuggable.

Signal Implementation Example metrics
Logs Structured JSON with trace ID, tool name, latency. Tool error rate, response size
Metrics Prometheus or Datadog exporters. Requests/sec, active sessions, success ratio
Traces OpenTelemetry spans client → server → db/api. End-to-end latency breakdown
Evals Synthetic tests of tools and prompts. Failure ratio, replay divergence

Centralize logs with session_id, tool_name, and status — and never log inputs that carry PII.

Runtime security checklist

  1. OAuth 2.1 + PKCE for remote HTTP.
  2. Validate the origin header against an allow-list.
  3. Rotate tokens on a short cycle.
  4. Rate-limit external calls per client.
  5. Sandbox tool execution that touches a shell or SQL.
  6. Enable audit logging with trace IDs.
  7. Never return raw stack traces to the model host.
  8. Encrypt temporary caches.

These map to the broader controls in MCP Security and Compliance.

Example topologies

Production cloud, multiple servers:

   AI Host (chat apps / IDEs)
            │  OAuth 2.1 token
            ▼
   MCP Client / Router
      │            │
  ┌───┴───┐    ┌───┴────┐
  │Server A│   │Server B │
  │  (DB)  │   │(Browser)│
  └───┬───┘    └───┬────┘
      │  Streamable HTTP
      ▼
   External APIs / DB / FS

Local hybrid (laptop + LAN):

Host (desktop app)
  └── Client (stdio) ──► Local Server (FastMCP)
                              └──► DB / files on localhost

Resilience and maintenance

Area Practice
Backups Export tool/resource catalogs and schemas with checksums.
Zero-downtime updates Blue-green deployments.
Failover Standby node on a secondary port.
Schema validation CI job runs tools/list and resources/list against the spec.
Version pinning Tag images with the protocol revision (e.g. 2025-06-18).

Performance levers worth applying: reuse long-lived sessions to amortize setup, compress HTTP bodies (not chunked streams), return URIs instead of large payloads, and pre-index frequently read resources.

Takeaways

  1. Runtime choices, not the spec, decide real-world stability — stdio for development, Streamable HTTP for scale.
  2. Containerize and apply standard security controls for any multi-user deployment.
  3. Observability (logs, metrics, traces) is as load-bearing as the tools themselves.
  4. Scale statelessly and cache read-only resources.
  5. Treat every server as a versioned, auditable microservice.
This entry was posted in . Bookmark the permalink.
Adam Bernard