Lesson 32 of 55: OAuth 2.0 and PKCE for Remote MCP Servers

Remote MCP servers exposed over HTTP need authentication. The MCP specification recommends OAuth 2.0 with PKCE for browser-based and CLI clients. This lesson covers the complete OAuth 2.0 flow for MCP: the authorization server setup, the protected resource server, the client-side PKCE dance, and the token refresh lifecycle. When you finish this lesson your MCP server will reject unauthenticated connections and correctly scope what each authenticated client can access.

OAuth 2.0 PKCE flow diagram for MCP server authentication showing authorization code flow with client tokens dark
MCP over HTTP uses OAuth 2.0 Authorization Code + PKCE: no client secrets, no password flow.

Why OAuth 2.0 for MCP

MCP servers are effectively APIs. They expose tools, resources, and prompts that can access sensitive data, execute code, or modify state. Without authentication, any client that knows the server URL can use those capabilities. OAuth 2.0 provides:

  • Authentication: Only clients that obtain a valid token can connect
  • Authorization: Tokens can carry scopes that limit which tools and resources a client can access
  • Delegation: A human user can authorize a client to act on their behalf without sharing passwords
  • Revocation: Access can be revoked immediately by invalidating the token

In practice, every MCP server you expose over HTTP is an unauthenticated attack surface until you layer on OAuth. Even internal servers benefit from token-based auth, because lateral movement between compromised services is one of the most common patterns in real-world breaches.

The MCP OAuth Flow

The flow follows OAuth 2.0 Authorization Code + PKCE (RFC 7636):

// Step 1: Client generates PKCE code verifier and challenge
import crypto from 'node:crypto';

function generatePkce() {
  const verifier = crypto.randomBytes(32).toString('base64url');
  const challenge = crypto.createHash('sha256').update(verifier).digest('base64url');
  return { verifier, challenge };
}

// Step 2: Client redirects user to authorization URL
function buildAuthUrl(config, pkce, state) {
  const params = new URLSearchParams({
    response_type: 'code',
    client_id: config.clientId,
    redirect_uri: config.redirectUri,
    scope: config.scopes.join(' '),
    state,
    code_challenge: pkce.challenge,
    code_challenge_method: 'S256',
  });
  return `${config.authorizationEndpoint}?${params}`;
}

// Step 3: User authorizes, gets redirected back with code
// Step 4: Client exchanges code for tokens
async function exchangeCode(config, code, pkce) {
  const response = await fetch(config.tokenEndpoint, {
    method: 'POST',
    headers: { 'Content-Type': 'application/x-www-form-urlencoded' },
    body: new URLSearchParams({
      grant_type: 'authorization_code',
      client_id: config.clientId,
      redirect_uri: config.redirectUri,
      code,
      code_verifier: pkce.verifier,
    }),
  });
  if (!response.ok) throw new Error(`Token exchange failed: ${response.status}`);
  return response.json();
}

This matters because MCP clients are often CLI tools or desktop apps that cannot safely store a client secret. PKCE lets these “public clients” prove they initiated the authorization request without holding a long-lived credential. Without PKCE, an attacker who intercepts the authorization code could exchange it for tokens before your client does.

PKCE code verifier challenge generation flow SHA256 hashing base64url encoding diagram dark security
PKCE prevents authorization code interception: the challenge proves ownership without a client secret.

Protecting an MCP Server with Bearer Tokens

import express from 'express';
import { McpServer } from '@modelcontextprotocol/sdk/server/mcp.js';
import { StreamableHTTPServerTransport } from '@modelcontextprotocol/sdk/server/streamable-http.js';

const app = express();

// Token validation middleware
async function requireAuth(req, res, next) {
  const authHeader = req.headers.authorization;
  if (!authHeader?.startsWith('Bearer ')) {
    return res.status(401).json({ error: 'unauthorized', error_description: 'Bearer token required' });
  }
  const token = authHeader.slice(7);
  try {
    // Validate with your auth server (introspection endpoint, or local JWT verification)
    const claims = await validateToken(token);
    req.auth = claims;  // { sub, scope, exp }
    next();
  } catch {
    res.status(401).json({ error: 'invalid_token', error_description: 'Token is invalid or expired' });
  }
}

// Apply auth to the MCP endpoint
app.use('/mcp', requireAuth);

app.post('/mcp', async (req, res) => {
  const transport = new StreamableHTTPServerTransport({ sessionIdGenerator: () => crypto.randomUUID() });
  const server = buildMcpServer(req.auth);  // Pass auth claims to server for per-user tool filtering
  await server.connect(transport);
  await transport.handleRequest(req, res, req.body);
});

The middleware above delegates validation to a validateToken function, which could call a remote introspection endpoint or verify a JWT locally. For most MCP deployments, local JWT verification is faster and avoids a network round-trip on every request. The next section shows how to do that with the jose library.

JWT Validation (Self-Contained Tokens)

import { createRemoteJWKSet, jwtVerify } from 'jose';

// Cache the JWKS (JSON Web Key Set) fetched from your auth server
const JWKS = createRemoteJWKSet(new URL('https://auth.yourcompany.com/.well-known/jwks.json'));

async function validateToken(token) {
  const { payload } = await jwtVerify(token, JWKS, {
    issuer: 'https://auth.yourcompany.com',
    audience: 'mcp-server',
  });
  return payload;
}

Token Refresh Lifecycle in MCP Clients

class TokenManager {
  #accessToken = null;
  #refreshToken = null;
  #expiresAt = 0;

  setTokens({ access_token, refresh_token, expires_in }) {
    this.#accessToken = access_token;
    this.#refreshToken = refresh_token;
    this.#expiresAt = Date.now() + (expires_in - 60) * 1000;  // 60s buffer
  }

  async getAccessToken(config) {
    if (Date.now() < this.#expiresAt) return this.#accessToken;
    if (!this.#refreshToken) throw new Error('Session expired - re-authentication required');
    
    const response = await fetch(config.tokenEndpoint, {
      method: 'POST',
      headers: { 'Content-Type': 'application/x-www-form-urlencoded' },
      body: new URLSearchParams({
        grant_type: 'refresh_token',
        client_id: config.clientId,
        refresh_token: this.#refreshToken,
      }),
    });
    if (!response.ok) throw new Error('Token refresh failed');
    this.setTokens(await response.json());
    return this.#accessToken;
  }
}

// Use it in the MCP transport
const tokenManager = new TokenManager();
const transport = new StreamableHTTPClientTransport(new URL(MCP_SERVER_URL), {
  requestInit: async () => ({
    headers: { Authorization: `Bearer ${await tokenManager.getAccessToken(oauthConfig)}` },
  }),
});

A subtle pitfall here: if the refresh token itself has expired or been revoked, the getAccessToken call will fail with no way to recover except re-authenticating the user. In long-running MCP clients like IDE extensions, you should catch this failure and prompt the user to re-authorize rather than silently failing all subsequent tool calls.

Using an Existing Auth Provider

For production, use an existing OAuth 2.0 provider rather than building your own authorization server:

  • Auth0: Managed OAuth + JWKS endpoint, simple Node.js SDK
  • Google OAuth 2.0: For Google Workspace integrations
  • GitHub OAuth: For developer-facing MCP tools
  • Keycloak: Self-hosted, enterprise IAM with fine-grained authorization

Whichever provider you choose, the MCP server side stays the same: validate the Bearer token, extract claims, and pass them into the server builder. The provider handles user management, consent screens, and token issuance so you can focus on MCP-specific authorization logic.

Client ID Metadata Documents (CIMD)

New in 2025-11-25

Dynamic Client Registration (DCR) requires every client to register with every authorization server it connects to. This creates friction: the client must exchange a registration request, store a per-server client_id, and handle registration failures. Client ID Metadata Documents (CIMD) replace DCR for most use cases by letting the client publish a metadata document at a well-known URL and use that URL as its client_id.

The flow works like this: the client chooses a URL it controls (e.g. https://my-mcp-client.example.com/.well-known/oauth-client) and serves a JSON document at that URL describing itself. When the client sends an authorization request to any MCP server, it uses the URL as its client_id. The authorization server fetches the metadata document, verifies it, and proceeds with the OAuth flow. No per-server registration step is needed.

// Client ID Metadata Document served at the client_id URL
// GET https://my-mcp-client.example.com/.well-known/oauth-client
{
  "client_id": "https://my-mcp-client.example.com/.well-known/oauth-client",
  "client_name": "My MCP Desktop Client",
  "redirect_uris": ["http://127.0.0.1:9876/callback"],
  "grant_types": ["authorization_code"],
  "response_types": ["code"],
  "token_endpoint_auth_method": "none",
  "scope": "mcp:tools mcp:resources"
}

CIMD is the recommended registration mechanism for public clients (desktop apps, CLI tools, browser extensions) that cannot securely store a client secret. For confidential server-to-server clients, traditional DCR or pre-registered credentials remain appropriate.

Incremental Scope Consent

New in 2025-11-25

When a client's current token does not have sufficient scope for a requested operation, the server can signal this via a WWW-Authenticate header with a scope parameter indicating the additional scopes needed. The client can then request the user's consent for just the additional scopes, rather than re-authorizing all scopes from scratch.

// Server responds 403 with WWW-Authenticate indicating needed scope
// HTTP/1.1 403 Forbidden
// WWW-Authenticate: Bearer scope="mcp:admin:delete"

// Client: request incremental consent for the new scope
const additionalScopes = parseWWWAuthenticate(response.headers['www-authenticate']);
const newToken = await requestIncrementalConsent(additionalScopes);
// Retry the request with the upgraded token

This is important for progressive authorization: start a session with minimal scopes (read tools, list resources), then ask for elevated scopes (write, delete, admin) only when the user actually tries to do something that needs them. It reduces the initial consent burden and follows the principle of least privilege.

Common Authentication Failures

  • Returning 403 instead of 401: 401 means "not authenticated" (present credentials), 403 means "authenticated but not authorized" (wrong scope). Use the right code or clients will not know to re-authenticate.
  • Not validating the audience claim: A token issued for your user service should not work on your MCP server. Always validate aud matches your server's identifier.
  • Not handling token expiry during long tool calls: An MCP tool that takes 5 minutes to execute may outlive a short-lived access token. Use the token manager pattern with a generous buffer.
  • Logging tokens: Never log full tokens in application logs. Log the token's sub (subject) and jti (token ID) instead for traceability without exposure.

What to Build Next

  • Add Bearer token validation to your existing Streamable HTTP MCP server. Test it with both valid and expired tokens.
  • Implement a simple OAuth client using PKCE that stores tokens in a local file and refreshes them automatically.

nJoy πŸ˜‰

Lesson 31 of 55: Choosing the Right LLM for MCP Applications (Cost and Quality)

Every team building MCP applications eventually faces the same question: which model should we use for this task? The wrong answer is “the most capable one” — that is how teams burn through their budget on GPT-6 Astra for queries that GPT-6 Luna could answer just as well. This lesson builds a systematic decision framework: a set of questions and criteria that map task characteristics to optimal model choices, plus the infrastructure to implement dynamic routing in production.

Decision framework flowchart for selecting between OpenAI Claude Gemini models based on task type dark
Model selection is a routing problem: match task characteristics to the cheapest model that meets quality requirements.

The Five Dimensions of Model Selection

Five dimensions determine the optimal model choice for an MCP task:

  1. Reasoning depth: Is the task multi-step, requires planning, or involves complex logic? Use Claude Opus 5.5 or GPT-6 Astra. Is it a simple lookup or classification? Use a mini/flash model.
  2. Context length: Does the task involve large documents, entire codebases, or long conversation history? GPT-6 Astra, Claude Opus 5.5, Gemini 4 Argon, and Gemini 3.8 Flash all offer about a 1M token window as of 20 October 2026. For standard tasks, 128K is still enough.
  3. Input modality: Does the task involve images, PDFs, or audio? Use Gemini (strongest multimodal support). Text only – any provider works.
  4. Output format: Does the task require guaranteed JSON schema output? Use OpenAI with zodResponseFormat. Free-form prose or code? Any provider.
  5. Volume and cost: Is this a high-throughput task called thousands of times per hour? Use GPT-6 Luna ($0.10/1M input) or Gemini 3.8 Flash ($0.75/1M input through 31 December 2026) before considering more expensive models.

These five dimensions are not equally weighted for every application. A customer-facing chatbot cares most about latency and cost. An internal compliance tool cares most about reasoning depth and output format. Before writing routing rules, rank these dimensions for your specific use case.

The Decision Framework

// Task routing decision table
// Use this as a starting point for your routing config

const ROUTING_RULES = [
  // Rule order matters - first match wins
  {
    name: 'multimodal',
    condition: (task) => task.hasImages || task.hasPDF || task.hasAudio,
    provider: 'gemini', model: 'gemini-3.8-flash',
    reason: 'Native multimodal support, cheapest multimodal option',
  },
  {
    name: 'large-context',
    condition: (task) => task.estimatedInputTokens > 100_000,
    provider: 'gemini', model: 'gemini-4-argon',
    reason: '1M token context window, best for whole-document/codebase analysis',
  },
  {
    name: 'deep-reasoning',
    condition: (task) => task.requiresPlanning || task.complexity === 'high',
    provider: 'claude', model: 'claude-opus-5-5',
    reason: 'Extended thinking mode, best instruction following',
  },
  {
    name: 'structured-output',
    condition: (task) => task.requiresStrictJSON,
    provider: 'openai', model: 'gpt-6-astra',
    reason: 'zodResponseFormat guarantees JSON schema adherence',
  },
  {
    name: 'high-volume-simple',
    condition: (task) => task.volume > 1000 && task.complexity === 'low',
    provider: 'gemini', model: 'gemini-3.8-flash',
    reason: 'Cheapest per-token, sufficient for simple tasks at scale',
  },
  {
    name: 'default',
    condition: () => true,
    provider: 'openai', model: 'gpt-6-luna',
    reason: 'Lowest OpenAI price in the current lineup for general tasks',
  },
];

export function selectModel(task) {
  const rule = ROUTING_RULES.find(r => r.condition(task));
  return { provider: rule.provider, model: rule.model, reason: rule.reason };
}

Rule order is critical in this table: the first matching rule wins. If you put the default rule at the top, every request would route to GPT-6 Luna regardless of complexity. When debugging unexpected routing, check rule ordering before anything else.

Model selection matrix showing task complexity vs cost tradeoff with provider recommendations in each quadrant dark
The cost-capability matrix: route high-volume simple tasks to cheap models and complex reasoning to capable models.

Estimating Task Complexity

// Simple heuristics for runtime complexity estimation
export function classifyTask(userMessage, context = {}) {
  const words = userMessage.split(/\s+/).length;
  const hasAnalyze = /analyz|evaluate|compare|assess|plan|strategy/i.test(userMessage);
  const hasSimple = /list|find|get|show|what is|how many/i.test(userMessage);

  return {
    complexity: hasAnalyze ? 'high' : (hasSimple ? 'low' : 'medium'),
    estimatedInputTokens: Math.ceil(words * 1.3) + (context.historyTokens ?? 0),
    hasImages: context.hasImages ?? false,
    hasPDF: context.hasPDF ?? false,
    hasAudio: context.hasAudio ?? false,
    requiresStrictJSON: context.requiresStrictJSON ?? false,
    requiresPlanning: hasAnalyze,
    volume: context.requestsPerHour ?? 0,
  };
}

These heuristics are a starting point, not a final solution. Keyword matching will misclassify some tasks – a user asking “analyze this simple list” triggers the complexity flag unnecessarily. Over time, replace these rules with a lightweight classifier trained on your actual query logs and quality ratings.

Cascading Fallback Strategy

// Try primary, fall back on quota or severe errors
export async function runWithFallback(task, providers) {
  const { provider: primaryKey, model } = selectModel(task);
  const fallbackKey = primaryKey === 'gemini' ? 'openai' : 'gemini';

  for (const key of [primaryKey, fallbackKey]) {
    const provider = providers[key];
    if (!provider) continue;
    try {
      return await provider.run(task.message, task.mcpClient);
    } catch (err) {
      const isQuota = err.status === 429 || err.message?.includes('RESOURCE_EXHAUSTED');
      if (!isQuota) throw err;
      console.error(`[router] ${key} quota hit, trying fallback`);
    }
  }
  throw new Error('All providers exhausted');
}

Fallback routing adds resilience but also introduces behavioral inconsistency. If your primary provider is Claude (optimized for reasoning) and your fallback is Gemini (optimized for speed), the quality of responses will shift when fallback activates. Log which provider handled each request so you can detect when fallback is firing too often.

Building a Cost Dashboard

// Track cost per provider, per task type, per hour
class CostTracker {
  #records = [];

  record({ provider, model, inputTokens, outputTokens, taskType }) {
    const costs = {
      // Checked 20 October 2026. Haiku is "from" these rates. Gemini rates are introductory.
      'gpt-6-astra': { input: 10, output: 50 },
      'gpt-6.1-sol': { input: 2, output: 10 },
      'gpt-6-luna': { input: 0.10, output: 0.50 },
      'claude-opus-5-5': { input: 4, output: 20 },
      'claude-sonnet-5-5': { input: 2, output: 10 },
      'claude-haiku-5-5': { input: 0.10, output: 0.50 },
      'gemini-3.8-flash': { input: 0.75, output: 3.75 },
      'gemini-4-argon': { input: 2, output: 10 },
    };
    const c = costs[model] ?? { input: 0, output: 0 };
    const cost = (inputTokens * c.input + outputTokens * c.output) / 1_000_000;
    this.#records.push({ provider, model, taskType, cost, ts: Date.now() });
  }

  summary() {
    return this.#records.reduce((acc, r) => {
      const key = `${r.provider}/${r.model}`;
      acc[key] = (acc[key] ?? 0) + r.cost;
      return acc;
    }, {});
  }
}

Even a simple cost tracker like this one reveals patterns that are invisible without data. You might discover that 80% of your spend comes from 5% of your queries, or that a particular task type routes to an expensive model when a cheaper one would suffice. Data-driven routing decisions consistently outperform intuition.

Common Routing Mistakes

  • Always routing to the most capable model: GPT-6 Astra input is $10 per million tokens. GPT-6 Luna input is $0.10. That is 100 times the input price, on the rates published 20 October 2026. Benchmark first, then route based on evidence.
  • Not accounting for caching: OpenAI’s automatic caching and Claude’s explicit cache_control can change the effective cost dramatically for repeated queries with the same prefix. Factor this into your cost model.
  • Routing on task type without measuring quality: A routing decision is only valid if you have measured that the cheaper model produces acceptable results for the task type. Build eval sets per task type and validate routing assumptions.
  • Ignoring latency: Cost is not the only dimension. GPT-6 Luna has much lower latency than GPT-6 Astra. Gemini 3.8 Flash is faster still. For user-facing real-time features, latency matters as much as cost.

What to Build Next

  • Run 20 real queries from your application through the framework above. Log provider, model, task complexity, cost, and a quality score (manual review). Use this data to refine the routing rules.
  • Set up a cost alert: if hourly spend exceeds a threshold, log a warning and automatically down-route to cheaper models.

nJoy πŸ˜‰

Lesson 30 of 55: Multi-Provider MCP Client Library in Node.js

The previous lesson established the differences between OpenAI, Claude, and Gemini. This lesson turns those differences into a Node.js abstraction layer that makes the provider transparent to the rest of your application. You write tool logic once, define a routing policy, and the layer handles schema conversion, message format, retry, and result normalization automatically. This is the architecture that makes multi-provider MCP applications maintainable at scale.

Multi-provider MCP abstraction layer diagram showing unified interface routing to OpenAI Claude Gemini dark
A provider abstraction layer routes MCP tool-calling requests to the appropriate LLM without changing application code.

The Core Interface

Define a common interface first. Every provider adapter must implement run(messages, tools) and return a normalized result:

// lib/providers/base.js

/**
 * @typedef {Object} ProviderResult
 * @property {string} text - The model's final text response
 * @property {number} inputTokens - Tokens consumed in input
 * @property {number} outputTokens - Tokens consumed in output
 * @property {number} turns - Number of tool-calling turns
 */

/**
 * Base class for LLM provider adapters.
 * Subclasses implement #callModel(messages, tools) and #extractToolCalls(response).
 */
export class BaseProvider {
  constructor(config = {}) {
    this.config = {
      maxTurns: config.maxTurns ?? 10,
      maxRetries: config.maxRetries ?? 3,
      ...config,
    };
  }

  /**
   * Run a complete tool-calling loop.
   * @param {string} userMessage
   * @param {import('@modelcontextprotocol/sdk').Client} mcpClient
   * @returns {Promise<ProviderResult>}
   */
  async run(userMessage, mcpClient) {
    const { tools: mcpTools } = await mcpClient.listTools();
    const providerTools = this.convertTools(mcpTools);
    return this._runLoop(userMessage, mcpClient, providerTools);
  }

  // Subclasses override these
  convertTools(mcpTools) { throw new Error('Not implemented'); }
  async callModel(messages, tools) { throw new Error('Not implemented'); }
  extractToolCalls(response) { throw new Error('Not implemented'); }
  extractText(response) { throw new Error('Not implemented'); }
  extractUsage(response) { return { inputTokens: 0, outputTokens: 0 }; }
  buildToolResultMessage(toolCallId, name, result) { throw new Error('Not implemented'); }
}

This base class defines the contract that every provider adapter must honor. By making convertTools, callModel, and extractToolCalls abstract methods, you guarantee that adding a new provider (like Mistral or Cohere) requires implementing a fixed set of behaviors rather than threading new logic through your entire application.

OpenAI Adapter

// lib/providers/openai.js
import OpenAI from 'openai';
import { BaseProvider } from './base.js';

export class OpenAIProvider extends BaseProvider {
  #client;

  constructor(config = {}) {
    super(config);
    this.#client = new OpenAI();
    this.model = config.model ?? 'gpt-6-astra';
  }

  convertTools(mcpTools) {
    return mcpTools.map(t => ({
      type: 'function',
      function: { name: t.name, description: t.description, parameters: t.inputSchema, strict: true },
    }));
  }

  async callModel(messages, tools) {
    return this.#client.chat.completions.create({
      model: this.model, messages, tools, tool_choice: 'auto',
    });
  }

  extractToolCalls(response) {
    const msg = response.choices[0].message;
    if (msg.finish_reason !== 'tool_calls') return [];
    return msg.tool_calls.map(tc => ({
      id: tc.id, name: tc.function.name,
      args: JSON.parse(tc.function.arguments),
    }));
  }

  extractText(response) {
    return response.choices[0].message.content ?? '';
  }

  extractUsage(response) {
    return { inputTokens: response.usage.prompt_tokens, outputTokens: response.usage.completion_tokens };
  }

  buildAssistantMessage(response) {
    return response.choices[0].message;
  }

  buildToolResultMessage(toolCallId, name, result) {
    return { role: 'tool', tool_call_id: toolCallId, content: result };
  }

  async _runLoop(userMessage, mcpClient, tools) {
    const messages = [{ role: 'user', content: userMessage }];
    let totalInput = 0, totalOutput = 0, turns = 0;

    while (true) {
      const response = await this.callModel(messages, tools);
      const usage = this.extractUsage(response);
      totalInput += usage.inputTokens; totalOutput += usage.outputTokens;

      const toolCalls = this.extractToolCalls(response);
      if (toolCalls.length === 0) {
        return { text: this.extractText(response), inputTokens: totalInput, outputTokens: totalOutput, turns };
      }

      if (++turns > this.config.maxTurns) throw new Error('Max turns exceeded');
      messages.push(this.buildAssistantMessage(response));

      const results = await Promise.all(toolCalls.map(async tc => {
        const result = await mcpClient.callTool({ name: tc.name, arguments: tc.args });
        const text = result.content.filter(c => c.type === 'text').map(c => c.text).join('\n');
        return this.buildToolResultMessage(tc.id, tc.name, text);
      }));
      messages.push(...results);
    }
  }
}
Three adapter classes OpenAI Claude Gemini extending BaseProvider interface with convertTools callModel extractToolCalls dark code diagram
Each adapter implements the same BaseProvider interface, hiding provider-specific message formats.

The OpenAI adapter is the most verbose of the three because OpenAI’s message format requires the most wrapping: tool calls live in tool_calls arrays, arguments arrive as JSON strings that need parsing, and results go back as separate role: 'tool' messages. The adapter absorbs all of this complexity so your application code stays clean.

Claude Adapter

// lib/providers/claude.js
import Anthropic from '@anthropic-ai/sdk';
import { BaseProvider } from './base.js';

export class ClaudeProvider extends BaseProvider {
  #client;

  constructor(config = {}) {
    super(config);
    this.#client = new Anthropic();
    this.model = config.model ?? 'claude-sonnet-5-5';
  }

  convertTools(mcpTools) {
    return mcpTools.map(t => ({ name: t.name, description: t.description, input_schema: t.inputSchema }));
  }

  async callModel(messages, tools) {
    return this.#client.messages.create({
      model: this.model, max_tokens: 4096, messages, tools,
    });
  }

  extractToolCalls(response) {
    if (response.stop_reason !== 'tool_use') return [];
    return response.content
      .filter(b => b.type === 'tool_use')
      .map(b => ({ id: b.id, name: b.name, args: b.input }));
  }

  extractText(response) {
    return response.content.filter(b => b.type === 'text').map(b => b.text).join('');
  }

  extractUsage(response) {
    return { inputTokens: response.usage.input_tokens, outputTokens: response.usage.output_tokens };
  }

  async _runLoop(userMessage, mcpClient, tools) {
    const messages = [{ role: 'user', content: userMessage }];
    let totalInput = 0, totalOutput = 0, turns = 0;

    while (true) {
      const response = await this.callModel(messages, tools);
      const usage = this.extractUsage(response);
      totalInput += usage.inputTokens; totalOutput += usage.outputTokens;

      const toolCalls = this.extractToolCalls(response);
      if (toolCalls.length === 0) {
        return { text: this.extractText(response), inputTokens: totalInput, outputTokens: totalOutput, turns };
      }

      if (++turns > this.config.maxTurns) throw new Error('Max turns exceeded');
      messages.push({ role: 'assistant', content: response.content });

      const results = await Promise.all(toolCalls.map(async tc => {
        const result = await mcpClient.callTool({ name: tc.name, arguments: tc.args });
        const text = result.content.filter(c => c.type === 'text').map(c => c.text).join('\n');
        return { type: 'tool_result', tool_use_id: tc.id, content: text };
      }));
      messages.push({ role: 'user', content: results });
    }
  }
}

Compare the Claude adapter’s _runLoop to the OpenAI version above. The structure is nearly identical, but the message format differs in subtle ways: tool results nest inside a user message as content blocks rather than standing alone as tool role messages. These small differences are exactly what the abstraction layer exists to hide.

The Provider Router

// lib/providers/router.js
import { OpenAIProvider } from './openai.js';
import { ClaudeProvider } from './claude.js';
import { GeminiProvider } from './gemini.js';

/**
 * Route a task to the appropriate provider based on task type.
 */
export class ProviderRouter {
  #providers;
  #defaultProvider;

  constructor(config = {}) {
    this.#providers = {
      openai: new OpenAIProvider(config.openai ?? {}),
      claude: new ClaudeProvider(config.claude ?? {}),
      gemini: new GeminiProvider(config.gemini ?? {}),
    };
    this.#defaultProvider = config.default ?? 'openai';
  }

  /**
   * Route based on task type.
   * @param {'reasoning' | 'multimodal' | 'highvolume' | 'default'} taskType
   */
  getProvider(taskType = 'default') {
    const routing = {
      reasoning: 'claude',      // Extended thinking, deep analysis
      multimodal: 'gemini',     // Images, PDFs, audio
      highvolume: 'gemini',     // Cheapest per-token option
      structured: 'openai',     // Strict JSON, Agents SDK
      default: this.#defaultProvider,
    };
    const key = routing[taskType] ?? this.#defaultProvider;
    return this.#providers[key];
  }

  async run(userMessage, mcpClient, taskType = 'default') {
    const provider = this.getProvider(taskType);
    return provider.run(userMessage, mcpClient);
  }
}

The router’s task-type mapping is deliberately simple. In production, you might extend it with quality scores from an eval harness, latency percentiles from your monitoring stack, or dynamic cost thresholds that shift routing as budgets tighten. The important thing is that routing logic lives in one place, not scattered across your codebase.

Using the Router

import { ProviderRouter } from './lib/providers/router.js';
import { Client } from '@modelcontextprotocol/sdk/client/index.js';
import { StdioClientTransport } from '@modelcontextprotocol/sdk/client/stdio.js';

const router = new ProviderRouter({
  default: 'openai',
  openai: { model: 'gpt-6-luna' },
  claude: { model: 'claude-opus-5-5' },
  gemini: { model: 'gemini-3.8-flash' },
});

const mcp = new Client({ name: 'multi-provider-host', version: '1.0.0' });
await mcp.connect(new StdioClientTransport({ command: 'node', args: ['./server.js'] }));

// Simple query -> cheapest OpenAI model
const r1 = await router.run('What products are low in stock?', mcp, 'default');

// Complex analysis -> Claude
const r2 = await router.run('Analyze our Q1 sales data and identify the top 3 growth opportunities', mcp, 'reasoning');

// Document analysis -> Gemini
const r3 = await router.run('Process the attached invoice PDF', mcp, 'multimodal');

console.log(r1.text);
console.log(`Tokens: ${r1.inputTokens} in / ${r1.outputTokens} out`);

await mcp.close();

Notice that the calling code never imports a provider SDK directly. It only knows about ProviderRouter and the MCP Client. This means you can swap providers, change models, or adjust routing rules without modifying any of your application’s business logic. That separation is what makes multi-provider systems maintainable over time.

Failure Modes in Multi-Provider Systems

  • Leaky abstractions: Avoid leaking provider-specific features (like OpenAI’s zodResponseFormat or Claude’s cache_control) through the abstraction layer. If you need them, expose them via provider-specific method extensions, not the base interface.
  • Tool schema compatibility: Not all JSON Schema features work equally across providers. Test your tool schemas against all target providers, especially nested objects, anyOf, and enum arrays.
  • Cost accounting per provider: Log result.inputTokens and result.outputTokens per provider and task type. Without this, you cannot measure whether your routing policy is saving money.

What to Build Next

  • Implement the Gemini adapter following the same pattern as the OpenAI and Claude adapters above.
  • Add a fallback option to the router: if the primary provider returns a 429, automatically retry on the fallback provider.

nJoy πŸ˜‰

Lesson 29 of 55: OpenAI vs Claude vs Gemini – MCP Tool Calling Compared

You have now built MCP integrations with all three major LLM providers. This lesson steps back and compares them head-to-head: the exact wire format differences, the tool schema quirks, the parallel calling behaviors, the error contracts, and the practical performance and cost characteristics. After this lesson you will know which provider to reach for first for a given task and how to justify that choice to a team.

Three provider logos OpenAI Claude Gemini side by side MCP tool calling comparison diagram dark
OpenAI, Claude, and Gemini each have a distinct tool-calling contract with MCP.

The Tool Schema: What Each Provider Expects

MCP tools expose a JSON Schema via inputSchema. The conversion to each provider’s format differs slightly:

Provider Tool format field Schema field name Extras required
OpenAI { type: 'function', function: { name, description, parameters } } parameters strict: true for deterministic calling
Claude { name, description, input_schema } input_schema None
Gemini { name, description, parameters } inside functionDeclarations parameters Must handle null parameters (no-arg tools)
// Unified converter: MCP tool -> provider-specific format
export function convertMcpTool(tool, provider) {
  switch (provider) {
    case 'openai':
      return {
        type: 'function',
        function: {
          name: tool.name,
          description: tool.description,
          parameters: tool.inputSchema,
          strict: true,
        },
      };
    case 'claude':
      return {
        name: tool.name,
        description: tool.description,
        input_schema: tool.inputSchema,
      };
    case 'gemini':
      return {
        name: tool.name,
        description: tool.description,
        parameters: tool.inputSchema ?? { type: 'object', properties: {} },
      };
    default:
      throw new Error(`Unknown provider: ${provider}`);
  }
}

This converter function is the single most reusable piece of code in a multi-provider MCP application. Build it once, test it against all three providers, and every tool you add to your MCP server will automatically work across all of them without per-provider adjustments.

The Tool Result: How Each Provider Expects It Back

// OpenAI: tool result goes in a new message with role 'tool'
// messages.push({ role: 'tool', tool_call_id: call.id, content: resultText });

// Claude: tool results go in the user message as tool_result content blocks
// messages.push({ role: 'user', content: [{ type: 'tool_result', tool_use_id: block.id, content: resultText }] });

// Gemini: function responses go directly to chat.sendMessage() as an array
// await chat.sendMessage([{ functionResponse: { name: fc.name, response: { result: resultText } } }]);
Three conversation flow diagrams showing how OpenAI Claude Gemini each expect tool results in different message structures dark
The message structure for returning tool results differs significantly across providers.

The tool result differences are where most cross-provider bugs hide. OpenAI uses a dedicated tool role, Claude nests results inside a user message, and Gemini sends them as functionResponse parts. If you mix up these formats, the model will either error out or silently ignore the tool results.

Parallel Tool Calling Behavior

Provider Parallel calls How to detect How to return results
OpenAI Yes, opt-in via parallel_tool_calls: true message.tool_calls.length > 1 Multiple role=’tool’ messages, one per call
Claude Limited (beta flag required) Multiple tool_use blocks in content Multiple tool_result blocks in one user message
Gemini Yes, default behavior candidate.content.parts.filter(p => p.functionCall) Array of functionResponse parts in one message

Parallel tool calling is more than a performance optimization. It changes how the model reasons about your tools. When a model can issue three independent lookups at once, it structures its approach differently than when it must chain calls sequentially. This means switching providers can subtly change your agent’s behavior even with identical prompts.

Stop Condition Comparison

// OpenAI: loop while finish_reason is 'tool_calls'
while (response.choices[0].finish_reason === 'tool_calls') { ... }

// Claude: loop while stop_reason is 'tool_use'
while (response.stop_reason === 'tool_use') { ... }

// Gemini: loop while any content part is a functionCall
while (candidate.content.parts.some(p => p.functionCall)) { ... }

Error Handling Patterns

Error type OpenAI Claude Gemini
Rate limit 429, check retry-after 429, check retry-after 429 / RESOURCE_EXHAUSTED, 5s base delay
Server error 500/503, retry 529 overloaded, 500, retry 500+, retry
Content blocked finish_reason: 'content_filter' N/A (usually surfaces as an error) finishReason: 'SAFETY'
Token limit hit finish_reason: 'length' stop_reason: 'max_tokens' finishReason: 'MAX_TOKENS'

A robust MCP client needs a unified error handler that normalizes these differences. Rather than scattering provider-specific error checks throughout your code, centralize them in your provider adapter so the rest of your application only sees a consistent set of error types: quota, server, content-blocked, and token-limit.

Performance and Cost Characteristics (March 2026)

Model Context Input $/1M Output $/1M Best for
GPT-6 Astra 1.05M $10.00 $50.00 Hardest reasoning and coding
GPT-6.1 Sol 1.05M $2.00 $10.00 Balance of intelligence and cost
GPT-6 Luna 1.05M $0.10 $0.50 High-volume tool calling
Claude Opus 5.5 1M $4.00 $20.00 Agentic coding and knowledge work
Claude Sonnet 5.5 1M $2.00 $10.00 Speed and intelligence
Claude Haiku 5.5 1M from $0.10 from $0.50 High-volume classification and routing
Gemini 3.8 Flash 1M $0.75 $3.75 Agents and high-volume multimodal work
Gemini 4 Argon 1M $2.00 $10.00 Flagship reasoning over large inputs

Prices and context windows checked 20 October 2026. Gemini 3.8 Flash is $0.75 / $3.75 per million tokens through 31 December 2026. Gemini 4 Argon is $2 / $10 through 30 November 2026. Claude Haiku 5.5 is listed from $0.10 / $0.50.

When to Use Which Provider

  • Use OpenAI (GPT-6 Astra) when you need strict JSON output via zodResponseFormat, the Responses API’s stateful sessions, or the Agents SDK’s built-in orchestration and handoffs.
  • Use Claude (Opus 5.5) when your agent needs deep reasoning, long-running coding, or instruction-following precision.
  • Use Gemini (3.8 Flash, or 4 Argon for the hardest jobs) when you need multimodal inputs alongside tool calls, or a 1M-token window for a whole codebase or document.

The best MCP application is not the one built on the “best” model. It is the one that routes tasks to the cheapest model that meets the quality bar for that specific operation.

In practice, most production MCP systems start with a single provider, discover its limitations on specific task types, and gradually adopt a multi-provider strategy. You do not need all three providers on day one. Start with the one that best fits your primary use case, measure its weaknesses, and add a second provider only where the data justifies it.

What to Build Next

  • Write a benchmark script that sends the same MCP tool-calling task to all three providers and measures latency, token count, and result quality.
  • Build the provider abstraction layer from the next lesson and route automatically based on task type.

nJoy πŸ˜‰

Lesson 28 of 55: Building a Production Gemini Client for MCP Agents

The previous three Gemini lessons gave you the building blocks: function calling, multimodal inputs, and Vertex AI deployment. This lesson assembles them into a production-grade client library you can drop into any Node.js MCP application. It covers the patterns that only show up after your agent has processed its first ten thousand requests: token budget management, graceful quota handling, automatic retry with jitter, structured response parsing, and observability hooks.

Production Gemini MCP client architecture diagram showing token budget retry circuit breaker observability dark teal
A production Gemini MCP client needs four layers: retry logic, budget enforcement, circuit breaking, and telemetry.

The Base Client Class

// gemini-mcp-client.js
import { GoogleGenerativeAI } from '@google/generative-ai';
import { Client } from '@modelcontextprotocol/sdk/client/index.js';
import { StdioClientTransport } from '@modelcontextprotocol/sdk/client/stdio.js';

export class GeminiMcpClient {
  #genai;
  #mcp;
  #model;
  #geminiTools;
  #config;

  constructor(config = {}) {
    this.#config = {
      model: config.model ?? 'gemini-3.8-flash',
      maxTokens: config.maxTokens ?? 8192,
      maxTurns: config.maxTurns ?? 10,
      maxRetries: config.maxRetries ?? 3,
      tokenBudget: config.tokenBudget ?? 100_000,
      onTokenUsage: config.onTokenUsage ?? null,
      onToolCall: config.onToolCall ?? null,
      ...config,
    };
    this.#genai = new GoogleGenerativeAI(process.env.GEMINI_API_KEY);
    this.#mcp = new Client({ name: 'gemini-prod-host', version: '1.0.0' });
  }

  async connect(serverCommand, serverArgs = []) {
    const transport = new StdioClientTransport({ command: serverCommand, args: serverArgs });
    await this.#mcp.connect(transport);
    const { tools } = await this.#mcp.listTools();
    this.#geminiTools = [{
      functionDeclarations: tools.map(t => ({
        name: t.name,
        description: t.description,
        parameters: t.inputSchema,
      })),
    }];
    this.#model = this.#genai.getGenerativeModel({
      model: this.#config.model,
      tools: this.#geminiTools,
      generationConfig: { maxOutputTokens: this.#config.maxTokens },
    });
  }

  async run(userMessage) {
    const chat = this.#model.startChat();
    return this.#runLoop(chat, userMessage);
  }

  async close() {
    await this.#mcp.close();
  }
}

Wrapping the Gemini SDK and MCP client into a single class gives you a single place to enforce all production concerns: retries, budgets, timeouts, and telemetry. Without this, those concerns leak across your entire codebase and become impossible to test or change consistently.

Retry Logic with Exponential Backoff and Jitter

  // Inside GeminiMcpClient class

  async #sendWithRetry(chat, content) {
    const { maxRetries } = this.#config;
    for (let attempt = 1; attempt <= maxRetries; attempt++) {
      try {
        return await chat.sendMessage(content);
      } catch (err) {
        const isQuota = err.status === 429 || err.message?.includes('RESOURCE_EXHAUSTED');
        const isServer = err.status >= 500;
        if ((!isQuota && !isServer) || attempt === maxRetries) throw err;

        const base = isQuota ? 5000 : 1000;
        const jitter = Math.random() * 1000;
        const delay = Math.min(base * Math.pow(2, attempt - 1) + jitter, 60_000);
        console.error(`[gemini] Attempt ${attempt} failed (${err.status ?? err.message}), retrying in ${Math.round(delay)}ms`);
        await new Promise(r => setTimeout(r, delay));
      }
    }
  }
Exponential backoff with jitter timing diagram for Gemini 429 quota errors 5 second base delay dark
Quota errors (429) get a 5-second base delay. Server errors (5xx) use a 1-second base. Jitter prevents thundering herd.

Retry logic is where most production Gemini integrations first break. Without jitter, multiple agent instances hitting quota limits at the same time will retry in lockstep, creating a thundering herd that keeps triggering 429 errors. The randomized delay breaks this cycle.

Token Budget Enforcement

  #totalTokensUsed = 0;

  #checkBudget(usage) {
    if (!usage) return;
    const total = usage.totalTokenCount ?? 0;
    this.#totalTokensUsed += total;
    if (this.#config.onTokenUsage) {
      this.#config.onTokenUsage({ total, cumulative: this.#totalTokensUsed });
    }
    if (this.#totalTokensUsed > this.#config.tokenBudget) {
      throw new Error(`Token budget exceeded: ${this.#totalTokensUsed} > ${this.#config.tokenBudget}`);
    }
  }

Token budgets prevent a common production failure: an agent enters a verbose loop, generates thousands of tokens per turn, and blows through your daily budget in minutes. Setting a per-request ceiling catches this early, before a single runaway conversation drains your account.

The Full Run Loop

  async #runLoop(chat, userMessage) {
    let response = await this.#sendWithRetry(chat, userMessage);
    this.#checkBudget(response.response.usageMetadata);
    let candidate = response.response.candidates[0];
    let turns = 0;

    while (candidate.content.parts.some(p => p.functionCall)) {
      if (++turns > this.#config.maxTurns) {
        throw new Error(`Max turns exceeded (${this.#config.maxTurns})`);
      }

      const calls = candidate.content.parts.filter(p => p.functionCall);
      const results = await Promise.all(calls.map(part => this.#executeToolCall(part.functionCall)));

      response = await this.#sendWithRetry(chat, results);
      this.#checkBudget(response.response.usageMetadata);
      candidate = response.response.candidates[0];
    }

    if (candidate.finishReason === 'SAFETY') {
      throw new Error('Response blocked by safety filters');
    }

    return candidate.content.parts.filter(p => p.text).map(p => p.text).join('');
  }

  async #executeToolCall(fc) {
    const start = Date.now();
    if (this.#config.onToolCall) {
      this.#config.onToolCall({ name: fc.name, args: fc.args, phase: 'start' });
    }
    try {
      const result = await this.#mcp.callTool({ name: fc.name, arguments: fc.args });
      const text = result.content.filter(c => c.type === 'text').map(c => c.text).join('\n');
      if (this.#config.onToolCall) {
        this.#config.onToolCall({ name: fc.name, durationMs: Date.now() - start, phase: 'done' });
      }
      return { functionResponse: { name: fc.name, response: { result: text } } };
    } catch (err) {
      if (this.#config.onToolCall) {
        this.#config.onToolCall({ name: fc.name, error: err.message, phase: 'error' });
      }
      return { functionResponse: { name: fc.name, response: { error: err.message } } };
    }
  }

Using the Production Client

import { GeminiMcpClient } from './gemini-mcp-client.js';

const client = new GeminiMcpClient({
  model: 'gemini-3.8-flash',
  tokenBudget: 50_000,
  maxTurns: 8,
  onTokenUsage: ({ total, cumulative }) => {
    console.error(`[tokens] +${total} total=${cumulative}`);
  },
  onToolCall: ({ name, durationMs, phase, error }) => {
    if (phase === 'done') console.error(`[tool:${name}] ${durationMs}ms`);
    if (phase === 'error') console.error(`[tool:${name}] ERROR: ${error}`);
  },
});

await client.connect('node', ['./servers/analytics-server.js']);

const answer = await client.run('What were the top 5 products by revenue last month?');
console.log(answer);

await client.close();

The callback hooks (onTokenUsage, onToolCall) are the foundation of your observability stack. In production, you would pipe these events to a metrics service like Datadog or Cloud Monitoring rather than console.error, giving you dashboards for tool latency, token burn rate, and error frequency.

Streaming Responses for Long Outputs

  // Add streaming support to the client
  async runStream(userMessage, onChunk) {
    const chat = this.#model.startChat();
    const stream = await chat.sendMessageStream(userMessage);

    for await (const chunk of stream.stream) {
      const text = chunk.candidates?.[0]?.content?.parts
        ?.filter(p => p.text)
        ?.map(p => p.text)
        ?.join('') ?? '';
      if (text && onChunk) onChunk(text);
    }

    const final = await stream.response;
    this.#checkBudget(final.usageMetadata);
    return final.candidates[0].content.parts.filter(p => p.text).map(p => p.text).join('');
  }

Monitoring Quota Usage

// Track requests-per-minute to avoid hitting quota limits proactively
class RateLimiter {
  #requests = [];
  #windowMs;
  #maxPerWindow;

  constructor(maxPerMinute = 60) {
    this.#windowMs = 60_000;
    this.#maxPerWindow = maxPerMinute;
  }

  async throttle() {
    const now = Date.now();
    this.#requests = this.#requests.filter(t => t > now - this.#windowMs);
    if (this.#requests.length >= this.#maxPerWindow) {
      const oldest = this.#requests[0];
      const wait = this.#windowMs - (now - oldest) + 100;
      await new Promise(r => setTimeout(r, wait));
    }
    this.#requests.push(Date.now());
  }
}

const limiter = new RateLimiter(55);  // 55 RPM leaves headroom
await limiter.throttle();
const answer = await client.run(userMessage);

Gemini vs OpenAI vs Claude: Production Comparison

Aspect Gemini 3.8 Flash GPT-6 Astra Claude Sonnet 5.5
Parallel tool calls Yes, aggressive Yes, optional Limited (beta)
Context window 1M tokens 1.05M tokens 1M tokens
Multimodal in same call Yes (text, image, PDF, audio) Yes (text, image) Yes (text, image)
Prompt caching Context caching (Vertex) Automatic (>1K tokens) Explicit cache_control
Schema conversion from MCP Pass through (JSON Schema) Wrap in function object Pass through (JSON Schema)
Stateful session object Chat object Manual messages array or Responses API Manual messages array

This comparison is a snapshot in time. Provider capabilities and pricing shift quarterly. The production-safe approach is to log your actual usage data – tokens, latency, error rates, cost per task type – and revisit your model choices every few months based on real numbers rather than marketing materials.

Failure Modes to Harden Against

  • RESOURCE_EXHAUSTED (429): Use a 5-second base delay with jitter. Gemini’s quota windows are per minute – log RPM metrics to catch spikes before they become errors.
  • Infinite tool loops: Gemini can get into cycles where it calls the same tool repeatedly with slightly different args. The maxTurns guard is essential. Log the tool name + args on each call to detect cycles early.
  • Large context accumulation: Gemini’s Chat session adds every turn to history. For multi-hour agent sessions, this can balloon token costs. Implement a sliding window or summarization strategy at 50K tokens.
  • Safety filter false positives: Check finishReason === 'SAFETY' and handle it distinctly from other errors – do not silently return an empty string to the user.

What to Build Next

  • Extract GeminiMcpClient into a reusable npm package with a clean API and write a node:test suite against it using a mock MCP server.
  • Deploy it to Cloud Run with Vertex AI credentials, Gemini 3.8 Flash, and a Cloud Monitoring dashboard tracking token usage, tool call latency, and error rates.

nJoy πŸ˜‰

Lesson 27 of 55: Google AI Studio, Vertex AI, and MCP Servers for Enterprises

Running Gemini through the free Google AI Studio API is fine for prototypes, but enterprise deployments require what Vertex AI provides: VPC-SC network boundaries, CMEK encryption, IAM-based access control, regional data residency, no prompts used for model training, and SLA-backed uptime. If your MCP server handles customer data, PII, or proprietary IP, Vertex AI is the correct target environment. This lesson covers the transition from AI Studio to Vertex AI and the MCP-specific patterns that differ between the two.

Vertex AI enterprise diagram with VPC security IAM CMEK regional data residency connected to MCP server dark
Vertex AI provides the enterprise security posture that production MCP deployments require.

AI Studio vs Vertex AI: The Key Differences

Feature AI Studio Vertex AI
Auth API key GCP Service Account / ADC
Network isolation Public internet VPC-SC, Private Service Connect
Data used for training May be used Never used
Encryption Google-managed Google-managed or CMEK
Regional control Limited Full (europe-west1, us-east4, etc.)
SLA No SLA 99.9% SLA
Pricing model Pay per token Pay per token + provisioned throughput option

For many teams, the “data used for training” row is the deciding factor. If your MCP tools process customer records, health data, or financial transactions, the guarantee that Vertex AI never trains on your prompts or responses is often a compliance requirement, not just a preference.

Setting Up Vertex AI in Node.js

Vertex AI uses Application Default Credentials (ADC) instead of API keys. In development, authenticate with gcloud auth application-default login. In production, attach a service account to your Compute Engine instance or Cloud Run service.

npm install @google-cloud/vertexai @modelcontextprotocol/sdk
import { VertexAI } from '@google-cloud/vertexai';
import { Client } from '@modelcontextprotocol/sdk/client/index.js';
import { StdioClientTransport } from '@modelcontextprotocol/sdk/client/stdio.js';

const vertex = new VertexAI({
  project: process.env.GCP_PROJECT_ID,
  location: process.env.GCP_REGION ?? 'us-central1',
});

// Connect to MCP server
const transport = new StdioClientTransport({ command: 'node', args: ['./servers/data-server.js'] });
const mcp = new Client({ name: 'vertex-host', version: '1.0.0' });
await mcp.connect(transport);
const { tools: mcpTools } = await mcp.listTools();

// Convert MCP tools to Vertex AI FunctionDeclarations
const vertexTools = [{
  functionDeclarations: mcpTools.map(t => ({
    name: t.name,
    description: t.description,
    parameters: t.inputSchema,
  })),
}];

const model = vertex.preview.getGenerativeModel({
  model: 'gemini-3.8-flash-001',  // Vertex uses versioned model names
  tools: vertexTools,
});

The key difference from AI Studio is that you never handle API keys directly. ADC resolves credentials from the environment – a service account JSON file locally, or Workload Identity on GKE. This eliminates an entire class of secret-management bugs that plague API key-based deployments.

Vertex AI authentication flow diagram showing service account ADC credential chain GCP IAM dark
Vertex AI authentication via ADC: no API keys in your code, credentials come from the GCP environment.

The Tool Calling Loop on Vertex AI

async function runVertexMcpLoop(userMessage) {
  const chat = model.startChat();
  let response = await chat.sendMessage(userMessage);
  let candidate = response.response.candidates[0];

  while (candidate.content.parts.some(p => p.functionCall)) {
    const calls = candidate.content.parts.filter(p => p.functionCall);
    const results = await Promise.all(
      calls.map(async part => {
        const fc = part.functionCall;
        const mcpResult = await mcp.callTool({ name: fc.name, arguments: fc.args });
        const text = mcpResult.content.filter(c => c.type === 'text').map(c => c.text).join('\n');
        return {
          functionResponse: {
            name: fc.name,
            response: { result: text },
          },
        };
      })
    );
    response = await chat.sendMessage(results);
    candidate = response.response.candidates[0];
  }

  return candidate.content.parts.filter(p => p.text).map(p => p.text).join('');
}

The tool calling loop is identical to the AI Studio version. The only differences are the SDK (@google-cloud/vertexai), the auth mechanism (ADC), and the model names (versioned rather than aliased).

This identical loop structure is a deliberate design choice by Google. Teams can prototype with AI Studio (free tier, API key) and then move to Vertex AI (production, IAM) by changing only the SDK import and initialization. Your MCP integration code, tool schemas, and business logic remain untouched.

Grounding with Google Search on Vertex AI

Vertex AI offers Grounding with Google Search – a built-in tool that adds real-time web search to Gemini responses. You can combine this with your custom MCP tools:

const modelWithGrounding = vertex.preview.getGenerativeModel({
  model: 'gemini-3.8-flash-001',
  tools: [
    { googleSearchRetrieval: {} },  // Enable Google Search grounding
    { functionDeclarations: mcpTools.map(t => ({  // And your MCP tools
      name: t.name, description: t.description, parameters: t.inputSchema,
    })) },
  ],
});

Grounding with Google Search is especially powerful for MCP agents that need both internal and external data. Your MCP tools handle proprietary databases and internal APIs, while Google Search fills in real-time public information – stock prices, weather, news, regulatory updates – without building additional tools for each source.

Deploying Your MCP Server Alongside Cloud Run

A common Vertex AI pattern: your MCP server runs as a container on Cloud Run (using Streamable HTTP transport), and your Node.js host service makes MCP calls to it over HTTPS. This pairs well with Vertex AI because both services can use the same VPC connector:

// Cloud Run MCP server URL (deployed with your service)
import { StreamableHTTPClientTransport } from '@modelcontextprotocol/sdk/client/streamable-http.js';

const transport = new StreamableHTTPClientTransport(
  new URL(process.env.MCP_SERVER_URL)  // e.g., https://my-mcp-server-xyz.run.app/mcp
);

const mcp = new Client({ name: 'vertex-cloud-host', version: '1.0.0' });
await mcp.connect(transport);

Provisioned Throughput for Predictable Latency

Vertex AI’s Provisioned Throughput option pre-allocates model capacity, eliminating the latency spikes that come from shared infrastructure. For MCP agents processing high-value business transactions (order processing, financial analysis, customer support), this is worth the cost:

// Configure provisioned throughput model endpoint
const model = vertex.preview.getGenerativeModel({
  model: 'projects/PROJECT_ID/locations/REGION/endpoints/ENDPOINT_ID',  // Provisioned endpoint
  tools: vertexTools,
});

“Vertex AI provides enterprise-grade security and privacy controls, ensuring your data is never used to train Google’s models and stays within your chosen regions.” – Google Cloud, Vertex AI Data Governance

Service Account Least-Privilege Setup

# Create a service account for your MCP host
gcloud iam service-accounts create mcp-vertex-host \
  --display-name="MCP Vertex AI Host"

# Grant only the roles required for Gemini API calls
gcloud projects add-iam-policy-binding PROJECT_ID \
  --member="serviceAccount:mcp-vertex-host@PROJECT_ID.iam.gserviceaccount.com" \
  --role="roles/aiplatform.user"

# For Cloud Run, set the service account at deploy time
gcloud run deploy my-mcp-host \
  --service-account=mcp-vertex-host@PROJECT_ID.iam.gserviceaccount.com \
  --region=europe-west1

In a real deployment, you will likely have two service accounts: one for the MCP host (needs aiplatform.user to call Gemini) and one for the MCP server on Cloud Run (needs access to your databases, APIs, and storage). Separating these accounts limits the blast radius if either service is compromised.

Failure Modes Specific to Vertex AI

  • Quota limits per region: Vertex AI quotas are per-region, not global. If you hit limits in us-central1, consider distributing across regions with a simple fallback.
  • ADC credential expiry: Service account tokens expire after 1 hour. The @google-cloud/vertexai SDK handles refresh automatically, but ensure the underlying credential source (Workload Identity, attached service account) is correctly configured.
  • VPC-SC policy blocking API calls: If your MCP server is behind a VPC Service Controls perimeter, ensure aiplatform.googleapis.com is in the allowed services list.
  • Model names are versioned: Unlike AI Studio’s gemini-3.8-flash alias, Vertex uses stable names like gemini-3.8-flash-001. Pin to a version in production to avoid unexpected breaking changes.

What to Build Next

  • Deploy a Cloud Run MCP server with Streamable HTTP transport and connect it to a Vertex AI host. Verify the full flow from user request to tool execution to response.
  • Set up a service account with least-privilege IAM and test that your MCP host can call Vertex AI and your Cloud Run MCP server without any extra roles.

nJoy πŸ˜‰

Lesson 26 of 55: Multimodal MCP – Images, PDFs, and Audio With Google Gemini

Gemini’s multimodal capabilities are not a bolt-on feature — they are a first-class part of the API. When you combine them with MCP, you unlock tool-calling patterns that no other provider can match today: an agent that reads a PDF invoice and simultaneously queries your accounting database; a vision pipeline that processes uploaded product photos and calls your inventory API; an audio transcription workflow that tags clips with taxonomy from your knowledge base. This lesson covers the full multimodal stack for MCP applications.

Gemini multimodal MCP diagram showing image PDF audio inputs flowing into model alongside tool calls dark
Gemini accepts images, PDFs, audio, and video alongside text and tool calls in the same request.

The Multimodal Parts System

Every Gemini request is built from an array of parts. A part can be text, inline data (base64), or a file URI from the Files API. This composability is what makes multimodal tool calling clean:

import { GoogleGenerativeAI } from '@google/generative-ai';
import fs from 'node:fs';
import path from 'node:path';

const genai = new GoogleGenerativeAI(process.env.GEMINI_API_KEY);

// Inline image (small files up to ~20MB)
function imageToInlinePart(filePath, mimeType = 'image/jpeg') {
  const data = fs.readFileSync(filePath).toString('base64');
  return { inlineData: { mimeType, data } };
}

// Inline PDF
function pdfToInlinePart(filePath) {
  const data = fs.readFileSync(filePath).toString('base64');
  return { inlineData: { mimeType: 'application/pdf', data } };
}

This parts-based architecture is why Gemini multimodal feels natural to work with. Rather than separate endpoints for vision and text, you compose a single array of parts, mixing modalities freely. When MCP tools are added on top, the model can reason across image content and tool results in the same conversation turn.

Image Analysis + MCP Tool Calls

A common pattern: the user uploads a product photo, the model identifies it visually, then calls an MCP tool to fetch live inventory and pricing data for that product:

import { Client } from '@modelcontextprotocol/sdk/client/index.js';
import { StdioClientTransport } from '@modelcontextprotocol/sdk/client/stdio.js';

const transport = new StdioClientTransport({
  command: 'node',
  args: ['./servers/inventory-server.js'],
});
const mcp = new Client({ name: 'vision-agent', version: '1.0.0' });
await mcp.connect(transport);
const { tools: mcpTools } = await mcp.listTools();

const model = genai.getGenerativeModel({
  model: 'gemini-3.8-flash',
  tools: [{ functionDeclarations: mcpTools.map(t => ({ name: t.name, description: t.description, parameters: t.inputSchema })) }],
});

async function analyzeProductImage(imagePath) {
  const chat = model.startChat();
  const imagePart = imageToInlinePart(imagePath);

  let response = await chat.sendMessage([
    imagePart,
    { text: 'Identify this product and check its current inventory and pricing using the available tools.' },
  ]);

  let candidate = response.response.candidates[0];

  // Tool calling loop (same pattern as lesson 24)
  while (candidate.content.parts.some(p => p.functionCall)) {
    const calls = candidate.content.parts.filter(p => p.functionCall);
    const results = await Promise.all(calls.map(async part => {
      const fc = part.functionCall;
      const result = await mcp.callTool({ name: fc.name, arguments: fc.args });
      const text = result.content.filter(c => c.type === 'text').map(c => c.text).join('\n');
      return { functionResponse: { name: fc.name, response: { result: text } } };
    }));
    response = await chat.sendMessage(results);
    candidate = response.response.candidates[0];
  }

  return candidate.content.parts.filter(p => p.text).map(p => p.text).join('');
}

const analysis = await analyzeProductImage('./uploads/product-photo.jpg');
console.log(analysis);

In practice, this pattern powers use cases like warehouse quality control (photograph a shelf, look up inventory), insurance claims processing (photograph damage, cross-reference policy), and retail product identification. The key insight is that the model handles the visual recognition while MCP tools provide the live data layer.

Gemini PDF invoice analysis with MCP database lookup tool call flow dark diagram boxes arrows
Invoice processing: Gemini reads the PDF, extracts line items, then calls MCP tools to verify each item against your ERP.

PDF Processing with MCP Enrichment

PDFs are particularly powerful. A 200-page contract, a scanned invoice, or a technical specification can be passed to Gemini as inline data. The model reads the entire document and can simultaneously call MCP tools to enrich or validate the extracted information:

async function processInvoice(pdfPath) {
  const model = genai.getGenerativeModel({
    model: 'gemini-4-argon',  // Pro for complex document understanding
    tools: [{ functionDeclarations: mcpTools.map(t => ({ name: t.name, description: t.description, parameters: t.inputSchema })) }],
  });

  const chat = model.startChat();
  const pdfPart = pdfToInlinePart(pdfPath);

  let response = await chat.sendMessage([
    pdfPart,
    {
      text: `Extract all line items from this invoice (product name, SKU, quantity, unit price, total).
For each line item, use the verify_product tool to check if the SKU exists in our system.
Flag any discrepancies between the invoice price and our current pricing.
Return a structured JSON summary.`,
    },
  ]);

  let candidate = response.response.candidates[0];
  while (candidate.content.parts.some(p => p.functionCall)) {
    const calls = candidate.content.parts.filter(p => p.functionCall);
    const results = await Promise.all(calls.map(async part => {
      const fc = part.functionCall;
      const result = await mcp.callTool({ name: fc.name, arguments: fc.args });
      const text = result.content.filter(c => c.type === 'text').map(c => c.text).join('\n');
      return { functionResponse: { name: fc.name, response: { result: text } } };
    }));
    response = await chat.sendMessage(results);
    candidate = response.response.candidates[0];
  }

  return candidate.content.parts.filter(p => p.text).map(p => p.text).join('');
}

Be careful with PDF size when using inline base64 encoding. A 50-page PDF might be 5-10MB, but base64 inflates that by roughly 33%. If your invoices or contracts regularly exceed 15MB, skip ahead to the Files API section below to avoid hitting request size limits.

The Files API for Large Files

Files over ~20MB (or videos) should use the Files API rather than inline base64. This also supports re-use across multiple requests without re-uploading:

import { GoogleAIFileManager } from '@google/generative-ai/server';

const fileManager = new GoogleAIFileManager(process.env.GEMINI_API_KEY);

// Upload once, use multiple times
const uploadedFile = await fileManager.uploadFile('./large-report.pdf', {
  mimeType: 'application/pdf',
  displayName: 'Q1 2026 Report',
});

console.log(`Uploaded: ${uploadedFile.file.uri}`);

// Wait for processing to complete
let fileState = await fileManager.getFile(uploadedFile.file.name);
while (fileState.state === 'PROCESSING') {
  await new Promise(r => setTimeout(r, 2000));
  fileState = await fileManager.getFile(uploadedFile.file.name);
}

if (fileState.state !== 'ACTIVE') {
  throw new Error(`File processing failed: ${fileState.state}`);
}

// Reference in any model call
const filePart = {
  fileData: {
    mimeType: fileState.mimeType,
    fileUri: fileState.uri,
  },
};

// Now use filePart in chat.sendMessage([filePart, { text: 'Summarize this report...' }])

The Files API also opens up video analysis. You can upload a product demo video, have Gemini analyze visual content frame by frame, and then call MCP tools to log findings or trigger workflows. Audio and video share the same upload-then-reference pattern shown above.

Audio Transcription + Tool Enrichment

async function processAudioWithEnrichment(audioPath) {
  const model = genai.getGenerativeModel({
    model: 'gemini-3.8-flash',
    tools: [{ functionDeclarations: mcpTools.map(t => ({ name: t.name, description: t.description, parameters: t.inputSchema })) }],
  });

  const audioData = fs.readFileSync(audioPath).toString('base64');
  const audioPart = { inlineData: { mimeType: 'audio/mpeg', data: audioData } };

  const chat = model.startChat();
  let response = await chat.sendMessage([
    audioPart,
    { text: 'Transcribe this audio. Then identify any product names or order numbers mentioned and look them up in our system.' },
  ]);

  let candidate = response.response.candidates[0];
  while (candidate.content.parts.some(p => p.functionCall)) {
    const calls = candidate.content.parts.filter(p => p.functionCall);
    const results = await Promise.all(calls.map(async part => {
      const fc = part.functionCall;
      const result = await mcp.callTool({ name: fc.name, arguments: fc.args });
      const text = result.content.filter(c => c.type === 'text').map(c => c.text).join('\n');
      return { functionResponse: { name: fc.name, response: { result: text } } };
    }));
    response = await chat.sendMessage(results);
    candidate = response.response.candidates[0];
  }

  return candidate.content.parts.filter(p => p.text).map(p => p.text).join('');
}
Three modality examples image analysis PDF processing audio transcription each with MCP tool call enrichment dark
Image, PDF, and audio inputs all follow the same parts-based pattern – the tool calling loop is identical.

Notice that all three modality examples – image, PDF, and audio – reuse the identical tool-calling loop from the function calling lesson. This is intentional. Multimodal inputs change what the model sees, not how it calls tools. Your MCP server code stays exactly the same regardless of input type.

Multimodal MCP Resources

MCP resources can also return binary content (type 'blob') – useful for image thumbnails, report PDFs, or audio clips. You can fetch these from a resource and pass them directly to Gemini:

// Fetch a binary resource from MCP and pass it to Gemini
const resource = await mcp.readResource({ uri: 'report://monthly/2026-03.pdf' });
const blobContent = resource.contents.find(c => c.mimeType === 'application/pdf');

if (blobContent) {
  const pdfPart = { inlineData: { mimeType: 'application/pdf', data: blobContent.blob } };
  const response = await chat.sendMessage([pdfPart, { text: 'Extract the key metrics from this report.' }]);
}

Audio Content Type in MCP

New in 2025-03-26

MCP added audio as a first-class content type alongside text and image in spec version 2025-03-26. Tool results and resource contents can now include audio blocks with base64-encoded data and a MIME type. This means an MCP server can return audio recordings, synthesised speech, or extracted audio clips directly as tool output – clients that support audio can play or process them without a separate download step.

// MCP tool returning audio content
server.tool('transcribe_meeting', '...', { recording_uri: z.string() }, async ({ recording_uri }) => {
  const audioBuffer = await downloadAudio(recording_uri);
  const transcript = await transcribe(audioBuffer);

  return {
    content: [
      { type: 'text', text: transcript },
      {
        type: 'audio',
        data: audioBuffer.toString('base64'),
        mimeType: 'audio/wav',
      },
    ],
  };
});

For Gemini specifically, you can pass MCP audio content directly into the inlineData format shown in the examples above. The audio type supports WAV, MP3, FLAC, OGG, and other standard MIME types. For files over ~10MB, use the Gemini Files API to upload first, then reference by file URI.

Failure Modes to Watch

  • File too large for inline: Base64 encoding a 50MB video inline will hit request size limits. Use the Files API for anything over ~10MB to be safe.
  • Unsupported MIME types: Not all MIME types work with all models. Test image/webp and application/x-pdf variants – stick to image/jpeg, image/png, and application/pdf for broadest support.
  • Files API cleanup: Uploaded files persist for 48 hours. For GDPR/CCPA compliance, explicitly delete files after processing with fileManager.deleteFile(name).
  • Audio length limits: Inline audio has a limit of about 20MB; use the Files API for longer recordings. Processing 1 hour of audio uses roughly 1,750 tokens per minute.

What to Build Next

  • Build an invoice processing MCP agent: takes a scanned PDF, extracts line items, calls a lookup_product MCP tool for each SKU, and outputs a reconciled JSON report.
  • Add a get_image resource to an MCP server that returns product photos as blob content – have Gemini analyze them and then call your tag_product tool.

nJoy πŸ˜‰

Gemini 4 Argon and 3.8 Flash + MCP: Function Calling at Scale

Google’s Gemini 3.8 Flash and Gemini 4 Argon bring a distinct approach to function calling that differs meaningfully from both OpenAI and Claude. Understanding those differences — particularly around parallel tool execution, the functionDeclarations schema, and how Gemini handles tool results — will save you hours of debugging when you first wire up an MCP server to the Gemini API.

Gemini 3.8 Flash and Gemini 4 Argon function calling with MCP server diagram dark teal
Gemini function calling bridges the Gemini API to MCP tools via a schema conversion layer.

The Gemini Function Calling Model

Gemini’s tool-calling API is part of its @google/generative-ai package (or the newer @google/genai unified SDK). The key primitives are:

  • FunctionDeclaration – describes a function with a name, description, and JSON Schema parameters
  • FunctionCall – the model’s request to invoke a function (name + args as a plain object)
  • FunctionResponse – your code’s reply to a function call (name + response object)
  • Tool – a wrapper around an array of FunctionDeclaration objects

Gemini can issue multiple FunctionCall parts in a single response, meaning it supports parallel tool calling natively. This is a significant performance advantage when your agent can execute tools concurrently.

Installing the SDK

npm install @google/generative-ai @modelcontextprotocol/sdk zod

Use Node.js 22+ with "type": "module" in your package.json. Store your API key as GEMINI_API_KEY in a .env file and load it with node --env-file=.env.

Converting MCP Tools to Gemini FunctionDeclarations

The MCP SDK returns tools as JSON Schema objects. Gemini’s FunctionDeclaration also uses JSON Schema for parameters, but the wrapper format differs. The conversion is straightforward:

import { GoogleGenerativeAI } from '@google/generative-ai';
import { Client } from '@modelcontextprotocol/sdk/client/index.js';
import { StdioClientTransport } from '@modelcontextprotocol/sdk/client/stdio.js';

const genai = new GoogleGenerativeAI(process.env.GEMINI_API_KEY);

// Connect to an MCP server
const transport = new StdioClientTransport({
  command: 'node',
  args: ['./servers/product-server.js'],
});

const mcp = new Client({ name: 'gemini-host', version: '1.0.0' });
await mcp.connect(transport);

// List MCP tools
const { tools: mcpTools } = await mcp.listTools();

// Convert to Gemini FunctionDeclarations
function mcpToolToGeminiDeclaration(tool) {
  return {
    name: tool.name,
    description: tool.description,
    parameters: tool.inputSchema,  // MCP already uses JSON Schema - pass through directly
  };
}

const geminiTools = [
  {
    functionDeclarations: mcpTools.map(mcpToolToGeminiDeclaration),
  },
];

This near-zero conversion cost is one of Gemini’s practical advantages over OpenAI. Because both MCP and Gemini use raw JSON Schema for parameters, you avoid the nested function wrapper that OpenAI requires. In a multi-provider setup, fewer transformations mean fewer bugs.

MCP JSON Schema parameters being converted to Gemini FunctionDeclaration format comparison dark diagram
MCP’s JSON Schema for tool parameters maps directly to Gemini’s parameters field – no deep transformation needed.

The Full Tool Calling Loop

async function runGeminiMcpLoop(userMessage) {
  const model = genai.getGenerativeModel({
    model: 'gemini-3.8-flash',
    tools: geminiTools,
  });

  const chat = model.startChat();
  const messages = [{ role: 'user', parts: [{ text: userMessage }] }];

  let response = await chat.sendMessage(userMessage);
  let candidate = response.response.candidates[0];

  while (candidate.finishReason === 'STOP' && hasFunctionCalls(candidate)) {
    const functionCalls = candidate.content.parts.filter(p => p.functionCall);

    // Execute all function calls in parallel (Gemini can issue multiple at once)
    const results = await Promise.all(
      functionCalls.map(async (part) => {
        const fc = part.functionCall;
        const mcpResult = await mcp.callTool({
          name: fc.name,
          arguments: fc.args,
        });
        const text = mcpResult.content
          .filter(c => c.type === 'text')
          .map(c => c.text)
          .join('\n');
        return {
          functionResponse: {
            name: fc.name,
            response: { result: text },
          },
        };
      })
    );

    // Send all function responses back in a single turn
    response = await chat.sendMessage(results);
    candidate = response.response.candidates[0];
  }

  return candidate.content.parts
    .filter(p => p.text)
    .map(p => p.text)
    .join('');
}

function hasFunctionCalls(candidate) {
  return candidate.content.parts.some(p => p.functionCall);
}

Note the key difference from OpenAI and Claude: Gemini uses a Chat session object (model.startChat()) with chat.sendMessage() instead of stateless messages arrays. The chat object maintains conversation history internally.

In production, this loop is the heartbeat of your agent. Every real-world Gemini MCP integration – from customer support bots to internal data dashboards – runs some version of this pattern. Getting it right here means the rest of your application can treat tool calling as a solved problem.

Gemini 4 Argon: Longer Context and Better Reasoning

Switch from gemini-3.8-flash to gemini-4-argon (or the latest stable alias) for tasks requiring deeper reasoning:

const model = genai.getGenerativeModel({
  model: 'gemini-4-argon',  // 1M token context window
  tools: geminiTools,
  generationConfig: {
    temperature: 0.7,
    maxOutputTokens: 8192,
  },
});

Gemini 4 Argon’s 1-million-token context window makes it ideal for MCP agents that need to analyze large datasets, entire codebases, or long document collections through resource-fetching tools.

With the model tier covered, the next question is speed. Even the best model is slow if it makes tool calls sequentially when it could run them in parallel. This is where Gemini’s default parallel calling behavior becomes especially valuable.

Parallel Tool Execution: The Gemini Advantage

When Gemini issues multiple function calls in one response, it means it has determined those calls can be satisfied independently. This enables true parallel execution at your application layer:

// Gemini may respond with multiple FunctionCall parts simultaneously
// Example: searching multiple databases at once
// candidate.content.parts = [
//   { functionCall: { name: 'search_products', args: { query: 'laptop' } } },
//   { functionCall: { name: 'get_inventory', args: { category: 'electronics' } } },
//   { functionCall: { name: 'get_pricing', args: { tier: 'enterprise' } } },
// ]

// Your Promise.all() handles them concurrently - real parallelism
const results = await Promise.all(functionCalls.map(callMcpTool));

Compare this to OpenAI (which also supports parallel calls) and Claude (which sequences calls unless you explicitly enable parallel tool use in the beta). Gemini’s default is to use parallelism aggressively when it makes sense.

Handling Errors in Tool Responses

async function callMcpToolSafe(fc, mcpClient) {
  try {
    const result = await mcpClient.callTool({
      name: fc.name,
      arguments: fc.args,
    });
    if (result.isError) {
      return {
        functionResponse: {
          name: fc.name,
          response: { error: result.content[0]?.text ?? 'Tool returned an error' },
        },
      };
    }
    const text = result.content.filter(c => c.type === 'text').map(c => c.text).join('\n');
    return {
      functionResponse: {
        name: fc.name,
        response: { result: text },
      },
    };
  } catch (err) {
    return {
      functionResponse: {
        name: fc.name,
        response: { error: `Execution failed: ${err.message}` },
      },
    };
  }
}

Getting error handling right is critical because MCP tools are external processes that can fail for reasons completely outside your control: a database connection drops, an API rate-limits you, or the tool process crashes. Wrapping every call in a safe executor ensures your agent loop never hangs on a single broken tool.

“Function calling lets you connect Gemini models to external tools and APIs. Rather than processing all data internally, the model generates structured function calls that your application executes.” – Google AI for Developers, Function Calling Guide

With the core integration, model selection, parallel execution, and error handling covered, it is worth cataloging the specific ways things break in practice. These failure modes are drawn from real Gemini MCP deployments, not theoretical edge cases.

Common Failure Modes

  • Null parameters schema: If a tool has no parameters, pass parameters: { type: 'object', properties: {} } – omitting the field entirely causes a 400 error.
  • Nested arrays in schemas: Gemini is stricter about nested array schemas than OpenAI. Test each tool schema independently with a simple test call before integrating.
  • Chat session state: The Chat object holds history in memory. For multi-user applications, create a new startChat() per session – do not share a chat instance across users.
  • finishReason misread: Always check candidate.finishReason. A value of 'SAFETY' or 'RECITATION' means the response was blocked – handle these as errors rather than silently continuing.

What to Build Next

  • Swap gemini-3.8-flash for gemini-4-argon and pass a 500KB document as a user message – observe how the model leverages full-context reasoning alongside tool calls.
  • Add a maximumTurns guard to your tool loop to prevent infinite agent loops.
  • Log response.response.usageMetadata.totalTokenCount to measure the cost of each agent run.

nJoy πŸ˜‰

Lesson 24 of 55: Production Patterns for Claude + MCP (Caching, Retries, Tool Guards)

Three months of production experience with Claude + MCP teaches you things that no documentation covers. The retry patterns that actually work. The system prompts that reduce hallucinated tool calls. The caching strategies that cut your bill in half. The error classes you will encounter and the ones that silently corrupt output. This lesson consolidates those hard-won patterns into a reference you can apply directly.

Production Claude MCP patterns overview diagram showing caching retry budget observability blocks dark
Production Claude + MCP: the patterns that separate reliable systems from ones that fail at 2am.

Prompt Caching for Cost Reduction

Anthropic’s prompt caching feature caches portions of the input prompt that do not change between requests. For MCP applications, the tool definitions and system prompt are perfect candidates – they are typically the same for every user in a session. Caching them can reduce costs by 50-90% on repeated calls.

// Enable prompt caching for stable content
const response = await anthropic.messages.create({
  model: 'claude-sonnet-5-5',
  max_tokens: 4096,
  system: [
    {
      type: 'text',
      text: `You are a helpful assistant with access to our product database and order management system.
Always verify product availability before confirming orders.
Format all prices in USD.`,
      cache_control: { type: 'ephemeral' },  // Cache this system prompt
    },
  ],
  tools: claudeTools.map(t => ({
    ...t,
    // Cache tool definitions - they rarely change
    ...(claudeTools.indexOf(t) === claudeTools.length - 1
      ? { cache_control: { type: 'ephemeral' } }  // Cache after last tool definition
      : {}),
  })),
  messages,
});

// Check cache performance in usage stats
const usage = response.usage;
console.error(`Cache: ${usage.cache_read_input_tokens} hit, ${usage.cache_creation_input_tokens} created`);

In practice, the 5-minute rolling cache window means that prompt caching works best for active sessions with frequent back-and-forth. For batch jobs or infrequent requests, the cache expires between calls and you will not see meaningful savings.

“Prompt caching enables you to cache portions of your prompt. Cached data is stored server-side for a rolling 5-minute period, after which it expires. Cache hits save 90% of input token costs for the cached portion.” – Anthropic Documentation, Prompt Caching

System Prompt Patterns That Work

Claude responds better to system prompts that describe the persona, define tool usage rules, specify output format, and set boundaries – in that order. Vague system prompts produce vague tool use.

const PRODUCTION_SYSTEM_PROMPT = `You are a precise product research assistant for TechStore.

TOOL USAGE RULES:
1. Always call search_products before making any recommendations
2. For price comparisons, call get_product_price for each product separately
3. If a product has less than 3 reviews, note "limited reviews" in your response
4. Never recommend products that are out of stock (use check_availability first)
5. If tools return errors, explain what you could not verify rather than guessing

OUTPUT FORMAT:
- Lead with the recommendation, then supporting evidence
- Include price, rating, and availability for each recommended product
- Use bullet points for product comparisons
- End with "Note: Stock and prices verified at [current timestamp]"

BOUNDARIES:
- You can only recommend products from our catalogue
- Do not speculate about products not in the search results
- If the user asks for something outside our catalogue, say so clearly`;

A well-structured system prompt is the single highest-leverage improvement you can make to a Claude + MCP integration. Vague prompts like “be helpful and use tools when needed” produce erratic tool usage. Specific rules about when to call which tool, in what order, and how to handle errors reduce hallucinated tool calls dramatically.

Anthropic prompt caching diagram showing system prompt and tool definitions cached versus messages uncached cost reduction
Prompt caching: static content (system prompt, tool definitions) cached at 90% discount; dynamic messages are not cached.

Production Error Taxonomy

// Claude API errors and how to handle them

// 429 - Rate limit: retry with exponential backoff
// 529 - Overloaded: retry with longer backoff (Anthropic load)
// 400 - Bad request: check tool schema, messages format, max_tokens
// 401 - Auth error: check ANTHROPIC_API_KEY
// 413 - Request too large: trim context or summarize conversation history

// Non-error patterns to watch:
// stop_reason === 'max_tokens' - response was cut off, increase max_tokens
// stop_reason === 'end_turn' but no text - model may be stuck, check context

async function callClaudeWithRetry(params, maxRetries = 3) {
  for (let attempt = 1; attempt <= maxRetries; attempt++) {
    try {
      return await anthropic.messages.create(params);
    } catch (err) {
      const shouldRetry = err.status === 429 || err.status === 529 || err.status >= 500;
      if (!shouldRetry || attempt === maxRetries) throw err;

      const delay = Math.min(1000 * Math.pow(2, attempt), 30000);
      const retryAfter = err.headers?.['retry-after']
        ? parseInt(err.headers['retry-after']) * 1000
        : delay;

      console.error(`[claude] Attempt ${attempt} failed (${err.status}), retrying in ${retryAfter}ms`);
      await new Promise(r => setTimeout(r, retryAfter));
    }
  }
}

Rate limiting (429) is the error you will hit most often during development and load testing. Anthropic’s rate limits are per-organization, so one runaway script can block your entire team. Always implement retry logic before you start scaling, not after.

Context Management for Long Conversations

// Summarise old conversation history when approaching context limits
// Claude Sonnet 5.5 context: 1M tokens (checked 20 October 2026)
// But cost grows linearly with context - summarize for efficiency

async function summariseHistory(messages, anthropicClient) {
  const summaryRequest = await anthropicClient.messages.create({
    model: 'claude-haiku-5-5',  // Use cheaper model for summarisation
    max_tokens: 500,
    messages: [
      ...messages,
      { role: 'user', content: 'Summarise our conversation so far in 3 bullet points, preserving all key facts found via tool calls.' },
    ],
  });
  return summaryRequest.content[0].text;
}

// In your main conversation loop, check token usage:
if (response.usage.input_tokens > 50000) {
  const summary = await summariseHistory(messages, anthropic);
  messages = [{ role: 'user', content: `Previous conversation summary:\n${summary}` }];
}

Context summarization is a cost optimization, not just a technical constraint. Even if your conversation fits within Claude’s 1M context window, sending 100K tokens per request is expensive. Summarizing early keeps your per-request cost predictable and your latency consistent.

Failure Mode: model Outputting Tool Calls That Do Not Exist

// Claude occasionally hallucinates tool names, especially if tool descriptions are vague
// Guard against this at the execution layer
const toolNames = new Set(mcpTools.map(t => t.name));

for (const toolUse of toolUseBlocks) {
  if (!toolNames.has(toolUse.name)) {
    console.error(`[warn] Claude called non-existent tool: ${toolUse.name}`);
    toolResults.push({
      type: 'tool_result',
      tool_use_id: toolUse.id,
      content: [{ type: 'text', text: `Tool '${toolUse.name}' does not exist. Available tools: ${[...toolNames].join(', ')}` }],
      is_error: true,
    });
    continue;
  }
  // ... execute valid tool
}

Hallucinated tool names are surprisingly common when your server exposes many tools with similar names, or when tool descriptions are ambiguous. The validation pattern above is cheap insurance: a few lines of code that prevent cascading failures from a single bad tool call.

What to Check Right Now

  • Enable prompt caching – add cache_control: { type: 'ephemeral' } to your system prompt and the last tool definition. Check the usage.cache_read_input_tokens to measure savings.
  • Add a tool existence check – validate every tool name Claude returns before attempting to execute it via MCP. Hallucinated tool calls happen in production.
  • Monitor stop reasons – log every stop_reason. A high rate of max_tokens stops means you need to increase max_tokens or summarize context sooner.
  • Measure prompt cache hit rates – aim for >70% cache hit rate in sustained conversations. Low hit rates mean your “static” content is actually varying between calls.

nJoy πŸ˜‰

Lesson 23 of 55: Claude Code, Agent Skills, and MCP for Autonomous Coding

Claude Code is Anthropic’s autonomous coding agent – and it is built on MCP. When Claude Code reads files, runs tests, executes commands, and browses documentation, it does all of this through MCP servers. The architecture is not a coincidence: it is Anthropic demonstrating exactly how a production-grade autonomous agent should integrate with external systems. Understanding how Claude Code uses MCP is one of the fastest ways to understand how you should build your own agents.

Claude Code architecture diagram showing MCP servers for filesystem bash tools web browser connected to Claude agent dark
Claude Code is an MCP host: it connects to filesystem, bash, and browser servers, and orchestrates them with Claude Sonnet 5.5.

Claude Code’s MCP Architecture

Claude Code (the CLI tool, claude) operates as an MCP host. When it starts, it connects to a set of built-in MCP servers that provide its core capabilities: computer-use (screen reading/clicking), bash (shell command execution), and files (filesystem read/write). You can extend Claude Code with your own custom MCP servers, making it immediately capable of working with your specific project tools.

# ~/.claude/config.json - Extend Claude Code with your MCP servers
{
  "mcpServers": {
    "my-project-tools": {
      "command": "node",
      "args": ["./tools/mcp-server.js"],
      "env": {
        "DATABASE_URL": "postgresql://localhost/mydb"
      }
    },
    "github-tools": {
      "command": "npx",
      "args": ["-y", "@anthropic-ai/mcp-github"]
    }
  }
}

Once configured, Claude Code can use your custom server’s tools as naturally as it uses its built-in bash or filesystem tools. Your create_github_issue tool becomes as usable as Bash(git commit).

Building Agent Skills for Claude Code

The most powerful Claude Code extension pattern is the “agent skill” – a specialised MCP server that encapsulates a complex workflow as a single callable tool. Instead of Claude figuring out the 20-step process to deploy a microservice, you encode those steps in a deploy_service tool that handles all the complexity.

// deploy-skill-server.js
import { McpServer } from '@modelcontextprotocol/sdk/server/mcp.js';
import { StdioServerTransport } from '@modelcontextprotocol/sdk/server/stdio.js';
import { z } from 'zod';
import { execFile } from 'node:child_process';
import { promisify } from 'node:util';

const exec = promisify(execFile);
const server = new McpServer({ name: 'deploy-skills', version: '1.0.0' });

server.tool(
  'deploy_service',
  `Deploy a microservice to Kubernetes. Handles build, push, and rollout.
  Returns the deployment status and the new pod count.`,
  {
    service_name: z.string().describe('Name of the service to deploy'),
    image_tag: z.string().describe('Docker image tag to deploy'),
    namespace: z.string().default('production').describe('Kubernetes namespace'),
    replicas: z.number().int().min(1).max(10).default(2),
  },
  async ({ service_name, image_tag, namespace, replicas }) => {
    const steps = [];

    // Step 1: Build
    const { stdout: buildOut } = await exec('docker', [
      'build', '-t', `${service_name}:${image_tag}`, './services/' + service_name
    ]);
    steps.push('Build: OK');

    // Step 2: Push
    await exec('docker', ['push', `myregistry/${service_name}:${image_tag}`]);
    steps.push('Push: OK');

    // Step 3: Deploy
    const manifest = generateK8sManifest(service_name, image_tag, namespace, replicas);
    await exec('kubectl', ['apply', '-f', '-'], { input: manifest });
    steps.push(`Deploy: OK (${replicas} replicas)`);

    // Step 4: Wait for rollout
    await exec('kubectl', ['rollout', 'status', `deployment/${service_name}`, '-n', namespace]);
    steps.push('Rollout: Complete');

    return {
      content: [{ type: 'text', text: steps.join('\n') + `\n\nService ${service_name} deployed successfully.` }],
    };
  }
);

const transport = new StdioServerTransport();
await server.connect(transport);
Claude Code agent skills diagram showing complex deployment workflow encapsulated as single MCP tool dark
Agent skills: encode complex multi-step workflows as atomic MCP tools. Claude calls one tool instead of orchestrating ten.

Permission Modes in Claude Code

Claude Code has a permission system that controls what actions it can take without asking for confirmation. MCP tools are subject to the same permission model. You can configure Claude Code to auto-approve specific tools, require confirmation for destructive operations, or run in fully supervised mode.

# .claude/settings.json (project-level)
{
  "permissions": {
    "allow": [
      "Bash(git *)",              # Allow all git commands
      "mcp:my-project-tools:read_*",  # Allow read-only tools from my server
      "Read(**)"                   # Allow reading any file
    ],
    "deny": [
      "mcp:my-project-tools:deploy_*",  # Always ask before deploying
      "Bash(rm -rf *)"            # Never auto-approve recursive deletes
    ]
  }
}

“Claude Code is designed to be an autonomous coding agent that can understand and work on complex codebases. It uses a set of built-in tools and can be extended with custom MCP servers to access domain-specific capabilities.” – Anthropic Documentation, Claude Code

Failure Modes with Claude Code MCP Extensions

Case 1: Tools That Are Too Granular

// BAD: Too granular - Claude has to call many tools in sequence and may make mistakes
server.tool('set_k8s_namespace', '...', { ns: z.string() }, handler);
server.tool('set_k8s_image', '...', { image: z.string() }, handler);
server.tool('apply_k8s_manifest', '...', { manifest: z.string() }, handler);
server.tool('watch_k8s_rollout', '...', { deployment: z.string() }, handler);

// BETTER: One atomic skill tool that handles the whole workflow
server.tool('deploy_service', 'Deploy a service to k8s...', { service: z.string(), ... }, handler);

Case 2: Forgetting to Handle Long-Running Operations

// Build + deploy can take minutes
// Don't timeout. Stream progress via notifications or use progress indicators
// Claude Code will wait, but it needs feedback to know the tool is running

server.tool('build_and_deploy', '...', { ... }, async ({ service }) => {
  // Send progress
  process.stderr.write(`[build] Starting build for ${service}...\n`);
  await buildService(service);  // May take 2-10 minutes
  process.stderr.write(`[deploy] Deploying ${service}...\n`);
  await deployService(service);
  return { content: [{ type: 'text', text: 'Done.' }] };
});

What to Check Right Now

  • Install Claude Code – npm install -g @anthropic-ai/claude-code. Then run claude in a project directory to see it in action.
  • Add your MCP server to Claude Code config – add it to ~/.claude/config.json or .claude/config.json (project-level). Then run claude and ask it to use your tool.
  • Design tools as atomic workflows – each tool should complete one meaningful unit of work end-to-end. Avoid exposing low-level implementation details as separate tools.
  • Review the permission system – set appropriate allow and deny rules for your project. Deny destructive tools by default and require explicit confirmation.

nJoy πŸ˜‰