Introduction

The Model Context Protocol (MCP) has revolutionized how AI assistants interact with products and services. As products increasingly integrate AI capabilities, they face a fundamental architectural decision: How should MCP servers be structured to support both external AI integrations and internal AI features?

We published a blog on Jun 2025 covering MCP Patterns, which highlighted three key patterns:

  • Capability-Level MCP Server for high-level business intent abstraction
  • Product-Level MCP Server for wrapping stable product APIs
  • Component-Level MCP Server for fine-grained, microservice-level automation

In this blog, we explore a critical architectural pattern: every product needs two distinct types of MCP servers, one for external integration and one for internal AI capabilities. This dual architecture enables products to leverage MCP both as an external integration point and as an internal AI infrastructure.

Product-Level MCP Server: Your External AI Gateway

The Product-Level MCP Server is an external-facing MCP server that enables your product to be integrated with external MCP hosts such as Claude Desktop, GitHub Copilot, ChatGPT connectors, and other AI platforms. This server acts as a bridge between AI assistants and your product’s capabilities, running in the user’s application host process.

Key characteristics:

  1. External Integration: Connects your product to external AI ecosystems
  2. API Gateway: Translates natural language requests into structured API calls to your product’s API
  3. Authentication: Maintains user credentials and security contexts
  4. Lightweight & Composable: Runs alongside other product MCP servers in the same host process
  5. User-Facing: Enables end-users to interact with your product through their preferred AI assistant

The Product-Level MCP Server is designed to be your product’s ambassador in the external AI ecosystem, making your product’s capabilities accessible to any AI assistant that supports MCP.

graph LR
    subgraph "Application Host Process"
        MH[External MCP Host]
        MS1[Your Product MCP Server]
        MS2[ProductA MCP Server]
        MS3[ProductB MCP Server]

        MH <--> MS1
        MH <--> MS2
        MH <--> MS3
    end

    subgraph "Remote Services"
        YI[Your Product Instance]
        PA[ProductA Instance]
        PB[ProductB Instance]

        MS1 <--> YI
        MS2 <--> PA
        MS3 <--> PB
    end

    subgraph "LLM"
        L[LLM]
        MH <--> L
    end

This Mermaid diagram illustrates how your Product-Level MCP Server operates within the broader external MCP ecosystem. The architecture is divided into three key sections:

  1. Application Host Process: The user’s local environment where MCP servers run:
    • External MCP Host: The central hub managing communication between AI assistants and MCP servers
    • Your Product MCP Server: Your product’s external-facing MCP server
    • Other Product MCP Servers: Additional MCP servers from other products (ProductA, ProductB, etc.)
    • All servers communicate bidirectionally with the MCP Host, enabling modular composition
  2. Remote Services: The backend services that MCP servers connect to:
    • Your Product Instance: Your product’s backend that the MCP server accesses via REST API
    • Other Product Instances: Backend instances for other products
    • Each MCP server communicates exclusively with its corresponding backend service

This architecture enables your product to seamlessly integrate with external AI ecosystems, allowing users to interact with your product through their preferred AI assistant.

Product Internal MCP Server: Your Internal AI Powerhouse

The Product Internal MCP Server is an internal-facing MCP server that runs within your product’s own backend infrastructure. This server is designed to power your product’s internal AI features, such as:

  1. Built-in AI Chat Bots: Enable your product to have its own internal AI chat interface
  2. AgenticOps Features: Support autonomous operations and intelligent automation within your product
  3. Internal AI Assistants: Provide AI capabilities to users directly within your product’s UI

Unlike the Product-Level MCP Server, the internal server has direct access to your product’s internal systems, data stores, and backend capabilities, bypassing the API layer entirely. This enables:

Key Advantages

  1. Deep System Access: Direct access to raw data, internal databases, and system-level capabilities
  2. Performance Optimization: Bypasses API overhead for data-intensive operations
  3. Privileged Operations: Can perform operations requiring elevated permissions or internal system knowledge
  4. Efficient Data Processing: Access to unprocessed, raw data for more sophisticated analysis
  5. Internal Workflows: Supports internal product tools, workflows, and business logic
  6. Component-Level Flexibility: Can be organized as one server or multiple servers based on components/capabilities

Architectural Variations

You can organize your internal MCP servers in two ways:

Single Internal MCP Server: One unified server that handles all internal capabilities, potentially using patterns like Claude Skill to selectively enable different capabilities.

Multiple Component-Level MCP Servers: Multiple specialized servers, each handling specific components or capabilities of your product, enabling fine-grained control and modularity.

The internal MCP server communicates with your product’s Internal MCP Client, which interfaces with your product’s own LLM or AI chat interface.

graph LR

    subgraph "Your Product"
        IMC[Internal MCP Host]
        IRA[Your Product API]
        IMS[Internal MCP Server]
        IB[Product Backend]
        IMS --> IB
        IMC <--> IMS
        IRA --> IB
    end

    subgraph "LLM"
        L[LLM]
        IMC <--> L
    end

    subgraph "End User"
        U[User]
        U --> IMC
    end
    
    %% Define styling
    classDef newCom fill:#b6d7a8,stroke:#333,stroke-width:1px;
    class IMC newCom;
    class IMS newCom;

This diagram illustrates the architecture of the Product Internal MCP Server operating within your product’s backend infrastructure. The architecture consists of three main sections:

  1. Your Product: Your internal backend environment:
    • Internal MCP Client (IMC): Interfaces between your product’s internal AI/chat bot and the internal MCP server
    • Your Product API: The standard API that external clients use
    • Internal MCP Server (IMS): Runs within your product’s backend, highlighted in green to indicate the internal AI infrastructure
    • Product Backend: Your core backend systems, data stores, and business logic
    • Internal MCP Server has direct backend access, bypassing the API
    • Internal MCP Client and Server communicate bidirectionally
  2. End User: The user interacting with your product:
    • Accesses your product’s internal AI features through the AI Interface

This internal architecture enables powerful AI-driven capabilities directly within your product, providing users with intelligent assistance without requiring external AI tools.

Key Differences

Understanding when to use each type of MCP server is crucial for architecting your product’s AI capabilities. Here’s a comparison:

Aspect Product-Level MCP Server Product Internal MCP Server
Location Runs in user’s environment Runs within your product’s backend
Access Method Via your product’s API Direct access to backend systems
Authentication User credentials/API tokens Internal service authentication
Data Access Limited to API capabilities Full access to raw data and internal systems
Use Context External AI integrations (Claude Desktop, Copilot, etc.) Internal AI features (product’s own chat bot, AgenticOps)
Deployment Alongside other product MCP servers in external hosts Integrated with your product’s services
Performance Subject to API throughput limits Optimized for internal data access
Security Context User’s permissions Elevated system permissions
Target Users External AI assistant users Your product’s end users
AI Interface External MCP hosts Your internal MCP client

How the Two MCP Servers Work Together

The power of the dual MCP architecture lies in how the two types of servers complement each other, enabling your product to provide both external AI integration and internal AI intelligence.

graph LR
    subgraph "Application Host Process"
        MH[External MCP Host]
        MS1[Your Product MCP Server]
        MS2[ProductA MCP Server]
        MS3[ProductB MCP Server]

        MH <--> MS1
        MH <--> MS2
        MH <--> MS3
    end

    subgraph "Your Product"
        IMC[Internal MCP Host]
        IRA[Your Product API]
        IMS[Internal MCP Server]
        IB[Product Backend]
        IMS --> IB
        IMC <--> IMS
        MS1 <--> IRA
        IRA --> IB
    end

    subgraph "Other Products"
        PA[ProductA Instance]
        PB[ProductB Instance]
        MS2 <--> PA
        MS3 <--> PB
    end

    subgraph "LLM"
        EL[LLM]
        MH <--> EL
        IMC <--> EL
    end

    subgraph "End Users"
        U2[User via External AI]
        U1[User via Product UI]
        U1 --> IMC
        U2 --> MH
    end
    
    %% Define styling
    classDef newCom fill:#b6d7a8,stroke:#333,stroke-width:1px;
    class IMC newCom;
    class IMS newCom;

This comprehensive diagram illustrates how the dual MCP architecture works together:

The Complete AI Ecosystem

  1. Application Host Process: External MCP servers running in users’ environments
    • Your Product MCP Server connects to your product via API
    • Other product MCP servers enable composable multi-product AI workflows
  2. Your Product: Your backend infrastructure with both MCP servers
    • Product-Level MCP Server (external): Handles external AI assistant requests via API
    • Internal MCP Server: Provides deep, privileged access for your internal AI features
    • Both servers access your Product Backend, but through different paths

  3. Other Products: Additional products in the AI ecosystem, each with their own MCP servers

  4. End Users: Two user paths
    • User via External AI: Uses their preferred external AI assistant to interact with your product
    • User via Product UI: Uses your product’s own chat bot or AI features directly

Benefits of the Dual Architecture

  • External Integration: Users can interact with your product through any external AI assistant they prefer
  • Internal Intelligence: Your product provides native AI capabilities directly within its interface
  • Unified Backend: Both MCP servers leverage the same backend infrastructure but through appropriate access patterns
  • Flexible AI Access: Users choose their preferred interaction method, external AI tool or your product’s native interface
  • Composable Ecosystem: Your product participates in the broader AI ecosystem while maintaining its own AI capabilities

This architecture enables your product to be both a consumer and provider of AI capabilities, creating a seamless bridge between external AI ecosystems and internal product intelligence.

Conclusion

The Dual MCP Architecture represents a fundamental pattern for modern products integrating AI capabilities. By implementing both a Product-Level MCP Server and a Product Internal MCP Server, your product can:

  • Integrate with external AI ecosystems through Claude Desktop, GitHub Copilot, ChatGPT connectors, and other emerging AI platforms
  • Provide native AI intelligence through built-in chat bots and AgenticOps features
  • Optimize for different use cases by choosing the right MCP server for each scenario
  • Maintain appropriate security boundaries between external integrations and internal operations

Whether you’re building a monitoring platform, a development tool, a CRM system, or any other product, adopting this dual MCP architecture positions you at the intersection of external AI innovation and internal AI excellence.

The future of product development isn’t about choosing between external AI integration or internal AI features, it’s about implementing both seamlessly through the power of two MCP servers.