How to Build an MCP Semantic Layer Server (Architecture, Code, and the No-Rip-and-Replace Case)

MCP in two minutes

The Model Context Protocol (MCP) is an open standard, introduced by Anthropic in November 2024, for connecting AI applications to external systems. As of June 2026 it is governed by the Linux Foundation's Agentic AI Foundation, with a stable spec revision dated 2025-11-25. It collapses the M*N integration problem (M agent frameworks each writing connectors for N data sources) into M+N: write your data source as one MCP server, and any MCP client can reach it.

MCP runs on JSON-RPC 2.0. A server exposes three primitives:

• Tools: model-controlled functions an agent can invoke (POST-like). This is where a query lives.
• Resources: application-controlled, read-only context (GET-like). This is where metadata, lineage, and entity catalogs live.
• Prompts: user-controlled reusable templates.

Three transports carry the JSON-RPC messages:

• stdio: server runs as a local subprocess. Best for local dev and IDE tools.
• SSE (HTTP+SSE): the legacy HTTP transport, being deprecated.
• Streamable HTTP: single HTTP endpoint (POST + optional SSE streaming), session management via the Mcp-Session-Id header, OAuth 2.1, stateless-friendly. This is the one you deploy to production behind a load balancer.

The lifecycle is: client sends initialize (negotiating protocol version and capabilities), server replies, client sends notifications/initialized, then the conversation proceeds with tools/list, tools/call, resources/read, and so on. That is the whole protocol surface you need to ship a semantic-layer server. Compare this to RAG vs semantic layer to understand where MCP fits in your architecture.

What the wire actually looks like

Here is a real tools/list response advertising one tool, with an input schema and an output schema. An agent invokes it with tools/call and the server returns both a human-readable content block and a typed structuredContent object. That is the shape that matters for a semantic layer: intent goes in, a typed result plus an audit trail (compiled SQL, graph version) comes back. Learn more about compile-time governance and how to enforce it at this layer.

The query flow, end to end

Here is what happens when an agent asks a governed question through a semantic-layer MCP server:

1. Agent sends intent: query_metric(metric="churn_risk", scope="finance", filters={segment:"EU BFSI"}, group_by=["quarter"]).
2. The semantic-layer server resolves every term against its typed graph. "Churn risk" is a versioned concept; "finance" selects the policy plane; "EU BFSI" resolves to entities with provable relationships.
3. It proves the join paths. If no valid path exists, compilation fails and the server returns isError: true with a structured message. The agent gets an error, not a fabricated number.
4. It injects compile-time governance: RBAC + ABAC + row/column predicates are evaluated before any SQL is emitted. Unauthorized intent fails compilation; data is never read.
5. It emits dialect-perfect SQL, executes against the warehouse, and returns structuredContent with rows, row_count, compiled_sql, and graph_version.
6. The agent receives a typed result and the audit trail. The next agent asking the same question in the same scope gets the same answer, because the compiler is deterministic.

The agent never sees a table. It never writes SQL. It expresses intent and gets governed truth.

Build it: a FastMCP semantic-layer server

The Python SDK (mcp, with the high-level FastMCP API) is the most mature path. FastMCP powers about 70% of MCP servers across all languages. The fastest path: return a Pydantic model, and FastMCP generates the outputSchema and validates the structuredContent for you. The docstring becomes the tool description that the agent reads to decide when to call it.

For governed errors, raise a tool error rather than crashing the process. The result comes back with isError: true and your message in the content block, which is exactly what lets the agent adapt. MCP defines two distinct failure modes: protocol errors (JSON-RPC error) for malformed calls, and tool execution errors (isError: true) for governed-away or out-of-scope queries. This distinction is the difference between an agent that gracefully degrades and one that throws an unhandled exception mid-orchestration.

One tool, not twelve

The natural MCP instinct is to expose many tools: one per metric, one per domain, one per table. For application integrations (Slack, GitHub, Linear) that granularity is right. For governed data it is wrong. Faced with twelve tools that all return revenue-shaped numbers, an agent picks one, often the wrong one, and runs it. Snowflake's own managed server caps at 50 tools and warns "Higher tool counts can degrade tool-selection accuracy."

A semantic execution layer inverts this. Expose a small set of intent tools (query_metric, get_entities, validate_join, list_metrics) and let the compiler choose definitions and join paths. Tool surface goes down; correctness goes up; the agent stops failing in the choose-a-tool layer. Expose metadata as MCP resources; expose the governed query as a small number of tools.

Wire it into your agents

The payoff: the same server works across every framework, with no per-framework rewrite. This is the foundation of the no-rip-and-replace argument. Claude API (MCP connector): Point the Messages API at your remote server. LangChain / LangGraph via langchain-mcp-adapters: the MultiServerMCPClient fetches tools and the agent uses them. LlamaIndex via llama-index-tools-mcp: straightforward get_tools_from_mcp_url integration.

Three frameworks. One server. No rewrite. That is the pitch to leadership: "Our Claude agents, our LangChain orchestrations, and our LlamaIndex retrievers all talk to the semantic layer the same way. When we change a metric definition, nothing in the agent layer changes. When we change semantic-layer vendors, we change a server URL, not a codebase."

Test it without a warehouse or an LLM

FastMCP's in-memory client connects directly to your server instance, with no subprocess, no network, and no model. Your tests exercise the real MCP protocol path in milliseconds. Layer your tests: in-memory unit tests on every save; schema-snapshot tests in CI to catch tool-contract drift; parameterized edge cases (empty filters, unknown metrics, denied scopes); and use the MCP Inspector for manual exploration and debugging. Mock the warehouse with an in-memory SQLite or httpx mock so compilation logic is tested deterministically. For observability, log the compile steps (resolved entities, proven joins, injected predicates, emitted SQL), attach the audit trail to every result, and trace agent -> server -> warehouse with correlation IDs.

MCP vs HTTP/JDBC: use the right door

MCP is not a replacement for your API. It is the agent door. Use MCP when AI agents need governed, auditable answers across multiple frameworks. Use HTTP/REST or JDBC for batch jobs, dashboards, scheduled exports, and direct-query workloads where there is no agent in the loop. A serious semantic layer exposes both, all compiling through the same graph so the answer is identical regardless of door. See the comparative analysis in SQL as the semantic layer compile target.

Honest limits

MCP is a transport, not a business-logic innovation. It will not make a bad agent good, and it does not, by itself, make integration free:

• If your agent is deeply coupled to a vendor-specific tool schema, swapping servers still means rewriting tool-handling code. Portability is a property of well-designed, task-shaped tools, not of the protocol alone.
• After a swap you will still need to re-test and re-validate behavior. MCP removes the rewrite, not the QA.
• MCP's base protocol lacks cryptographic client-server authentication, and sandboxing is the client's responsibility. Prompt injection and tool poisoning are real. Connect only to trusted servers, allowlist tools, keep a human in the loop for sensitive actions, and put governance in the semantic layer rather than trusting the agent to behave.
• MCP solves connector fragmentation. It does not solve definitional fragmentation. Two agents over the same MCP fleet with no semantic layer behind it will still disagree about revenue. The protocol made every connector look the same; the semantic layer makes every answer mean the same thing.

Where Colrows fits

Colrows is engineered to be the semantic execution layer behind your MCP fleet. Agents emit intent over MCP; Colrows resolves it against a typed semantic graph, proves every join, enforces RBAC + ABAC + row/column governance at compile time (unauthorized intent fails compilation, data is never read), emits dialect-perfect SQL across 16+ engines, and returns a structured result with a full audit trail (graph version, definitions used, executed SQL, identity context). It can ingest dbt, Cube, Looker, or Unity Catalog definitions alongside its own graph. Because MCP servers are stateless and portable, the build vs buy decision favors MCP: moving from one semantic-layer vendor to Colrows means changing a server URL, not rewriting agents. MCP is the transport. The semantic layer is the logic. You need both. See Colrows docs for additional technical details.

Frequently asked questions

What is an MCP semantic layer?

A semantic layer exposed as an MCP server, so AI agents query governed business meaning (metrics, entities, join paths, policies) through standardized MCP tool calls instead of writing raw SQL against tables.

Is MCP stable enough to build on in 2026?

Yes. The current stable spec revision is 2025-11-25, MCP is governed by the Linux Foundation's Agentic AI Foundation, and per maintainer figures there are over 10,000 active public servers and 97M+ monthly SDK downloads across Python and TypeScript.

What tools should a semantic-layer MCP server expose?

A small set of intent tools (query_metric, get_entities, validate_join, list_metrics) plus metadata as resources, not one tool per metric. Fewer, intent-shaped tools improve agent tool-selection accuracy.

How does an MCP server return errors to an agent?

Two ways: JSON-RPC protocol errors for malformed calls, and isError: true inside the result for governed-away or out-of-scope queries, with an actionable message the agent can act on.

Does MCP avoid rip-and-replace?

For agent integration, largely yes: agents written against MCP tools work with any MCP server, so swapping the semantic layer is a URL change, not an agent rewrite. You still re-test behavior, and deeply vendor-coupled agents may still need work.

MCP or HTTP/JDBC for my semantic layer?

MCP for multi-framework agent integration; HTTP/REST or JDBC for batch, dashboards, and direct-query workloads. A good semantic layer offers both over the same graph.

Which frameworks support MCP today?

Claude (web, desktop, Code, API connector), Claude Agent SDK, LangChain/LangGraph, LlamaIndex, the Vercel AI SDK, OpenAI's Responses API, Cursor, and VS Code, among others.

How do I test an MCP semantic-layer server?

Use FastMCP's in-memory client for deterministic pytest runs (no network, no LLM), snapshot-test schemas in CI, mock the warehouse, and use the MCP Inspector for manual debugging.