Systems | Development | Analytics | API | Testing

Kong's AI Gateway applies the same architectural pattern as the API Gateway — now governing LLM, MCP, and agent traffic at the infrastructure layer. Just as API gateways abstracted rate limiting, auth, and caching across microservices, AI gateways do the same for large language models and agents — with token budgets, semantic caching, and semantic routing replacing their REST equivalents. Kong breaks this into three layers: LLM Gateway, MCP Gateway for tool calls, and Agents Gateway for agent-to-agent traffic.#Shorts.

Most AI revenue will flow through tokens — and the two bottlenecks are tokens per watt (energy cost) and tokens per second (throughput). Tokens per watt determines how much output you can generate from a fixed energy supply — already constrained and getting tighter. Tokens per second sets the ceiling on how fast that revenue can flow. Kong's AI Gateway optimizes both at the connectivity layer: semantic caching and semantic routing increase token output without adding watts or latency.#Shorts.

LLMs are absorbing the business logic of microservices for agentic use cases — but both patterns will coexist in enterprise infrastructure for a long time. Cloud-native infrastructure (microservices + APIs) keeps powering web and mobile experiences. The agentic layer — LLMs, MCP tool calls, and context traffic — runs in parallel, activating the same APIs and CRUD operations underneath. Kong manages both swim lanes: the API traffic between clients and microservices, and the context traffic flowing between agents and LLMs.#Shorts.

CLI offers speed and developer freedom for API access; MCP provides centralized security, governance, and observability at enterprise scale. With CLI, credentials live on the developer's local machine and audit trails are shell-only — fast, but ungoverned. MCP adds authentication, centralized policy enforcement, and observability across all API calls, at the cost of some speed and higher token consumption. Kong's MCP Gateway is built for teams that need the governance trade-off without giving up too much velocity.#Shorts.

Software is going headless: the internet is shifting from GUIs built for humans to APIs, MCP servers, and CLIs built for machines and agents. Machines will consume the internet at a scale 1,000x greater than humans — more agents will exist than people, and programmatic access moves far more data than any click ever could. This transition requires API and AI infrastructure capable of moving terabytes at a scale never built before. Kong provides the connectivity layer for this machine internet — the infrastructure between agents, LLMs, and the services they consume.#Shorts.

The context graph — not the UI layer or system of record — is the true competitive IP of the AI era, and Kong built Context Mesh to help companies govern it. Without the right context layer, AI agents are generic and interchangeable regardless of which LLM is underneath. Companies that own and protect their context graph can differentiate their agentic workflows; those that don't are left with legacy CRUD backends that don't translate to agentic use cases. Context Mesh gives enterprises policy and governance over what agents can consume — the rulebook for all context flowing in and out.#Shorts.

Want to bill customers for the AI tokens they actually use? This video shows you how to set up a LangChain app that meters LLM token usage and streams it to Kong Konnect Metering & Billing as CloudEvents — turning every prompt and response into invoiced usage, automatically.