A LiteLLM gateway that runs safely and gives every team a budget and clear access rules

For management, a gateway turns AI spend from a surprise into a budget: every team has its own limit, spend is reported per team and application, and a revoked key stops working everywhere. Applications only talk to the gateway, so switching models means changing configuration, and you are not tied to one vendor.

For the data protection officer and the works council, the gateway is the one place that decides which data may go to which model and what is logged. In Bitkom's 2026 survey, 45% of companies that use AI named concern that company data falls into the wrong hands as a barrier. An EU-only model list and request logs in your own store answer that concern in one place.

Once several teams call language models, keys end up in many places and nobody can say which team caused which part of the bill. A gateway gives every application one API, whichever model answers, and gives you one place to decide who may use which model and how much they may spend.

LiteLLM is a common choice because it is open source, runs in your own environment and supports most commercial providers and self-hosted models. It is not the only option: Kong, Azure API Management and several cloud services also offer AI gateway features. No vendor pays us, so if one of those fits your stack better, we say so.

A single container on a laptop is easy. A gateway that every team depends on needs more: a Postgres database for keys, teams and spend, Redis as soon as more than one instance runs, several replicas behind a load balancer, health checks, and a plan for the spend and log tables that grow with every request.

• Without Redis, each instance counts rate limits and budgets on its own, so limits can be exceeded and revoked keys keep working until local caches expire.
• The admin interface and management endpoints are reachable only from your internal network.
• Provider keys live in your secret manager, not in configuration files.
• Request logs follow a retention rule your data protection officer has agreed to.

LiteLLM had three serious incidents in 2026. In March, malicious releases appeared on PyPI. In May, an actively exploited vulnerability in its MCP test endpoints was fixed. In October, a compromised release pipeline pushed a malicious version to PyPI and to its Docker images. All three were fixed, and LiteLLM rotated its signing key after October. They still show what running it takes: versions pinned by image digest and verified, advisories watched, upgrades rolled out in stages with tests, provider keys that can be rotated quickly, and as little of the gateway exposed as possible.

If you already run LiteLLM, we start by checking which versions were ever installed, which endpoints are reachable and where provider keys are stored. Our guide on running LiteLLM in production lists the checks.

Every team or application gets its own virtual key with the models it may use, a budget and rate limits. Spend is reported per team, application and task, so cost questions get answered from data. Routing to a cheaper model is switched on only after regression tests show that quality holds for that task, and an EU-only model list keeps requests where your rules require.

Coding agents are often the largest model users in a company. Claude Code and other tools that accept your own API endpoint can send their requests through the gateway, so their spend shows up per team and their keys can be revoked in one place. Some tools route requests through their vendor's servers and cannot reach a gateway on your internal network. We check each tool you use before promising anything.

Keys, budgets, routing, fallbacks and spend tracking are part of the open-source version. Some features, such as single sign-on for more than five users and several audit and governance functions, need LiteLLM's paid enterprise licence. We map your requirements to the feature list before you buy anything, and we use your cloud provider's identity and logging services where they close a gap.

We deploy LiteLLM on the Kubernetes you already run, on premise or on EKS or AKS, or as a managed container service in your cloud account. Behind it can sit Azure OpenAI and Amazon Bedrock in your own accounts, models you serve yourself with vLLM, and public APIs where your rules allow them.