Running LiteLLM in production after the 2026 incidents

Is LiteLLM safe to run after the 2026 incidents

Yes, if you can patch and verify it like the critical service it is. The incidents were real, but each had a clearly bounded set of versions and a clear way out. Two were attacks on how LiteLLM is distributed and one was a flaw in the code. The table summarises them from LiteLLM's own incident posts and the CVE record, as of 7 November 2026.

The October case carries the most useful lesson for operators. According to LiteLLM's root cause analysis, it began with an administrative key to an internal deployment of LiteLLM's Agent Platform that was committed to a public repository on 27 May 2026 and was still valid more than four months later. Through that service the attacker obtained a GitHub token belonging to an organisation owner. Signing did not stop it either: the malicious images carried a valid signature from the previous signing key, so LiteLLM advises checking digests as well. Our reading is that the same habits protect you with any gateway: pin exact artefacts, keep keys short-lived and notice when something you run changes. LiteLLM has since published a list of changes to its release pipeline, identity and detection controls.

How to check whether you were affected

Start with an inventory of every place LiteLLM runs: the proxy deployments, CI jobs, developer laptops and any Python application that imports the SDK. Then work through each incident against that list.

1. 1

   Record versions and digests

   For each deployment, record the LiteLLM version and the digest of the image that is running, which may differ from the tag in your manifest. In a Python environment that may contain 1.107.0, LiteLLM warns against starting with pip install, because any Python start there can rerun the malicious code. Inspect the files in site-packages instead.

2. 2

   March: look for 1.82.7 and 1.82.8

   Search CI logs, build logs and lock files for these versions, and site-packages for a file named litellm_init.pth. LiteLLM also lists outbound requests to models.litellm.cloud and checkmarx.zone as indicators, and its incident post treats 10:39 to 16:00 UTC on 24 March 2026 as the window to check.

3. 3

   CVE-2026-42271: check versions and endpoint calls

   Any proxy from 1.74.2 up to but not including 1.83.7 was vulnerable. If one was reachable by people or services you do not fully trust, search access logs for POST requests to /mcp-rest/test/connection and /mcp-rest/test/tools/list and treat unexpected calls as a compromise of the host. This log check is our own suggestion. LiteLLM does not publish it as a procedure.

4. 4

   October: look for 1.107.0 and its files

   Look for litellm 1.107.0, the files cook.pth, cook.py and sitecustomize.py in site-packages, a Python process holding /tmp/remote_code_loader.lock, and the image digest LiteLLM published as an indicator. LiteLLM notes that finding nothing proves little, because containers and temporary directories may already have been cleaned up, so also check whether any deployment pulled latest, main-latest, main-stable or v1.107.0 on 21 October.

5. 5

   Rotate what the host could reach

   Where exposure is plausible, rebuild from a known-good version and rotate everything the host could reach. For October, LiteLLM lists the instance or task role, every provider key stored in the proxy database, the master and virtual keys, the database password and any secret in the environment of any process on the host. Revoke the old credentials, do not just issue new ones.

Do you need Redis and Postgres

You need Postgres as soon as you issue virtual keys, and Redis as soon as more than one worker process serves traffic. LiteLLM's virtual keys require a Postgres database set through DATABASE_URL. Its Redis requirements page calls Redis highly recommended for any deployment that runs more than one worker, which includes a single container started with several workers and a Kubernetes deployment with more than one replica. Without Redis, each worker keeps its own state:

• Rate limits are counted per worker, so the documentation's example limit of 100 requests per minute across four workers lets 400 through
• Hard budgets can be overshot, because concurrent requests on different workers are counted before the database catches up
• Key deletions, budget edits and team changes reach other workers only when their local cache entry expires, so a revoked key stays usable on workers that did not process the revocation
• Response cache hits only land on the worker that stored the response
• Admin UI SSO login can fail when the browser redirect reaches a different worker than the one that started it

That third point is the usual answer to why revoked keys keep working for a while. The setting user_api_key_cache_ttl controls how long, in seconds, keys are cached in memory, and shared Redis state lets a revocation apply across workers. Write an offboarding test that revokes a key and then calls every replica with it, and run it before go-live and after every upgrade. The documentation also points out that setting REDIS_HOST and REDIS_PORT in the environment alone is not enough: both the router settings and the cache block must point at Redis.

For LiteLLM on Kubernetes, the production best practices page recommends one Uvicorn worker per pod and scaling with pods, about 1 vCPU and 4 GiB of memory per worker as a floor, Redis 7.0 or newer once you run more than one instance, and a cap on database connections through database_connection_pool_limit, which defaults to 10. The readiness endpoint /health/readiness keeps returning 200 while the database is unavailable, so pods stay in service. Decide deliberately whether that is what you want during a database outage.

How to pin, verify and upgrade versions

Pin the image digest, verify its signature, and move one environment at a time after your own regression tests pass. LiteLLM's image security page recommends digest pinning because it fixes the exact image content regardless of tag changes, warns against latest in production and has deprecated main-stable and main-latest. Images on GHCR have been signed with cosign since v1.83.0. After the October incident LiteLLM rotated the signing key and says to trust the old key only for images published before 1.107.0. In our reading, a deployment pinned by digest would have stayed on its existing image throughout the October incident, whatever the tags pointed to.

The release rhythm has also changed. As of 7 November 2026, release candidates and stable releases ship weekly on Tuesday mornings, starting the week of 2 November 2026, and release candidates get a seven-day window for early testers before promotion. LiteLLM recommends pinning production to the latest stable release. It is also piloting a long-term support line, starting with 1.108, for one month, and has paused its policy of patching the four most recent stable lines while the pilot runs. Check the version support page before you assume your line still receives fixes.

1. 1

   Pick a target and read its notes

   Choose a stable release, read the release notes for changed defaults and record the digest. Two recent examples: since 1.104.0 the proxy does not start without a strong master key, and STDIO MCP servers are off unless LITELLM_ENABLE_MCP_STDIO=true. Since 1.105.0 the /sso/debug route is off.

2. 2

   Verify the artefact

   Run cosign verify against the public key at a pinned commit, as LiteLLM's documentation shows, and compare the digest with the one you recorded. Mirror the verified image into your own registry so that production never pulls from a public tag.

3. 3

   Run regression tests

   In staging, replay a fixed set of requests for each model route and check status codes, streaming, tool calls, cost records, key revocation and budget enforcement. LiteLLM's long-term support line ships patches for LLM translation and cost tracking, which suggests where upgrades most often change behaviour.

4. 4

   Roll out in stages

   Move one environment or a share of traffic at a time, watch error rates and latency per provider, and keep the previous digest ready for rollback. Schema migrations can make a rollback harder, so back up the database first. The production guide also lets you stop pods from running migrations with DISABLE_SCHEMA_UPDATE and run them as a separate step.

Exposing LiteLLM safely and handling its logs

Expose only the routes your clients call, from a private network where possible, and keep everything administrative internal. That is also the core of LiteLLM's security best practices page. In practice:

• Run the proxy behind an ingress or internal load balancer, with Kubernetes network policies that admit only the client namespaces and allow egress only to your model providers, Postgres and Redis
• Serve the admin UI only on an internal hostname or VPN, or switch it off on the public deployment with DISABLE_ADMIN_UI=True. Once admins sign in through SSO or named admin users, set disable_env_credential_login: true, because otherwise UI_USERNAME and UI_PASSWORD act as a shared admin login, and with UI_PASSWORD unset the master key itself is accepted
• Set NO_DOCS, NO_REDOC and NO_OPENAPI to True together. By default the Swagger UI, ReDoc and the OpenAPI schema are served without authentication, and setting only one leaves the others readable
• Keep the master key and LITELLM_SALT_KEY in a secret store, give each workload its own virtual key, and do not rotate the salt key once credentials are stored, as LiteLLM warns
• For MCP, leave STDIO servers disabled unless you need them and mark each MCP server explicitly with available_on_public_internet. The documentation is inconsistent about the default, and callers count as internal only when their IP address falls inside mcp_internal_ip_ranges
• Behind a reverse proxy, set PROXY_BASE_URL to the exact https origin users see, so that session cookies carry the Secure flag

Logging and log retention

By default LiteLLM writes a spend log and error logs per request but does not store prompt and response content. The setting store_prompts_in_spend_logs switches content storage on, and disable_spend_logs stops writing spend logs to the database. Retention is off until you set it: with maximum_spend_logs_retention_period unset, nothing is ever deleted. The cleanup settings work without an enterprise licence, and aggregated daily tag spend has a separate retention setting. Deleting rows does not return disk space to the operating system, so size Postgres storage for the peak rather than the steady state. Decide with your data protection officer whether prompts may be stored at all. A gateway that keeps request bodies holds whatever clients send, including credentials pasted into prompts.

# config.yaml (excerpt)
general_settings:
  master_key: os.environ/LITELLM_MASTER_KEY
  disable_env_credential_login: true
  store_prompts_in_spend_logs: false
  maximum_spend_logs_retention_period: "30d"
  maximum_spend_logs_cleanup_cron: "0 4 * * *"
  database_connection_pool_limit: 10

# Redis: both blocks are needed for multi-worker state
router_settings:
  redis_host: os.environ/REDIS_HOST
  redis_port: os.environ/REDIS_PORT
  redis_password: os.environ/REDIS_PASSWORD
litellm_settings:
  cache: true
  cache_params:
    type: redis
    host: os.environ/REDIS_HOST
    port: os.environ/REDIS_PORT
    password: os.environ/REDIS_PASSWORD

# Environment of the client-facing pods
LITELLM_MODE=PRODUCTION
NO_DOCS=True
NO_REDOC=True
NO_OPENAPI=True
DISABLE_ADMIN_UI=True

Which features need the enterprise licence

As of 7 November 2026, the open-source gateway covers routing, virtual keys and spend logs, and a licence key on the same image switches on the governance features. The one most teams hit first is SSO: it is free for the admin UI for up to five users and needs an enterprise licence beyond that. LiteLLM says pricing depends on deployment size and usage, offers a 30-day trial and sells through the AWS and Azure marketplaces. The table pairs the enterprise features you are most likely to need with an open-source way around them. The right-hand column is our own judgement.

Our rule of thumb: if you need SSO for more than five people in the UI, audit evidence for an auditor or regulator, or separate logging destinations per team, compare the licence with the engineering time needed to rebuild those features around the gateway and keep them working through upgrades. One more point belongs in that comparison. According to LiteLLM's security best practices page, enterprise customers receive email notice of major security updates seven days before public disclosure, which matters if you cannot patch within a day.

Go-live checklist for a self-hosted LiteLLM gateway

Before you self-host LiteLLM for real traffic, give each item below an owner and keep evidence that it is done.

1. 1

   Version and supply chain

   A stable release pinned by digest, signature verified, image mirrored to your own registry, and no latest or main-stable tags anywhere in manifests or CI.

2. 2

   Known incidents closed

   Inventory checked for 1.82.7, 1.82.8, 1.107.0 and any version below 1.83.7, with rotations documented wherever exposure was plausible.

3. 3

   State

   Postgres with backups and a tested restore, Redis 7.0 or newer for any deployment with more than one worker, and an offboarding test that passes on every replica.

4. 4

   Exposure

   Admin UI internal or disabled, API docs off, environment credential login off, STDIO MCP off, every MCP server marked public or private on purpose, network policies in place.

5. 5

   Secrets

   Master key, salt key and provider keys in a secret store, one virtual key per workload with an expiry, and no application using the master key.

6. 6

   Logs

   Retention period set, prompt storage decided with data protection, and logs shipped to your observability stack with its own retention.

7. 7

   Tests

   Regression suite per model route, a load test at expected peak with headroom, and failover between providers exercised at least once.

8. 8

   Operations

   A named patch owner who follows LiteLLM's security posts and release notes, a target time for applying security fixes, and a runbook for rotating every credential the gateway holds.

Cloudsail sets up and operates LiteLLM on clients' own Kubernetes clusters and cloud accounts, including acceptance tests like the ones in this checklist.

LiteLLM gateway setup and operationLLM platform engineeringAI system testing