LiteLLM Is a CVSS 10.0 Liability. LangChain Leaks Your Secrets.
Langfuse Inherits Both.
Your AI gateway shouldn't be the breach. Here's how to switch — one sub-product at a time.

Most teams adopted this stack for convenience: one base URL to route across OpenAI, Anthropic, Bedrock, Azure, and Vertex; one framework to chain prompts and tools; one dashboard to watch tokens and latency. The convenience is real. But the security model underneath it was never designed for a world where a poisoned model response, a manipulated Host header, or a single error path can hand an attacker remote code execution or a wallet full of provider keys.

LiteLLM: From AI Gateway to Unauthenticated RCE

LiteLLM is the most widely deployed open-source AI gateway/proxy — a single OpenAI-compatible endpoint that fans out to dozens of providers. That central position is exactly why a flaw in it is catastrophic: it sits in the path of every model call, and it holds every provider key. In 2026 it accumulated a set of vulnerabilities that, taken together, turn the gateway itself into the breach.

Individually these are serious. Chained, they are terminal. Researchers at Horizon3.ai confirmed that combining CVE-2026-42271 with CVE-2026-48710 yields a CVSS 10.0, zero-credential, unauthenticated remote code execution path on vulnerable LiteLLM deployments. The exploit chain reads like a checklist of everything an AI gateway should never allow:

The fix arrived in LiteLLM 1.83.7 (May 8, 2026), which added authorization controls and updated Starlette. But patching one chain does not change the posture, and 2026 was not LiteLLM's first incident.

The March 2026 supply-chain compromise

In March 2026, attackers slipped malicious code into official LiteLLM releases on PyPI. The payload was a credential stealer — designed to exfiltrate secrets across cloud environments, CI/CD pipelines, and developer machines that installed the poisoned package. Trend Micro, Snyk, and HeroDevs all reported on the compromise. For a package that, by design, is wired into the most secret-rich path in your infrastructure, a single tainted release is a direct line to your provider keys and cloud credentials.

LangChain: Prompt Injection That Becomes Code Execution

LangChain (and LangGraph) is an orchestration library, not a gateway — it lives inside your application process, chaining prompts, tools, and state. That makes its vulnerabilities a different shape: instead of an external network attacker, the threat rides in on the data your application already trusts — specifically, model output.

The vector is what makes LangGrinch so dangerous in agentic systems. The primary entry point is the LLM response itself — fields like additional_kwargs and response_metadata — which are controlled via prompt injection and then serialized and deserialized during streaming. The chain looks like this:

The patches are the right shape: a new allowed_objects allowlist in load()/loads(), Jinja2 blocked by default, and secrets_from_env now defaulting to False. But the lesson stands — in an agentic system, model output is untrusted input, and any layer that deserializes it as trusted is one prompt-injection away from code execution. That defense belongs in front of orchestration, not inside it.

Langfuse & the Coupling Problem

Langfuse is the observability layer many of these stacks bolt on — traces, token counts, latency, prompt history. The problem is not a single dramatic RCE; it is coupling. Observability is only as secure as the gateway it watches, and the gateway is only as secure as its worst error path.

CVE-2025-0330 above is the clearest illustration: a flaw in the LiteLLM gateway leaks the Langfuse public and secret keys through an error path. A bug in one product burns the credentials of the adjacent one. When you bolt observability onto an insecure gateway, the observability layer inherits the gateway's blast radius — and its keys.

The deployment surface adds to it. Self-hosted Langfuse Docker images shipped with vulnerable Node/npm packages; the issue was addressed in release 3.143.0. The broader pattern is unmistakable: the adjacent low-code AI tool Langflow — a different product, not to be confused with Langfuse — also had a vulnerability added to the CISA KEV catalog in May 2026. Across the AI middleware ecosystem, unsecured tooling is being actively exploited.

Thorough Comparison

Here is a fair, factual comparison across the four layers. LangChain is an orchestration library, not a gateway, so several gateway-specific rows are marked N/A for it — that is not a knock, it is a category difference. The point is the security posture each layer brings by default.

And on the feature axis people actually adopt LiteLLM for — multi-provider routing and cost control — RuntimeAI's Secure LLM Router matches the convenience and adds the governance that was missing:

How the Router Handles a New Model Drop

A fair question when you put a gateway in your critical path: what happens the day OpenAI or Anthropic ships a new model? With many setups the answer is “wait for an SDK bump or a code change.” With the Secure LLM Router, the answer is nothing.

The router is a governed passthrough. The model name in your request is forwarded to the provider you configured, so a new model from a provider you already use works the moment that provider exposes it — no redeploy, no SDK upgrade. Three governance layers wrap that passthrough without getting in its way:

• Policy gate — per-tenant and per-agent allow/deny lists decide which models are usable. Default-open, so new models flow through; lock specific models down (or pin an allowlist) when governance requires it.
• Cost attribution — register the new model’s price so spend and budgets stay accurate. Routing works either way; this just keeps the ledger complete.
• Model-deprecation policy — schedule retirement of an old model with automatic migration to its replacement, so you are never pinned to a model a provider is sunsetting.

Wired providers today: OpenAI, Anthropic, Azure OpenAI, Google Gemini & Vertex, Mistral, Cohere, Groq, Together, Fireworks, Perplexity, DeepSeek, xAI, Cerebras, SambaNova, OpenRouter, and AI21 — plus any OpenAI-compatible self-hosted endpoint (vLLM, Ollama, TGI) and RuntimeAI’s on-prem Sovereign LLM Platform. Providers with non-standard auth get a dedicated adapter; OpenAI-compatible providers route through the unified path. The honest version of “works with everything” is this: any model from a provider you’ve connected works without a code change — and connecting a new OpenAI-compatible provider is a config entry, not a release.

It's Not Just These Three — Map for the Whole Middleware Layer

LiteLLM, LangChain/LangGraph, and Langfuse are the most visible names, but the same governance gap runs through the entire AI-middleware category. Whatever you run today, there is a governed RuntimeAI layer to swap it for — one at a time:

You Don't Have to Rip It Out — Switch One Layer at a Time

This is the part that matters most. You do not have to do a forklift migration to fix this. The AI middleware stack is modular, and so is the fix. Keep your application, your prompts, and your agents exactly as they are — and swap out the risky layer underneath them. Adopt a single RuntimeAI sub-product to close one specific CVE class, or adopt the whole platform for defence-in-depth. The migration is incremental and reversible.

The smallest possible change is a base-URL swap. The Secure LLM Router is OpenAI-compatible, so most apps move with one environment variable — and the provider key stops living in plaintext:

# Before — LiteLLM proxy: shared key in env, unauthenticated control plane
OPENAI_API_BASE=http://litellm-proxy:4000/v1
OPENAI_API_KEY=sk-litellm-master-key        # long-lived, sits in .env

# After — RuntimeAI Secure LLM Router (drop-in, OpenAI-compatible)
OPENAI_API_BASE=https://router.runtimeai.io/v1
OPENAI_API_KEY=${RUNTIMEAI_TOKEN}           # short-lived JWT, minted by QuantumVault
# every call now carries tenant context + Bot-CA mTLS; your routing
# and cost policy are unchanged — only the security model is.

RuntimeAI Take

Most Advanced AI Security Zero Trust · Defence in Depth

The middleware CVEs of 2025–2026 are not a string of unlucky bugs — they are the predictable result of running trust-everything convenience layers in the most sensitive position in the stack. RuntimeAI's answer is four independent control layers, each of which would have blunted these specific CVE classes:

• Discovery / Inventory. Every model call, MCP server, and agent is registered and identity-bound through the Secure LLM Router and MCP Gateway. There is no anonymous caller and no shadow MCP test endpoint — the Host-header bypass and command-injection surface (CVE-2026-48710 / 42271) have nothing unauthenticated to reach.
• Behavioural enforcement. Flow Enforcer applies OPA/Rego policy to every tool call and route. Shell-spawning configs, unsanitized parameters, and out-of-policy provider routes are rejected before execution — the exact decision LiteLLM's test endpoint never made.
• Flow / egress control. The bidirectional AI Firewall + PII Shield scan inbound and outbound content, redact PII, and detect injection patterns. A poisoned LLM response field — the LangGrinch (CVE-2025-68664) vector — is caught before it reaches a vulnerable deserializer, and provider keys live in QuantumVault, not plaintext env, so a stealer or error path (CVE-2025-0330, the supply-chain payload) finds nothing to exfiltrate.
• Immutable audit trail. The Audit Black Box records every call, route, and tool invocation in a Merkle-chained, PQ-Sign-timestamped log — the secure replacement for Langfuse-style observability, with no plaintext-key coupling. If a provider, agent, or credential is compromised, the Kill Switch revokes it instantly at L1/L2/L3.

Sources

• LiteLLM RCE Vulnerability Exploited — Cybersecurity News
• Inside the LiteLLM Supply-Chain Compromise — Trend Micro Research
• LiteLLM Vulnerability CVE-2026-42271 — CybelAngel
• Poisoned Security Scanner Backdooring LiteLLM — Snyk
• Critical LangChain Core Vulnerability — The Hacker News
• LangGrinch — LangChain Core CVE-2025-68664 — Cyata
• CVE-2025-68664 — NVD
• LangChain / LangGraph Flaws Expose Files — The Hacker News
• GHSA-879v-fggm-vxw2 — GitHub Advisory