March 30, 2026
5 Min Read

When Your Security Scanner Becomes The Weapon | LiteLLM Supply Chain Attack Breakdown

Madan Singhal
Head of AI

On March 24, 2026, a supply chain attack hit LiteLLM - one of the most widely used AI infrastructure libraries in the world, downloaded 3.4 million times every day. But this wasn't a typical attack. Threat actor TeamPCP compromised the very security scanning tool that LiteLLM used to protect itself, then used those stolen credentials to backdoor the library directly.

The result: two poisoned PyPI versions (litellm 1.82.7 and 1.82.8) containing a credential stealer that swept environment variables, SSH keys, cloud provider credentials, Kubernetes tokens, and database passwords, then silently exfiltrated them to an attacker-controlled server. The compromised packages were live for approximately three hours before PyPI quarantined them.

Keep reading to understand how this attack was executed, who is affected, what to do right now, and how Singulr Agent PulseTM provides the runtime governance layer that would have detected and contained it.

3.4M
Daily PyPI Downloads
3 hrs
Packages Live Before Quarantine
9.4
CVSS4B Score (Near Max)
88
AI Bot Comments in 102 Seconds
5
Supply Chain Ecosystems Hit
480M
Total PyPI Lifetime Downloads

It's Already in Your AI Stack

LiteLLM is not a niche library. It is the routing layer underpinning a significant fraction of all production AI infrastructure, serving as the universal proxy that routes calls to OpenAI, Anthropic, Azure OpenAI, Google Vertex, AWS Bedrock, and dozens of other LLM providers. Its adoption extends far beyond direct installation: it’s pulled in as a transitive dependency by LangChain integrations, CrewAI, AutoGen, and dozens of MCP server implementations.

The exposed library is not a dev tool or a niche utility. It is core AI infrastructure. Any team building on top of agents, pipelines, or multi-provider LLM routing has a very high probability that LiteLLM is somewhere in their dependency graph, and in many cases, they may not even know it is there.

⚠️ TRANSITIVE RISK
Even if you did not explicitly install LiteLLM, you may have received the compromised version through an AI agent framework, MCP server, or LLM orchestration library that lists LiteLLM as a dependency. Check your full dependency tree — not just your direct installs.

How a Security Tool Became the Attack Vector

The sophistication of this attack lies in its indirection. TeamPCP did not attack LiteLLM directly. They attacked Trivy, Aqua Security's open-source vulnerability scanner, the tool LiteLLM was using to check for security issues in its own CI/CD pipeline. Once Trivy was compromised, the attacker had access to LiteLLM's publishing credentials.

"TeamPCP did not need to attack LiteLLM directly. They compromised Trivy, a vulnerability scanner running inside LiteLLM's CI pipeline without version pinning. That single unmanaged dependency handed over the PyPI publishing credentials, and from there the attacker backdoored a library that serves 95 million downloads per month." — Jacob Krell, Senior Director for Secure AI Solutions, Suzu Labs

The attack unfolded over five weeks, targeting five separate supply chain ecosystems in sequence:

01
Mid-February 2026
Initial Trivy Compromise via Misconfigured GitHub Actions
An automated bot ("hackerbot-claw") exploits a misconfigured workflow in Trivy's own repository, stealing a privileged Personal Access Token. This grants write access to the trivy-action repository and its release infrastructure.
02
March 19, 2026
Tag Hijacking — Trivy-Action v0.69.4 Poisoned
Attackers rewrite existing Git tags to point to a malicious release. Because most CI pipelines reference tags rather than commit SHAs, thousands of pipelines silently began executing attacker-controlled code.
03
March 23, 2026
Lateral Move — Checkmarx KICS Compromised
The same credential-harvesting infrastructure pivots to Checkmarx KICS. A fake C2 domain is registered to impersonate the legitimate Checkmarx security company.
04
March 24, 2026 — 10:30 UTC
PYPI_PUBLISH Token Stolen → LiteLLM 1.82.7 Published
LiteLLM's CI runs Trivy via apt without a pinned version, pulling the compromised action. The token was exfiltrated and attackers published a malicious version.
05
March 24, 2026 — 10:52 UTC
LiteLLM 1.82.8 Published with .pth Persistence
Thirteen minutes later, a follow-up version adds persistence via a Python path configuration file that executes malicious code at interpreter startup.
06
March 24, 2026 — 12:44–12:46 UTC
AI Bot Spam Floods GitHub Issue #24512
Attackers deploy dozens of bot comments within seconds to suppress disclosure discussion.
07
March 24, 2026 — ~16:00 UTC
PyPI Quarantines Both Packages
After several hours, PyPI quarantines both malicious versions and a formal advisory is issued.

The Credential Loot Box

The malware embedded in proxy_server.py was purpose-built for maximum damage against AI infrastructure operators, the teams who, by definition, hold API keys for every cloud provider and LLM vendor they work with.

The credential stealer targeted:

  • All environment variables, swept in a single pass
  • SSH private keys from ~/.ssh/ and common key paths
  • AWS credentials (~/.aws/credentials, ~/.aws/config)
  • GCP application default credentials and service account keys
  • Azure CLI tokens and service principal credentials
  • Kubernetes service account tokens and kubeconfig files
  • Database connection strings and passwords from configuration files

All harvested secrets were encrypted and transmitted via POST request to models.litellm[.]cloud — a domain deliberately crafted to look like an official LiteLLM endpoint. Encrypted HTTPS payloads meant network monitoring tools flagged nothing unusual.

Why Standard Defences Failed

This attack exposes a critical gap in how we verify package integrity. The malicious content was published to PyPI using legitimate stolen credentials. The package passed every standard integrity check.

⚠️ HASH VERIFICATION ILLUSION
pip install --require-hashes would have passed. The wheel's RECORD file correctly declared litellm_init.pth with a matching hash. No hash mismatch. No suspicious domain in the package metadata. No typosquatted package name. The only detection path was behavioral, inspecting what the package does at runtime rather than what it is on disk.

The persistence mechanism in version 1.82.8 illustrates exactly why perimeter defenses and static analysis are insufficient for AI supply chain threats. The litellm_init.pth file uses Python's legitimate path configuration mechanism to execute arbitrary code before any user application code runs - before your logging framework initializes, before your SIEM agent starts, before any application-level security control is active.

This is not a zero-day. It is not an obfuscated exploit. It is a well-documented Python feature used as a persistence mechanism. Static analysis tools do not flag it as malicious because, in isolation, a .pth file is not malicious. The only way to detect it is through by monitoring runtime behavior at the time of execution.

Similarly, your DLP investment is blind to this threat. The exfiltration pathway, an encrypted HTTPS POST to a plausible-looking domain, is indistinguishable from legitimate LiteLLM telemetry to any network-level DLP tool. Only an egress allowlist enforced at the process level would have blocked the exfiltration at the network layer.

Are You Affected?

⚠️ YOU MAY BE AFFECTED IF...
  • You ran pip install litellm on March 24 between 10:39 - 16:00 UTC
  • You received v1.82.7 or v1.82.8 with no version pin
  • You built a Docker image with an unpinned pip install during this window
  • Any AI agent framework, MCP server, or orchestration tool pulled LiteLLM as a transitive dependency
  • Your CI/CD auto-updated LiteLLM during the incident window

✔️ YOU ARE SAFE IF...
  • You use LiteLLM Cloud (managed service)
  • You run the official Docker image: ghcr.io/berriai/litellm
  • v1.82.6 or earlier, and did not upgrade
  • You installed from the source via GitHub (repo was not compromised)
  • Your install was pinned to a specific safe version hash

Indicators of Compromise (IoCs)
Indicator Type Value Severity
Filesystem Artifact litellm_init.pth in site-packages CRITICAL
C2 Domain (LiteLLM) models.litellm[.]cloud CRITICAL
C2 Domain (Residual) checkmarx[.]zone HIGH
Compromised Versions litellm==1.82.7, litellm==1.82.8 CRITICAL
Trivy Binary (tag-hijacked) trivy-action v0.69.4 CRITICAL
Malicious File proxy_server.py (modified in 1.82.7 & 1.82.8) CRITICAL

Recommendations: A Three-Phase Response

The AI threat landscape has fundamentally shifted. The risk is no longer limited to employees pasting sensitive data into ChatGPT. Today, any unpinned AI dependency in a CI/CD pipeline can become a vector for credential theft at scale, and the library doing the most damage may be a transitive dependency you never explicitly installed.

1. Detect & Assess Your Exposure

The compromised litellm_init.pth file runs inside your Python environment at interpreter startup — before application code. Traditional perimeter defenses don't see it. Security teams must extend visibility to the dependency layer of their AI stack.

IMMEDIATE DETECTION STEPS

  • Run: find /usr/lib/python*/site-packages/ -name "litellm_init.pth" across all systems
  • Check DNS and SIEM logs for any outbound lookups to models.litellm[.]cloud or checkmarx[.]zone
  • Run pip show litellm in every environment and CI runner to verify the installed version
  • Audit your GitHub Actions workflows for any reference to trivy-action@v0.69.4 by tag (not pinned SHA)
  • Scan your full dependency tree using tools like pip-audit or Snyk, not just direct dependencies

THE CONTINUOUS DISCOVERY CHALLENGE

Manual signature maintenance is unsustainable. LiteLLM is neither the first nor the last AI library to be targeted. For every tool you identify and audit today, new AI infrastructure components will emerge tomorrow, many of them pulling in sensitive dependencies without explicit version pinning.

✦ SINGULR AGENT PULSE
Agent Discovery continuously maps all AI agents, tool connections, MCP server bindings, and the full dependency graph across your enterprise, including transitive LiteLLM relationships you didn't know existed. When a new agentic AI component exhibits high-risk access patterns, Singulr automatically surfaces it and pushes updated risk intelligence to your integrated EDR/XDR platforms.

2. Contain & Remediate

Whether your scan revealed active exposure or a clean environment, the next step is the same: establish hard controls before the next incident.

IF YOU WERE AFFECTED, ROTATE IMMEDIATELY

Assume any credentials available to the LiteLLM environment may have been exposed. Do not wait for forensic confirmation before rotating:

  • All LLM API keys (OpenAI, Anthropic, Google, Azure OpenAI, AWS Bedrock)
  • AWS/GCP/Azure IAM credentials and service account keys
  • Database passwords and connection strings
  • SSH private keys are present on affected systems
  • Kubernetes service account tokens
  • Any secrets stored in environment variables or .env files

IMMEDIATE HARDENING — VERSION PINNING

Pin LiteLLM to v1.82.6 or earlier immediately. Do not upgrade until LiteLLM announces a verified clean release. Extend this discipline to every AI dependency in your stack:

  • requirements.txt: Use exact version pins (litellm==1.82.6), not ranges
  • GitHub Actions: Pin every action by full commit SHA, never by tag name
  • Docker builds: Pin all pip install commands; prohibit unpinned installs in CI/CD
  • PyPI trusted publishing: Migrate from static PYPI_PUBLISH tokens to OIDC-based JWT trusted publishing

BLOCK AT NETWORK LEVEL

Block outbound traffic to models.litellm[.]cloud and checkmarx[.]zone at your network perimeter immediately.

✦ SINGULR AGENT PULSE
Agent Runtime Controls enforce real-time egress allowlists for AI infrastructure processes. An unauthorized outbound POST from a Python process to an unrecognized domain would have been blocked at execution, before a single credential left the environment. The C2 exfiltration to models.litellm[.]cloud would not have succeeded.

3. Govern with Runtime Controls

Blocking and patching alone are insufficient. The LiteLLM incident is part of a deliberate campaign that already hit Trivy, Checkmarx KICS, and multiple other ecosystems. Organizations that respond with one-time fixes will remain reactive.

GOVERNANCE REQUIREMENTS FOR AI DEPENDENCIES

  • Approved dependency inventory with verified version pins and hash locks
  • Mandatory CI/CD gates that reject any unpinned AI library installation
  • Runtime egress allowlists scoped to each AI infrastructure component
  • SBOM generation at build time for all AI-enabled services
  • Continuous drift monitoring - alert when runtime behavior deviates from the approved baseline

CONTINUOUS MONITORING

Even in environments where strict dependency pinning is enforced today, monitoring must continue. AI dependency trees are complex and shift rapidly. Shadow AI thrives in visibility gaps.

✦ SINGULR AGENT PULSE
Agent Governance detects configuration drift the moment it occurs, a modified proxy_server.py or a new .pth file in site-packages triggers a policy violation alert against your approved baseline. Combined with Agent Risk Intelligence , powered by the Singulr Trust Feed, your teams receive continuous risk-scoring updates as the threat landscape evolves, not just at install time.

How Agent Pulse Maps to the LiteLLM Attack

Every stage of the TeamPCP attack chain had a corresponding Agent Pulse control that would have detected or blocked it. This is not retrofitting, it is what Singulr Runtime GovernanceTM at the agentic layer was designed to provide.

Attack Stage What Happened Agent Pulse Control
Trivy CI Compromise Unpinned security scanner exfiltrated the PYPI_PUBLISH token Agent Discovery maps CI/CD tool connections; Risk Intelligence flags unpinned dependency risk
Credential Harvesting Malware swept env vars, SSH keys, AWS/GCP/Azure credentials Runtime Controls block unauthorized file access; anomalous credential sweeps trigger instant alert
.pth Persistence litellm_init.pth executed malicious code at Python startup Agent Governance detects new .pth creation as configuration drift against the approved baseline policy
C2 Exfiltration Encrypted POST to models.litellm[.]cloud exfiltrated secrets Runtime Controls enforce egress allowlists; unauthorized outbound POST is blocked before any data leaves
Transitive Exposure MCP servers and agent frameworks silently pulled in LiteLLM Agent Discovery surfaces the full dependency graph, including every transitive LiteLLM relationship

Four Integrated Capabilities

01 · AGENT DISCOVERY
Continuous Visibility & Context Graph
Maps all AI agents, tool connections, MCP server bindings, and permission chains, including agents and dependencies you didn't know existed.
↳ Would have mapped every pipeline with a transitive LiteLLM dependency
02 · RISK INTELLIGENCE
Singulr Trust Feed + Red-Teaming
Continuously evaluates agent risk posture through Singulr Trust Feed intelligence and AI red-teaming simulations against adversarial scenarios.
↳ Would have flagged LiteLLM's credential access patterns as anomalous
03 · AGENT GOVERNANCE
Policy Enforcement + Drift Detection
Defines and enforces policies aligned to agent type, data sensitivity, and tool access scope. Alerts on configuration changes and runtime drift.
↳ Would have detected proxy_server.py modification as policy drift
04 · RUNTIME CONTROLS
Real-Time Enforcement at Execution
Enforces controls across agent interactions in real time, blocking unauthorized access, prompt injection, and data leakage as they happen.
↳ Would have blocked the POST to models.litellm[.]cloud in real time

Integration-First by Design

Agent Pulse supports the full modern AI stack and is deployable without replacing existing security infrastructure:

Copilot Studio  ·  AWS Bedrock  ·  Azure AI Foundry  ·  GCP Vertex AI  ·  Databricks  ·  ServiceNow Now Assist  ·  CrewAI  ·  LangGraph  ·  OpenTelemetry Agents  ·  ChatGPT GPTs  ·  n8n  ·  MCP Servers  ·  EDR / XDR / SIEM

"As AI becomes more autonomous, invoking tools and operating across increasingly agentic workflows, governance has to be something you apply continuously. The only way to successfully manage this at scale is through a unified approach that enforces governance controls, provides visibility across agents, models, and tools, and continuously measures whether safeguards are working as designed." — Alex Green, CISO, Delta Dental Plans Association

Conclusion

The TeamPCP campaign spanning Trivy, Checkmarx KICS, and LiteLLM across five supply chain ecosystems in under five weeks represents a structural shift in how AI infrastructure is targeted. These are not opportunistic attacks. They are deliberate campaigns that identify tools with elevated access to automated pipelines and compromise the trust infrastructure surrounding them.

The real challenge is not any single compromised library. The problem is the loss of visibility and control over an AI dependency graph that is too large, too dynamic, and too interconnected for manual security processes to track. LiteLLM is the backbone of modern AI infrastructure. When it was poisoned, it became an infection hub for every pipeline that consumed it.

Organizations that respond with one-time patching, one-time version pinning, or one-time blocking will stay reactive. Security leaders need a programmatic response that treats AI dependencies as a new class of infrastructure risk, applying the same rigor to identity, endpoints, and cloud workloads.

That response requires:

  • Continuous discovery to surface AI dependencies and agents that employees deploy, not just those that IT approves.
  • Runtime governance that enforces policy at the moment an agent or dependency takes action, not on the next scan cycle.
  • Adaptive enforcement that keeps pace with the velocity of AI library development and threat actor sophistication.

The window to establish control over agentic AI infrastructure risk is closing.

The time to act is now.

Newsletter

Occasional updates from  Singulr

See how Singulr helps you stay ahead in AI innovation

In your personalized 30-minute demo, discover how Singulr helps you:
eye logo

Gain complete visibility across all three AI vectors in your environment

meter logo

Experience Singulr Pulse™ intelligence that keeps you ahead of emerging AI risks

search logo

See AI Red Teaming in action as it identifies vulnerabilities in real-time

tick logo

Witness Singulr Runtime Protection™ that safeguards your data without slowing AI innovation

Gradient background transitioning from deep purple to a lighter violet shade.