How to respond to a supply chain compromise

You've received an alert. A package you installed has been identified as malicious — or Veln blocked an install that turned out to be a real threat. What do you do now?

This is the step-by-step incident response process for a supply chain compromise involving npm or Python packages.

Step 0: Determine if you were actually affected

Before doing anything else, establish whether the malicious package was actually installed and executed on any of your systems.

# Check if the package is currently installed
npm list <package-name>
pip show <package-name>

# Check the git history of your lockfile for when the package appeared
git log -p -- package-lock.json | grep "<package-name>"
git log -p -- requirements.txt | grep "<package-name>"

# Check if the package was in any built Docker images
docker history <image-name> | grep "npm install\|pip install"

If the package was never installed, you're done. If it was installed, continue.

Step 1: Determine what the malicious code did

Before rotating credentials, you need to know what you're rotating. Find the malicious package's source code — it will have been preserved in security advisories, researcher posts, or Veln's audit log if you use Veln.

The malicious code will typically reveal:

Which environment variables it accessed (tells you what credentials to rotate)
Which files it read (tells you what secrets may have been exposed)
Where it sent data (the exfiltration endpoint)

If you can't find the source code, assume the worst: all environment variables, all credential files, and all secrets available in the execution context are compromised.

Step 2: Identify the blast radius

Developer machines. Which developers ran npm install or pip install during the window when the malicious package was available? Check git history for commits that would have triggered a dependency install. Check your CI logs for build timestamps.

CI/CD environments. Which CI pipelines ran during the window? Which pipelines install the affected package? What secrets were available in those pipelines?

Production systems. Did any production deployment pull the malicious package? Check deployment logs.

For each affected context, list every credential, token, and secret that was available in the process environment.

Step 3: Rotate all exposed credentials

This is non-negotiable. Assume all credentials available in any affected context are compromised. Rotate them all.

GitHub personal access tokens:

Settings → Developer settings → Personal access tokens → Revoke and regenerate

AWS credentials:

IAM → Users or Roles → Security credentials → Deactivate and create new access keys

npm auth tokens:

npm token revoke <token-id> for each token

Docker registry credentials:

Regenerate via your registry's account settings

Database credentials:

Rotate via your database provider's console

API keys (Stripe, SendGrid, Twilio, etc.):

Rotate via each service's API keys dashboard

Do this rotation in parallel if possible — if the attacker is actively using exfiltrated credentials, time matters.

Step 4: Check for persistence

Sophisticated malware doesn't just exfiltrate credentials. It also establishes persistence — mechanisms to maintain access even after the credentials are rotated.

On developer machines:

# Check for new cron jobs
crontab -l
cat /etc/crontab
ls /etc/cron.d/

# Check for new startup items (macOS)
ls ~/Library/LaunchAgents/
ls /Library/LaunchDaemons/

# Check for new systemd services (Linux)
systemctl list-units --state=enabled --user
systemctl list-units --state=enabled

# Check shell profile modifications
diff ~/.bashrc ~/.bashrc.backup  # if you have a backup
cat ~/.bashrc | grep -v "^#" | tail -20

On CI runners: Most CI runners are ephemeral — they're created fresh for each build and destroyed after. If you use self-hosted runners (persistent machines), apply the same checks as developer machines.

Step 5: Remove the malicious package and rebuild

# Remove the package
npm uninstall <package-name>
pip uninstall <package-name>

# Update lockfile to remove the package
npm install
pip-compile requirements.in

# If you need to keep the package functionality,
# pin to a known-safe version
npm install <package-name>@<safe-version>

# Rebuild any affected Docker images from scratch
docker build --no-cache -t <image-name> .

Do not use a Docker image that had the malicious package installed. Even if you removed the package, the filesystem state of the image is unknown. Rebuild from scratch.

Step 6: Check your audit trail

If you use Veln, check the audit log for:

Every machine that installed the affected package
The exact version and timestamp of each install
Any other anomalous installs that may have been related

# Veln audit log (in local Console or via the API)
# Shows all install events for the affected package name

If you don't use Veln, check whatever logs you have:

npm build logs in CI
pip install output in Docker build logs
System package manager logs

Step 7: Notify affected parties

If the compromise involved production credentials with access to customer data, you may have breach notification obligations. Consult your legal team.

Notify your security team and, if you're part of an organization, your CISO.

If you discovered a previously-unreported malicious package, report it to the registry (npm security: security@npmjs.com, PyPI security: security@pypi.org).

Step 8: Prevent recurrence

The immediate response is over. Now prevent it from happening again:

Add Veln to every machine and CI pipeline. Install-time verification means future incidents are caught before the malicious code executes.
Switch to npm ci everywhere. If you're using npm install in CI, change it.
Add --require-hashes to pip installs. Or switch to uv or poetry with lockfiles.
Limit CI secrets. Use step-level secrets, not job-level. Use OIDC where possible.
Add Dependabot for security updates. For after-the-fact CVE patches.

The best incident response is the one you never have to run. Veln catches compromised packages before they execute.