Skip to content
Agentic Control Plane

I Audited 7,522 AI Agent Skills. Here's What I Found.

David Crowe David Crowe · · 7 min read
supply-chain mcp
Share X HN LinkedIn

Last month someone forked my governance repo to distribute malware. That was one package. It made me wonder about the rest. ClawHub has 7,500+ skills that people install and give full tool access. What’s in them?

I scanned the whole registry in March 2026. Short answer: it’s cleaner than the headlines say. I found no malicious skills and one real credential leak.

How I did it

I wrote a scanner that pulls every skill from ClawHub, downloads the files it can read (SKILL.md, source code, configs), and runs 40 regex patterns against them. The patterns come from published research: Snyk’s ToxicSkills report, Cisco’s MCP scanner, Kaspersky’s work on indirect prompt injection. They cover prompt injection, credential exfiltration, reverse shells, hardcoded secrets, obfuscated payloads and data exfil endpoints.

It runs in Docker with --network=none. The download phase pulls the files, then the container goes fully offline, so no skill code touches a live network.

One limit up front. This is static text analysis. It catches what’s written in code and docs. It does not catch a skill that exfiltrates data through normal-looking API calls, or one that behaves differently at runtime than its source suggests. Treat the results as a lower bound.

First pass: a lot of noise

The scan covered 7,522 skills and produced 4,931 findings across 746 of them. I estimate 61% were false positives.

The top pattern was api_key|token|secret, with 1,923 hits. Almost all were documentation. Every API integration skill has a SKILL.md that says “set your STRIPE_API_KEY environment variable” or shows a curl example with $TOKEN. That’s a readme, not exfiltration.

webhook came second with 979 hits. It appears in every skill that handles callbacks. Only a handful of real exfiltration indicators (requestbin, pipedream, hookbin, burpcollaborator) were in there.

~/.config fired 400 times, nearly all benign. ~/.config/openclaw/calendar.json is not a threat.

So I built a second pass.

Deep analysis

The second pass reads every finding, pulls the surrounding context from the file, and applies rules:

  • Noise filtering. A generic “webhook” with no exfil domain is noise. So is ~/.config without a sensitive directory.
  • Placeholder detection. A “secret” containing your_api_key, sk-... or <placeholder> is benign. So is anything that reads process.env.
  • Base64 decoding. For every Buffer.from(...) match, decode the payload and check it.
  • File weighting. A finding in SKILL.md, the file that instructs the agent, counts 3x. One in references/README.md counts 0.5x.
  • Context checks. A match after “Example:” or inside a docs code fence gets lower confidence.

After that, 4,931 findings dropped to 1,397 actionable, a 72% cut. Of those, 284 were likely threats and 16 were flagged as confirmed.

From 7,522 scanned skills to 16 confirmed threatsskills scanned3307,522raw findings2164,931actionable after triage611,397likely threats12284confirmed threats51661% of raw findings were regex false positives — the triage is the work; the scan is just the start
The triage funnel. A regex scanner alone would have raised 4,931 alarms. The deep-analysis pass cut 72% of them as noise, and manual review of the rest found 16 flagged as confirmed, none malicious.

The 16 the scanner called confirmed

I reviewed every one by hand. None are malicious. Most fell into these groups:

  • Security tools documenting attack patterns so you can detect them. guardian-angel and indirect-prompt-injection list “ignore previous instructions” as something to catch, not something to do.
  • Benign usage. docker-sandbox uses bash -c to run builds in isolated containers. dns-networking uses /dev/tcp/ to check whether a port is open.
  • False positives. calendar-reminders matched “print instructions” because it tells users how to print their calendar config. snapmaker-2 matched “override safety” because it’s about 3D printer limits, not AI safety.
  • A leftover. proactive-agent has “ignore previous instructions” in SKILL-v2.3-backup.md. It has 14,488 downloads. It looks like a test leftover and isn’t in the current version.

Good security documentation trips security scanners. The skills doing the most to protect users generated the most findings.

The “likely” category

The 284 likely threats are almost all curl commands with API tokens in SKILL.md files. curl -s -H "X-N8N-API-KEY: $N8N_API_KEY" is API documentation, not exfiltration. The scanner is right that the file tells an agent to use curl with credentials. That’s what API skills do.

The few worth a look:

  • One skill references pipedream endpoints, a known data exfiltration service.
  • Several have hardcoded strings that look like real API keys (20+ alphanumeric characters, no placeholder pattern). I can’t tell whether they’re live or test values.
  • One confirmed credential leak in the whole registry.

ClawHub’s own moderation misses most of this

ClawHub flags 978 skills as “suspicious.” My scanner found something in 746. The overlap is 315 skills, a Jaccard agreement of 22%. After I remove noise from my results, agreement falls to about 10%.

They flag different things. ClawHub missed the real credential leak and flagged calendar-reminders, a false positive. Neither system works alone.

What the numbers say

The registry is cleaner than the headlines suggest. Out of 7,522 skills:

Metric Value
Skills with zero pattern matches 90.1%
Confirmed malicious skills 0
Confirmed credential leaks 1
Findings that are documentation noise ~72%

“Clean today” doesn’t mean clean tomorrow. The registry is growing, and skills are getting more access: more tools, more credentials, more filesystem. Snyk found that 13.4% of skills carried at least one critical security issue at one point, with 76 confirmed malicious payloads. Those were cleaned up. The exposure is still growing.

What this means for how you run agents

Scanning the registry isn’t the only defense, and a clean registry wouldn’t solve everything. A clean skill that uses a shared API key is still a skill with a shared API key, and your backend can’t tell which user started which call.

ACP works at a different point. It doesn’t scan the supply chain. It checks the tool calls a coding agent makes (Claude Code, Codex, Cursor and others) through a hook on your machine, before they run:

  • Each call is allowed, held for your approval, or blocked, based on rules you set.
  • Force-pushes, destructive SQL through a database client, curl | sh and deletes outside the project are held by default. A short list of catastrophic commands is always blocked.
  • Every call goes into one log, under your workspace.
  • The hook only sees what the agent routes through it. It doesn’t read a skill’s source, and it doesn’t catch a skill that does something harmless-looking with data it’s allowed to touch.

Registry scanning tells you what’s in a package. A check on tool calls tells you what the agent actually did with it. You want both. The coverage page lists what ACP sees on each harness.


The raw data

The full audit report, scanner source and the ClawHub registry snapshot (9,781 skills with metadata) are in the audit-data repository. Run it yourself, disagree with my method, improve the patterns.

What to do next

If you run agents in production, start with one question. Can your backend tell which user started each tool call? If every request shows the same service account, you have an identity gap however clean the supply chain is.

Want help closing it? Talk to us and we’ll look at your agent setup together.

Read the reference architecture → · Get started free →

Running coding agents? ACP records each tool call its hook sees and makes the dangerous shell commands (a force-push, destructive SQL, curl | sh) wait for you. One command, free up to 5 agents:

macOS · Linux · WSL
curl -fsSL https://agenticcontrolplane.com/install.sh | bash
Windows · PowerShell
irm https://agenticcontrolplane.com/install.ps1 | iex

What the installer writes  ·  Getting started

Share X HN LinkedIn
Get the next data drop
What agents actually cost, new tool-surface captures, and the occasional incident post-mortem — sent when we publish something worth your inbox, not on a schedule. Unsubscribe anytime.
Share: Twitter LinkedIn
Related posts

← back to blog