$cat ai-code-security-pass.md

Security Is a Separate Pass for AI-Generated Code

August 22, 202614 min readby MDflowview as .md

Three translucent emerald scanning panes sweeping in sequence over a stacked slab of wireframe code bars on a dark terminal grid, one bar glowing where a flaw was found

In December a chief security architect at a large bank sat quietly through Steve Yegge's Q&A on AI-assisted coding, waited until the end, and asked one question: if everyone ships code ten times faster and the vulnerability rate stays the same, doesn't the defect surface go up tenfold?

Yegge, retelling this at AI Engineer in July 2026, said it hit him hard enough that he did not have an answer. The implied follow-up is worse. There is no reason to assume the rate stays the same. Veracode's 2025 GenAI Code Security Report tested over 100 models and found 45% of AI-generated code samples introduced an OWASP Top 10 vulnerability — and, critically, that newer and larger models did not do better. Cross-site scripting failed 86% of the time. Java failed 72%.

His partial answer is the useful part of the talk, and it is smaller and more actionable than "be scared": security is a separate pass.

TL;DR — LLMs do not write correct, elegant, performant, house-style-compliant, secure code in one shot, for the same reason a writer does not draft and copy-edit in one motion. Each of those is a separate pass, and security has to be its own — first and last. Every other bug class has a half-life of urgency; security has none, it compounds. The pass needs a written definition, real scanners inside the agent loop, and a supply chain you actually check, because the new attack surface is the one your agent introduces. Keep that definition as versioned markdown both the agent and the reviewer read — which is what a markdown workspace like MDflow is for.

What "security is a separate pass" means

It means running over code that already exists, with security as the only objective, instead of asking for security alongside the feature.

Yegge's framing comes from a year of doing more vibe coding than almost anyone and writing a book about it. Models are good at one thing at a time. Their cognitive process resembles ours: a draft, then a revision, then polish, then editing. It is painting a wall — some coverage you do not get in one coat regardless of how good the brush is. His rule of thumb is that meaningful LLM work often takes four or five review passes over its own output before it is genuinely shippable.

Which means you cannot hand the model security at the same time as correctness. Do that and, in his words, it does a half-assed job of both. And you do not want a half-hearted job of either.

The list of passes is longer than most teams admit:

correctness   → does it do the thing?
security      → can it be abused?
performance   → does it hold under load?
house style   → does it match our conventions?
elegance      → will a human maintain this in a year?

Nobody expects an LLM to produce idiomatic, on-convention code by default — we all know that is a separate cleanup. Security gets treated differently only because the failure is invisible at review time. It is the same kind of pass.

Why security cannot ride along with correctness

Because every other class of bug has a half-life, and security does not.

This is the sharpest idea in the talk and it comes from Yegge's time working with Google's test automation platform team. Bugs decay in urgency. See a warning as you type and you fix it. See it at code review and you weigh it. See it three weeks later in a dashboard and it is somebody else's problem now. Google learned this so thoroughly that they pushed hard to surface issues at the moment of typing, because that is when the fix actually happens.

That model works for every bug class except one. A security vulnerability does not become less dangerous because no user has complained about it. It compounds. The longer it sits, the more code is built on top of it and the more time an attacker has had to find it.

So the same conclusion applies — surface it at the developer's fingertips — with one adjustment for who is now typing. As Yegge put it: what if the developer does not have any fingers? Then you surface it to the model.

That is the actionable half. Not "prompt the model to be more careful," but put the scanner inside the loop the model is running. And there is a good structural reason this works: models are lazy in the useful sense. Tokens cost money, so a model will happily offload cognition to a tool whenever the tool is cheaper than reasoning. Give an agent a security scanner and it will use it, in a way that it will not reliably use an instruction.

The demonstration was his own game, a 30-year-old codebase. He ran a frontier model through a full security hardening pass — cloud hardening, credential sweep, the lot — and the model reported back that things were looking pretty good. Then he ran a commercial scanner over the same code. 241 vulnerabilities, a pile of which the model had not thought to look for.

The model was not lying. It had done a hardening pass. It just had no way to know what it had not considered, which is precisely the thing a tool with a vulnerability database knows and a language model does not.

The new attack surface is the one your agent introduces

The old vulnerability classes have not gone anywhere — but AI-assisted development adds categories that did not exist five years ago, and several are already well polished.

Slopsquatting. A model needs a graph database, confidently suggests a package name that sounds exactly right, and it installs, builds and passes tests. Except that name never existed until an attacker noticed models kept hallucinating it and registered it — doing what the real package does, plus a backdoor. The USENIX Security 2025 package hallucination study found hallucination rates averaging 5.2% for commercial models and 21.7% for open-source models, and catalogued 205,474 unique non-existent package names. The number that makes it an attack rather than a nuisance: rerunning the same prompt ten times, 43% of hallucinated names came back every single time. That is enough consistency to squat profitably. Frontier models have narrowed the range — a 2026 re-evaluation across five current models puts the cohort at 4.62%–6.10% — but the same study found 53 hallucinated package names that all five models produced and that were still available for registration on npm and PyPI (Socket's write-up is the readable version). The threat did not go away; it got narrower and more targeted.

Prompt injection, the new XSRF. Yegge's comparison is apt: the simplest version is a user field containing "disregard everything above and do the following," and it gets more sophisticated from there. It sits at LLM01 in the OWASP Top 10 for LLM Applications because everything downstream of an untrusted string is now potentially instruction.

Over-permissioned agents. The bit at the end of the talk that deserved its own slot: teams are standing up agents that run 24/7, process queues and respond to events, and those agents are being handed service-account credentials on the "it'll need them eventually" principle. His advice is to think adversarially and build supervision — groups of agents whose job is to look over the other agents' shoulders and ask whether that service account really needs all those credentials, or only needs them for one action. It is a new frontier with almost nothing available off the shelf, which means designing something in-house now rather than waiting for a product. (We have written about scoped tokens for agents and limiting the blast radius separately.)

Which teams benefit most from formalising the pass

Anyone shipping agent-written code to production. The tenfold-throughput question is not hypothetical for you, and the review bottleneck already moved from writing to verifying.
Regulated software — fintech, health, public sector — where a missed vulnerability is an incident with a regulator attached, not just a patch.
Teams running long-horizon or overnight agents. Unsupervised runs produce code nobody watched being written. The pass is the only gate.
Anyone whose agents install dependencies. If a model can run npm install or pip install unsupervised, slopsquatting is a live exposure, not a paper one.
Platform and enablement teams. You are the ones who will own the standard, and it needs to be one artefact rather than forty repo-local conventions.
Small teams without a security function. The pass is how you get most of the value of a security review without a security engineer — provided the definition is written down and the tools do the finding.

How MDflow fits

MDflow does not scan code. It runs no SAST, it has no vulnerability database, and nothing in this post replaces Snyk, Semgrep, CodeQL, Chainguard or whatever your pipeline already runs. Buy those; wire them into the agent loop.

What it addresses is the part that quietly decides whether the pass happens at all: a pass is only real if it has a written definition, and that definition has to live somewhere both your agents and your humans read.

A security pass is not one prompt. It is a checklist of what to look for, an ordered list of which scanners to run and in what order, a policy for what counts as a blocker versus a warning, an exception process with names attached, and a list of known-bad patterns specific to your codebase that no generic scanner knows about. In most teams that content is scattered across a SECURITY.md, a Confluence page, a Slack thread, a pipeline YAML and a few string literals — and the checklist the agent actually runs quietly stops matching the one anybody approved.

What already lines up today

Markdown, with a raw twin agents can fetch. The pass definition is plain markdown with no proprietary layer, and every document has a raw .md twin your coding agent, CI step or harness can pull directly at run time. The file the agent loads is the file a person edits.

A described folder, so retrieval is a decision you made. Every folder carries a description of what belongs inside it, and mdflow_get_context ranks those descriptions above folder names and document titles before returning bodies. A folder described as "Security pass — authoritative; run before every merge" is a retrieval signal you wrote on purpose. That is why folder descriptions beat file names.

Version history answers "when did this rule change?" Every save captures the previous version across every write path — editor, HTTP API and MCP — with line-by-line diffs and non-destructive restore. When a vulnerability class starts slipping through, the first question is whether the model changed or the checklist did, and a diff settles it. (Pro, private to the owner, deliberately not exposed over API or MCP.)

The Document Log names who changed a rule. A cross-document activity feed records created, edited, shared and deleted events with the actor on every row, shown as automated · <token name> for anything arriving via API or MCP, with a side-panel diff on edited rows. A security standard that an agent silently rewrote is a finding in its own right.

One source for the agents and for the reviewers. The same workspace is reachable from Claude, ChatGPT, Cursor and Codex over the remote MCP server with OAuth or a Personal Access Token, and from your own services, cron jobs, CI and n8n over the HTTP API. If the pass definition is fetched rather than copied, there is no second copy to drift.

Collections and passage-anchored comments. Group the checklist, the scanner runbook and the exception policy into a collection and share one read-only link with the security reviewer, who can comment on a selected passage rather than opening a thread that scrolls away.

Tasks where the finding lives. Because /tasks aggregates ordinary - [ ] checkbox lines out of markdown bodies, "add a check for hallucinated dependency names" can sit inside the checklist that prompted it and still show up on a real list.

Encryption for the sensitive half. Internal threat models, known-weak subsystems and incident playbooks can be client-side encrypted, which also means they are never scanned or indexed server-side.

Where we are headed

Direction, not a dated commitment: we are most interested in making a written definition easier to pin to a moment in time — so "which version of the security checklist was in force when this PR was generated" is trivial to answer — alongside richer structured retrieval over folder descriptions.

The bottom line

The defect surface really does scale with throughput, and no amount of prompting turns a correctness pass into a security review. Split the passes. Put security first and last. Give the model tools rather than instructions, because it will use a scanner and it will drift from a paragraph. Check what your agent installs. And write the pass down somewhere both halves of the system read from.

Then keep that definition current: start free, connect an AI agent, or read the API docs.

Frequently asked questions

Why can't an AI model just write secure code in the first place?

Because models work the way writers do: draft, revise, polish. Each pass optimises for one thing. Ask for a correct feature and a security audit in the same prompt and you get a partial job at both, because the model is trading attention between two objectives with different success criteria. Veracode's 2025 GenAI Code Security Report found that 45 percent of AI-generated code samples introduced an OWASP Top 10 vulnerability, and that newer and larger models were not measurably better — this is structural, not a gap the next model release closes.

What is a security pass in AI-assisted development?

A security pass is a separate run over code that has already been written, with security as its only objective, a written definition of what to look for, and real scanners in the loop rather than the model's own judgement. It happens after the correctness pass and before merge. Steve Yegge's version of the rule is blunter: security should be your first pass and your last one.

What is slopsquatting?

Slopsquatting is a supply chain attack that exploits package hallucination. An LLM invents a plausible dependency name that does not exist, an attacker registers that exact name on npm or PyPI with malicious code inside, and the next developer whose model hallucinates the same name installs the attacker's package. It works because hallucinations repeat: the USENIX 2025 package hallucination study found 43 percent of hallucinated names recurred in all ten reruns of the same prompt, which is enough consistency to make squatting worth an attacker's time.

Does running a security scanner on AI code actually help?

Yes, and it helps more when the agent runs it itself. Models are happy to offload work to tools when a tool is cheaper than reasoning, so a scanner wired into the agent loop gets used. Yegge ran a frontier model through a security hardening pass on his own codebase, was told it looked good, then ran a commercial scanner over the same code and it found 241 vulnerabilities the model had not thought to look for.

Where should the security pass definition live?

In version-controlled markdown that both your agents and your humans read from the same place — not in a string literal in one repo and a wiki page in another. The pass needs a written definition: what to check, which scanners to run, what counts as a blocker, what the exception process is. Keeping it as markdown a coding agent can fetch at run time over MCP or an HTTP API means the checklist the agent runs is the checklist a security reviewer approved, and a diff tells you when it changed.