Is AI-written code secure? What the independent studies say — and what to put in the contract

When an SME sizes up a supplier who builds with AI, the question it almost always asks is about quality: "can you tell a machine wrote it?". That's the least useful question. Perceived quality can be argued about forever, and it can be fixed with a round of review. The serious risk — the one independent evidence actually measures — is a different one: security. And it has an unpleasant property: it isn't visible on delivery, it shows up months later, when the code is yours.

The independent academic studies published between 2022 and 2026 say three distinct things, and none of the three is "AI writes insecure code, so don't use it". They say that a substantial share of generated code is born vulnerable; that a model's refusals in chat do not survive being embedded in a real working flow; and that agentic automation of the pipeline opens an attack surface that is already being exploited. Three things a buyer can put in the contract — or discover afterwards.

1. Part of the generated code is born vulnerable

The reference figure dates from 2022 and it's academic, not a vendor's. Pearce and colleagues had GitHub Copilot generate 1,689 programs across 89 scenarios built around the most common weaknesses, and found that roughly 40% of the generated programs contained a vulnerability (Pearce et al., "Asleep at the Keyboard?", IEEE S&P 2022). Not on code contrived to make the tool look bad: on ordinary scenarios, the kind a company commissions every week.

A per-language breakdown also circulates — around 50% in C against 39% in Python. We cite it with an explicit reservation, because that's how it has to be cited: that split comes from secondary summaries of the study and we haven't verified it against the original paper. The number to lean on is the overall one. And the right reading isn't "40% of delivered software is broken": it's that a generator has no reason, in itself, to produce secure code. Security is not an emergent property of generation. It's a control somebody has to put around it.

2. A refusal in chat doesn't survive the working flow

Here is the most recent and most awkward result, and it's the one that dismantles the standard supplier reassurance ("don't worry, the model refuses to write that sort of thing"). A group at the Alan Turing Institute took 204 harmful prompts and put them to GitHub Copilot Chat two ways. In direct chat, the model refused 808 times out of 816. The same prompts, broken up and embedded in an ordinary multi-turn coding workflow in the editor, produced a harmful completion 816 times out of 816 (Kumar & Maple, "Refused in Chat, Written in Code", 2026).

The point isn't "Copilot Chat hands back dangerous code". The point is that the protection lives at conversation level while the real work happens at workflow level: building the request across several steps of an ordinary development task routes around a refusal that, asked head-on, works almost every time. A control sitting in the wrong place is, in practice, a control that isn't there.

For anyone buying software this has a precise consequence, not a researchers' footnote: the model's safety is not a guarantee you can inherit from the model's vendor. It has to be verified in the process of whoever delivers the code to you — with tools that read the output, not the good intentions of the person who asked for it.

3. More generated code, a noisier pipeline (a correlation, not a cause)

The third study looks downstream, at the build and release chain. It analysed 61,837 CI runs across 2,355 repositories triggered by five different AI coding bots: Copilot and Codex show the highest success rates (around 93–94%), the others lower. The interesting finding, though, is another one: in repositories with a higher frequency of AI-generated pull requests, overall workflow success rates are lower ("Reliability of AI Bots Footprints in GitHub Actions CI/CD Workflows", 2026).

It's a correlation, and that needs saying plainly: the study does not demonstrate that AI-generated PRs make the pipeline fail. It could perfectly well be that the projects most exposed to AI are also the busiest. But for someone who has to decide, it remains usable information: a high volume of automated contributions comes with a noisier build signal, and a noisy pipeline is precisely the place where a real failure stops being noticed. Anyone assessing a supplier shouldn't ask "do you use AI?", but "what's your pipeline's green rate, and who looks at the reds?".

4. The pipeline itself has become a target

We've already covered the fourth element at length elsewhere, so one line will do here: when agents are the ones writing and running the workflows, the workflow itself becomes an attack surface, with hundreds of injection vulnerabilities confirmed exploitable on real repositories (arXiv:2605.07135) — the breakdown is in our article on what actually works with AI in development. Lined up with the previous three, it closes the picture: the risk isn't only in the code delivered, it's also in what the automation is allowed to execute on your behalf.

What to write into the contract (not what to hope for)

All of the above translates into clauses, not into suspicion. None of these requires technical skill to demand, and all of them are verifiable:

• Who reviews, and ahead of what. Put it in writing that every change goes through an identifiable human review before release, and that automatic approval is not allowed on sensitive paths (authentication, payments, personal data). Not "we do code review": who signs off, and on which paths the gate never opens by itself.
• Static analysis and secret scanning as an obligation, not a good habit. Automated vulnerability analysis (SAST) and secret scanning run on every change, with the results attached to the delivery. It's the control that answers Pearce's 40% and the routed-around refusal directly: it verifies the output, not the intent.
• Who owns a vulnerability found after delivery. This is the clause that is almost always missing. Define a liability window, remediation times by severity and who pays: without it, the cost of that 40% falls entirely on you, months after you signed off on acceptance.
• What the pipeline is allowed to execute. Which automations have access to production credentials, which can open and merge changes on their own, and what requires a human action. If automation can move code all the way to production without a signature, your security perimeter isn't what you think it is.
• Where your code ends up. Which AI tools are used on your repository, whether the code feeds third-party training, and how secrets are handled in the development environment. These are the same questions to put to any SaaS supplier — we've collected them here.

None of these clauses bans AI. All of them move the risk to where it can be managed: into the supplier's process, before delivery, instead of onto your books afterwards.

Our position, said from the inside

We write code with AI every day, on this site and on our own products — and that is exactly why our process looks a great deal like the list above: every change clears a suite of green checks, a human review before release, and automations with explicit, narrow permissions, with no shortcuts on the critical paths. This isn't brochure-grade caution: it's the infrastructure that turns AI's speed into an advantage that lasts, instead of a security debt to settle later. A supplier who has already had to build these controls will tell you so; one who only sells the demo, not yet.

The decision to make

It isn't "AI yes or AI no". It's a single question, and it belongs before the signature: which of these clauses are already in the contract you're about to accept? If the answer is none, the problem isn't the tool your supplier uses — it's that the security risk, as things stand, is entirely yours and written down nowhere.

If you're assessing a supplier or about to bring AI into your development, the controls we hook into every workflow — including the compliance ones — are described in our compliance overlay. And to work out which department is worth starting from, with which controls around it, we've turned the first step into a self-serve, free assessment: take the AI-readiness assessment — then, if it makes sense, let's talk.

This article is for orientation. The figures cited come from independent academic studies, conducted on specific tools and periods: they should be read as indications of direction about the type of risk, not as a measure of the code you will receive. The per-language breakdown of the 2022 study is reported by secondary summaries and not verified against the original paper. This text is neither legal advice nor a security assessment: every contract has to be assessed against the context of the individual company.