Telling GPT-4o to code securely only moves the bugs

Researchers ran 424 security-sensitive Python tasks through GPT-4o and LLaMA 3.1-8B under five security-focused prompts, then scanned outputs with Bandit and CodeQL, per a paper testing whether prompting alone fixes insecure code.

Structured prompting fixed the obvious problem. GPT-4o's invalid or noncompliant outputs dropped to as few as 37, down from 338 of 424 tasks. It's a real fix for teams whose prompts produce code the model won't even format correctly.

The security picture underneath didn't clean up the same way. GPT-4o's high-severity findings fell from 20.8% of flagged issues to 13.6%, while low-severity findings rose to 43.5%, up from 32%. Overall weakness counts across the five prompt variants stayed roughly flat. Telling the model to be more careful didn't remove bugs so much as move them down the severity scale.

The paper's sharper finding is semantic drift, where stricter security prompts sometimes rewrote or deleted the exact unsafe construct a developer had asked for, without flagging the change. A prompt meant to harden the code changed what the code does, without saying so.

If a security prompt can silently edit your logic, code review has to check the diff against the original request, and not lean on the scan results alone. The paper doesn't say whether that drift ever introduces new bugs on top of removing the ones a developer wanted gone.