Gemini's SQL validity collapsed to 2.33% on a foreign agent spec

Gemini's SQL syntax validity collapsed to 2.33% when it worked from a migration spec written by a different AI coding agent. Token F1 came out to 0.035, AST similarity to 0.015, in a test that swapped specs between agents to see what survived the handoff.

The numbers come from a paper posted to arXiv on Oracle-to-PostgreSQL migrations. Researchers ran 1,802 Oracle PL/SQL scripts, pulled from a corpus of 1,006 files, through three tools: Amazon Kiro, Gemini, and Copilot. Then they cross-tested, feeding a spec one agent wrote into a different agent's execution step. The worst pairing was Gemini running on a Kiro-authored spec.

Rewriting the foreign spec into Gemini's own format before execution closed most of the gap.

The gap matters for anyone chaining different vendors' coding agents on the same migration job, since this test covered only a few of many possible agent pairings. It's a narrow slice of the broader spec-interchange problem, but the collapse was steep enough to notice.