SQLMorph Shows Heavy Abbreviation in a Question Costs Text-to-SQL Systems Up to 17% Accuracy

arXiv 2609.08950·low signalXBlueskyLinkedInCopy link

SQLMorph generates evaluation sets by mutating queries rather than collecting new ones, using Join Query Expansion to systematically raise structural complexity through valid join additions and Textual Query Augmentation to produce controlled natural-language perturbations. Applied to state-of-the-art systems, JQE reveals accuracy degrading as join count grows while TQA shows linguistic brittleness from heavy abbreviation reducing accuracy by up to 17%. It also replaces binary Execution Accuracy with Execution Precision and Execution Recall combined into an F1, which separates over-prediction from under-prediction and exposes system differences that binary metrics hide.