GitHub cut secret-scanning false positives 95% and published the eval methodology.

The three-tier structure is worth copying: false-positive reduction and precision as the primary outcome, recall held inside a defined guardrail as the safety constraint, and latency, cost, reliability and compatibility as operational guardrails (GitHub). The transferable practices are versioning prompt, model and system config on every run, changing one major variable at a time against a known baseline, and manually classifying failures by source: model reasoning, prompt framing, input construction, pipeline logic, dataset quality, labeling error. That last taxonomy is the part most eval setups lack, and it's the difference between "the model is bad" and knowing which of six things to fix.