XRepoTest benchmarks repository-level test generation in Rust, Go, Julia, PHP and Ruby, and adds an Invocation Rate metric.

Fourteen models including Claude 4.5, GPT-5.2, DeepSeek V4-Pro and the Qwen families run under realistic repository constraints in a containerized framework with file-level, LSP-based and retrieval-based context strategies (arXiv 2608.25939). Invocation Rate is the metric to steal: it checks whether a generated test actually exercises the function it targets, rather than passing vacuously. Results show a substantial gap between standalone and repository-level performance, and richer context does not monotonically improve test reliability.