Skip to content

Wire Briefings Search

Fetching from the wire…

01

02

03

04

05

06

07

08

Wire Briefings Search

Entity trail

StarCoder-7B

Source-backed findings, relationship evidence, citations, and briefing history from the public MindPattern archive.

Briefing refs

0

Findings

1

Edges

1

Sources

1

Corpus findings

2026-09-10 / skill-finderDetecting benchmark leakage with code-specific features beat perplexity-only methods on eight code benchmarksCGMIA fine-tunes a shadow model on a subset of benchmark samples to build labeled member and non-member data, then extracts expert features per sample (CodeBLEU, edit distance, test pass rate, perplexity) alongside CodeBERT semantic embeddings, and combines them in a learned classifier. It outperformed eight existing membership-inference methods on most of eight code generation benchmarks, and detected known leaked APPS samples in StarCoder-7B's training data. The argument against the incumbent, DetectLeak, is that perplexity mostly reflects general familiarity with code patterns and degrades on complex or rare samples.

Graph relationships

USES
APPS
StarCoder-7B training data includes leaked APPS samples.
Source finding

Source trail

arXiv 2609.09865 — Keep Evaluation Fair: Detecting Data Leakage in Code Generation Benchmarks via Membership Inference Attacks

Graph sources

entity graphfindings textkg edgeskg entities