Same Score, Different Evidence: Decodability, Surface Sufficiency, and Causal Relevance in Code Models

Interpretability experiments must distinguish model mechanisms from measurement artifacts. We present a case-study-backed reporting matrix for linear probes over five identifier properties and roles in code models. The study includes three models and collectively covers all seven XLCoST programming languages without forming a complete factorial design. The matrix binds each claim to an estimand, comparator, matching unit, uncertainty statement, outcome, and falsifier. On Python, boolean occurrence type is decodable at 0.981–0.988 macro-F1, while a masked source-line classifier reaches 0.983 without a language model. Paired problem-clustered differences are −0.003 to +0.002, with intervals excluding probe advantages above 0.014–0.017. A full-residual patch at a class or structure query-name site fails its specified bidirectional gate in Qwen2.5-1.5B, recovering 0.009 to 0.020 of the matched behavioral gap. A third case exposes a failed control. Role-conditioned iterator renaming yields 0.998–0.999 F1 at embedding layer zero, consistent with a lexical code introduced by the intervention. These findings require distinct claims about decodability, surface sufficiency, and causal relevance.

A classifier can predict a property from a model's hidden states with almost perfect accuracy. That result leaves a central question unanswered. What, exactly, has the experiment established about the model?

A linear probe is a small classifier trained on representations from a frozen model. Its success shows that information about the target can be recovered under a particular readout. As work on the limits of probing has emphasized, this does not establish that the model computes an abstract concept or uses the decoded information to produce its answers. Experiments on task relevance make this distinction explicit.

Code offers a useful setting for examining the gap. Variable names, nearby syntax, and program behavior can all predict an identifier's role. A high score may reflect any combination of them. Prior code-probing research therefore provides a starting point for asking which comparisons can separate these explanations.

Our study uses three comparisons to examine what high probe scores establish. The boolean comparison asks whether local source syntax is already sufficient to predict the label. The iterator experiment examines whether a renaming procedure removes a naming cue or introduces a new one. The class-name intervention tests whether changing a model state changes a specified behavioral outcome. Each comparison constrains a different claim about what the model represents or uses.

Our contribution is a reporting matrix that connects each claim to its comparator, matched sample, uncertainty, and observed outcome. It records how a negative result narrows a claim and how a confounded control leaves a question unresolved. The three cases show why reporting a high score together with a list of completed controls is insufficient.

We study identifier properties in XLCoST, a corpus containing C, C++, C#, Java, JavaScript, PHP, and Python. The models are Qwen2.5-1.5B, Qwen2.5-Coder-1.5B, and StarCoder2-7B.

The targets cover accumulators, indices or keys, iterators, boolean occurrences, and class or structure names. These are related case studies, rather than interchangeable measurements of a single concept. Boolean labels, for example, distinguish assignment, conditional, loop, and return occurrences. Class or structure labels identify declared type names and references.

Coverage varies by target. Iterator experiments cover all three models and seven languages. Boolean experiments cover four languages, as do the class or structure experiments, with different language subsets. Earlier accumulator and index experiments use one model and a weaker evaluation protocol. The study is therefore not a complete model-by-target-by-language comparison.

What the controls ask

Randomized-label control tasks test whether a probe predicts the intended labels better than a specified random-label task. Renaming tests sensitivity to naming cues, provided the replacement procedure does not introduce label information. A surface classifier asks how much can be predicted without model activations. An intervention asks whether changing a model state changes a specified behavioral outcome.

These comparisons answer different questions. Passing one does not establish the claim tested by another.

On Python, the three boolean probes achieve 0.982, 0.981, and 0.988 macro-F1. Macro-F1 averages the F1 scores of the target classes. The probes also outperform their specified randomized-label controls.

Yet a character-based classifier using the enclosing source line reaches 0.983 after the variable name is masked. It has no language model or hidden states. Local syntax is sufficient for a similarly high score on this prediction task.

Comparing the same occurrences

We compare probe and surface predictions on aligned occurrences, with uncertainty computed by resampling whole source problems. The paired analysis covers 915 problems, with probe-minus-surface differences ranging from −0.003 to +0.002 macro-F1. All three 95% intervals include zero.

The result bounds the possible advantage under this comparison. It does not demonstrate exact equality. The intervals exclude probe advantages larger than approximately 0.014–0.017 macro-F1 over the selected masked-line classifier.

The choice of comparator matters. The masked line was the best of the reported surface feature families on this sample. Against the next-best Python comparator, a masked enclosing statement, the probe differences are +0.010 to +0.017. The sign also varies across languages, where the study does not supply paired uncertainty bounds. The supported conclusion is specific to Python and the selected comparator. It does not show that the model itself relies only on local syntax.

Iterator probes score 0.976–0.987 on the original Python programs. After a role-conditioned renaming intervention, scores increase to 0.998–0.999, and all three models select embedding layer zero.

This initially looks like strong robustness to misleading names. The construction of the intervention gives a different explanation. Target identifiers receive names from one pool, while other identifiers receive names from another. The replacement vocabulary therefore encodes target membership. Because a new probe is trained after each intervention, it can learn the new lexical rule.

Layer zero contains the input embeddings, before contextual processing by the transformer layers. Near-perfect prediction there is compatible with the naming shortcut. It does not establish that the probe has recovered a semantic role independently of names. The figure also includes random-noun renaming, which draws replacement names independently of variable role.

Role-agnostic single-character and numeric renaming reduce F1 by 0.046–0.072. A name-only classifier reaches 0.918 on the same 427 Python programs. These observations make naming cues relevant to the interpretation, although the surface comparison lacks a paired probe-minus-surface interval.

These experiments test what a refitted probe can recover after identifier names change. They do not directly test whether the language model memorized programs during pretraining.

A stronger renaming test would use a one-to-one replacement independent of the target label and preserve program meaning. It would also distinguish refitting a probe after renaming from asking whether the original probe continues to work. Those are separate claims about recoverable information and a stable readout.

Class or structure names are also highly decodable, with baseline F1 of 0.979–0.982 across the three models. To test a behavioral contribution, we intervene on the full residual state of Qwen2.5-1.5B at layer 18 at a query-name site. This follows the distinction between decoding information and testing it through activation patching.

The experiment uses 288 matched class-versus-function prompt pairs in 48 template clusters. The readout is the logit difference between True and False. Denoising inserts a class-source state into a function prompt. Noising reverses the intervention. Recovery divides the mean signed effect by the mean behavioral gap between matched class and function prompts.

The specified gate requires positive effects separated from controls, at least 5% recovery, and a mean effect of at least 0.10 logit-difference units in both directions. These thresholds are experimental design choices.

Denoising recovers 0.89% of the matched gap, with a 95% interval from −0.07% to 1.95%. Noising recovers 2.04%, with a 95% interval from 1.11% to 3.08%. These intervals come from resampling the 48 template clusters and both lie below the 5% recovery threshold. Their mean effects, 0.0087 and 0.0199, also fall below the 0.10 requirement.

The failed gate bounds the contribution at the tested site, layer, and readout. It does not establish that class information is absent or unused elsewhere. Nor does patching the full residual state identify a causal contribution from the particular direction learned by a probe.

The behavioral readout has a further limitation. Both prompt classes prefer True. The matched gap therefore measures a relative distinction, rather than a well-calibrated binary decision. The intervention tests whether that relative distinction transfers.

The contribution is a reporting unit that ties each claim to the quantity being estimated, its comparator, the matching unit, uncertainty, the observed outcome, and a test that could change the conclusion. This follows the motivation for falsifiable interpretability research.

A record of which controls ran is insufficient. The outcome must change the claim. A confounded control should remain visible as a confounded control. A negative intervention should state the effect it can exclude and the conditions under which that bound applies.

These cases illustrate the matrix, rather than validate it as a universal framework. Their common lesson is to report the comparison that makes a score interpretable.

The experiments use heterogeneous targets, incomplete language coverage, and language-specific label extractors without a common multilingual gold audit. Earlier accumulator and index results use token splits and test-selected layers, so they remain exploratory. Cross-language iterator transfer does not enforce one global problem partition and cannot establish language-independent abstraction.

The boolean bound is conditional on the selected surface comparator, and prediction-level inputs for its paired summary were not retained. The class or structure intervention covers one model, one principal layer and site, and one behavioral readout. Its gate was not registered in advance.

Within these limits, the study shows why high probe accuracy is not a common unit of interpretability evidence. Decodability, surface sufficiency, and causal relevance each require their own comparison.