Scaling Up Interaction Detection for Large Language Models
Interpretability tools often fail as models and inputs scale. SPEX (Spectral Explainer) and its refined algorithm ProxySPEX address this by exploiting sparsity, low-degreeness, and hierarchy to identify influential interactions with far fewer ablations. Experiments show SPEX maintains faithfulness on thousands of features, explains a GPT-4o mini trolley-problem failure via high-order synergies, uncovers redundant and synergistic data interactions in ResNet, and improves attention head pruning on MMLU. Code is available in the SHAP-IQ repository; SPEX was published at ICML 2025.
Key Takeaways
- SPEX and ProxySPEX scale interaction detection in interpretability: They identify influential feature interactions in models with thousands of features—far beyond the dozens previous methods could handle—by treating the problem as sparse recovery using tools from signal processing and coding theory.
- Sparsity, low-degreeness, and hierarchy enable efficiency: The framework assumes only a small fraction of low-order interactions matter, and ProxySPEX further exploits hierarchical structure to match SPEX's performance with roughly 10x fewer costly ablations.
- Superior faithfulness at scale: On sentiment analysis, SPEX matched existing methods (Faith-Shap, Faith-Banzhaf) on short inputs but uniquely maintained performance as inputs scaled to thousands of features, where marginal methods like LIME degrade.
- It uncovers interactions that standard methods miss: For example, it explained a GPT-4o mini failure on a modified trolley problem as a high-order synergy among the words "trolley," "pulling," and "lever"—insight SHAP missed—and found redundant and synergistic training-data interactions in a CIFAR-10 ResNet.
- Practical applications and future directions: ProxySPEX-informed attention head pruning outperformed competing methods on MMLU (with interactions concentrated in later layers), and the team aims to unify the three attribution lenses and validate findings against established science in genomics and materials science. Code is available in the SHAP-IQ repository.
Interpretability researchers have long struggled with a scaling problem: the tools that explain what a model is doing tend to break down as inputs grow longer and models grow larger. A new framework called SPEX (Spectral Explainer), together with a refined algorithm called ProxySPEX, tackles this by finding the influential interactions inside a model's decisions using far fewer experiments than before, extending interaction discovery from dozens of components to thousands.
Why Interactions Are the Hard Part
Modern interpretability research splits into three lenses: feature attribution (which input parts drove a prediction), data attribution (which training examples shaped the behavior), and mechanistic interpretability (which internal components matter). Across all three, the same obstacle recurs: behavior emerges not from isolated parts but from dependencies among them.
The math is brutal. As the number of features, data points, or components grows, the number of potential interactions grows exponentially, making exhaustive analysis computationally infeasible. The researchers' approach rests on ablation — measuring what changes when a component is removed, whether that means masking input segments, retraining on data subsets, or intervening on the forward pass. But each ablation is costly, so the goal is maximum insight per perturbation.
The Structural Bet: Sparsity and Hierarchy
SPEX exploits two structural observations. First, sparsity: only a small fraction of possible interactions truly drive the output. Second, low-degreeness: influential interactions typically involve only a few features. These properties let the team reframe an intractable search as a sparse recovery problem, borrowing tools from signal processing and coding theory — strategically chosen ablations combine many candidate interactions into a single signal, and decoding algorithms then disentangle them.
ProxySPEX adds a third observation: hierarchy — if a high-order interaction matters, its lower-order subsets likely matter too. That assumption pays off dramatically: ProxySPEX matches SPEX's performance with around 10x fewer ablations.
What the Experiments Show
On a sentiment analysis task, SPEX matched the faithfulness of existing interaction techniques (Faith-Shap, Faith-Banzhaf) on short inputs, but uniquely kept that performance as context scaled to thousands of features. Marginal methods like LIME and Banzhaf also operate at scale but show significantly lower faithfulness because they miss interactions.
The framework also explained a striking failure: on a modified trolley problem stripped of moral ambiguity, GPT-4o mini answered correctly only 8% of the time. Standard SHAP blamed individual instances of the word "trolley," yet swapping in synonyms like "tram" changed little. SPEX instead identified a high-order synergy between both instances of "trolley" plus "pulling" and "lever." Replacing those four words with synonyms drove the failure rate to near zero.
Applied to a ResNet trained on CIFAR-10, ProxySPEX uncovered both redundant training-data interactions (a cluster of dog images with horse-like silhouettes influencing a horse prediction) and synergistic ones (an automobile test image defined jointly by a sports car, a yellow truck, and a red delivery vehicle). And for attention head pruning on the MMLU highschool-us-history task, a ProxySPEX-informed strategy outperformed competing methods and even improved performance on the target task. Analysis across depth showed early layers behaving largely linearly, with head interactions concentrated in later layers.
Where This Goes Next
The team points to two open directions: unifying the three attribution perspectives into a holistic view of a machine learning system, and validating interaction discovery against established science in fields like genomics and materials science. Code for both algorithms is available in the SHAP-IQ repository, with SPEX published at ICML 2025 and ProxySPEX at NeurIPS 2025.
Meta description: SPEX and ProxySPEX use sparse recovery to find influential interactions in LLMs, scaling interpretability from dozens to thousands of features with 10x fewer ablations.
Tags: interpretability, SPEX, ProxySPEX, LLM, feature attribution
Featured image: abstract technology style — layered translucent network grids with sparse highlighted nodes, no people, no logos, no text.
Comments
No comments yet. Be the first to comment.