Study Finds Inconsistent Functional Auto-Consistency in LLM Self-Explanations Across Model Families

Automated self-explanations frequently decouple from underlying model behavior under targeted interventions, creating operational blind spots across automated sentiment and review pipelines.

• A new research framework tested behavioral alignment. Researchers Alisson Rosa Pereira and Bruno Iochins Grisci evaluated whether token-highlighting explanations provided by models match their internal decision mechanics across IMDB and Steam datasets.
• Model families demonstrated stark variations in stability. Closed-source GPT-4o-mini aligned with expected consistency patterns, while open-source alternatives like Gemma3, Granite8B, and DeepSeek variants showed structural contradictions or over-sensitivity under intervention.
• Academic publication validated the findings. The 6-page study, featuring 2 figures and 1 table, was submitted to arXiv on October 23, 2026, and accepted into the IJCNN 2026 conference proceedings.

• Plausibility has historically masked functional inconsistency. Traditional evaluation protocols often focused on whether an AI's self-generated justification sounded logical to human readers, ignoring whether modifying those highlighted tokens actually altered the model's prediction.
• Three core metrics determine real explanation fidelity. The study operationalizes auto-consistency through sufficiency (retaining only highlighted tokens), comprehensiveness (removing highlighted tokens), and counterfactuality (inverting token semantics).
• Architecture scale and family dictate behavior. Open models such as Gemma3 and Granite8B held baseline sufficiency coherence but broke down under stricter perturbations, while DeepSeek variants failed baseline sufficiency or overreacted to semantic inversion.

The deployment of Large Language Models in classification pipelines rests on an unverified operational assumption: when a model highlights specific tokens to justify its classification, those tokens represent the actual drivers of the output. The paper submitted by Pereira and Grisci demonstrates that for multiple contemporary model architectures, this premise does not hold.

The experimental setup tests consistency through direct behavioral interventions. When an enterprise configures an LLM to classify customer sentiment or analyze product reviews on datasets like IMDB or Steam, the system produces both a category decision and a self-explanation highlighting influential words. The researchers isolated these highlighted tokens to measure sufficiency, stripped them to test comprehensiveness, and inverted their meaning to track counterfactual sensitivity. If a model genuinely relied on the highlighted tokens, retaining only those tokens should preserve the classification, while removing or reversing them should systematically degrade or shift the output.

The empirical results diverge sharply along architectural lines. GPT-4o-mini behaved according to functional expectations across all three test conditions. When presented with only its highlighted justifications, it maintained its conclusions; when those justifications were inverted, its behavior adjusted proportionately.

By contrast, open-weight deployments exhibited structural failures. Gemma3 and Granite8B preserved baseline coherence when restricted strictly to highlighted tokens, but their stability degraded once exposed to counterfactual and comprehensiveness interventions. DeepSeek model variants displayed more pronounced divergence: certain runs failed basic sufficiency tests entirely—meaning the model could not reproduce its own verdict when limited to the exact tokens it claimed were decisive—while other runs displayed erratic over-sensitivity under counterfactual adjustments.

The incentive structure behind model deployment creates a systemic friction point. Engineering teams deploy token-highlighting features because human operators require interpretability before automating classification decisions. However, because generating an explanation is an act of token generation rather than an introspection of the model's internal weights, the resulting output often mimics human rationalization rather than recording mechanistic causality. For enterprise systems relying on open-weight alternatives to manage classification costs, relying on self-explanations introduces unquantified functional risk into production pipelines.

• Machine learning engineers configuring open-weight pipelines: Teams deploying Granite8B or DeepSeek variants face unpredictable shifts if automated fallbacks depend on token-level explainability checks.
• Product operations leads relying on sentiment scoring: Platforms using automated Steam or customer review classification risk processing outputs where the documented explanation does not correspond to the actual classification trigger.
• Enterprise compliance auditors: Risk teams verifying automated classification accuracy cannot accept self-generated token attribution as formal proof of algorithmic fairness or operational safety.

• Independent replication across non-review benchmarks: Watch whether the functional consistency gap between GPT-4o-mini and open architectures persists across structured tabular, legal, or financial text datasets.
• Adoption of intervention-based evaluation metrics: Track whether production observability platforms replace static token attribution with sufficiency and counterfactuality stress tests.
• Fine-tuning frameworks targeting auto-consistency: Look for developer updates to Granite8B, Gemma3, and DeepSeek that explicitly optimize explanation faithfulness rather than superficial text coherence.