Preprint proposes test for when LLM hidden-state selection beats voting

A new arXiv preprint reports that CASE, a hidden-state selection method for large language model answers, outperformed majority voting by up to 19 points on medium-difficulty questions and 16.8 points on hard questions. The method trains a linear gate on answer-token hidden states and uses a decodability measure to predict when selection should outperform voting. The paper reports a 0.75 correlation between decodability and accuracy gains on held-out data, while warning that conventional probes can leak question identity.