Amazon Paper Finds Tax and Legal AI Benchmarks Often Grade Correct Answers as Wrong
New Amazon paper finds that benchmarks used to grade AI models on tax, legal and business rules often mark correct answers wrong, because the tests’ own answer keys or grading code contain mistakes. Benchmarks like RuleArena and LegalBench test whether models apply written rules to tax, legal, and business cases. If the reference answer or the scorer is wrong, the score measures the benchmark's bugs, not the model. They audited 9 public benchmarks by tracing references back to the source rules, rerunning reference programs, and testing scorers with controlled answers. Then they rescored the same saved model responses after each fix. LegalBench's tax scorer deleted decimal points, so $259,487.13 was read as 25,948,713. Fixing it raised accepted answers from 8 or 9 to between 87 and 91 out of 96 for each of 3 models. Before trusting a low score on a rule-following benchmark, read the scorer code and check a few failed answers by hand.
Context
Bugs in answer keys and scorers can make models look far worse than they are, so low scores on these benchmarks may mislead.
Rohan Paul summarizes an Amazon paper that audited 9 public benchmarks for applying tax, legal and business rules, including RuleArena and LegalBench. It found mistakes in answer keys and grading code; LegalBench's tax scorer dropped decimal points, and fixing it raised accepted answers from 8 or 9 to 87 to 91 out of 96 for each of 3 models. The paper itself was not opened.