Submission history From: Michael Bock [view email] [v1] Tue, 22 Jul 2025 00:37:59 UTC (6,688 KB)

Full-text links: Access Paper: View PDFHTML (experimental)TeX Source view license

BibTeX formatted citation × loading… Data provided by:

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Title:TaxCalcBench: Evaluating Frontier Models on the Tax Calculation Task

Focus to learn more arXiv-issued DOI via DataCite

Current browse context: cs.AI new | recent | 2025-07 Change to browse by: cs

BibTeX formatted citation × loading… Data provided by:

Bibliographic Explorer Toggle Bibliographic Explorer (What is the Explorer?)

Connected Papers Toggle Connected Papers (What is Connected Papers?)

scite.ai Toggle scite Smart Citations (What are Smart Citations?)

Links to Code Toggle CatalyzeX Code Finder for Papers (What is CatalyzeX?)

Huggingface Toggle Hugging Face (What is Huggingface?)

Links to Code Toggle Papers with Code (What is Papers with Code?)

ScienceCast Toggle ScienceCast (What is ScienceCast?)

Replicate Toggle Replicate (What is Replicate?)

Spaces Toggle Hugging Face Spaces (What is Spaces?)

Link to Influence Flower Influence Flower (What are Influence Flowers?) Core recommender toggle CORE Recommender (What is CORE?)

Link to Influence Flower Influence Flower (What are Influence Flowers?)

Core recommender toggle CORE Recommender (What is CORE?)

[Submitted on 22 Jul 2025] Title:TaxCalcBench: Evaluating Frontier Models on the Tax Calculation Task Authors:Michael R. Bock, Kara Molisee, Zachary Ozer, Sumit Shah View PDF HTML (experimental) Abstract:Can AI file your taxes? Not yet. Calculating US personal income taxes is a task that requires building an understanding of vast amounts of English text and using that knowledge to carefully compute results. We propose TaxCalcBench, a benchmark for determining models’ abilities to calculate personal income tax returns given all of the necessary information. Our experiment shows that state-of-the-art models succeed in calculating less than a third of federal income tax returns even on this simplified sample set. Our analysis concludes that models consistently misuse tax tables, make errors in tax calculation, and incorrectly determine eligibility. Our findings point to the need for additional infrastructure to apply LLMs to the personal income tax calculation task. Subjects: Artificial Intelligence (cs.AI) Cite as: arXiv:2507.16126 [cs.AI]   (or arXiv:2507.16126v1 [cs.AI] for this version)   https://doi.org/10.48550/arXiv.2507.16126 Focus to learn more arXiv-issued DOI via DataCite Submission history From: Michael Bock [view email] [v1] Tue, 22 Jul 2025 00:37:59 UTC (6,688 KB)

[Submitted on 22 Jul 2025] Title:TaxCalcBench: Evaluating Frontier Models on the Tax Calculation Task Authors:Michael R. Bock, Kara Molisee, Zachary Ozer, Sumit Shah View PDF HTML (experimental) Abstract:Can AI file your taxes? Not yet. Calculating US personal income taxes is a task that requires building an understanding of vast amounts of English text and using that knowledge to carefully compute results. We propose TaxCalcBench, a benchmark for determining models’ abilities to calculate personal income tax returns given all of the necessary information. Our experiment shows that state-of-the-art models succeed in calculating less than a third of federal income tax returns even on this simplified sample set. Our analysis concludes that models consistently misuse tax tables, make errors in tax calculation, and incorrectly determine eligibility. Our findings point to the need for additional infrastructure to apply LLMs to the personal income tax calculation task. Subjects: Artificial Intelligence (cs.AI) Cite as: arXiv:2507.16126 [cs.AI]   (or arXiv:2507.16126v1 [cs.AI] for this version)   https://doi.org/10.48550/arXiv.2507.16126 Focus to learn more arXiv-issued DOI via DataCite

[Submitted on 22 Jul 2025] Title:TaxCalcBench: Evaluating Frontier Models on the Tax Calculation Task Authors:Michael R. Bock, Kara Molisee, Zachary Ozer, Sumit Shah View PDF HTML (experimental) Abstract:Can AI file your taxes? Not yet. Calculating US personal income taxes is a task that requires building an understanding of vast amounts of English text and using that knowledge to carefully compute results. We propose TaxCalcBench, a benchmark for determining models’ abilities to calculate personal income tax returns given all of the necessary information. Our experiment shows that state-of-the-art models succeed in calculating less than a third of federal income tax returns even on this simplified sample set. Our analysis concludes that models consistently misuse tax tables, make errors in tax calculation, and incorrectly determine eligibility. Our findings point to the need for additional infrastructure to apply LLMs to the personal income tax calculation task. Subjects: Artificial Intelligence (cs.AI) Cite as: arXiv:2507.16126 [cs.AI]   (or arXiv:2507.16126v1 [cs.AI] for this version)   https://doi.org/10.48550/arXiv.2507.16126 Focus to learn more arXiv-issued DOI via DataCite

Subjects: Artificial Intelligence (cs.AI) Cite as: arXiv:2507.16126 [cs.AI]   (or arXiv:2507.16126v1 [cs.AI] for this version)   https://doi.org/10.48550/arXiv.2507.16126 Focus to learn more arXiv-issued DOI via DataCite

Bibliographic and Citation Tools Bibliographic Explorer Toggle Bibliographic Explorer (What is the Explorer?) Connected Papers Toggle Connected Papers (What is Connected Papers?) Litmaps Toggle Litmaps (What is Litmaps?) scite.ai Toggle scite Smart Citations (What are Smart Citations?)

Bibliographic Explorer Toggle Bibliographic Explorer (What is the Explorer?) Connected Papers Toggle Connected Papers (What is Connected Papers?) Litmaps Toggle Litmaps (What is Litmaps?) scite.ai Toggle scite Smart Citations (What are Smart Citations?)

alphaXiv Toggle alphaXiv (What is alphaXiv?) Links to Code Toggle CatalyzeX Code Finder for Papers (What is CatalyzeX?) DagsHub Toggle DagsHub (What is DagsHub?) GotitPub Toggle Gotit.pub (What is GotitPub?) Huggingface Toggle Hugging Face (What is Huggingface?) Links to Code Toggle Papers with Code (What is Papers with Code?) ScienceCast Toggle ScienceCast (What is ScienceCast?)