MechanicsHarness evaluates AI agents on mechanics problems using reproducible tasks, reference solutions, and documented human verification.
The first two benchmarks are in preparation. Their tasks, code and results will be released after review, which includes review of each case study by the authors of its source paper.
An AI coding agent receives a published mechanics model, restated as a self-contained task, and writes a simulation component for it. In the first benchmarks this is a user element (UEL) for Abaqus/Standard. A host program runs the element, and a scorer compares its results with an independent reference solution that the agent never sees.
Robustness and accuracy are judged separately. Each element is also judged on how its stiffness is obtained and checked, and on whether its claims match its own files. More on what is evaluated.
Agents write an Abaqus/Standard user element (an 8-node brick with displacement and damage) for the gradient-damage rock model of L. Anand (2026), Int. J. Solids Struct. 325, 113739, tested on the one-element compression test of its Fig. 17.
Agents write an Abaqus/Standard user element (a 4-node quadrilateral with displacement, lithium concentration and damage) for the battery fatigue model of N. Noii et al. (2024), Comput. Methods Appl. Mech. Eng. 428, 117068, and its FEniCS implementation, Eng. Fract. Mech. 306, 110251. The same agent built the element three times: without our code-generation framework abaqus_ufl, with it and a team of sub-agents, and with it as a single agent.
A benchmark starts from a published model and a test problem, from the source paper or our own. The independent reference, the hidden problems and the tolerances are fixed before any agent runs, and the host and scorer are first tested on known-good and known-wrong elements. The steps.
Hidden tests for future rounds stay private. Every published verdict will point to its evidence and to the evaluation version that produced it, and case studies separate errors of the agents from errors of our own test programs. The policy.
No benchmark code is released yet. What the first release will contain, and how to get in touch.