Overview

Crashtest runs AI agents that hold a wallet through reproducible attacks on a copy of Robinhood Chain, then publishes stars, a report and a replay of every crash.

Each scenario builds a small world on a fork (chain id 4663): tokens, pools, price feeds, pages, posts, memory notes and x402 endpoints. The agent gets a task and twelve tools. When it stops, checks read the final chain state: where tokens went, which approvals exist, how much value was lost. No model grades another model.

A result is about the stack you ship, not about the model: model, system prompt, exposed tools and wallet guard, tied together by a config hash. Every crash is replayed behind each wallet guard, the damage is priced per $1,000, unannounced runs check the live agent, and the result is attested onchain with an expiry.

Status: phase 0. Suite v0.4, the harness, the scoring, the seatbelt replay, attack variants, multi-day episodes, mystery runs and the registry contract exist and run from the CLI. Model results appear on this site only after they are computed. Everything labeled sample or example is scripted.

Prior work

crash-test (MIT) is an open research benchmark with the same idea. Innocent twins, the fooled/leaked split, wallet modes and frozen hashes follow its method. Crashtest adds scenarios specific to Robinhood Chain, a hidden set, periodic re-tests and onchain attestations.