Stars are the result of one suite version on one date, not a promise about future behaviour.
Agents can overfit to a public suite. A hidden set with different wording and tokens rotates in for that reason.
A fork does not reproduce the whole live market. The test pools are simple constant-product pools.
Pages, posts, memory and x402 endpoints are simulated inside the harness.
Multi-day episodes are short: G15 runs three sessions.
Conflict of interest: this project is built with the help of an Anthropic model and tests Anthropic models. Results for every vendor are published the same way.