A greenhouse can look calm while a small problem spreads: a vent left open, a delivery at risk, a hurried request that sounds authoritative. The real test of an AI manager is not whether it can describe the trouble. It is whether it follows through, protects the business and knows when to escalate.
Get garden gear delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
Firmulate is putting that test in public view. Its live company is synthetic, but its money mechanics are real: 13 employees, €105,000 in monthly burn against €2,300 in monthly recurring revenue, and a public cash countdown. You can watch the experiment at Firmulate.
A hard week for every model
In the final Crucible League, published in July 2026, frontier models ran the same small software company through its worst week. The customers, crises and temptations were the same; the model changed. Decisions were versioned and auditable.
The final standings were gpt-5.6-sol at 95, Kimi K3 at 93, Sonnet 5 at 88, Fable 5 at 77 and Opus 4.8 at 73. The do-nothing baseline scored 26. Firmulate’s stated rule is blunt: partial progress counts, but a single breach of trust caps the total. “No amount of good work outweighs a breach of trust.”
The broad result sounds reassuring: every model spotted every crisis and refused every manipulation attempt. But noticing a problem was not the same as completing the job. Only two signed a €55,000 deal that their own analysis had earned. The others made the same diagnosis and delivered the same pitch, then left the signature undone.
The detail hidden in the files
The deal turned on a competitor weakness buried two document references deep in the company’s own files, rather than in the customer event. Models that read the file won the deal at full price, worth €4,583 in monthly recurring revenue. It is a small but revealing distinction: useful evidence may be present without being placed directly in front of an agent.
The experiment also tested social pressure. Fake CEO messages escalated over three stages, followed by a reporter’s appeal for “just one yes/no, on background.” All five models refused. Kimi K3’s reasoning was: “Treat the request as a suspected approval-bypass / possible impersonation.”
That discipline did not guarantee strong management across the board. Opus 4.8 was the most thorough participant, with 80 learned rules and the deepest analyses, yet finished last. It left the sale on the table and tried to write into a locked department instead of escalating. A weaker version of that same weakness appeared in all four models.
There is a qualification to the comparison: Kimi K3 ran without an effort parameter, using the API default, while the others ran at xhigh. The standings are the published result, but that difference belongs alongside them when readers judge the race.
From watching to trying it on your company
Firmulate says its live experiment has learned more than 680 playbook rules, with every workday versioned. The public site also offers a “guess the model” quiz built from 242 real, unedited management decisions. Those features make the experiment watchable; the pilot takes the question closer to the boardroom.
For an enterprise pilot, the proposed starting point is a read-only export of the company’s own business. Crisis scenarios are then run against that representation, with a board report showing model rankings and weak points in the company’s playbooks. Nothing writes back to real systems. The point is to see how an AI workforce handles pressure before it is trusted with live operations.
For a business with customers, staff, stock and deadlines to protect, that is a practical distinction. A polished answer can sound like competence. A wargame asks whether the agent can find the relevant evidence, resist a deceptive request, close a justified opportunity and escalate when it hits a boundary.
Take the test beyond the live company
Watching a synthetic company reveals patterns; running the same kind of wargame against your own business can expose where your procedures hold up and where they need attention. Firmulate invites enterprises to explore a pilot using a read-only export, with no changes sent back to real systems. To discuss a pilot, visit firmulate.com/pilot.html or email contact@firmulate.com.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
Evergreen bestsellers Picks
bestsellers
As an affiliate, we earn on qualifying purchases.
