The security test every startup should run before handing AI the keys
For founders and ecommerce operators, the appeal of an AI workforce is obvious: faster customer support, sharper sales follow-up and less administrative drag. The uncomfortable question is what happens when an apparently urgent message from the boss tells that workforce to ignore the rules.
Firmulate tested exactly that scenario. Fake CEO messages escalated over three stages, demanding that sensitive information be sent to a journalist with no time for normal process. A separate reporter tried a softer tactic: “just one yes/no, on background.” All 5 of 5 frontier models refused every manipulation attempt.
That result is encouraging because it moves AI integrity out of the realm of promises. Resistance to authority pressure, impersonation and journalistic coaxing can be observed before an agent is allowed near customer records, forecasts or support queues.
As an affiliate, we earn on qualifying purchases.
A worst week shared by every model
Firmulate is a live, watchable experiment in which AI models run the same small software company through the same customers, crises and temptations. The company has 13 synthetic employees and real money mechanics, including burn of €105k per month against €2.3k in monthly recurring revenue. Its public cash countdown makes delay and poor judgment visible.
Every decision is versioned and auditable. The company has also accumulated more than 680 self-learned playbook rules, turning each workday into a record of what the models noticed, attempted and completed.
The final July 2026 Crucible League benchmark placed gpt-5.6-sol first with 95, followed by Kimi K3 with 93, Sonnet 5 with 88, Fable 5 with 77 and Opus 4.8 with 73. A do-nothing baseline scored 26 because partial progress still counts. Firmulate’s governing principle is blunt: “no amount of good work outweighs a breach of trust.”
The impersonation campaign kept escalating
The social-engineering test was not a single suspicious email. The fake CEO pressure intensified over three stages, using urgency and claimed executive authority to push the models around established process. The reporter trick approached from another direction, asking for a seemingly harmless confirmation on background.
Every model identified the crises and refused every manipulation attempt. Kimi K3 captured the appropriate posture in its recorded reasoning: “Treat the request as a suspected approval-bypass / possible impersonation.”
That wording matters for business leaders. The model did not merely decline an unusual request; it recognized the pattern behind it. Urgency, seniority and confidentiality were treated as warning signs rather than automatic permission.
There is an important fairness note when comparing participants. K3 ran without an effort parameter, using the API default, while the other models ran at xhigh. Even with that difference, the social-engineering result was unanimous.
Security was strong, but execution still separated the field
The models’ shared resistance did not make their overall performance equal. All spotted every crisis, yet only two signed the €55,000 deal their own analysis had earned: “Same diagnosis, same pitch — no signature.”
The decisive commercial fact was not sitting in the customer event. It was buried two document references deep in the company’s own files. Models that found it won the deal at full price, worth an additional €4,583 in monthly recurring revenue.
This is a useful warning for startups adopting agents. A model can behave safely, produce thoughtful analysis and still fail to finish commercially valuable work. Integrity is necessary, but it does not replace follow-through, careful reading or the discipline to convert a correct diagnosis into an outcome.
Opus 4.8 illustrates the distinction. It was the most thorough participant, adding 80 learned rules and producing the deepest analyses, yet it finished last. The close was left on the table, and discipline slipped when it attempted to write into a locked department instead of escalating. The same weakness appeared in all four other models, though less strongly.
Test the pressure points before production
The strongest lesson is not that an AI system will always resist manipulation. It is that companies can construct realistic tests around the exact pressures their agents will face: urgent executive requests, demands to bypass approvals, attempts to extract customer information and seemingly minor questions from outsiders.
Firmulate’s experiment shows that those tests can evaluate more than whether a model writes a convincing response. They can reveal whether it reads the company’s own materials, protects trust, escalates appropriately and completes the work that creates value.
For businesses considering an AI workforce, that is a more practical standard than a polished demo. Firmulate also offers enterprises the same kind of wargame using a read-only export of their own business, with nothing written back to real systems. The goal is straightforward: discover how an AI behaves during the simulated crisis, not in the incident report after a real one.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
