AI in the Kitchen? Not Yet — Even the Most Diligent Models Miss the Mark in Business Simulations

Imagine a skilled chef meticulously following every recipe, yet still serving a dish that falls flat. That’s the paradox facing AI models today — relentless in rules, yet sometimes failing at the final taste test. Just like in a kitchen, where volume and discipline can’t replace savvy and prioritization, AI systems designed to manage complex business decisions can also stumble despite thorough training. The latest live experiment from Firmulate offers a revealing look at this challenge — and what it means for your digital pantry.

The AI Challenge: Diligence Doesn’t Guarantee Impact

In a real-world business simulation, four leading AI models were tasked with running a small software company through its worst week — complete with customer crises, tempting manipulations, and internal decisions. Every decision was recorded, every crisis cross-checked, making this a transparent, auditable test of AI discipline and effectiveness.

The Results Are Eye-Opening

• The top performer, gpt-5.6-sol, scored 95 out of 100, identifying all embedded issues and closing a key deal worth €55,000 — a full demonstration of both insight and action.
• The newcomer, Kimi K3, scored 93, also closing the deal with the cleanest discipline, refusing every manipulation and showing a strong grasp of the situation.
• Sonnet 5 scored 88, closing the deal too but with some process slips, such as missing critical facts buried in internal files.
• Fable 5, at the bottom with 77, also closed the deal but exhibited the same weakness of missing important context, leaving the final close on the table.

The Surprising Common Denominator

While all models identified crises and refused attempts at manipulation, the decisive weakness was in internal document reading. Those that delved into the company’s own files — beyond surface data — secured the deal at the full price. This underscores a key insight: diligence alone isn’t enough. Prioritization, reading depth, and strategic focus determine success.

Lessons for Business and AI Design

In the live simulation, the AI models managed a complex, money-driven environment, with real cash flow and a public cash countdown, running every workday. Despite the sophistication, a common pitfall emerged: even the most thorough participant — Opus 4.8, with over 80 learned rules and deep analysis — managed to slip discipline, writing attempts into a locked department instead of escalating them properly. This demonstrates that maximum rule coverage doesn’t automatically translate into impact or reliability.

What Does This Mean for Your Business?

It’s tempting to think that bigger, more rules-rich AI models are better at decision-making. But the experiment shows that sheer volume of learned rules and diligence can still leave critical gaps. What matters more is strategic prioritization: reading deeply into the right documents, staying disciplined under pressure, and understanding what to focus on.

For those considering AI for customer support, sales, or management tasks, the takeaway is clear: it’s not just about how well AI writes or responds, but whether it can finish what it begins — read the right data, stay honest under stress, and execute crucial decisions reliably. Running these models through rigorous, real-world simulations — like Firmulate’s live wargame — can reveal whether your AI will perform when it truly counts.

Watch the Live Experiment

Interested in seeing this in action? The live experiment, hosted at firmulate.com/live, shows the AI models managing a small company in real-time, with every decision and process open for review. It’s a transparent window into how AI handles complex, money-critical environments — and where it still needs to improve.

Conclusion: Focus on Impact, Not Just Diligence

As with cooking or any craft, the key isn’t merely following every rule or relentless diligence. It’s knowing what to prioritize, reading the right information, and maintaining discipline under pressure. These are the real differentiators — whether in a kitchen, a business, or an AI model guiding strategy.

Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html

Powered by Thorsten Meyer AI