Just Train AI with AI Output: What Could Go Wrong?
January 9, 2026
AI is still dumb technology and needs to be trained to improve. Unfortunately AI training datasets are limited. Patronus AI claims it has a solution to training problem and the news is told on VentureBeat in the article, “AI Agents Fail 63% Of The Time On Complex Tasks. Patronus AI Says Its New ‘Living’ Training Worlds Can Fix That.” Patronus AI is a new AI startup with backing from Datadog and Lightspeed Venture Partners.
The company’s newest project is called “Generative Simulators” that creates simulated environments that continuously generate new challenges for AI algorithms to evaluate. AI Patronus could potentially be a critical tool for the AI industry. Research discovered that AI algorithms with a 1% error rate per step compound a 63% chance of failure.
Patronus AI explains that traditional datasets and measurements are like standardized tests: “they measure specific capabilities at a fixed point in time but struggle to capture the messy, unpredictable nature of real work.” The new Generative Simulators produces environments and assignments that adapt based on how the algorithm responds:
“The technology builds on reinforcement learning — an approach where AI systems learn through trial and error, receiving rewards for correct actions and penalties for mistakes. Reinforcement learning is an approach where AI systems learn to make optimal decisions by receiving rewards or penalties for their actions, improving through trial and error. RL can help agents improve, but it typically requires developers to extensively rewrite their code. This discourages adoption, even though the data these agents generate could significantly boost performance through RL training.”
Patronus AI said that training has improved AI algorithm’s task completion by 10-20%. The company also says that Big Tech can’t build all of their AI training tools in house because the amount of specialized training needed for niche fields is infinite. It’s a natural place for third party companies like Patronus AI.
Patronus AI founds its niche and is cashing in! But that failure rate? No problem.
Whitney Grace, January 9, 2026
Comments
Got something to say?
-
- Subscribe to Beyond Search
Feature archive
News archive
-
