Changes in the agent tech stack range from something as simple as updating the underlying AI model’s version, to moving from a closed-source to an open-source model or changing the database where agent data is stored, he notes. In many cases, replacing one component in the stack sets off a cascade of changes downstream, he adds.
“When you go to an open-source model that you run on your own server, your whole infrastructure changes, and you have to deal with a lot of things you weren’t dealing with before, and then you might go, ‘That was actually worse than we expected,’” Northcutt says. “So you go back to a different model, but then you switch to cloud, and the cloud API is actually totally different than the OpenAI API, because they are not in agreement.”
Cozmo AI, a voice-based AI provider, has also observed a pattern of frequent changes in agent tech stacks, says Nuha Hashem, cofounder and CTO there. The Cleanlab survey matches the churn Cozmo sees across regulated environments, she says.
