San Jose CA, commercial hub of silicon valley and its network of freeways.
Steve Proehl | The Image Bank | Getty Images
Despite the C-suite’s enthusiasm over artificial intelligence agents that can plow through office tasks like never-sleeping interns, the underlying technology is still rickety and a potential cost-sucker.
That much was clear this week during two separate events held in Silicon Valley, during which executives and engineers discussed the current excitement and challenges involving AI agents.
Kevin McGrath, the CEO of the AI startup Meibel said during a session that “the biggest problem that we’re working with in AI right now,” involves the misguided idea that everything needs to be processed by a large language model, or LLM.
“Just give all of your tokens and all of your money to an AI Claw bot that will just waste millions and millions of tokens,” McGrath said, before explaining how companies need to be more deliberate when deciding which tasks are best suited for AI agents.
Since the recent rise of OpenClaw, a so-called “harness” that lets developers use various AI models to create and manage fleets of digital assistants, the tech industry has been pushing AI agents as the next big thing.
Nvidia CEO Jensen Huang told CNBC’s Jim Cramer in March that it “is definitely the next ChatGPT.”
But on Wednesday at the Generative AI and Agentic AI Summit in San Jose, technical staff from companies like Google and its DeepMind AI unit, Amazon, Microsoft and Meta revealed that creating and operating AI agents is not an easy task.
One session led by Google software engineer Deep Shah focused on new techniques intended to help manage the operational costs of running tons of AI agents.
It costs money to run AI agents, and a poorly designed and maintained system to monitor those digital assistants and their actions could potentially end up burning cash instead of saving it.
“If you think of a machine learning system or any multi-agent system, there are multiple challenges you will find when you try to deploy that system at scale,” Shah said. “The first one is the inference cost.”
Ravi Bulusu, CEO of the startup Synchtron, pointed to the problem of complexity, noting the various ways companies organize data, choose tech platforms, and build and run software and their workforces.
Because running AI agents significantly touches all those points, “No single dimension is solved in isolation and the interdependencies are what make this hard, in fact chaotic even,” Bulusu said.
The theme of AI agent complexity continued on Thursday during an AI event held in Mountain View, Calif., that featured ThinkingAI and MiniMax, both headquartered in Shanghai, China.
ThinkingAI recently rebranded as an AI agent management platform, moving away from its genesis as a mobile game analytics company when it was known as ThinkingData.
As part of its rebranding, ThinkingAI partnered with MiniMax, which in January went public in Hong Kong. It is one of China’s leading AI labs and has released powerful models for free to the open-source community, becoming one of the country’s so-called “AI Tigers.”
ThinkingAI co-founder Chris Han said the shift to AI agent management tech is part of its efforts to expand from the video game sector to other industries that are excited about AI agents, but lack the expertise.
And despite OpenClaw’s growing popularity in China, Han said that it’s too complicated and too prone to security flaws for businesses.
“OpenClaw is a good tool for personal things, but definitely cannot reach the enterprise level,” Han said. “In terms of the enterprise level, you have to figure out a lot of things, your memory, how to manage your agents, teams, communications; there are a lot of things you have to figure out.”
Han declined to comment on any possible national security concerns over Chinese AI models that might impact ThinkingAI’s strategy, but said that the service can also support AI models from companies like OpenAI and Google.
If the U.S. government were to ban Chinese open-weight AI models in the country, Han joked he might take that as a good sign.
“If that happens, maybe we are successful,” Han said.
WATCH: AI demand metrics are broken and only Anthropic is being realistic.
