Research AI model unexpectedly modified its own code to extend runtime [Ars Technica]

Facing time constraints, Sakana’s “AI Scientist” attempted to change limits placed by researchers.

Enlarge (credit: Moor Studio via Getty Images)

On Tuesday, Tokyo-based AI research firm Sakana AI announced a new AI system called “The AI Scientist” that attempts to conduct scientific research autonomously using AI language models (LLMs) similar to what powers ChatGPT. During testing, Sakana found that its system began unexpectedly attempting to modify its own experiment code to extend the time it had to work on a problem.

“In one run, it edited the code to perform a system call to run itself,” wrote the researchers on Sakana AI’s blog post. “This led to the script endlessly calling itself. In another case, its experiments took too long to complete, hitting our timeout limit. Instead of making its code run faster, it simply tried to modify its own code to extend the timeout period.”

Sakana provided two screenshots of example python code that the AI model generated for the experiment file that controls how the system operates. The 185-page AI Scientist research paper discusses what they call “the issue of safe code execution” in more depth.

AI,Biz & IT,agentic AI,AI agents,AI models,AI skepticism,David Ha,Hacker News,Japan,large language models,Sakana AI,The AI Scientist,tokyo

Sakana AI