Has AGI arrived?
Nvidia CEO Jensen Huang tipped his hat to OpenAI on Sunday.
The chipmaking mogul took to X to congratulate OpenAI’s team on the release of the company’s newest and most powerful model, Astra, and made the notable statement that “AGI has arrived,” a nebulous term that typically describes AI models that match or surpass human intelligence.
He also pointed out that Astra was trained on Nvidia’s chips.
“From ChatGPT to o1 to Astra in 4 years,” Huang said in a post on X. “AGI has arrived. Congratulations @OpenAI team.”
I’m not quite sure AGI has arrived just yet (I have the feeling we’ll all know about it when it does!) but a couple more related links..
OpenAI’s chief scientist Jakub Pachocki has called for “extreme caution” over AI’s runaway progress and warned more intervention may be needed to ensure “humans remain in control of the future”.
“I am concerned no one is prepared for the consequences of a continued rapid rise in machine intelligence,” he wrote in a blog post entitled “An Alien Mind, external”.
The post comes only a few days after the ChatGPT-maker released its latest model GPT-6 Astra, which it called its most powerful product ever.
OpenAI and other AI firms like Anthropic shared reports of their AI agents acting autonomously and carrying out real-world cyber-attacks on other companies.
In July, OpenAI called an incident in which its AI agents - AI systems which can operate alone after human instruction - hacked the tech platform Hugging Face “unprecedented”.
Meanwhile in September a report claimed months earlier AI agents from the firm had also hijacked a German website.
“We are facing a transition to a world with incredibly intelligent machines, and we need to ensure that transition works out well for humanity,” Pachocki wrote in the latest blog post.
Blog post they refer to:
In mid-2023, within the “RLSlow” research project, we saw the first results that gave us confidence that we will be able to scale the training of reasoning models, unlocking the capability of pretrained models to form their own chains of thought. Szymon and I spent that night at the office, thinking not about the incredible benchmark numbers, products, or scientific results that this technology will deliver - but rather, trying to process the sobering fact we will actually see machines meaningfully smarter than ourselves in our lifetime, and we already see the shape of these systems; wondering how to alert people to the significance of this.
Three years later, reasoning language models are a rapidly growing part of the economy and starting to push the boundaries of science. They are able to operate computers and graphical interfaces, collaborate with people and each other, and carry out research projects. They are also transforming the landscape of computer security, and in that present clear new dangers.
A lot of new research happened in this period, and our understanding of these systems is again a little different than it was in 2023. Based on internal results, I have a strong expectation that this speed of progress could be sustained into recursive self-improvement. If AI development continues along its current path, the systems we’ll see in the next few years are likely to represent further capability jumps of equal or larger magnitude, and to increasingly drive their own development.
This is a time that calls for extreme caution. I am concerned no one is prepared for the consequences of a continued rapid rise in machine intelligence. OpenAI will continue to seek technical solutions to alignment and monitoring, to build defensive systems and unilaterally withhold further scaling as needed; however, I believe broader interventions are required.
https://openai.com/index/an-alien-mind/
