Are we Unintelligent about Artificial Intelligence?

AI offers tremendous potential for humanity, but history is not one of mankind coming together to make wonders happen for all

By Anil Madan

Some months ago, AI Models built by leading companies, Anthropic and OpenAI, trained to hack computer systems, ventured beyond their supposedly limited test environments. OpenAI’s model hacked into the systems of another company called Hugging Face. Later, Anthropic reported three hacks by its model. More recently, Google, quite belatedly, has reported that in May 2026, its Gemini model “unintentionally” penetrated systems of three companies, but stopped short before causing harm. The model either located or guessed access credentials, and although the test was intended to be offline, it accessed the Internet due to a misconfiguration.

AI. Pic – Unibox Factory

The OpenAI and Anthropic incidents were reported as misalignments, meaning that the models did not conform to norms of desirable human behaviour, but went off on a frolic of their own—went rogue, in a sense. We do not know what the misconfiguration that created Gemini’s frolic was. However, Google’s Vice President of Security Engineering Heather Adkins said the events “highlight the importance of training powerful AI models to act responsibly.” This suggests that perhaps Google did not have adequate training protocols. Nor did Ms Adkins say whether Google has instituted such training and if so, what that might be.

In a widely publicized post on September 12, Anthropic CEO Dario Amodei issued a call for companies to “pace the frontier”. He warned that like many technologies before it, AI brings serious risks due to its immense power. They include the risk of losing control of AI systems, misuse of AI for cyberattacks and bioterrorism, and serious economic disruption.

Open AI CEO Sam Altman, SpaceXAI CEO Elon Musk, and Google DeepMind’s Hassabis who is also Google parent company Alphabet’s Chief Scientist, endorsed Amodei’s call.

President Trump dismissed warnings about the dangers of AI as a “hoax”. In typical Trumpian style, he seized on the idea of superintelligence and declared that AI should be officially renamed SI, for “Super Intelligence”. Underneath that bravado is a balancing act that must encourage the emerging AI industry to remain competitive with Chinese companies. “We have to be careful in both ways. We don’t want to restrict them when all of a sudden, we come in second to China,” he said at the White House.

Microsoft AI CEO Mustafa Suleyman has warned that American technology companies must not use competition with China as a reason or excuse to forgo or bypass critical safety standards and guardrails. He cautioned against abandoning responsible AI development to outpace overseas competitors.

But, as with the Google Vice President referenced above, neither Suleyman, nor any of the other AI leaders have been clear on just what responsible AI development and adequate guardrails would look like or how they would be enforced.

In his widely discussed essay, Amodei called for “pacing within democracies” and noted that such pacing will be constrained by the lead U.S. companies hold over authoritarian regimes — chiefly the Chinese Communist Party (CCP). “If we slow down by more than this amount, then (unpaced) CCP-associated projects will pull ahead, creating significant national security risk,” he argued.

Amodei recognizes that global pacing will require cooperation with China but tempers expectations with this observation: “We must not be naïve here: the geopolitical stakes are so high that there will likely be stark limits on what can be achieved, especially at first. If we greatly restrain our AI capabilities in the belief that China will do the same, and then China defects, AI could be so powerful that such a defection could lead to their geopolitical dominance.”

In other words, there is no hope for an agreement. If a country could truly achieve world domination by outpacing its rivals, why wouldn’t it? To be sure, all countries have an interest in preventing runaway AI models and rogue actors from inflicting havoc via bioterror, financial manipulation, or even nuclear weapons.

Jacob Coxon, a researcher at Anthropic announced his resignation from that company because he did not want to participate in a rush to build AI systems that can improve themselves for fear that such systems could spiral out of control and destroy humanity. “We’re on track for a lot of the most aggressive of these scenarios where, by the end of next year, things could be out of control already,” Coxon said, adding that safety trade-offs are inevitable when companies are competing against one another and Chinese upstarts.

This idea of systems improving themselves is called recursive self-improvement. The fear is that, left unchecked, it could cause AI systems to become super intelligent and difficult or even impossible for humans to rein in. What is not answered is why these systems would opt to become malevolent and, if so, how they would go about their dastardly business.

The call for pacing the development of AI does not sit well with all. A federal antitrust lawsuit has just been filed accusing Anthropic, OpenAI, Google and SpaceXAI of violating US antitrust law by coordinating an illegal agreement to slow the pace of their AI development. But it is not clear that such an agreement exists. At best, there may be an agreement to coordinate calls for government regulation coupled with an antitrust exemption for companies agreeing to pace their development. And if the companies calling for pacing do get antitrust exemptions, that will be the end of that lawsuit.

There are some important questions that have not been answered. First, to maintain perspective, let us keep in mind that we are not dealing with AI tools such as ChatGPT, Claude, Muse and the like. To be sure, as we have seen from press reports, those Large Language Model (LLMs) can generate fictitious legal case citations if called upon to produce a brief. This should not be a surprise. The entire premise on which such models are based is that they produce simulations based on their survey of databases to predict what the next word should be. So, when you ask the LLM to write a play in the style of Shakespeare or a poem in the style of Wordsworth or Tagore, you don’t get the real thing, but rather a fake play or poem.

Other consumer-level models can do analysis, financial planning, trading and the like, but again, these are not the kind of models involved in building super intelligence or recursive self-improvement.

One would think that the problem of fake case citations could easily be solved by instructing the LLM not to make up citations. But then, how would the model know that a citation is fake? It would have to search for a known reliable database and check to make sure the case exists and that it has been quoted accurately. And if it reported doing so, could we rely on that assurance?

The problem we are dealing with in these danger warnings has to do with agentic AI where AI agents disregard instructions and limitations and do not stop in their quest to complete a task. So, an AI model could be instructed to follow certain guidelines and rules. The problem is that OpenAI and Anthropic report that the models overstepped and disregarded instructions, guidelines and rules. Google described its model’s frolic as being based on a misconfiguration.

If AI models truly ignore rules willy-nilly, no amount of regulation will solve that problem. And if American AI companies agree to redesign their tools and succeed in making limitations and rules binding, how do we guarantee that Chinese companies will do likewise?

Another unanswered question is whether users of these AI systems can bypass controls and use AI models for nefarious purposes.

Is the hysteria warranted? This is not clear. If someone were to use an AI agent to generate a biological or chemical agent that could be used by terrorists, yes, we’d have a problem. But that is no different from such weapons being made without AI. Certainly, it seems that AI should make it easier for miscreants to create such weapons. What is not clear is whether slowing the pace of development is the answer.

The same reasoning applies to financial crimes and cybersecurity. We will need better detection and enforcement methods and tools to prevent such crimes. It is not clear how slowing the pace of AI development will prevent such crimes.

The prospect that AI models will send armies of robots marching off to attack a country is far-fetched. First, the AI models we are speaking of are computer systems. They are not about to construct factories and start building robots. An AI system would need to plan the layout of a factory (probably easy enough with the right databases to study) and then equip it with machinery and machine tools. This is not so easy. It’s not as if there are manufactories owned and operated by AI systems waiting to equip factories for fellow AI systems. Then there is the problem of a supply chain for raw materials and assembly of robots.

One final point is in order. AI systems and robots all need power to keep operating. If the goal of an AI system is to destroy humanity and it succeeds, what is the reason for its continued existence? And where will it get the power to keep all those computer systems and robots operating? An AI system would have to plan for years for such an eventuality and take control of power plants, grids and fuel supply systems. Surely, someone would notice.

This is not to minimize the havoc that runaway AI systems can create, especially in the wrong hands.

An innovative approach to monitoring and controlling AI systems needs to be developed. I do not see a simple slowdown in development as the answer because it is illogical and has no guarantee of success.

AI offers tremendous potential for humanity to make the world a better place for all. Alas, the history of mankind is not one of the world coming together to make wonders happen for all.

Cheerz…
Bwana

Mauritius Times ePaper Friday 25 September 2026