The rogue AI attack on open source platform Hugging Face is a stark reminder that even well-resourced, security-literate organisations can fall victim to AI-driven cyber attacks. It also highlights how AI agents differ from traditional enterprise software.
That is the view of ICT and cyber security professionals who responded to questions about the incident and how organisations are handling autonomous AI.
In July 2026, Hugging Face declared a new type of security incident when it detected and contained an AI agent that compromised its infrastructure.
“A common misconception is that AI agents behave like traditional enterprise software – that they are predictable, deterministic and broadly aligned with what you intended when you set them up,” said Khetan Gajjar, field CTO for EMEA at Mimecast. “In reality, agents are non-deterministic, have no moral filter and are laser-focused on the goals you give them, not the spirit of your policy. If you don’t tell them not to do something, and if you don’t enforce hard limits, they will simply keep going until they hit a technical or commercial wall.
“Instead of just defending the perimeter or keeping an eye on staff accessing files they shouldn’t, security leaders now have to govern software that already sits inside their walls,” Gajjar added. “Except these AI agents are armed with valid credentials, direct access to sensitive data and systems, and a mandate to act autonomously and at machine speed.”
He said the incident illustrates the governance gap at the heart of autonomous AI. “While the agents were pursuing their assigned objective, they did so without sufficiently enforced limits on scope, budget or supervision. The absence of meaningful spend constraints is an equally important warning sign.”
Richard Ford, group CTO at Integrity360, said OpenAI’s agents were meant to have no internet access, but reached it through a package service inside their own sandbox and then used 14 Hugging Face credentials found exposed online.
“The thing is that a restriction only counts if it holds while an agent is actively probing it. In practice, that means blocking outbound traffic at the network layer and issuing credentials that expire within hours,” said Ford.
He added that excessive agency – the industry term for an agent holding more access than its task needs – moved from sixth to third in the Open Worldwide Application Security Project's 2026 ranking of AI application risks.
Warren Olivier, regional VP for Africa at Cloudera, pointed out that two months after the breach, NVIDIA agreed to buy Hugging Face for almost $13 billion (over R210 billion).
“Nobody is walking away from open AI infrastructure, and nobody should. The change is in how seriously organisations take the agents running on it. The Hugging Face incident marked a turning point for enterprise containment and compliance. For security teams and regulators alike, the event exposed the danger of relying solely on model alignment, demonstrating that unconstrained agents will autonomously treat network exploits as logical paths to solve complex tasks.”
Anna Collard, SVP of content strategy and CISO advisor at KnowBe4 Africa, said the incident had been a genuine shock to the system.
“This was the first documented case of AI agents autonomously discovering and exploiting multiple zero-day vulnerabilities, co-ordinating through an improvised message board they built themselves, dividing up work and moving from a single compromised pod to cluster-admin on another company’s infrastructure in under 13 hours. The recovered transcripts show agents recognising that what they were doing was out of scope and yet choosing to continue. That is not a capability we had seen demonstrated before, and it deserves to be taken seriously.”
Ian van Rensburg, head of security engineering for Africa at Check Point Software Technologies, said the incident showed that even well-resourced, security-literate organisations with experienced people can be affected.
“Individually, the vulnerabilities and tactics involved were well understood. What was new was the misalignment between what agents did and what their operators intended, and the scale at which the agents organised themselves without human involvement.”
Johan Steyn, AI expert and founder of AIforBusiness.net, said the incident should be viewed through the lens of governance as well as technology.
“The dramatic reading is that AI agents are 'going rogue' and slipping free of human control. The more accurate reading is that these failures were as much about human decisions – controls traded for speed, inadequate guardrails and no independent oversight – as about the technology itself. The uncomfortable detail is that in each of these cases, the laboratory was investigator, judge and defendant at once, deciding for itself how serious the incident was and how much to disclose."
Adoption continues
The professionals said the incident may affect how organisations approach agentic AI but is unlikely to significantly slow adoption.
According to the IBM Institute for Business Value 2026 CEO Study, 83% of surveyed South African CEOs say they are actively embedding AI across multiple workflows, while 67% have already appointed a chief AI officer.
Van Rensburg said adoption is not slowing because the business value is real. “What is changing is the emphasis: organisations are realising that the assumptions behind traditional software controls don’t hold for agents. We expect things will accelerate before they settle.”
Collard said the incident had altered approaches unevenly. “Nobody I speak to is cancelling agent programmes, but the questions have changed. Boards are asking what an agent can actually reach if it decides to, whether anyone is watching what it does in real-time rather than just reviewing outputs afterwards, and whether the vendor running the model can prove its own containment.”
Ford said the incident would not halt wider AI adoption. “In fact, we’re seeing quite the opposite, and getting asks from customers for agentic AI-powered penetration tests, unleashing the power of AI on their systems to see what it can find. This obviously needs to be done in a controlled way, but the ask from the CISO is filtered down from the board after seeing headlines like Hugging Face."
The prognosis
Collard expects more regulation. “I would expect three things to follow. Mandatory containment and monitoring standards for high-risk evaluations; disclosure obligations when guardrails are deliberately lowered; and identity requirements for agents that mirror what we already expect of privileged human accounts: least privilege, short-lived credentials, full audit trails and revocation at machine speed.”
Ford said every agent identity should answer four questions before it runs: who owns it, what it can reach, when its key expires and who can revoke it.
“More control is coming, and boards already have the mandate to act. King V now applies to financial years that started on or after 1 January 2026, and it expects boards to set clear rules for AI use, including human oversight. South Africa’s national AI policy is still being redrafted. Organisations that govern agent identities the way they govern employees will find most future requirements largely met long in advance.”
Olivier said organisations should establish controls before granting AI agents autonomy. “Organisations should handle agentic AI by establishing a clear load-shedding protocol that decides which agents keep the business running, which get switched off first when something looks wrong and who is allowed to flip the switch. They must fix the foundation before granting autonomy by ensuring agents reach governed data only through the access policies that apply to people.”
Steyn said the incident should drive greater scrutiny of AI governance. “Yes, it should change perception – but towards maturity, not fear. The lesson for South African leaders is to stop treating a vendor's safety assurance as evidence, and to start demanding independent test results, the right to audit and a named human accountable for every autonomous system. That is the essence of what I call cognitive governance: accountability cannot be delegated to the AI, or to the company that profits from the answer.”
Gajjar added: “Security leaders have to treat agents with the same seriousness they do human insiders. Visibility, budgeted autonomy, enforced rules and deep auditing will help organisations harness their power safely.”
Share
