Artificial intelligence (AI) hallucinations pose a serious threat to the integrity of the legal profession, with lawyers risking their professional credibility if they rely on fabricated case law and unverified information generated by the emerging technology.
This was the word from former politician Dr Matthews Phosa, director and attorney at Frans Schutte & Mathews Phosa Inc, speaking last week at the LexisNexis Future of Law Conference, at the Sandton Convention Centre, in Johannesburg.
SA’s legal fraternity must embrace AI and digital technologies to modernise the administration of justice, but should guard against surrendering human judgement, accountability and constitutional principles to automated systems, warned Dr Phosa, who is the former Mpumalanga premier.
Responding to a question from ITWeb about the growing number of AI hallucination incidents in the legal fraternity globally and locally, Phosa said lawyers must exercise professional responsibility and uphold ethical standards when incorporating AI into their work.
“Presenting fabricated legal information to a court could expose practitioners to accusations of dishonesty and professional misconduct. How do you convince a judge with a lie? They [judges] are going to check, and you'll be exposed for lack of professional ethics," he asserted.
SA has already seen two significant AI hallucination incidents, highlighting the risks of relying on AI-generated information without verification.
In April, the Department of Communications and Digital Technologies withdrew the country's draft National AI Policy after up to six AI-generated references, including fictitious academic journals, were discovered in the document.
A few days later, the Department of Home Affairs suspended two officials after fabricated references appeared in its Revised White Paper on Citizenship and Immigration.
Growing concern
Phosa specifically highlighted the dangers of AI generating case law that does not exist, warning that legal practitioners cannot simply accept the technology's responses without independently interrogating their accuracy.
He said he actively encourages young legal professionals to examine AI-generated information, particularly when preparing legal memoranda and conducting legal research.
Phosa’s comments underscore the risks associated with the technology's tendency to produce convincing but factually-incorrect information, commonly referred to as AI hallucinations.
“For the legal profession, such inaccuracies could have serious consequences, particularly when fabricated judgements, legal precedents or references find their way into court submissions. We must teach our young ethical class to use AI responsibly.”
He added that deliberately presenting false information is unethical and undermines the responsibility that comes with being a member of the legal profession.
While raising concerns about the risks, Phosa maintained the legal fraternity should not approach AI with fear, or resist its integration into professional practice.
Instead, he believes the technology should be treated as a tool that can support legal professionals, provided they retain responsibility for the quality and accuracy of their work.
He pointed to the potential for AI to improve legal research, recalling the difficulties he experienced conducting research in the law library before the emergence of modern AI tools.
"Artificial intelligence presents extraordinary opportunities for the legal profession, and we must embrace it. It is not coming, it is here. It can improve legal research, accelerate court processes, strengthen compliance, and make legal information more accessible.”
Technology can now generate legal arguments within seconds, analyse complex contracts and process vast quantities of information at a scale that was previously unimaginable, he noted.
“These capabilities could help reduce the time and costs associated with legal services, while enabling legal professionals to handle increasingly complex workloads. However, you must never confuse a technologically-sophisticated answer with a legally-sound or morally-sound decision. There’s a big difference.”
Phosa also addressed concerns about AI replacing jobs, arguing that education institutions must adapt their teaching approaches to prepare students for a changing professional environment.
Also speaking at the event, Ian Andrews, LexisNexis CEO Southern and Eastern Africa, noted the future of law is a shared responsibility − one the legal profession cannot shape in isolation.
Law, he said, shapes how institutions govern, how organisations operate and how people experience justice, rights and opportunity.
"When we ask what the future of law looks like, we are also asking what kind of institutions, economy and society SA will have," said Andrews.
"Technology changes the possibilities. Trust and human judgement determine what we do with them.”
He emphasised that accountability using AI does not outsource professional responsibility, and urged institutions to build the capacity to change course as evidence changes, rather than try to forecast the next decade. “Stop predicting. Start adapting."
Share
