A local judge has ordered an investigation into the conduct of lawyers who used generative artificial intelligence (GenAI) to present non-existent citations in court.
This, after in a Johannesburg High Court case, Northbound Processing brought an urgent application seeking interim relief to compel the South African Diamond and Precious Metals Regulator to release a refining licence that had already been issued but was withheld pending certain conditions.
The licence was tied to Northbound’s acquisition of a processing business from Rappa Resources in 2024. The regulator required confirmation that Rappa’s licence had been returned before releasing Northbound’s.
In court documents seen by ITWeb, on Monday, acting judge DJ Smit noted a troubling feature of this case was the inclusion of fake case citations in Northbound’s legal arguments, which the court attributed to “hallucinated” outputs generated by AI tools.
He criticised this practice, highlighting the dangers of relying on GenAI without proper verification.
The non-existent references undermined the credibility of Northbound’s submissions and were noted as an unfortunate misuse of AI in legal proceedings.
While preparing the judgement, Smit discovered that two cases cited in Northbound Processing’s heads of argument – offered in support of important points related to mandamus relief – were fictitious.
The acting judge sought clarification from the parties, and junior counsel for Northbound, Giles Barclay-Beuthin, initially explained that the wrong version of the heads had been submitted, attributing the issue to confusion caused by shorthand citations used during drafting. He also provided a corrected version in which the problematic paragraphs were replaced.
However, the attorney for the opposing respondents pointed out that the revised version contained two additional incorrect citations. He further questioned whether the existing legal references accurately supported the propositions they were meant to back. Northbound’s counsel was given another chance to respond.
Not so genius
In response to a direct inquiry from the court, Barclay-Beuthin acknowledged that the non-existent citations likely originated from an AI tool.
He said that due to pressing deadlines – including the urgency of the case, limited time for finalising the heads of arguments and the earlier withdrawal of another junior counsel − he had used a tool called “Legal Genius”, which claimed to be trained solely on South African legal sources.
The acting judge asked whether this situation aligned with a recent case in the KwaZulu-Natal Division (Mavundla), where non-existent cases were cited based on research conducted by a candidate attorney.
In Mavundla vs MEC: Department of Co-Operative Government and Traditional Affairs KwaZulu-Natal, the Pietermaritzburg High Court dealt with the consequences of legal practitioners submitting false case citations, most likely generated by AI, in their court filings.
According to law firm Michalsons, the Mavundla judgement is a warning to legal professionals about the uncritical use of AI tools in legal research and drafting. “It shows why thorough fact-checking remains essential, regardless of the source of the information, to maintain professional standards and uphold justice.”
Barclay-Beuthin argued that it was distinguishable from the Mavundla case, partly because the senior counsel, Arnold Subel SC, had not referenced the fabricated cases in court and no one suffered prejudice due to the mistakes.
Subel took full responsibility for the oversight, apologised, and expressed disbelief that the information had been generated as described. He said he had relied on his experienced legal team, including two junior counsel, and had only conducted a general review (sense-check) of the heads before submission. He believed the legal propositions involved were well-established and didn’t require citation. His oral submissions were independently prepared and did not rely on the written heads.
The court reiterated the professional obligation of legal practitioners not to mislead the court, even through negligence.
The acting judge endorsed reasoning from Mavundla and a recent English High Court decision, which warned of the dangers of relying on GenAI for legal research without verification.
The English court stressed that AI can generate convincing but false outputs, including made-up sources and quotes, and that legal professionals have a duty to verify any such content before using it in practice.
Smit emphasised that misuse of AI in legal proceedings can seriously harm the justice system and public trust.
The English High Court stated that disciplinary measures – including public reprimands, cost orders and referrals to regulatory bodies – may be appropriate, depending on the circumstances.
Apology accepted
Smit concluded that while he accepted the legal team’s apologies and found no intent to deceive the court, even unintentional errors in this context can have severe implications.
In line with the Mavundla case, he ordered that the conduct of Northbound’s legal practitioners be referred to the Legal Practice Council for investigation.
“In the context of legal research, the risks of using artificial intelligence are now well-known. Such tools can produce apparently coherent and plausible responses to prompts, but those coherent and plausible responses may turn out to be entirely incorrect,” said Smit.
“The responses may make confident assertions that are simply untrue. They may cite sources that do not exist. They may purport to quote passages from a genuine source that do not appear in that source. Those who use artificial intelligence to conduct legal research notwithstanding these risks have a professional duty to check the accuracy of such research by reference to authoritative sources, before using it in the course of their professional work (to advise clients or before a court, for example).”
The application by Northbound Processing is heard as a matter of urgency, the acting judge ruled.
Share