Frequent Hallucinations Still Common in A.I. Models
With the rapid development, hype, and proliferation of artificial intelligence, many non-lawyers may be asking themselves the question of whether or not they even need to hire a lawyer at this point to resolve their legal disputes or represent them in court. While artificial intelligence has rapidly transformed the legal landscape, offering powerful tools for research, drafting, and analysis for lawyers and laypeople alike and is undeniably exciting, the technology still has its problems, as we have previously covered. Even with the technology advancing, a recent ruling from the Ninth Circuit Court of Appeals serves as a stark reminder that A.I. is far from infallible, and that the consequences of over-reliance can be severe. Recently, the Ninth Circuit sanctioned two licensed California attorneys for submitting briefs riddled with AI-generated hallucinations, fabricated case citations, invented quotations, and misrepresented legal authority. The court suspended them from practice before the court for six months, fined each $2,500, and required them to disclose A.I. use in all future filings. “Lawyers using generative A.I. must thus be aware of the tendency of generative A.I. to make these mistakes and guard against them,” the ruling said. If seasoned lawyers can fall into this trap, and face serious professional consequences for it, the risks for an untrained person acting on AI-generated legal advice are exponentially greater.
Appearing “Pro Se” and Relying on A.I.
Self-representation is indeed a constitutional right for anyone facing charges in the United States, otherwise known as appearing “pro se.” But the option is one that judges and legal experts consistently caution against, particularly in criminal proceedings. Criminal law is a complex web of procedural rules, evidentiary standards, constitutional protections, and strategic considerations that take years to master. An A.I. chatbot, however sophisticated, cannot assess the specific facts of a case with the judgment of a trained advocate, negotiate with prosecutors, read a courtroom, strategize with any real sophistication, or provide the kind of relationship-driven counsel that often determines outcomes. Worse, as the Ninth Circuit did and countless other cases have, individuals were sanctioned for overreliance on the technology because it is far from perfect. A.I. can confidently present fabricated legal authority as real, even when prompted not to, and a defendant with no legal training would have no reliable way to know the difference. Given the stakes in a criminal proceedings, including the potential loss of freedom, abuse of constitutional rights, reputational harm, and the countless other factors that are held in the balance of the outcome, the simple truth is that a defendant’s right to counsel exists precisely because the legal system is too consequential and too complex to navigate relaying on A.I. alone and without the benefit of a seasoned advocate. That being said, it is undoubtedly a beneficial learning tool or starting point for understanding some basic legal concepts, but is not an adequate substitute for an experienced criminal defense attorney in any way.
