top of page
Search

Lawyers Continue to Cite AI “Hallucinated” Cases

  • 8 minutes ago
  • 2 min read

On June 22, 2023, the federal district court Judge in Mata v. Avianca, Inc., 678 F. Supp. 3d 443 (S.D.N.Y. 2023) fined a party’s counsel $5,000 for submitting case citations that had been “hallucinated” by ChatGPT and then engaging in conduct that the Court found to constitute bad faith in avoidance and in statements to the Court. This instance of lawyers citing AI-hallucinated cases possibly was the first recorded instance of such an occurrence, but it would not be the last.  (This situation is not an issue unique to the legal field.)  According to a database of legal decisions aggregated by Damien Charlotin, a senior research fellow Paris School of Advanced Business Studies (“HEC Paris”), the use of AI-hallucinated citations has been established or alleged in 1502 cases internationally since June 2023.  Although pro se litigants (individuals representing themselves without counsel) make up over half of the results, 394 cases involve lawyers in the U.S.  S. Melendez, “AI Keeps inventing fake cases.  Lawyers keep citing them”, Scientific American (online) (May 22, 2026), available at https://www.scientificamerican.com/article/why-lawyers-keep-citing-fake-cases-invented-by-ai/.


The use of AI in this way continues despite its now widely publicized penchant for inventing cases that do not exist.  A Boston University study involving students found that even forewarning AI users of its shortcomings did not completely eliminate their reliance on the tool for certain tasks, especially when things like deadlines created competing external pressure.  Id.


Although unverified trust in AI legal answers appears to be a widespread issue, lawyers have ethical requirements that make the uncritical use of AI legal results problematic. The American Bar Association’s Model Rules of Professional Conduct (“MRPC”), which are a highly influential model largely adopted (with varying modifications) throughout the U.S., include several relevant sections.  MRPC 1.1 requires that lawyers provide competent representation, with Comment [8] specifically stating to, “…keep abreast of changes in the law and its practice, including the benefits and risks associated with relevant technology…” MRPC 3.3 meanwhile requires that lawyers not knowingly offer false statements or evidence.


Further actions to address this problem also have been taken.  The very month that the lawyers in Mata were fined, a federal judge in Texas issued “Mandatory Certification Regarding Generative Artificial Intelligence,” a standing order prohibiting the use of AI to generate filings without a human verifying the results, the first of several such orders around the country.  Many state-level legal institutions have begun to issue specific guidelines to prevent AI misuse.  For example, in response to a December 2024 release by the Supreme Court of Illinois, which stated, among other things, that “Attorneys, judges, and self-represented litigants are accountable for their final work product,” the Illinois Attorney Registration and Disciplinary Commission (“ARDC”) released the “Illinois Attorney’s Guide to Implementing AI” in an effort to ensure the responsible use of AI tools.


Additional Resources



Read further about the Boston University study here: https://www.sciencedirect.com/science/article/pii/S2666557326000066





Read the Illinois ARDC’s The Illinois Attorney’s Guide to Implementing AI here:  https://iardc.org/Files/Implementing-AI-Guide/?page=1

© 2026 by The Law Office of John Ratnaswamy, LLC

10 S. Riverside Plaza, Suite 875, Chicago, Illinois 60606

View Accessibility Statement

The Supreme Court of Illinois does not recognize certifications of specialties in the practice of law and certificates, awards, and recognitions are not a requirement to practice law in Illinois.

bottom of page