Legal AI hallucinations: how LIA marks the gap

A legal AI hallucination is an answer that sounds authoritative but is not supported: a case that does not exist, a quotation a judge never wrote, or a fact the record does not contain. Language models produce fluent text by predicting what usually comes next. Fluency is not the same as having a source.

What has gone wrong in court

In Mata v. Avianca, Inc. (S.D.N.Y. 2023), lawyers filed a brief citing decisions that did not exist. The citations had been generated by a chatbot, and the court sanctioned the lawyers.

In Australia, in Dayal [2024] FedCFamC2F 1166, a solicitor tendered a list of authorities prepared with an AI tool that contained citations that could not be found. The court referred the conduct to the Victorian Legal Services Board.

Courts have since issued guidance. The Supreme Court of New South Wales issued Practice Note SC Gen 23 on the use of generative AI, and the Supreme Court of Victoria has published guidelines for litigants. Check the current version of the rules in your court before you rely on any AI output.

Why it happens

• The model is asked for an answer, and an answer is what it produces, whether or not the material supports one.
• Citations have a predictable shape, so a plausible fake is easy to generate.
• Most tools blend fact, inference and guess into one confident paragraph, so the reader cannot see where support ends.

How LIA approaches it

LIA reads the record you give it and keeps three things on different lines. Evidence is what a document in the file actually shows. Inference is what may reasonably be drawn from it. Gap is what the record does not establish. Every answer has Show working, so you can see which document each line rests on.

When the file does not support a proposition, LIA marks it as a gap instead of filling it. That is the main defence against hallucination: an honest gap is more useful to counsel than a fluent guess.

LIA is not legal advice, and it does not replace checking authorities in an official reporter or database. Human judgment remains the authority.

A worked example (fictional matter)

The following matter is invented for illustration.

Question: Did the employer receive the complaint before the dismissal on 14 March?

Evidence: an email from the employee to HR dated 9 March attaches the complaint (Document 12). The dismissal letter is dated 14 March (Document 19).

Inference: if the HR inbox was monitored, the complaint was likely received before the dismissal.

Gap: nothing in the file shows that anyone at HR opened or read the 9 March email. No read receipt, reply or file note is in the record.

A fluent tool might write "the employer was aware of the complaint before dismissing the employee." LIA writes what the file shows, what it suggests, and what still has to be proved.

A checklist before you rely on AI output

• Open every cited authority yourself in an official source.
• Check every quotation against the original text.
• Ask which document each factual statement rests on.
• Treat any statement without a source as a gap.
• Follow your court's practice note on generative AI and disclose where it requires.