AI and the legal function: where AI invents a ruling that doesn't exist — and how to keep a filing defensible

Of all the departments, legal carries the highest liability bar. It's the only one where the way to fail isn't slow adoption or uncertain ROI: it's accuracy. An AI tool that generates a legal text can produce a citation that is perfect in form — party names, court, docket number, holding — and entirely invented in substance. Not a typo: a ruling that doesn't exist, filed as though it did. It's the case where a single act can expose a practice to a sanction, a lost client and a disciplinary proceeding all at once.

This isn't theory. The now-textbook case is Mata v. Avianca (Southern District of New York, 2023): a lawyer used ChatGPT for research, filed a brief with model-generated, nonexistent precedents, and was sanctioned by the court. Since then the mechanism has a name — «hallucination» — and a literature. And this is where the only sensible conversation about AI in legal opens: not «how much time does it save me», but «how do I trust what it produces».

Even paid legal AI hallucinates: the Stanford RegLab numbers

You'd think the problem only affects those using a generic chatbot. It's the opposite, and it's the finding that should steer every choice. The Stanford RegLab measured the error rate of the AI legal-research tools from the two dominant incumbents: around 33% for Westlaw's AI and over 17% for Lexis+ AI. These are vertical tools, expensive, purpose-built for lawyers and sold precisely as «reliable» — and they hallucinate anyway, at rates that matter. One error in three, one in six: that's not a margin you can wave away in a filed document.

The practical reading is blunt: no tool — not even a premium legal one — shifts the responsibility for verification. It's the reason that, in this department, human control isn't a compliance accessory but the heart of the workflow. And the reason that legal research, though it's the most common use case — industry data puts it at around 74% of lawyers using AI for research — is also the most dangerous when left without structured control.

The 2026 sanction wave: a reported trend, to be read for the mechanism

Around this mechanism, the US legal press documented a genuine wave of sanctions across 2025–2026. They should be read for what they are — a reported trend, from specialist blogs and trackers, not facts we can independently verify from here — but the direction is consistent and instructive:

• As reported by the legal press, in Whiting v. City of Athens before the Sixth Circuit Court of Appeals attorneys were reportedly sanctioned around 30,000 dollars over 24 false or misrepresented citations — described as the highest federal appellate sanction on record for AI-fabricated citations.
• In the case involving the Morgan & Morgan firm (February 2025), again according to the coverage, of 9 precedents cited in a motion 8 were nonexistent.
• A tracker cited by the trade press reportedly counted over 1,200 cases worldwide in which AI-hallucinated content was filed in court; for the first quarter of 2026 alone the figure is around 145,000 dollars in sanctions tied specifically to AI-generated fake citations.

Taken singly they're anecdotes; taken together, and set beside the error rates Stanford measured, they describe a systemic risk, not an episodic one. The American Bar Association has made it the subject of explicit guidance. The point for an Italian SME or practice isn't the size of the individual American sanction — it's that the failure comes from the exact same gesture: trusting the output without verifying it.

The answer is governance: the human signature on every citation

The good news is that the antidote is known, and it isn't «ban AI». It's an operating principle as simple as it is strict: an AI-generated draft doesn't advance until an attorney has independently verified every citation and every source — checked one by one against the official database, not «read for plausibility». It's exactly what the ABA Formal Opinion 512 (July 2024) requires, the first comprehensive US national guidance: generative AI doesn't change existing ethical obligations — competence, candor to the tribunal, supervision. It only makes them more explicit.

It's a double-lock gate whose bolts must both throw together: the independent verification of the sources and the attorney's supervision. Neither one, on its own, is enough.

On top of this, Italy adds a formal duty the rest of the world is still chasing: the Legge 132/2025 (Italy's 2025 AI law) requires clients to be informed when AI tools are used in delivering a professional service, and the Consiglio Nazionale Forense (the National Bar Council) has issued a modello di informativa — a disclosure template lawyers must give clients. A generic consent clause in the engagement letter isn't enough — the same principle the US guidance made explicit: consent must be informed and specific, not boilerplate.

There's also an upstream mistake that, as reported, a US court has already penalised in the Heppner case: using a consumer-grade AI tool, whose terms of service allow data retention and sharing with third parties, reportedly caused attorney-client privilege and work-product protection to fall away. Translated for a practice: uploading a client's documents into the free chatbot isn't just a question of output quality — it can destroy the very confidentiality the relationship rests on. The tool must be chosen, not improvised.

And the EU AI Act? A gray zone, not automatically «high-risk»

An honest read of the EU AI Act avoids the alarmism. Annex III classifies the «administration of justice» as high-risk only when AI is used by a judicial authority to research and interpret facts and law and apply them. Law-firm tools — contract review, research assistance, drafting, scheduling — don't fall automatically into the high-risk category, unless they materially influence a decision affecting a person. The prudent reading: it's a gray zone, neither a ban nor a heavy obligation — but a case-by-case, DPIA-style assessment remains the wise choice, especially because the practice is the data controller for the client's personal data. Our compliance overlay hooks onto exactly this: which controls fit which use case, without either understating or inflating the obligation.

The economics for an SME: enterprise tools aren't for you

Here the industry note is clear and worth stating plainly: enterprise legal tools are too expensive and too seat-minimum for an SME or a small practice. Harvey AI starts at 100–200 dollars per user per month and climbs to 1,000–2,000 for the mid-market band, on quote only and with minimums of dozens of seats. Thomson Reuters CoCounsel effectively runs at 300–600 dollars and up per user per month once you add the mandatory Westlaw subscription, without which you can't buy it. These are figures built for large firms.

The band genuinely within reach of an SME is a different one, and it exists:

• Spellbook — around 99–199 dollars per user per month, a drafting and review plugin inside Word: the most realistic entry point for a small practice.
• Genie AI — free plan with 500+ templates, then 75 dollars per month for Pro (Enterprise from 600), with a template library across more than 150 jurisdictions.
• TheLawGPT — tiers from 19.99 to 89.99 dollars per month, the cheapest tool of those surveyed.

Where the return is most tangible is contract review: the first pass over a standard commercial contract drops 80–85% in time — and it's the case where, per industry data, 78% of legal teams say they're comfortable delegating it to an AI agent, but under attorney supervision. The other area where the price has already collapsed is e-discovery: per-document review cost has gone from around 1.50–3.00 dollars of manual work to 0.11–0.50 dollars with AI. An honesty caveat: these time savings are US and global figures — Italian-specific time or cost savings for legal AI are not available, so they should be taken as an order of magnitude, not as a promise calibrated to the Italian market.

One last caution on adoption, because it's easy to misread. AI use is sharply skewed by firm size: it's used by 46% of firms with more than 100 attorneys, 30% of those with 10 to 49, and just 18% of solo practitioners. These are three independent rates — they photograph different populations, they don't sum. But they tell a single story: the smaller the practice, the more adoption is left to the individual and the less it's governed. And it's precisely the small practice that a fake-citation sanction can damage the most.

The Italian market: already ahead, and the rules arrive fast

Unlike other sectors, on Italian legal the adoption data is rich. According to the Rapporto Censis–Cassa Forense 2026, 55.3% of Italian lawyers already use AI. The digital spend of professional practices is worth around 2.01 billion euro (+3% year on year), averaging around 10,500 euro per practice (Politecnico di Milano Observatories). AI in legal, in short, is no longer a frontier bet: it's already inside the majority of practices.

And it's perhaps the only field where Italy has legislated faster than expected. Beyond the Legge 132/2025, the Consiglio Nazionale Forense issued a client-disclosure template on the use of AI (the art. 13 schema), distributed to the local Bar councils from autumn 2025. The substantive principle running through it is always the same: AI for support and instrumental functions only, the attorney's intellectual contribution predominant, and the full ethical responsibility staying theirs. Whoever adopts AI in an Italian practice doesn't start from a regulatory vacuum — they start from an already-written framework, which must be known before the tool is chosen.

Where to start, in practice

If you work in the legal function — in a practice or in-house — and you want to bring in AI without exposing yourself, the sensible path is ordered and puts defensibility before speed:

• Put verification before adoption — decide now, before choosing the tool, that no AI-generated citation or source enters a filing without independent checking against the official database. It's the control that makes everything else sustainable.
• Choose the tool, don't improvise it — avoid consumer chatbots for client documents: professional confidentiality can depend on the terms of service. Start from the SME band (contract review) where the return is clear and the risk governable.
• Inform the client in writing — use the CNF template: in Italy the AI-use disclosure isn't a courtesy, it's a duty (Legge 132/2025), and a generic clause isn't enough.
• Keep the human signature on the filing — AI prepares and speeds up; the judgement, the responsibility and the signature stay the attorney's. That's what makes the result defensible, not just fast.

Even before choosing the tool, though, it's worth knowing where you are: our AI-readiness assessment helps you understand where to start with more return and less friction, and which controls to put around the first pilot. If the theme is the compliance of what touches client data, professional secrecy or duties to the Bar, our compliance overlay explains how we wire the controls to every design.

We've turned the first step into a self-serve, free assessment: a few questions and an indication of where to start, with what controls around it. Take the AI-readiness assessment — then, if it makes sense, let's talk.

This article is for orientation and does not constitute legal advice. The error rates of AI legal-research tools (~33% for Westlaw's AI, over 17% for Lexis+ AI) are measurements attributed to the Stanford RegLab; the Mata v. Avianca case (2023) is the established, verifiable precedent cited as an example of the mechanism. The US sanctions of 2025–2026 (Whiting v. City of Athens and the roughly 30,000 dollars over 24 citations, the Morgan & Morgan firm and the 8 nonexistent precedents out of 9, the more than 1,200 censused cases, the roughly 145,000 dollars of the first quarter of 2026) and the Heppner ruling on professional secrecy are reported by the US legal press and the American Bar Association: they should be read as a documented trend, not as independently verified facts. The Italian adoption and spend data come from the Rapporto Censis–Cassa Forense 2026 (55.3% of lawyers) and the Politecnico di Milano Observatories (digital spend at 2.01 billion, around 10,500 euro per practice); the duties cited derive from the Legge 132/2025 and the art. 13 client-disclosure template issued by the Consiglio Nazionale Forense. Tool prices and time savings (contract review −80–85%, e-discovery from 1.50–3.00 to 0.11–0.50 dollars per document) are US and global data: specific Italian savings figures are not available and should be taken as an order of magnitude. Amounts, thresholds, requirements and duties must be verified against the official texts and with a licensed professional before any decision; every tool choice and every automation that touches client data or professional secrecy must be assessed against the context of the individual practice.