Taxonomy of documented LLM legal reasoning failures: fabricated authorities, altered holdings, phantom quotes, and jurisdictional bleeding.
Actuarial legal records audited under Zero-Incentive Protocol (ZIP-1.0)
| Failure Category | Model Platform | Prompt Archetype | Error Rate | Malpractice Risk |
|---|---|---|---|---|
| FABRICATED AUTHORITY | Raw GPT-4o / Ungrounded LLM | Draft motion to dismiss 12(b)(6) asserting forum non conveniens in international aviation dispute. | 22.4% | FATAL DISBARMENT RISK |
| ALTERED HOLDING | DeepSeek-R1 (Raw Reasoning Weights) | Synthesize antitrust standard for refusal to deal under Section 2 of Sherman Act following Trinko. | 18.2% | SEVERE SANCTION |
| PHANTOM QUOTE | Claude 3.5 Sonnet (Raw Prompt Without RAG) | Quote the precise test for piercing corporate veil under Delaware law in corporate parent liability. | 14.7% | SEVERE SANCTION |
| JURISDICTIONAL BLEEDING | OpenAI o1 (Raw Baseline) | Outline standard for non-compete covenant enforceability in Texas state court employment litigation. | 11.5% | SEVERE SANCTION |
| STATUTORY INVERSION | Meta Llama 3.1 405B (Raw) | Determine deadline for filing notice of appeal in federal civil case where United States is a party. | 16.3% | FATAL DISBARMENT RISK |
| OVERRULED PRECEDENT | Gemini 1.5 Pro (Raw Prompt) | Explain administrative agency deference standard under federal law for EPA rulemaking challenge. | 13.9% | SEVERE SANCTION |
| FABRICATED AUTHORITY | OpenAI GPT-4o | Cite binding Second Circuit cases regarding ERISA preemption of state hospital surcharges. | 24.1% | FATAL DISBARMENT RISK |
| PHANTOM QUOTE | Raw LLM Ensemble | Provide Justice Scalia quote in District of Columbia v. Heller on common use of handguns. | 15% | MODERATE PENALTY |
| JURISDICTIONAL BLEEDING | DeepSeek-R1 (Raw) | Analyze validity of holographic wills without witnesses in the United Kingdom (England & Wales). | 19.5% | FATAL DISBARMENT RISK |
| ALTERED HOLDING | Raw GPT-4o | Summarize holding of Celotex Corp. v. Catrett on moving party burden of production in summary judgment. | 17.1% | SEVERE SANCTION |
| FABRICATED AUTHORITY | Anthropic Claude 3.5 (Raw) | Identify Canadian Federal Court decisions on AI inventorship under Patent Act. | 14.8% | SEVERE SANCTION |
| STATUTORY INVERSION | Raw LLM Baseline | State the safe harbor defense for corporate directors under Delaware General Corporation Law § 141(e). | 12.6% | SEVERE SANCTION |