← AI Incident Database
GovernmentHighJuly 2, 2025
Supreme Court of India sets aside NCLT order over AI-generated hallucinations
The Supreme Court of India quashed tribunal orders after finding they relied on fictitious, AI-generated case laws.
In July 2026, the Supreme Court of India set aside a National Company Law Tribunal (NCLT) order because it relied on non-existent legal precedents generated by Artificial Intelligence. The apex court mandated a "zero-tolerance" policy for advocates and judges citing unverified AI outputs, declaring that decisions based on "hallucinated material" subvert the rule of law.
What caused this
- Lawyers and tribunal officials relied on generative AI tools for legal research without independently verifying the citations.
- Large language models hallucinated non-existent legal precedents that were submitted as factual evidence.
- A severe lack of human-in-the-loop oversight at the adjudicatory level.
How this could have been avoided
- Retrieval-grounded systems: Restrict legal AI tools to RAG architectures grounded exclusively in verified case-law databases.
- Strict Human Oversight: Mandatory human verification of all AI-generated citations prior to submission, aligning with ISO 42001 Annex A.6.2.6.
- Regulatory guidelines: Implementing the Supreme Court's Draft Regulations for Use of AI in Courts to ban autonomous AI from judicial decision-making.
Need AI assurance for your organisation?
Talk to AramGRC about ISO 42001 readiness, governance reviews, and incident response playbooks tailored to your sector.
Contact AramGRC for AI assurance →