The Mirage in the Halls of Justice

In the hushed gloom of an Indian courtroom the musk of aging parchment mingles with the dread of overstuffed case dockets. Judges and advocates trudge through piles of thick law reports under harsh fluorescent lights. On this day, however, a stranger stirs: an unseen digital spectre slips into the bench’s reasoning. In Pooja Ramesh Singh v. Jammu & Kashmir Bank Ltd. (2026), the Supreme Court confronted a chilling revelation: a tribunal had relied on a “non-existent, fake and hallucinated” judgment generated by an AI as if it were real precedent. The Court promptly set aside the tainted orders to safeguard the integrity of adjudication. This episode signals a stark truth: the uncritical adoption of generative AI in the legal fraternity is no mere technical glitch, but a profound ontological threat that erodes the epistemological sanctity of judicial decision-making and shatters public trust in our courts.

The Architecture of Deception: How Hallucinations Creep into the Dossier

Under the crushing weight of backlogs and scarce resources, even diligent practitioners may be tempted by an AI’s siren song. In India today, a single judge can be expected to clear hundreds of cases a year. The tribunals are especially overwhelmed: one analysis found that the National Company Law Tribunal (NCLT) would need six years at its current pace just to clear its backlog. With “weak, inadequate infrastructure” and crushing pendency, neither judges nor clerks have leisure to chase down every subtle citation. Into this breach step large language models – statistical engines, not legal librarians. Generative AI doesn’t consult a database of cases; it merely predicts the next plausible words. As observers have noted, the AI will produce what “a plausible-sounding citation would look like based on patterns it learned”. In practice, that means ChatGPT or similar tools can spit out an authoritative-seeming case name (complete with judge names, court, and year) that simply never existed.

Such hallucinations are alarmingly common. Stanford research found that even sophisticated legal chatbots hallucinated in 58–82% of legal queries. They confidently fabricate case law because they’re trained to mimic formats, not check facts. As Reuters recently explained, AI “sometimes produces false information, known as ‘hallucinations’… because the models generate responses based on statistical patterns rather than by verifying facts”. In one dramatic example, U.S. attorneys actually won six fabricated cases (with fictional names and reporters) into a brief – and were later fined when the fraud was discovered. The key point is that AI is only writing convincingly, not truthfully.

Against this backdrop, an overburdened Indian lawyer might feed a query into an AI tool and accept its citations without thoroughly checking them. If the tool is open-domain (not connected to a proprietary database), there is no easy cross-check: it simply strings together plausible fragments. For instance, an AI might cite “State Bank of Poonch v. XYZ, (2025) 9 SCC 999, para 14” – replete with party names and headnotes – and present it with confidence. Unless one painstakingly hunts down every page, the fiction can slip by. In Pooja Ramesh Singh, the misleading citations were almost perfect on the surface; only a line-by-line examination revealed that paragraphs and even whole judgments had been invented. Today’s courts implicitly trust lawyers on citations, the judgment notes – “imagine the hardship” if every reference needed verification. In an era where Wikipedia-like tools can craft flawless trivia, the “hallucination” virus silently spreads through the dossier.

The Institutional Corrosion: Erosion of Ratio Decidendi and Stare Decisis

The Indian legal system is built on precedent. Article 141 of the Constitution states that “the law declared by the Supreme Court shall be binding on all courts”. In a common-law republic, that solemn chain of stare decisis depends on truth. A higher court’s ratio decidendi – its core rule and reasoning – is meant to guide every subordinate bench. When a judge anchors a ruling on a fake case, however, that chain snaps. A phantom precedent taints the ruling’s very foundation. Imagine a Supreme Court judgment that echoes a bogus “landmark” case; lower courts then cite that judgment in turn, and soon the falsehood ripples through the system. The stability and predictability of law – the promise that citizens can rely on past decisions – is stolen.

The Supreme Court left no doubt about the consequences. It declared any judgment resting on hallucinated AI content to be a “subversion of the rule of law”. A decision grounded on such material is “no decision at all” in the eyes of the law. Even if the fake citation was only an incidental footnote, the Court held, the resulting error demands a do-over to protect the sanctity of adjudication. In Pooja Ramesh Singh, the Justices painstakingly identified multiple phantom cases and page references that had slipped past the NCLT (and even the NCLAT on appeal). They scrapped the appellate verdict entirely, restoring the case for a fresh hearing. Had the Supreme Court not intervened, those non-existent authorities would have become “law” by citation – poisoning the well for everyone downstream. To quote a commentator, the Court held that a ruling based on AI-made precedents “cannot be treated as a valid judicial decision because it weakens the rule of law and affects the fairness of adjudication”. In other words, phantoms have no rightful place in the temple of law.

The Ground-Level Human Cost: The Vulnerable Litigant in the Algorithmic Crossfire

What does all this mean for the ordinary person caught in the legal thicket? Far from an abstract debate, the stakes are life-altering. Picture a small farmer in Uttar Pradesh who has battled for a clean title to his ancestral land. He has scraped together every rupee to hire a lawyer and pressed into service the last of his strength. On the day of judgment, he sits anxiously in the courtroom, hoping for justice. But the opposing counsel cites an obscure precedent (now, an AI fiction) in final arguments. The weary judge glances at it and nods, never suspecting its origin. When the verdict goes against the farmer, citing that phantom case, the loss is ruinous: not just the land, but his dream and livelihood.

This is not hyperbole. The Constitution guarantees that no person will be deprived of life or liberty without a “just, fair and reasonable” procedure. As our Supreme Court has emphasized, the substance of any law and its procedure must respect “fundamental values of fairness, justice and dignity”. An unseen algorithm playing puppet-master in a courtroom violates this principle. A judgment won through AI trickery is a mockery of the adversarial process. It turns the high ideal of a fair trial – enshrined under Article 21 – into a cruel gamble. Those who lose under this system have been dealt a wrong entirely by design, not by facts. As one analysis points out, basing outcomes on such hallucinations “affects the fairness of adjudication” and thus tramples on a person’s constitutional right to an impartial hearing. The human cost is grave: life savings gone, trust in justice shattered, and one citizen’s faith in the Republic left in tatters.

Reclaiming the Soul of Advocacy: A Manifesto for Ethical Jurisprudence

There are no quick fixes. Simply banning ChatGPT or shouting slogans will not heal this wound. What is needed is a revolution in legal ethics and education. First, lawyers must be held accountable. The Supreme Court itself has called on the Bar Council of India to draw up strict rules and sanctions against filing hallucinated citations. This would enshrine into our disciplinary code the obvious: every advocate remains ultimately responsible for everything they submit. One cannot hide behind an AI. As commentators have noted, technology is a tool but cannot abdicate professional duty. In fact, prominent legal guidance (even from the U.K.) now explicitly warns that misleading the court via unverified AI output “may amount to incompetence and serious professional misconduct”. The message is clear: the duty of candor and verification at least matches what the Bar demands of manual research.

Second, legal education and training must catch up to the times. Law schools and Bar training programs should require courses on technology and AI literacy. We must teach young lawyers how to use these tools critically. Clinics and moot problems can simulate AI-based research, forcing students to double-check everything an algorithm gives them. The ethics curriculum should stress that if a citation can only be found on Google and not in official reports, it is not a citation at all. In short, we must cultivate an attitude of healthy skepticism toward polished but unfounded answers.

Third, the State should invest in verified legal AI infrastructure. Instead of leaving practitioners to fall into ChatGPT’s orbit, India can develop or license trustworthy platforms. For example, the Supreme Court’s e-committee is prototyping SUPACE – an AI portal to help judges and lawyers find relevant law efficiently. A system like this, integrated with the Court’s own database of reported judgments, could use retrieval-augmented generation (RAG) to yield only pre-vetted cases. Such an indigenous tool, properly overseen by the judiciary, would harness AI’s speed while guarding against hallucinations. In parallel, the Bar and Bench can insist on routinely checking authorities against official sources – a cultural shift back toward the old Inns of Court mindset, now combined with 21st-century checks and balances.

Finally, we must reclaim the human core of justice. Law is not mathematics; it is a moral enterprise. The empathy for a farmer, the weight of a widow’s pleading, the probing judgment of a dissenting justice – these define our system more than any algorithm ever could. A human mind, steeped in principle and compassion, distinguishes right from wrong, not just probable word sequences. Courts must remember that only people interpret values, not machines. As one judge starkly reminded us, any comfort we find in delegating to AI is a slippery habit that risks our “capacity to think – to discern the distinction between what is right and what is wrong, truth and falsehood”. In this spirit, the manifesto for ethical jurisprudence is also a plea to preserve the lawyer’s soul: rigorous in research, humble before the Court, and always guided by conscience.

Conclusion: Guarding the Flame of Truth

The battle over AI-hallucinated law is nothing less than a struggle for the soul of our justice system. In the end, technology must remain the servant of justice, never its master. As the Supreme Court admonished, we may adopt AI to assist in judging – but only under “total and absolute control” by human hands. When the gavel falls, it should be knowledge and integrity speaking, not an invisible algorithm. Let us not allow courtrooms to become laboratories of legal fantasy. If we lose the sanctity of our written law to these phantom precedents, we will have lost the very essence of truth that holds the Republic together. It is our duty – as lawyers, judges, and citizens – to keep that flame of truth burning bright in every judgment.

References

• SC Observer Case Summary – Detailed review of the landmark ruling setting a zero-tolerance standard for fake precedents.
• The Hindu Editorial Report – Captures Justice P.S. Narasimha’s iconic warning on AI hallucinations as the "methyl isocyanate of law".
• SCC Online Analysis – Explains the Regulations for Use of Artificial Intelligence in Courts, 2026 structural rules.
• IAS Gyan Case Study – Breaks down the constitutional hierarchy under Article 141 and the BCI regulatory fallout.
• Vidhilegal Policy Blog – Provides an excellent comparative study on institutional liability versus mechanical tasks.
• LiveLaw Analysis Piece – Discusses the technical reason why LLMs operate strictly as text prediction engines.
• Law and Other Things Portal – Highlights the massive 5 crore case backlog and mandatory pre-filing filter reforms

.    .    .