India’s existing issues with the rise of artificial intelligence (AI) have largely centred on content-related risks such as deepfakes, misinformation, synthetic media and harmful online content. But a recent Supreme Court case has brought a different concern into focus: who is responsible when AI-generated information influences a decision and turns out to be wrong?

 

The issue came to the fore after the Supreme Court last week set aside an order of the National Company Law Tribunal (NCLT) that relied on judgments which were later found to be non-existent and allegedly generated through artificial intelligence.

 


The case has raised a risk that goes beyond content creation to the use of AI outputs in decision-making. As organisations increasingly deploy generative AI tools, questions of accountability are beginning to surface across sectors.

 
 


What happens if an AI tool fabricates financial information used in a lending decision? Or generates incorrect compliance references in a regulatory filing? What if an auditor relies on AI-generated legal interpretations, or a hospital uses an AI-generated medical summary that contains inaccuracies?

 


In each case, the question remains the same: who bears responsibility for the consequences?

 


The answer, legal and industry experts say, is far from settled.


No dedicated liability framework


Currently, India does not have a law that specifically assigns liability for AI-generated errors.

 


Ashwini Kumar, advocate and founder of My Legal Expert (MLE), said existing laws such as the Information Technology Act, contract law, tort law, consumer protection provisions, intellectual property law and sectoral regulations may apply depending on the circumstances, but none were designed for generative AI systems.

 


“Any imposition of liability today is determined through conventional legal principles such as negligence, duty of care, etc.,” Kumar said, adding that courts are likely to determine responsibility on a case-by-case basis until a clearer framework emerges.

 


Shreya Suri, partner at CMS INDUSLAW, said liability questions currently fall back on existing legal provisions, including information technology laws, the Bharatiya Nyaya Sanhita, intellectual property laws and, in some situations, intermediary liability provisions if adequate safeguards are not in place.

 


The result is a patchwork approach in which responsibility depends heavily on the facts of each case rather than a single legal standard.


If AI gets it wrong, who is liable?


According to experts, there is no universal answer.

 


Kumar said liability may extend to multiple stakeholders depending on their role, degree of control and the level of human intervention involved. Courts are likely to examine factors such as foreseeability, negligence and contractual allocation of risk rather than adopting a blanket rule.

 


Suri similarly said responsibility could rest with the user, institution or software provider, or all three, depending on the nature of the violation.

 


The legal position becomes even more complex because AI itself cannot be held responsible.

 


Venkatesh Naidu, chief executive officer of BajajCapital Insurance Broking, said organisations using AI would likely face scrutiny first if an AI-generated error causes financial loss.

 


“If a bank, broker or advisor acts on a recommendation from an AI tool and it turns out to be wrong, the natural question is: who signed off on this?” Naidu said.

 


He added that developers could also come under scrutiny in certain situations involving contractual obligations or product liability claims, although that area remains largely untested.


The lawyer remains responsible


The Supreme Court matter originated in a legal context, but lawyers say the judgment put emphasis on an existing principle rather than creating a new one.

 


Gauhar Mirza, partner at Saraf and Partners, said the responsibility for citing authentic legal precedents remains with the advocate regardless of the tools used during research.

 


“AI changes the tool, not the duty. The advocate remains accountable for every authority placed before the court,” Mirza said.

 


He added that professional responsibility cannot be delegated to an AI system. While AI can assist with research and drafting, it cannot replace a lawyer’s judgement or ethical obligations.

 


Mirza also said professional negligence standards are likely to evolve as AI adoption increases.

 


Failing to verify AI-generated citations or relying blindly on AI outputs could itself become evidence of negligence.

 


The Supreme Court’s draft Regulations for Use of Artificial Intelligence in Courts, 2026, follow a similar approach by treating AI as an assistive technology while retaining human oversight and accountability, he said.


Beyond courts: The enterprise challenge


The liability debate is not limited to lawyers.

 


As companies integrate AI into finance, compliance, human resources, customer service and internal decision-making processes, questions around accountability are becoming increasingly important.

 


Madhu Rajputra Peravalli, chief executive officer of Troogue.ai, said many organisations still lack a clearly documented chain of accountability for AI-assisted decisions.

 


“Companies may know which AI tool they are using, but they often do not have a clear chain of accountability around who approved the input, who reviewed the output, who acted on it and who owns the consequence,” he said.

 


According to Peravalli, AI should remain a decision-support tool rather than a decision-maker.

 


“AI handles scale and speed; humans own judgement and consequence,” he said.

 


He argued that organisations should build safeguards directly into AI workflows through evidence trails, source validation, confidence scores and exception-based human review for high-risk decisions.

 


“The final accountability must remain with the human or institution using it,” he said.


A growing concern for insurers


The uncertainty is also creating challenges for insurers.

 


Naidu said most professional liability and errors-and-omissions policies were drafted before AI became part of everyday business decision-making and were designed primarily around human mistakes.

 


“AI creates a different kind of risk altogether, and it doesn’t always fit cleanly into the language of these older policies,” he said.

 


While some international insurers have begun developing AI-specific endorsements and products covering risks such as “algorithmic bias” and “faulty automated decisions”, India remains at an early stage.

 


For now, insurers are placing greater emphasis on governance, oversight and documentation when evaluating AI-related risks.

 


Naidu said one of the most difficult questions arises when several entities are involved in a single AI-assisted decision, such as a model developer, software provider, deploying company and intermediary.

 


“If something goes wrong, it’s rarely going to be one party’s fault alone,” he said.

 


Courts and insurers, he added, are likely to examine who had the final opportunity to verify the AI-generated recommendation before acting on it.


From content regulation to accountability


Kumar said that rather than treating every AI issue as merely a content moderation problem, the law should distinguish between AI-assisted decision-making, autonomous systems, generative AI, and high-risk applications.

 


He added that India would benefit from a framework that addresses accountability, explainability, testing standards and human oversight alongside innovation.

 


The Supreme Court case has exposed a gap in India’s legal framework. As AI-generated outputs increasingly shape decisions in courts, hospitals, banks and businesses, one important question remains unanswered: who is responsible when AI gets it wrong? For now, there is no clear rule, and liability is likely to be decided on a case-by-case basis.



Source link

YouTube
Instagram
WhatsApp