Can, and should, AI make decisions?

Imagine stepping into a courtroom where the judge is not a person but a sophisticated AI system. Could this be the future of our legal system? As artificial intelligence continues to weave itself into the fabric of various industries, we are faced with a compelling question: Can—and should—AI make decisions in judicial and arbitration settings?

This issue leads to heated debates at every AI-Law conference. While AI excels in legal research, contract drafting, and predictive analytics, its role in decision-making processes remains controversial. It is not just about technological capability; it’s about navigating the ethical labyrinth and practical realities of integrating AI into a system traditionally governed by human judgment.

In this post, we will explore the advantages and challenges of employing AI in judicial and arbitration decision-making briefly. We will look into how AI might mitigate inherent human biases, examine the nature of AI’s decision-making process, and uncover the potential pitfalls that could arise when machines step into roles once reserved for humans.

Are Humans Perfect Decision-makers?

First, I will look at a different question. Are humans programmed to make complex decisions on a daily basis? Dr. Ula Cartwright-Finch at a recent panel discussion hosted by TrialView at London Disputes Week 2024 on ‘Arbitration in Alchemy’ answered this question in the negative. She noted that inherent cognitive biases coupled with information overload humans make it harder for humans to make complex decisions.

Cognitive biases are a genuine issue that affects decision-making. The human brain is not meant to handle complex decisions daily like judges/arbitrators do. This results in cognitive shortcuts (reliance on our biases) due to information overload, which creates room errors. Nobel laureate Kahneman identifies these biases as “intuitive preferences that consistently [violate] the rules of rational choice”, highlighting that our decision-making abilities are not as rational as we believe. If you ever get a chance to read Kahneman’s Thinking Fast and Thinking Slow, you will realise just how flawed our so-called rational mind is. Judges and arbitrators alike are susceptible to these cognitive flaws such as anchoring, hindsight bias, confirmation bias, etc. Studies have even indicated how hunger, the performance of a sports team, weather and the use of complex numbers can impact the way judges make decisions. These biases are rarely addressed under traditional notions of independence and impartiality.

This is not to say judges are not good decision-makers, years of experience and practice have allowed judges to improve this art. However, judges are still human and can suffer from these biases unconsciously.

One of the most compelling arguments for integrating AI into judicial and arbitration decision-making is its potential to reduce these biases that plague human decision-making. AI, in theory, could help eliminate these biases. Unlike humans, AI does not get tired, hungry, or emotionally swayed by persuasive rhetoric. AI can process information uniformly, relying on data rather than subjective judgment. AI can help ensure that similar cases are treated similarly, a fundamental principle of fairness that human judges sometimes overlook due to unconscious bias. By standardising the way cases are analysed, AI could improve consistency and fairness across judicial and arbitration systems. AI can also handle most mundane and administrative tasks associated with decision-making allowing human judges to focus on the more nuanced issues. For example, Trialview’s AI allows parties and judges to search the brief for various factual issues and to highlight contradictions. This makes the process of perusing the brief much more convenient.

How does AI Make Decisions?

AI models, particularly those used in legal contexts, are often powered by machine learning (ML) and large language models (LLMs). Unlike humans, who use intuition, emotional intelligence, and experience to make decisions, AI operates on brute-force pattern recognition. A machine-learning algorithm does not ‘think’ as humans do; instead, it calculates probabilities based on the data used to train the AI. For instance, if an AI is trained on a dataset of spam and non-spam emails, it learns to recognise patterns, such as identifying certain phrases like “Nigerian Prince” or “reward” in spam emails. It will then flag emails containing these phrases as having a high probability of being spam emails.

In the legal context, this means feeding AI systems vast repositories of case law, statutes submissions, etc. The AI then ‘learns’ to predict outcomes or provide legal interpretations by finding patterns in that data. A basic example of this is the studies done to predict court decisions. Most notably, the study by DM Katz in 2017 to predict US Supreme Court Decisions managed to predict overall court decisions with a 70.2% accuracy and individual judges’ votes with a 71.9% accuracy. A similar study on the European Court of Human Rights managed to predict the decisions of the court with a 79% accuracy.

The introduction of Generative AI by OpenAI has made these models capable of not only predicting human decision-making but also mimicking human reasoning. ChatGPT Strawberry or o1 is probably the strongest LLM model that can mimic human reasoning. When I asked ChatGPT o1 whether it can mimic human reasoning, this was the response it gave. Frankly, I could not have said it better.

Risks Associated with AI

Despite these advantages, allowing AI to play a sole or predominant role in legal decision-making raises concerns that cannot be ignored. One of the most significant is the issue of accountability. If an AI makes an incorrect or biased decision, who is held accountable? Can a party appeal an AI’s decision? These are thorny questions that legal systems are not yet equipped to handle.

Moreover, there is a societal and ethical dimension to justice that AI cannot replicate. The law is not merely a set of rules to be mechanically applied but a living system that evolves with societal values, cultural norms, and ethical considerations. Human judges and arbitrators interpret the law not just as it is written but as it should be applied in the context of real-world complexities. AI, for all its computational power, lacks the empathy, ethical judgment, and human intuition necessary to make decisions that reflect societal values.

The ‘black box’ decision-making nature of AI does not resonate well with the public. While AI can provide a decision or prediction, it doesn’t always explain how it reached that conclusion, making its outputs difficult to scrutinise. In contrast, when a human judge makes a decision, that decision must be explained and justified. This lack of transparency undermines humans in the AI-driven adjudication process.

AI may offer a solution to human bias but it also comes with its own set of biases; most of which stem from the data used to train these systems. The algorithms behind AI are only as good as the data they are fed. “Garbage-in-garbage out”. If the dataset includes biased outcomes, the AI will perpetuate those biases. For instance, if an AI model is trained on sentencing data that reflects racial or gender biases, it may replicate these patterns in its recommendations. An infamous example of this is the COMPAS AI system used for sentencing by US Courts. The COMPAS has been widely criticised for showing racial biases, often predicting a higher recidivism risk for Black defendants compared to white defendants.

LLMs (Generative IA) can also generate convincing and yet false content (hallucinations), which are not only ethically questionable but can lead to dire consequences in the legal sphere. Hallucinations occur because the purpose of Generative AI is not to think intelligently but to generate content, regardless of its veracity. In a famous US case, lawyers faced severe consequences for using false cases and judgements hallucinated by ChatGPT in their written submissions.

The recent EU AI Act has categorised AI used in legal decision-making as high-risk AI for the above reasons. Addressing these biases in AI is a significant technical and ethical challenge. Moreover, developers are now working on creating ‘explainable AI’ (XAI), which seeks to make AI decision-making processes more transparent. XAI provides reasons for its decisions, allowing human overseers to scrutinise and question them. Still, it’s an open question whether AI can ever completely overcome the biases present in the data on which it depends. Thus, AI and humans alike have their own vices, idea is that they can complement each other’s weaknesses if utilised well.

How can AI Assist with Decision-making?

One of the most significant benefits of AI is its ability to handle complex and vast datasets without experiencing fatigue. A judge or arbitrator may take months to review hundreds of pages of documents. Whereas AI can do the same in minutes if not seconds. For example, Brazil’s VICTOR AI has reduced the time taken to do a preliminary examination of an appeal case lodged in the Brazilian Supreme Court from 30 minutes (by a human) to 5 seconds. Similarly, the Frankfurt courts are currently developing the ‘FraUKe-AI’ to assist with air-passenger claims. This AI is supposedly capable of identifying common fact patterns in air-passenger claims and formulating draft decisions which can then be reviewed by a human judge.

The COMPASS AI system used to assist sentencing in criminal cases by US Courts is another unique example of how AI can be used to assist decision-making. The COMPAS can rate the defendant’s potential to re-offend(recidivism) and this rating is used by judges when sentencing. Despite the controversy associated with COMPASS AI provides judges with a statistical outlook regarding the potential to re-offend, without which judges will have to mostly rely on their own intuition. AI tools can in this manner be used to assist with decision-making regarding complex mathematical and statistical issues. AI is already being used for the valuation of company assets and to predict the movement of stocks. It would not be too far-fetched of an idea to consider an AI tool that can assist with DCF calculations/quantum stages of arbitration awards. Tribunal’s own quantum expert.

Beyond AI systems specifically designed for decision-making, even LLMs such as ChatGPT can assist with decision-making, particularly in interpreting statutes and contracts. For common law judges and arbitrators, interpretation is a large part of their decision-making alchemy. In the recent case of Snell v. United Specialty Insurance Company, U.S. Circuit Judge Kevin Newsom issued an unusual opinion where he used LLMs to interpret the term ‘landscaping’—specifically, to determine whether building a trampoline falls under this concept. He stated that LLMs are quite good at providing interpretations as they are trained on real-world linguistic data. According to him, “put simply, ordinary-meaning interpretation aims to capture how normal people use language in their everyday lives—and the bulk of the LLMs’ training data seem to reflect exactly that.”

For example, AI could assist in contract interpretation by analysing thousands of similar contracts and how courts or tribunals have interpreted key clauses. It could then offer possible interpretations based on existing case law, leaving the arbitrator to apply human reasoning and judgment to the specific context of the dispute. This could reduce the time taken to render decisions, making dispute resolution more accessible, efficient and affordable.

While the prospect of AI completely taking over decision-making from human judges or arbitrators seems unlikely in the near future, AI can serve as a powerful assistant to human decision-makers. AI can analyse case law, statutes, and precedents to provide judges with a range of possible outcomes. It can help identify relevant case similarities that a human judge may overlook, particularly in complex or niche legal areas.
The Way Forward: A Hybrid Approach

So, should and can AI be used in judicial and arbitration decision-making? Should AI replace human decision-makers? Perhaps the answer lies not in choosing one over the other but in harmoniously blending AI’s capabilities with human wisdom. AI’s ability to process vast amounts of data quickly and accurately makes it an invaluable tool for legal research, document review, and even interpretations. However, the final decision should remain in human hands, ensuring that justice is not reduced to mere data points. There is a reason why every case is judged on its own facts, it is because people need to feel justice is being done to them. Reducing people’s issues to mere data will distance people from the judicial process.

This discussion is far from over… The weight of technological innovation will force us to reconsider how we approach justice and decision-making. In a recent novel I read called ‘Blink of an Eye’, an AI detective raises an intriguing question: why do we consider human decision-making the gold standard when it’s proven to be flawed?

By Minul Muhamdiramge, Chevening Scholar | Junior Counsel