LIDW24 – Arbitration Alchemy

Arbitration Alchemy: AI, Psychology and Decision-Making Dynamics

Speakers: Dr Ula Cartwright, Toby Landau KC, David Blayney KC, Myfanwy Wood, Stephen Dowling.

As part of an exciting week at LIDW 2024, TrialView’s director Stephen Dowling chaired a stimulating discussion on the psychological aspects of AI adoption in international arbitration, kindly hosted by Serle Court. He was joined by Toby Landau KC, Dr. Ula Cartwright-Finch, David Blayney and Myfanwy (Miffy) Wood, who each provided a unique view as to which cognitive biases affect legal decision-making and the potential of AI technology in reducing their negative effects.

Dr. Ula Cartwright-Finch opened the discussion by explaining the difference between unconscious bias and cognitive bias. While unconscious biases are automatic, unintentional judgements which humans make based on their own background and experiences; cognitive biases are systematic errors in thinking which can affect our decision-making and judgements. This often occurs in situations when brains are overwhelmed with enormous amounts of incoming information. Since we do not have an infinite capacity to process all of that information, we rely on mental shortcuts to help us analyse the information and to make decisions. The reliance on mental shortcuts can lead to flaws in our reasoning processes. By being unaware of such biases in professional environments, decision-makers run the risk of making unfair and unjust decisions.

Dr. Ula Cartwright-Finch explained that, in the context of legal decision-making, overconfidence bias, confirmation bias, anchoring bias, and recency bias are most commonly found. For example, overconfidence bias occurs when an individual overestimates their own abilities, knowledge and the accuracy of their predictions. Legal professionals may over-rely on their experience and expertise and thus be overconfident in the accuracy of an assessment, for example in the context of predicting case outcomes. During arbitration proceedings, which often span many months, the tribunal is expected to process and retain substantial amounts of information. This is where confirmation, anchoring and recency biases, which all relate to our memory, can heavily affect how tribunal members interpret information and ultimately reach their final decision on a matter. Confirmation bias can lead to individuals favouring, interpreting, and remembering information in a way that confirms their preexisting beliefs. Anchoring bias causes an individual to rely too heavily on the first pieces of information they process (ie. the ‘anchor’) which influences how they process all information that follows. This can cause them to make errors in any subsequent judgements by relying too heavily on the original ‘anchor’. Recency bias relates to individuals recalling recent events more easily and vividly, which causes them to have a greater influence over their decisions than events occurring earlier in the past.

Cognitive Biases in Practice

Due to his background as both an advocate and arbitrator, Toby Landau KC explained how the above-mentioned biases manifest themselves in the context of international arbitration proceedings. Arbitrations based on the adversarial model of adjudication, often associated with common law jurisdictions, prove particularly susceptible to cognitive biases. This is based on the assumption that the tribunal is unprepared; and hence, requires an extensive presentation of all of the facts about a case by the parties. In practice, this leads to an ‘information overload’ which the tribunal is expected to digest, analyse, and translate into a fair outcome. Many rely on creating a map of the case with ‘anchor points’ which aid in their comprehension of the information they are met with. In Toby Landau KC’s experience, it is usually at this initial stage of proceedings that the fate of a case may be determined. The ‘anchor points’ set by the tribunal within their case map influence how they perceive and analyse certain evidence and information. When individuals have a preconceived image of a case in mind, they are more likely to accept information which confirms their initial view. This occurs because the anchors set at the beginning of the proceedings shape their perspective of the case going forward. Consequently, they may become less inclined to revise or ‘remap’ their case understanding.

The tribunal’s decision-making may be particularly affected by cognitive biases at the stage of the post-hearing brief. Often occurring at a much later point in time to the initial proceedings, the tribunal’s memory of the case and its facts will naturally subside. At this stage, the tribunal most likely would refer back to the ‘anchor points’ they set initially, highlighting the importance of mental anchoring and adjustment (MAO). This increases the risk of their decision-making being affected by confirmation bias and other cognitive biases related to memory.

Another interesting point brought up by the panel was whether the ultimate goal of the legal profession should be to entirely eliminate cognitive biases. Myfanwy (Miffy) Wood highlighted that arbitral proceedings represent a process that both parties willingly enter and consent to. Depending on which parties you ask, some may view arbitration as a more reliable dispute resolution method, as opposed to litigating in front of domestic courts. Given the increasing awareness of how cognitive biases influence humans’ decision-making, other parties may choose to use this fact to their advantage; whether during the arbitrator selection process or main proceedings.

AI To The Rescue?

This leads us to an important question: (How) should AI technology be used to help legal professionals manage information overload? Is there a risk that using AI in this context could exploit existing human limitations, especially when it comes to setting the right ‘anchor points’? Would AI increase the risks related to cognitive biases?

Drawing on his background as both an advocate and founder of the LegalTech analysis and reasoning tool ‘Associo’, David Blayney KC illustrated how the concept of ‘decision hygiene’ should guide the development of AI solutions within this context. The awareness of biases, in combination with structured decision-making, which takes into account diverse perspectives, is integral to reducing the negative effect of cognitive biases. One practical adoption of AI tools could be the production of a visual, rather than written, map which sets out all disputed points in a case. This could aid arbitrators in analysing complex information and may reduce their reliance on mental shortcuts to synthesise the information in question. Opting for visual tools, rather than solely relying on written submissions, can help with reconciling each party’s submissions more effectively. Decision trees may also be an effective aid in this context. By breaking down a complex case into manageable parts, decision trees help visualise different potential outcomes and thereby allow the quantification of risk attributed to different points in an analytical manner. This may be particularly useful relating to overconfidence and confirmation bias.

As AI technology advances, we may find ourselves at a point in the future where, e.g. in relation to costs, parties may be increasingly scrutinised as to the reasons for not utilising technology to aid legal proceedings and decision-making. Ultimately, this illustrates the importance of transparency regarding the (non)use of AI technology, as this will have an influence on public trust in legal proceedings going forward.

Concluding Thoughts

Rounding off this exciting discussion, the panel collectively agreed that the ultimate focus of future AI solutions should be on aiding and supporting, rather than replacing, human decision-making. Nevertheless, there remain significant practical issues, notably relating to how an AI system is trained, which may influence the efficacy and legitimacy of adopting AI technology in its current form within international arbitration proceedings.