Event Title
Assessing AI Output in Legal Decision-Making with Nearest Neighbors
Speaker Bio
Professor Biedermann obtained a BSc and a MSc in Forensic Science at UNIL (2002), and worked as a scientific advisor in the Federal Department of Justice and Police (Berne, Switzerland) in cases investigated by the Office of the Attorney General of Switzerland (2003–2010). His PhD thesis focused on graphical models and probabilistic inference for evaluating scientific evidence in forensic science (2007). Alex conducted visiting research stays in Italy, the UK, Australia and the US. He teaches scientific evidence interpretation and decision analysis in Lausanne, with visiting appointments at partner institutions in Australia (University of Adelaide Law School) and China (China University of Political Science and Law, Beijing). Alex is currently the principal investigator of the research project NORMDECS (Normative Decision Structures of Forensic Interpretation in the Legal Process), funded by the Swiss National Science Foundation (2016–2020). This project aims to study fundamental questions of forensic interpretation through probability and decision theory. The project features a highly multidisciplinary perspective by connecting core forensic science and the law with computational statistics and philosophy of science acting as supporting disciplines. Alex has authored and co-authored well over 100 publications (peer-reviewed articles, commentaries, chapters and books). He is a committee member of the Statistics and Law Section of the Royal Statistical Society, and a councillor of the International Association of Evidence Science.
Timothy Lau is a research associate at the Federal Judicial Center. He serves as the Center's liaison to the Advisory Committee on the Federal Rules of Evidence. His work at the Center also touches on issues concerning the intersection of law and technology. He was the author of the Center's report, "Trade Secret Seizure Best Practices under the Defend Trade Secrets Act of 2016," that was developed and submitted to Congress pursuant to the Act. At present, he is developing primer to educate federal judges about artificial intelligence, and he also interfaces with the Law Committee of the IEEE Global Initiative on Ethics of Autonomous and Intelligent Systems. Dr. Lau earned his J.D. at Stanford Law School and his Sc.D. in materials science and engineering from MIT. He clerked for Judge Raymond C. Clevenger III of the U.S. Court of Appeals for the Federal Circuit.
Presentation Type
Article
Start Date
20-3-2020 11:30 AM
End Date
20-3-2020 12:00 PM
Description
Artificial intelligence (“AI”) systems are widely used to assist or automate decision-making. However, while there are general metrics for performance of AI systems, there is as yet no well-established gauge to assess the quality of particular AI recommendations or decisions. This presents a serious problem in the emerging use of AI in legal applications because the legal system aims at good performance not only in the aggregate but also in individual cases. This article presents the concept of using nearest neighbors to assess individual AI output. The method has the benefit of being easy to understand and apply for judges, lawyers, and juries. In addition, it is fundamentally compatible with existing AI methodologies. The paper explains how the concept could be applied for probing AI output in a number of use cases, including civil discovery, risk prediction, and forensic comparison while also presenting its limitations.
Assessing AI Output in Legal Decision-Making with Nearest Neighbors
Artificial intelligence (“AI”) systems are widely used to assist or automate decision-making. However, while there are general metrics for performance of AI systems, there is as yet no well-established gauge to assess the quality of particular AI recommendations or decisions. This presents a serious problem in the emerging use of AI in legal applications because the legal system aims at good performance not only in the aggregate but also in individual cases. This article presents the concept of using nearest neighbors to assess individual AI output. The method has the benefit of being easy to understand and apply for judges, lawyers, and juries. In addition, it is fundamentally compatible with existing AI methodologies. The paper explains how the concept could be applied for probing AI output in a number of use cases, including civil discovery, risk prediction, and forensic comparison while also presenting its limitations.