The Organization of Scientific Area Committees for Forensic Science (OSAC), was established to promote and develop forensic-science standards based on sound scientific principles. One of the first standards to be approved deals with declaring fragments of glass to be either distinguishable or indistinguishable in their chemical composition. This determination is important when it is suspected that small fragments associated with a defendant came from the scene of a crime involving broken glass. Because of instrumental measurement error, even fragments with identical elemental concentrations will display some differences. To account for measurement error, the standard uses statistical hypothesis tests that presume that two fragments are identical unless the measurements are far apart. This procedure seems to give an advantage to the prosecution position that the fragments came from the crime scene.
A member of OSAC's Forensic Science Standards Board therefore asked the organization's Legal Resource Committee (LRC) if the criminal law’s concern with avoiding false convictions at the expense of false acquittals should influence what counts as a statistically significant difference. The LRC concluded that the high burden of persuasion in criminal cases does not dictate a particular significance level but that the law does require an expert witness to provide a statistically valid explanation of how probative a conclusion of "indistinguishable" really is. This paper introduces the memorandum and places it in the broader context of the three main statistical approaches to reasoning about the implications of evidence—classical hypotheses testing, likelihood comparisons, and Bayesian inference. It notes that the hypothesis testing procedure in the OSAC standard may not be the most suitable way to interpret and present the results of the chemical measurements.
David H. Kaye, Hypothesis Testing in Law and Forensic Science: A Memorandum, 130 Harv. L. Rev. F. 127 (2017).