Empirical Truth and Probability

There are relatively few absolute truths. The findings of science and technology are empirical — based on evidence and/or observation and verifiable through evidence and observation — and empirical truths are typically nuanced. It is possible to compute the probability that patterns in software and data are random, and this result can be used to form an opinion.

Rule 401 and Probability

Rule 401 of the US Federal Rules of Evidence states:

Evidence is relevant if it has the tendency to make the existence of any fact that is significant for determinating if the action is more or less probable than it would be without the evidence. 

I spoke with a corporate attorney that I have worked with about this, and he said “Rule 401, in my understanding, is not referring to ‘probability’ as a mathematical matter, but merely tries to reflect that there must be some logical connection between evidence being offered and the claims or defenses.”

Notwithstanding my learned friend’s explanation, there is general agreement that the relevance of expert evidence that arises from the hard sciences requires particularly rigorous reasoning. That reasoning should be provided as a quantitative result, after establishing that the evidence is authentic. Computing probabilities can be most helpful in this regard. Analysis showing strong mathematical probability can form the basis for well-founded opinions.

For coherence, empirical truth requires a proper suite of propositions that mutually support each other. Experts must adhere to this practice when reviewing technology. As a result, a competent expert should be able to opine on the abilities and work habits of the people who worked together to develop the technology in question, even though they never met, and on whether the technology was created organically or misappropriated.

Certainty

The purpose of probability is to quantify uncertainty. For example, what is the probability of rolling a six-sided die and obtaining a 1?

For empirical truth, certainty is generally assumed once a threshold of probability is achieved. This is the basis, for example, for matching DNA profiles, fingerprint matching and matching retina scans. With due consideration for prosecutor’s fallacy, a higher probability leads one to believe that the possibility is more likely.

There are 7.5 x 10¹⁸ grains of sand in the world, and there are estimated to be 10²¹ stars in the observable universe. In a recent case, I, in my capacity as a software expert, found and presented evidence that was more unique than all the grains of sand in the entire world, and the number of stars in the observable universe. As a result, I felt confident that the opinions I derived from my analysis of the evidence were well-founded.

Subsequently, in that same case I was invited to provide technical support during the deposition of one of the other side’s experts. I computed the likelihood or probability of many statements that the other expert made mere seconds after he uttered them. These real-time probability computations provided guidance for further questions posed by the attorney that I was supporting.

Summary

This article summarizes the importance of empirical study in the analysis of evidence, and the role that the mathematical determination of probability plays in determining the value of evidence.

Analyzing evidence for unlikely contents can yield important information that may be the basis for an expert opinion. It is vital that an expert be able to quantify the probabality that anomalies are merely due to random coincidence, and that the expert’s report reflect this in a defensible manner. Since correlation does not imply causation, probabilities must be combined with other factors to have meaning.