# A Simple, Statistically Robust Test of Discrimination

Submitted, 2024.

ArXiv: 2407.06539.

## Abstract

In observational studies of discrimination, the most common statistical
approaches consider either the rate at which decisions are made (benchmark
tests) or the success rate of those decisions (outcome tests). Both tests,
however, have well-known statistical limitations, sometimes suggesting
discrimination even when there is none. Despite the fallibility of the
benchmark and outcome tests individually, here we prove a surprisingly strong
statistical guarantee: under a common non-parametric assumption, at least one
of the two tests must be correct; consequently, when *both* tests agree,
they are guaranteed to yield correct conclusions. We present empirical evidence
that the underlying assumption holds approximately in several important
domains, including lending, education, and criminal justice—and that our
hybrid test is robust to the moderate violations of the assumption that we
observe in practice. Applying this approach to 2.8 million police stops across
California, we find evidence of widespread racial discrimination.