Journal Article 3 Mentions
Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing
Yoav Benjamini1995
Yosef Hochberg
Top 1% · 99th Percentile
99,817 citations · Statistics and Probability

TLDR

When people test many questions, some answers may look right by chance. A good method can limit wrong answers while still finding more real answers.

Summary

1 Study Aim

The authors address the multiplicity problem (the increased chance of misleading results when many questions are tested). They compare controlling the familywise error rate (FWER, the chance of at least one false positive) with controlling the false discovery rate (FDR, the expected share of rejected hypotheses that are false). They seek a method that controls FDR while improving power (the ability to detect real effects), when FWER control is unnecessarily strict. The study seeks a safer way to test many questions without missing as many real answers.

2 Study Design

The study develops a sequential Bonferroni-type procedure (an ordered, step-by-step threshold rule) for independent test statistics (results from tests that do not influence one another). The authors prove that this procedure controls FDR. They use simulation (computer-based repeated testing) to compare its performance and demonstrate its use through examples. The source does not report a human or biological participant sample. They created a step-by-step rule, tested it repeatedly by computer, and showed examples of how to use it.

3 Findings

The authors show that FDR equals FWER when every tested hypothesis is true, but becomes smaller when some hypotheses are false. They report that the sequential procedure controls FDR for independent test statistics. Their simulations show a substantial gain in power compared with stricter error control. The examples illustrate when FDR is appropriate and how the procedure can be used. The study recommends FDR control when researchers can accept some false positives (incorrect claims of an effect) while seeking more genuine effects. The method can find more genuine effects while keeping mistaken findings at a controlled level.

Abstract

SUMMARY The common approach to the multiplicity problem calls for controlling the familywise error rate (FWER). This approach, though, has faults, and we point out a few. A different approach to problems of multiple significance testing is presented. It calls for controlling the expected proportion of falsely rejected hypotheses — the false discovery rate. This error rate is equivalent to the FWER when all hypotheses are true but is smaller otherwise. Therefore, in problems where the control of the false discovery rate rather than that of the FWER is desired, there is potential for a gain in power. A simple sequential Bonferronitype procedure is proved to control the false discovery rate for independent test statistics, and a simulation study shows that the gain in power is substantial. The use of the new procedure and the appropriateness of the criterion are illustrated with examples.

Referenced In