journal article Nov 02, 2018

Multiple Testing with the Structure-Adaptive Benjamini–Hochberg Algorithm

View at Publisher Save 10.1111/rssb.12298
Abstract
SummaryIn multiple-testing problems, where a large number of hypotheses are tested simultaneously, false discovery rate (FDR) control can be achieved with the well-known Benjamini–Hochberg procedure, which a(0, 1]dapts to the amount of signal in the data, under certain distributional assumptions. Many modifications of this procedure have been proposed to improve power in scenarios where the hypotheses are organized into groups or into a hierarchy, as well as other structured settings. Here we introduce the ‘structure-adaptive Benjamini–Hochberg algorithm’ (SABHA) as a generalization of these adaptive testing methods. The SABHA method incorporates prior information about any predetermined type of structure in the pattern of locations of the signals and nulls within the list of hypotheses, to reweight the p-values in a data-adaptive way. This raises the power by making more discoveries in regions where signals appear to be more common. Our main theoretical result proves that the SABHA method controls the FDR at a level that is at most slightly higher than the target FDR level, as long as the adaptive weights are constrained sufficiently so as not to overfit too much to the data—interestingly, the excess FDR can be related to the Rademacher complexity or Gaussian width of the class from which we choose our data-adaptive weights. We apply this general framework to various structured settings, including ordered, grouped and low total variation structures, and obtain the bounds on the FDR for each specific setting. We also examine the empirical performance of the SABHA method on functional magnetic resonance imaging activity data and on gene–drug response data, as well as on simulated data.
Topics

No keywords indexed for this article. Browse by subject →

References
34
[1]
Barber "Controlling the false discovery rate via knockoffs" Ann. Statist. (2015) 10.1214/15-aos1337
[2]
Barber "Rocket: robust confidence intervals via Kendall’s tau for transelliptical graphical models" Ann. Statist. (2018) 10.1214/17-aos1663
[3]
Barber "The p-filter: multilayer false discovery rate control for grouped hypotheses" J. R. Statist. Soc. (2017) 10.1111/rssb.12218
[4]
Barlow (1972)
[5]
Bartlett "Rademacher and Gaussian complexities: risk bounds and structural results" J. Mach. Learn. Res. (2003)
[6]
Selective Inference on Multiple Families of Hypotheses

Yoav Benjamini, Marina Bogomolov

Journal of the Royal Statistical Society Series B:... 2014 10.1111/rssb.12028
[7]
Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing

Yoav Benjamini, Yosef Hochberg

Journal of the Royal Statistical Society Series B:... 1995 10.1111/j.2517-6161.1995.tb02031.x
[8]
The control of the false discovery rate in multiple testing under dependency

Yoav Benjamini, Daniel Yekutieli

The Annals of Statistics 2001 10.1214/aos/1013699998
[9]
Borovkov (1999)
[10]
Boyd "Distributed optimization and statistical learning via the alternating direction method of multipliers" Foundns Trends Mach. Learn. (2011)
[11]
Cheng (2015)
[12]
Chouldechova (2014)
[13]
Coser "Global analysis of ligand sensitivity of estrogen inducible and suppressible genes in mcf7/bus breast cancer cells by DNA microarray" Proc. Natn. Acad. Sci. USA (2003) 10.1073/pnas.2235866100
[14]
GEOquery: a bridge between the Gene Expression Omnibus (GEO) and BioConductor

Sean Davis, Paul S. Meltzer

Bioinformatics 2007 10.1093/bioinformatics/btm254
[15]
Ferreira "On the Benjamini–Hochberg method" Ann. Statist. (2006) 10.1214/009053606000000425
[16]
Genovese "False discovery control with p-value weighting" Biometrika (2006) 10.1093/biomet/93.3.509
[17]
"Sequential selection procedures and false discovery rate control" J. R. Statist. Soc. (2016) 10.1111/rssb.12122
[18]
Hu "False discovery rate control with groups" J. Am. Statist. Ass. (2010) 10.1198/jasa.2010.tm09329
[19]
Hütter "Optimal rates for total variation denoising" Proc. Mach. Learn. Res. (2016)
[20]
Keller "Reading span and the time-course of cortical activation in sentence-picture verification" A. Conv. Psychonomic Society (2001)
[21]
Lei "AdaPT: an interactive procedure for multiple testing with side information" J. R. Statist. Soc. (2018) 10.1111/rssb.12274
[22]
Lei (2016)
[23]
Lei (2017)
[24]
Li "Accumulation tests for FDR control in ordered hypothesis testing" J. Am. Statist. Ass. (2017) 10.1080/01621459.2016.1180989
[25]
Liu "The nonparanormal: semiparametric estimation of high dimensional undirected graphs" J. Mach. Learn. Res. (2009)
[26]
Ramdas (2017)
[27]
Schildknecht "More specific signal detection in functional magnetic resonance imaging by false discovery rate control for hierarchically structured systems of hypotheses" PLOS One (2016) 10.1371/journal.pone.0149016
[28]
Schwartzman "Multiple testing of local maxima for detection of peaks in 1d" Ann. Statist. (2011) 10.1214/11-aos943
[29]
Siegmund "Detecting simultaneous variant intervals in aligned sequences" Ann. Appl. Statist. (2011) 10.1214/10-aoas400
[30]
Srebro (2010)
[31]
A Direct Approach to False Discovery Rates

John D. Storey

Journal of the Royal Statistical Society Series B:... 2002 10.1111/1467-9868.00346
[32]
Storey "Strong control, conservative point estimation and simultaneous conservative consistency of false discovery rates: a unified approach" J. R. Statist. Soc. (2004) 10.1111/j.1467-9868.2004.00439.x
[33]
Sun "Oracle and adaptive compound decision rules for false discovery rate control" J. Am. Statist. Ass. (2007) 10.1198/016214507000000545
[34]
Zhao "A powerful fdr control procedure for multiple hypotheses" Computnl Statist. Data Anal. (2016) 10.1016/j.csda.2015.12.013
Cited By
125
Journal of the American Statistical...
Journal of the American Statistical...
Metrics
125
Citations
34
References
Details
Published
Nov 02, 2018
Vol/Issue
81(1)
Pages
45-74
License
View
Funding
National Science Foundation award Award: DMS-1654076
Cite This Article
Ang Li, Rina Foygel Barber (2018). Multiple Testing with the Structure-Adaptive Benjamini–Hochberg Algorithm. Journal of the Royal Statistical Society Series B: Statistical Methodology, 81(1), 45-74. https://doi.org/10.1111/rssb.12298
Related

You May Also Like

Regression Shrinkage and Selection Via the Lasso

Robert Tibshirani · 1996

50,685 citations

Maximum Likelihood from Incomplete Data Via the EM Algorithm

A. P. Dempster, N. M. Laird · 1977

49,275 citations

Regression Models and Life-Tables

D. R. Cox · 1972

38,899 citations

Regularization and Variable Selection Via the Elastic Net

Hui Zou, Trevor Hastie · 2005

20,401 citations