Fisher–Schultz Lecture: Generic Machine Learning Inference on Heterogeneous Treatment Effects in Randomized Experiments, With an Application to Immunization in India

Victor Chernozhukov; Mert Demirer; Esther Duflo; Iván Fernández-Val

doi:10.3982/ecta19303

journal article Open Access Jan 01, 2025

Fisher–Schultz Lecture: Generic Machine Learning Inference on Heterogeneous Treatment Effects in Randomized Experiments, With an Application to Immunization in India

Victor Chernozhukov Mert Demirer Esther Duflo Iván Fernández-Val

Econometrica Vol. 93 No. 4 pp. 1121-1164 · JSTOR

View at Publisher Save 10.3982/ecta19303

Abstract

We propose strategies to estimate and make inference on key features of heterogeneous effects in randomized experiments. These key features include
best linear predictors of the effects using machine learning proxies,
average effects sorted by impact groups, and
average characteristics of most and least impacted units. The approach is valid in high‐dimensional settings, where the effects are proxied (but not necessarily consistently estimated) by predictive and causal machine learning methods. We post‐process these proxies into estimates of the key features. Our approach is generic; it can be used in conjunction with penalized methods, neural networks, random forests, boosted trees, and ensemble methods, both predictive and causal. Estimation and inference are based on repeated data splitting to avoid overfitting and achieve validity. We use quantile aggregation of the results across many potential splits, in particular taking medians of
p‐values and medians and other quantiles of confidence intervals. We show that quantile aggregation lowers estimation risks over a single split procedure, and establish its principal inferential properties. Finally, our analysis reveals ways to build provably better machine learning proxies through causal learning: we can use the objective functions that we develop to construct the best linear predictors of the effects, to obtain better machine learning proxies in the initial step. We illustrate the use of both inferential tools and causal learners with a randomized field experiment that evaluates a combination of nudges to stimulate demand for immunization in India.

Topics

No keywords indexed for this article. Browse by subject →

References

92

[1]

Abadie The Review of Economic Studies (2005) 10.1111/0034-6527.00321

[2]

Abadie, Alberto, Matthew M. Chingos, and Martin R. West (2017): “Endogenous Stratification in Randomized Experiments,” Technical report, National Bureau of Economic Research.

[3]

Agarwal (2020)

[4]

Alatas, Vivi, Arun G. Chandrasekhar, Markus Mobius, Benjamin A. Olken, and Cindy Paladines (2019): “When Celebrities Speak: A Nationwide Twitter Experiment Promoting Vaccination in Indonesia,” Technical report, National Bureau of Economic Research. 10.3386/w25589

[5]

Angrist (2008) 10.2307/j.ctvcm4j72

[6]

Athey Proceedings of the National Academy of Sciences (2016) 10.1073/pnas.1510489113

[7]

Athey Econometrica (2021) 10.3982/ecta15732

[8]

Banerjee, Abhijit, Arun Chandrasekhar, Esther Duflo, Suresh Dalpath, John Floretta, Matthew Jackson, Loza Francine, Harini Kannan, and Anna Schrimpf (2021): “Inference on Winners,” Technical Report 28726, NBER Working Paper.

[9]

Banerjee, Abhijit, Arun Chandrasekhar, Esther Duflo, Suresh Dalpath, John Floretta, Matthew Jackson, Harini Kannan, Anna Schrimpf, and Mahesh Shrestha (2019): “Leveraging the Social Network Amplifies the Effectiveness of Interventions to Stimulate Take up of Immunization.”

[10]

Banerjee BMJ (2010)

[11]

Barber, Rina Foygel, Emmanuel J. Candes, Aaditya Ramdas, and Ryan J. Tibshirani (2022): “Conformal Prediction Beyond Exchangeability.” 10.1214/23-aos2276

[12]

Barnard Journal of the Royal Statistical Society. Series B (Methodological) (1974)

[13]

Bassani BMC Public Health (2013)

[14]

Program Evaluation and Causal Inference With High-Dimensional Data

A. Belloni, V. Chernozhukov, I. Fern�ndez-Val et al.

Econometrica 2017 10.3982/ecta12723

[15]

Inference on Treatment Effects after Selection among High-Dimensional Controls

A. Belloni, V. Chernozhukov, C. Hansen

The Review of Economic Studies 2014 10.1093/restud/rdt044

[16]

Belloni, Alexandre, Victor Chernozhukov, and Kengo Kato (2013): “Uniform Post Selection Inference for lad Regression Models,” arXiv preprint. arXiv:1304.0282. 10.1920/wp.cem.2013.2413

[17]

Simultaneous analysis of Lasso and Dantzig selector

Peter J. Bickel, Ya’acov Ritov, Alexandre B. Tsybakov

The Annals of Statistics 2009 10.1214/08-aos620

[18]

Bishop (2006)

[19]

Chen, Qizhao, Vasilis Syrgkanis, and Morgane Austern (2022): “Debiased Machine Learning Without Sample-Splitting for Stable Estimators,” arXiv preprint. arXiv:2206.01825.

[20]

Chernozhukov The Econometrics Journal (2017)

[21]

Fisher–Schultz Lecture: Generic Machine Learning Inference on Heterogeneous Treatment Effects in Randomized Experiments, With an Application to Immunization in India

Victor Chernozhukov, Mert Demirer, Esther Duflo et al.

Econometrica 10.3982/ecta19303

[22]

Chernozhukov, Victor, Iván Fernandez-Val, and Ye Luo (2015): “The Sorted Effects Method: Discovering Heterogeneous Effects Beyond Their Averages,” ArXiv e-prints 10.1920/wp.cem.2015.7415

[23]

Chernozhukov American Economic Review: Papers and Proceedings (2015) 10.1257/aer.p20151022

[24]

Chernozhukov, Victor, Whitney Newey, and Rahul Singh (2018): “Debiased Machine Learning of Global and Local Parameters Using Regularized Riesz Representers,” arXiv preprint. arXiv:1802.08667.

[25]

Chernozhukov, Victor, Whitney K. Newey, and Rahul Singh (2021): “A Simple and General Debiased Machine Learning Theorem With Finite Sample Guarantees,” arXiv preprint. arXiv:2105.15197. 10.1093/biomet/asac033

[26]

Chernozhukov Journal of the American Statistical Association (2021) 10.1080/01621459.2021.1920957

[27]

Cox Biometrika (1975) 10.1093/biomet/62.2.441

[28]

Crepon, Bruno, Esther Duflo, Huillery Elisa, William Pariente, Juliette Seban, and Paul-Armand Veillon (2021): “Cream Skimming and the Comparison Between Social Interventions Evidence From Entrepreneurship Programs for at-Risk Youth in France,” Report.

[29]

Crump The Review of Economics and Statistics (2008) 10.1162/rest.90.3.389

[30]

Davis Review of Economics and Statistics (2020) 10.1162/rest_a_00850

[31]

Deryugina American Economic Review (2019) 10.1257/aer.20180279

[32]

Dezeure, Ruben, Peter Bühlmann, and Cun-Hui Zhang (2016): “High-Dimensional Simultaneous Inference With the Bootstrap,” arXiv preprint. arXiv:1606.03940.

[33]

DiCiccio Statistics & Probability Letters (2020) 10.1016/j.spl.2020.108865

[34]

Domek Vaccine (2016) 10.1016/j.vaccine.2016.03.065

[35]

Dudley (2000)

[36]

Duflo Handbook of development economics (2007) 10.1016/s1573-4471(07)04061-2

[37]

Fan Journal of Business & Economic Statistics (2022) 10.1080/07350015.2020.1811102

[38]

Fithian, William, Dennis Sun, and Jonathan Taylor (2014): “Optimal Inference After Model Selection,” arXiv preprint. arXiv:1410.2597.

[39]

Foster, Dylan J., and Vasilis Syrgkanis (2019): “Orthogonal Statistical Learning,” arXiv preprint. arXiv:1901.09036.

[40]

On regression adjustments to experimental data

David A. Freedman

Advances in Applied Mathematics 2008 10.1016/j.aam.2006.12.003

[41]

Friedman (2001)

[42]

Genovese The Annals of Statistics (2008)

[43]

Gibson The Lancet Global Health (2017) 10.1016/s2214-109x(17)30072-4

[44]

Giné Ann. Statist. (2010) 10.1214/09-aos738

[45]

Gneezy The Journal of Legal Studies (2000) 10.1086/468061

[46]

Goodfellow (2016)

[47]

Hansen, Christian, Damian Kozbur, and Sanjog Misra (2017): “Targeted Undersmoothing,” arXiv preprint. arXiv:1706.07328. 10.2139/ssrn.3167373

[48]

Hartigan Journal of the American Statistical Association (1969) 10.1080/01621459.1969.10501057

[49]

Hastie (2015) 10.1201/b18401

[50]

Hirano Econometrica (2003) 10.1111/1468-0262.00442

Showing 50 of 92 references

Cited By

10

Fisher–Schultz Lecture: Generic Machine Learning Inference on Heterogeneous Treatment Effects in Randomized Experiments, With an Application to Immunization in India

Victor Chernozhukov, Mert Demirer · 2025

Econometrica

Metrics

10

Citations

92

References

Details

Published: Jan 01, 2025
Vol/Issue: 93(4)
Pages: 1121-1164
License: View

Authors

V

Victor Chernozhukov

Department of Economics and Center for Statistics and Data Sciences, MIT

M

Mert Demirer

Sloan School of Management, MIT

E

Esther Duflo

Department of Economics, MIT

I

Iván Fernández-Val

Department of Economics, Boston University

Funding

National Science Foundation

Cite This Article

Victor Chernozhukov, Mert Demirer, Esther Duflo, et al. (2025). Fisher–Schultz Lecture: Generic Machine Learning Inference on Heterogeneous Treatment Effects in Randomized Experiments, With an Application to Immunization in India. Econometrica, 93(4), 1121-1164. https://doi.org/10.3982/ecta19303

Fisher–Schultz Lecture: Generic Machine Learning Inference on Heterogeneous Treatment Effects in Randomized Experiments, With an Application to Immunization in India

You May Also Like