Anomaly detection by robust statistics

Peter J. Rousseeuw; Mia Hubert

doi:10.1002/widm.1236

journal article Open Access Nov 23, 2017

Anomaly detection by robust statistics

Peter J. Rousseeuw Mia Hubert

WIREs Data Mining and Knowledge Discovery Vol. 8 No. 2 · Wiley

View at Publisher Save 10.1002/widm.1236

Abstract

Real data often contain anomalous cases, also known as outliers. These may spoil the resulting analysis but they may also contain valuable information. In either case, the ability to detect such anomalies is essential. A useful tool for this purpose is robust statistics, which aims to detect the outliers by first fitting the majority of the data and then flagging data points that deviate from it. We present an overview of several robust methods and the resulting graphical outlier detection tools. We discuss robust procedures for univariate, low‐dimensional, and high‐dimensional data, such as estimating location and scatter, linear regression, principal component analysis, classification, clustering, and functional data analysis. Also the challenging new topic of cellwise outliers is introduced.WIREs Data Mining Knowl Discov2018, 8:e1236. doi: 10.1002/widm.1236This article is categorized under:Algorithmic Development > Spatial and Temporal Data MiningTechnologies > ClassificationTechnologies > Structure Discovery and ClusteringTechnologies > Visualization

Topics

No keywords indexed for this article. Browse by subject →

References

78

[1]

10.1214/088342307000000087

[2]

10.1002/0471725382

[3]

10.1002/0470010940

[4]

10.1214/aoms/1177693054

[5]

Donoho DL (1983)

[6]

10.1002/wics.34

[7]

Hampel FR (1986)

[8]

10.1080/01621459.1993.10476408

[9]

Robust Estimation of a Location Parameter

Peter J. Huber

The Annals of Mathematical Statistics 10.1214/aoms/1177703732

[10]

10.1016/j.csda.2007.11.008

[11]

10.1198/106186004x12632

[12]

10.1080/01621459.1984.10477105

[13]

10.1007/978-94-009-5438-0_20

[14]

Hubert M "Minimum covariance determinant and extensions" Wiley Interdiscip Rev Comput Stat (2017)

[15]

10.1080/00401706.1999.10485670

[16]

10.1080/10618600.2012.672100

[17]

BoudtK RousseeuwPJ VanduffelS VerdonckT. The minimum regularized covariance determinant estimator. Technical Report arXiv:1701.07086 2017. 10.2139/ssrn.2905259

[18]

Stahel WA. Robuste Schätzungen: infinitesimale Optimalität und Schätzungen von Kovarianzmatrizen. PhD Thesis ETH Zürich 1981.

[19]

Donoho DL. Breakdown properties of multivariate location estimators. PhD Qualifying paper Department of Statistics Harvard University Boston 1982.

[20]

10.1080/01621459.1995.10476517

[21]

Rousseeuw PJ "A measure of directional outlyingness with applications to image data and video" J Comput Graph Stat (2017)

[22]

10.1214/aos/1176343347

[23]

10.1214/aos/1176350505

[24]

Tatsuoka KS "On the uniqueness of S‐functionals and M‐functionals under nonelliptical distributions" Ann Stat (2000) 10.1214/aos/1015956714

[25]

10.1198/004017002188618509

[26]

10.1007/s10618-005-0024-4

[27]

10.1007/s001840200191

[28]

10.1016/j.csda.2004.03.018

[29]

10.1007/s10260-007-0048-6

[30]

10.1080/01621459.1990.10474920

[31]

Brownlee KA (1965)

[32]

10.1016/j.csda.2007.01.007

[33]

10.1198/016214507000000950

[34]

10.1214/12-aoas575

[35]

Robust Statistics

Peter J. Huber

Wiley Series in Probability and Statistics 10.1002/0471725250

[36]

10.1214/aoms/1177693245

[37]

10.2307/2288796

[38]

10.1007/978-1-4615-7821-5_15

[39]

10.1214/aos/1176350366

[40]

10.1093/biomet/87.3.603

[41]

10.1198/016214506000000096

[42]

10.1016/s0169-7439(01)00188-5

[43]

10.1016/j.chemolab.2007.01.004

[44]

10.1198/004017004000000563

[45]

10.1002/1099-128x(200009/12)14:5/6<751::aid-cem622>3.0.co;2-d

[46]

10.1007/bf02595862

[47]

10.1080/00401706.2015.1093962

[48]

Steinwart I (2008) 10.1007/978-0-387-77242-4

[49]

10.1007/s11634-010-0068-1

[50]

10.2307/2291457

Showing 50 of 78 references

Cited By

193

Anomaly Detection on Data Streams for Smart Agriculture

Juliet Chebet Moso, Stéphane Cormier · 2021

Agriculture

A Robust Bootstrap Test for Mediation Analysis

Andreas Alfons, Nüfer Yasin Ateş · 2021

Organizational Research Methods

A New Ensemble Method for Detecting Anomalies in Gene Expression Matrices

Laura Selicato, Flavia Esposito · 2021

Mathematics

Machine Learning Analysis of the Bleomycin Mouse Model Reveals the Compartmental and Temporal Inflammatory Pulmonary Fingerprint

Natalie Bordag, Valentina Biasin · 2020

iScience

Metrics

193

Citations

78

References

Details

Published: Nov 23, 2017
Vol/Issue: 8(2)
License: View

Authors

P

Peter J. Rousseeuw

Department of Mathematics KU Leuven Leuven Belgium

M

Mia Hubert

Department of Mathematics KU Leuven Leuven Belgium

Cite This Article

Peter J. Rousseeuw, Mia Hubert (2017). Anomaly detection by robust statistics. WIREs Data Mining and Knowledge Discovery, 8(2). https://doi.org/10.1002/widm.1236

Anomaly detection by robust statistics

You May Also Like