BishtRef
Articles Journals Publishers Authors Subjects Most Cited New Papers Year Stats Open Access
journal article Jan 31, 2022

A Survey on Active Deep Learning: From Model Driven to Data Driven

Peng Liu ORCID Rajiv Ranjan ORCID Guojin He Lei Zhao ORCID
ACM Computing Surveys Vol. 54 No. 10s pp. 1-34 · Association for Computing Machinery (ACM)
View at Publisher Save 10.1145/3510414
Abstract
Which samples should be labelled in a large dataset is one of the most important problems for the training of deep learning. So far, a variety of active sample selection strategies related to deep learning have been proposed in the literature. We defined them as Active Deep Learning (ADL) only if their predictor or selector is a deep model, where the basic learner is called the predictor and the labeling schemes are called the selector. In this survey, we categorize ADL into model-driven ADL and data-driven ADL by whether its selector is model driven or data driven. We also introduce the different characteristics of the two major types of ADL, respectively. We summarized three fundamental factors in the designation of a selector. We pointed out that, with the development of deep learning, the selector in ADL also is experiencing the stage from model driven to data driven. The advantages and disadvantages between data-driven ADL and model-driven ADL are thoroughly analyzed. Furthermore, different sub-classes of data-drive or model-driven ADL are also summarized and discussed emphatically. Finally, we survey the trend of ADL from model driven to data driven.
Topics

No keywords indexed for this article. Browse by subject →

References
129
[1]
Deep Residual Learning for Image Recognition

Kaiming He, Xiangyu Zhang, Shaoqing Ren et al.

2016 IEEE Conference on Computer Vision and Patter... 10.1109/cvpr.2016.90
[2]
ImageNet: A large-scale hierarchical image database

Jia Deng, Wei Dong, Richard Socher et al.

2009 IEEE Conference on Computer Vision and Patter... 10.1109/cvpr.2009.5206848
[4]
Peng Liu Liping Di Qian Du and Lizhe Wang. 2018. Remote sensing big data: Theory methods and applications. Remote Sensing 10 5 (2018) 711. 10.3390/rs10050711
[6]
Burr Settles. 2009. Active Learning Literature Survey. Technical Report. University of Wisconsin—Madison Department of Computer Sciences.
[8]
Fredrik Olsson. 2009. A literature survey of active machine learning in the context of natural language processing. https://www.ccs.neu.edu/home/vip/teach/MLcourse/4_boosting/materials/SICS-T--2009-06--SE.pdf.
[11]
Pengzhen Ren Yun Xiao Xiaojun Chang Po-Yao Huang Zhihui Li Xiaojiang Chen and Xin Wang. 2020. A survey of deep active learning. arXiv:2009.00236. Retrieved from https://arxiv.org/abs/2009.00236.
[12]
Christopher Schröder and Andreas Niekler. 2020. A survey of active learning for text classification using deep neural networks.arXiv:2008.07267. Retrieved from https://arxiv.org/abs/2008.07267.
[13]
Samuel Budd Emma C. Robinson and Bernhard Kainz. 2021. A survey on active learning and human-in-the-loop deep learning for medical image analysis. Medical Image Analysis 71 (2021) 102062. 10.1016/j.media.2021.102062
[16]
Jordan T. Ash and Ryan P. Adams. 2019. On the difficulty of warm-starting neural network training. arxiv:1910.08475. Retrieved from http://arxiv.org/abs/1910.08475.
[20]
Yarin Gal Riashat Islam and Zoubin Ghahramani. 2017. Deep bayesian active learning with image data. arXiv:1703.02910. Retrieved from https://arxiv.org/abs/1703.02910.
[22]
Mark Woodward and Chelsea Finn. 2017. Active one-shot learning. arXiv:1702.06559. Retrieved from http://arxiv.org/abs/1702.06559.
[23]
Kunkun Pang Mingzhi Dong Yang Wu and Timothy M. Hospedales. 2018. Meta-learning transferable active learning policies by deep reinforcement learning. arxXiv:1806.04798. Retrieved from http://arxiv.org/abs/1806.04798.
[24]
Sachin Ravi and Hugo Larochelle. 2018. Meta-learning for batch mode active learning. In Proceedings of the 6th International Conference on Learning Representations (ICLR’18). OpenReview.net.
[25]
Gabriella Contardo Ludovic Denoyer and Thierry Artières. 2017. A meta-learning approach to one-step active learning. arXiv:1706.08334. Retrieved from http://arxiv.org/abs/1706.08334.
[26]
Jia-Jie Zhu and José Bento. 2017. Generative adversarial active learning. arXiv:1702.07956. Retrieved from http://arxiv.org/abs/1702.07956.
[28]
Melanie Ducoffe and Frederic Precioso. 2018. Adversarial active learning for deep networks: A margin based approach. arXiv:1802.09841. Retrieved from https://arxiv.org/abs/1802.09841.
[30]
Vít Ruzicka, Stefano D’Aronco, Jan Dirk Wegner, and Konrad Schindler. 2020. Deep active learning in remote sensing for data efficient change detection. CoRR abs/2008.11201.
[38]
Stacked Sparse Autoencoder (SSAE) for Nuclei Detection on Breast Cancer Histopathology Images

Jun Xu, Lei Xiang, Qingshan Liu et al.

IEEE Transactions on Medical Imaging 10.1109/tmi.2015.2458702
[42]
Ozan Sener and Silvio Savarese. 2017. A geometric approach to active learning for convolutional neural networks. arXiv: abs/1708.00489. Retrieved from https://arxiv.org/abs/1708.00489.
[43]
Ozan Sener and Silvio Savarese. 2017. Active learning for convolutional neural networks: A core-set approach. arXiv:1708.00489. Retrieved from https://arxiv.org/abs/1708.00489.
[44]
Prateek Munjal Nasir Hayat Munawar Hayat Jamshid Sourati and Shadab Khan. 2020. Towards robust and reproducible active learning using neural networks (unpublished).
[45]
R. A. Fisher. 1992. On the Mathematical Foundations of Theoretical Statistics. Springer, New York, NY, 11–44.
[47]
Tong Zhang. 2000. The value of unlabeled data for classification problems. In Proceedings of the 17th International Conference on Machine Learning. Morgan Kaufmann, 1191–1198.
[48]
Burr Settles, Mark Craven, and Soumya Ray. 2008. Multiple-instance active learning. In Advances in Neural Information Processing Systems. MIT Press, 1289–1296.
[50]
Kamalika Chaudhuri, Sham M. Kakade, and Praneeth Netrapalli, et al. 2015. Convergence rates of active learning for maximum likelihood estimation. In Advances in Neural Information Processing Systems 28: Annual Conference on Neural Information Processing Systems, Corinna Cortes, Neil D. Lawrence, and Daniel D. Lee, et al. (Eds.). 1090–1098.

Showing 50 of 129 references

Metrics
130
Citations
129
References
Details
Published
Jan 31, 2022
Vol/Issue
54(10s)
Pages
1-34
License
View
Authors
P
Peng Liu

Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing, China

R
Rajiv Ranjan

the School of Computing, Newcastle University, Newcastle, UK

G
Guojin He

Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing, China

L
Lei Zhao

School of Information Science and Technology, Beijing Forestry University, Beijing, China

Funding
NSFC Award: 61731022, and 41971397
Cite This Article
Peng Liu, Rajiv Ranjan, Guojin He, et al. (2022). A Survey on Active Deep Learning: From Model Driven to Data Driven. ACM Computing Surveys, 54(10s), 1-34. https://doi.org/10.1145/3510414
Related

You May Also Like

Data clustering

A. K. Jain, M. N. Murty · 1999

9,568 citations

Anomaly detection

Varun Chandola, Arindam Banerjee · 2009

8,799 citations

Machine learning in automated text categorization

Fabrizio Sebastiani · 2002

5,027 citations

Object tracking

Alper Yilmaz, Omar Javed · 2006

3,632 citations

A Survey on Bias and Fairness in Machine Learning

Ninareh Mehrabi, Fred Morstatter · 2021

3,466 citations