Robust object detection under harsh autonomous‐driving environments

Youngjun Kim; Hyekyoung Hwang; Jitae Shin

doi:10.1049/ipr2.12159

journal article Open Access Mar 04, 2021

Robust object detection under harsh autonomous‐driving environments

Youngjun Kim

Hyekyoung Hwang Jitae Shin

IET Image Processing Vol. 16 No. 4 pp. 958-971 · Institution of Engineering and Technology (IET)

View at Publisher Save 10.1049/ipr2.12159

Abstract

Abstract
In the autonomous driving environment, object instances in an image can be affected by various factors such as camera, driving state, weather, and system component. However, the deep learning‐based vision systems are vulnerable to perturbation, which contains noise. Thus, robust object detection under harsh autonomous‐driving environments is a more difficult than the generic situation. In this paper, it is found that not only the accuracy, but also the speed of the non‐maximum suppression‐based detector can be degraded under harsh environments. Therefore, object detection is handled under a harsh situation with adversarial mechanisms such as adversarial training and adversarial defence. Adversarial defence modules are designed to improve robustness in feature extraction level and define perturbations under a harsh environment for training object detectors to improve the robustness of the model's decision boundary. The proposed adversarial defence and training mechanisms improve the object detector in both accuracy and speed. The proposed method shows a 43.7% mean average precision for the COCO2015 dataset in generic object detection and 39.0% mean average precision for the BDD100K dataset in a driving environment. Furthermore, it achieves a real‐time capability of 23 frames per second.

Topics

No keywords indexed for this article. Browse by subject →

References

42

[1]

Yu F.et al.:Bdd100k: A diverse driving video database with scalable annotation tooling. arXiv:1805.04687 (2018)

[2]

Choi J.et al.:Gaussian yolov3: An accurate and fast object detector using localization uncertainty for autonomous driving. In:Proceedings of the IEEE International Conference on Computer Vision pp.502–511.IEEE Piscataway NJ(2019) 10.1109/iccv.2019.00059

[3]

Vision meets robotics: The KITTI dataset

A Geiger, P Lenz, C Stiller et al.

The International Journal of Robotics Research 10.1177/0278364913491297

[4]

10.1016/j.patrec.2019.11.004

[5]

10.3390/s20020349

[6]

Boloor A.et al.:Simple physical adversarial examples against end‐to‐end autonomous driving models. In:2019 IEEE International Conference on Embedded Software and Systems pp.1–7.IEEE Piscataway NJ(2019) 10.1109/icess.2019.8782514

[7]

Michaelis C.et al.:Benchmarking robustness in object detection: Autonomous driving when winter is coming. arXiv:1907.07484 (2019)

[8]

Eykholt K.et al.:Robust physical‐world attacks on deep learning visual classification. In:Proceedings of the IEEE Conf. on Computer Vision and Pattern Recognition pp.1625–1634.IEEE Piscataway NJ(2018) 10.1109/cvpr.2018.00175

[9]

10.1109/tnnls.2018.2886017

[10]

Zhao Q.et al.:M2det: A single‐shot object detector based on multi‐level feature pyramid network. In:Proc. of the AAAI Conf. on Artificial Intelligencevol.33 pp.9259–9266.AAAI Press Palo Alto CA(2019) 10.1609/aaai.v33i01.33019259

[11]

Tian Z.et al.:Fcos: Fully convolutional one‐stage object detection. In:Proceedings of the IEEE International Conference on Computer Vision pp.9627–9636.IEEE Piscataway NJ(2019) 10.1109/iccv.2019.00972

[12]

Najibi M. Singh B. Davis L.S.:Autofocus: Efficient multi‐scale inference. In:Proceedings of the IEEE International Conference on Computer Vision pp.9745–9755.IEEE Piscataway NJ(2019) 10.1109/iccv.2019.00984

[13]

Mask R-CNN

Kaiming He, Georgia Gkioxari, Piotr Dollar et al.

2017 IEEE International Conference on Computer Vis... 10.1109/iccv.2017.322

[14]

Zhou P.et al.:Scale‐transferrable object detection. In:Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition pp.528–537.IEEE Piscataway NJ(2018) 10.1109/cvpr.2018.00062

[15]

Rethinking the Inception Architecture for Computer Vision

Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe et al.

2016 IEEE Conference on Computer Vision and Patter... 10.1109/cvpr.2016.308

[16]

Chen L.‐C.et al.:Rethinking atrous convolution for semantic image segmentation. arXiv:1706.05587 (2017)

[17]

Deformable Convolutional Networks

Jifeng Dai, Haozhi Qi, Yuwen Xiong et al.

2017 IEEE International Conference on Computer Vis... 10.1109/iccv.2017.89

[18]

Liu S.et al. Receptive field block net for accurate and fast object detection. In:Proc. of the European Conf. on Computer Vision pp.385–400.Springer Berlin(2018) 10.1007/978-3-030-01252-6_24

[19]

Baldi P.:Autoencoders unsupervised learning and deep architectures. In:Proceedings of ICML workshop on unsupervised and transfer learning pp.37–49.MIT Press Cambridge MA(2012)

[20]

Residual Attention Network for Image Classification

Fei Wang, Mengqing Jiang, Chen Qian et al.

2017 IEEE Conference on Computer Vision and Patter... 10.1109/cvpr.2017.683

[21]

Fu C.‐Y.et al.:DSSD: Deconvolutional single shot detector. arXiv:1701.06659 (2017)

[22]

Wang R.J. (2018)

[23]

Dhillon G.S.et al.:Stochastic activation pruning for robust adversarial defense. arXiv:1803.01442 (2018)

[24]

Bijelic M.et al.:Seeing through fog without seeing fog: Deep sensor fusion in the absence of labeled training data. arXiv:1902.08913 (2019) 10.1109/cvpr42600.2020.01170

[25]

Technology Nov 2019. [Online].https://www.sony.net/SonyInfo/technology/stories/IMX490/ Retrieved March 3 2021

[26]

10.1109/lsens.2018.2860990

[27]

Identity Mappings in Deep Residual Networks

Kaiming He, Xiangyu Zhang, Shaoqing Ren et al.

Lecture Notes in Computer Science 10.1007/978-3-319-46493-0_38

[28]

Soft-NMS — Improving Object Detection with One Line of Code

Navaneeth Bodla, Bharat Singh, Rama Chellappa et al.

2017 IEEE International Conference on Computer Vis... 10.1109/iccv.2017.593

[29]

10.1109/access.2018.2807385

[30]

Sinha A. Namkoong H. Duchi J.:Certifying some distributional robustness with principled adversarial training. arXiv:1710.10571 (2017)

[31]

Chakraborty A.et al.:Adversarial attacks and defences: A survey. arXiv:1810.00069 (2018)

[32]

Vincent P. "Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion" J. Machine Learning Res. (2010)

[33]

Su H.et al.:Robust cell detection and segmentation in histopathological images using sparse reconstruction and stacked denoising autoencoders. inInternational Conference on Medical Image Computing and Computer‐Assisted Intervention pp.383–390.Springer Berlin(2015) 10.1007/978-3-319-24574-4_46

[34]

Samangouei P. Kabkab M. Chellappa R.:Defense‐gan: Protecting classifiers against adversarial attacks using generative models. arXiv:1805.06605 (2018)

[35]

Lee H. Han S. Lee J.:Generative adversarial trainer: Defense to adversarial perturbations with gan. arXiv:1705.03387 (2017)

[36]

Tramèr F.et al.:Ensemble adversarial training: Attacks and defenses. arXiv:1705.07204 (2017)

[37]

Microsoft COCO: Common Objects in Context

Tsung-Yi Lin, Michael Maire, Serge Belongie et al.

Lecture Notes in Computer Science 10.1007/978-3-319-10602-1_48

[38]

Zhang S.et al.:Single‐shot refinement neural network for object detection. In:Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition pp.4203–4212.IEEE Piscataway NJ(2018) 10.1109/cvpr.2018.00442

[39]

Ross T.‐Y. et al.:Focal loss for dense object detection. In:Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition pp.2980–2988.IEEE Piscataway NJ(2017)

[40]

Tianzhi0549 tianzhi0549/fcos. [Online].https://github.com/tianzhi0549/FCOS Retrieved March 3 2021

[41]

Tianzhi0549 qijiezhao/m2det. [Online].https://github.com/qijiezhao/M2Det Retrieved March 3 2021

[42]

Bethgelab bethgelab/imagecorruptions. [Online].https://github.com/bethgelab/imagecorruptions Retrieved March 3 2021

Metrics

23

Citations

42

References

Details

Published: Mar 04, 2021
Vol/Issue: 16(4)
Pages: 958-971
License: View

Authors

Y

Youngjun Kim

College of Information and Communication Engineering Sungkyunkwan University Suwon Korea

H

Hyekyoung Hwang

College of Information and Communication Engineering Sungkyunkwan University Suwon Korea

J

Jitae Shin

College of Information and Communication Engineering Sungkyunkwan University Suwon Korea

Funding

National Research Foundation of Korea Award: NRF‐2017R1D1A1B03031752

Cite This Article

Youngjun Kim, Hyekyoung Hwang, Jitae Shin (2021). Robust object detection under harsh autonomous‐driving environments. IET Image Processing, 16(4), 958-971. https://doi.org/10.1049/ipr2.12159

Robust object detection under harsh autonomous‐driving environments

You May Also Like