SGC-VSLAM: A Semantic and Geometric Constraints VSLAM for Dynamic Indoor Environments

Guohao Fan; Cheng Zhao; Dexin Li

doi:10.3390/s20082432

journal article Open Access Apr 24, 2020

SGC-VSLAM: A Semantic and Geometric Constraints VSLAM for Dynamic Indoor Environments

Guohao Fan Cheng Zhao

Dexin Li

Sensors Vol. 20 No. 8 pp. 2432 · MDPI AG

View at Publisher Save 10.3390/s20082432

Abstract

As one of the core technologies for autonomous mobile robots, Visual Simultaneous Localization and Mapping (VSLAM) has been widely researched in recent years. However, most state-of-the-art VSLAM adopts a strong scene rigidity assumption for analytical convenience, which limits the utility of these algorithms for real-world environments with independent dynamic objects. Hence, this paper presents a semantic and geometric constraints VSLAM (SGC-VSLAM), which is built on the RGB-D mode of ORB-SLAM2 with the addition of dynamic detection and static point cloud map construction modules. In detail, a novel improved quadtree-based method was adopted for SGC-VSLAM to enhance the performance of the feature extractor in ORB-SLAM (Oriented FAST and Rotated BRIEF-SLAM). Moreover, a new dynamic feature detection method called semantic and geometric constraints was proposed, which provided a robust and fast way to filter dynamic features. The semantic bounding box generated by YOLO v3 (You Only Look Once, v3) was used to calculate a more accurate fundamental matrix between adjacent frames, which was then used to filter all of the truly dynamic features. Finally, a static point cloud was estimated by using a new drawing key frame selection strategy. Experiments on the public TUM RGB-D (Red-Green-Blue Depth) dataset were conducted to evaluate the proposed approach. This evaluation revealed that the proposed SGC-VSLAM can effectively improve the positioning accuracy of the ORB-SLAM2 system in high-dynamic scenarios and was also able to build a map with the static parts of the real environment, which has long-term application value for autonomous mobile robots.

Topics

No keywords indexed for this article. Browse by subject →

References

38

[1]

Vidal "Ultimate SLAM? Combining Events, Images, and IMU for Robust Visual SLAM in HDR and High-Speed Scenarios" IEEE Robot. Autom. Lett. (2018) 10.1109/lra.2018.2793357

[2]

Li "Ongoing Evolution of Visual SLAM from Geometry to Deep Learning: Challenges and Opportunities" Cogn. Comput. (2018) 10.1007/s12559-018-9591-8

[3]

Klein, G., and Murray, D. (2007, January 13–16). Parallel Tracking and Mapping for Small AR Workspaces. Proceedings of the 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality, Nara, Japan. 10.1109/ismar.2007.4538852

[4]

ORB-SLAM: A Versatile and Accurate Monocular SLAM System

Raul Mur-Artal, J. M. M. Montiel, Juan D. Tardos

IEEE Transactions on Robotics 2015 10.1109/tro.2015.2463671

[5]

ORB-SLAM2: An Open-Source SLAM System for Monocular, Stereo, and RGB-D Cameras

Raul Mur-Artal, Juan D. Tardos

IEEE Transactions on Robotics 2017 10.1109/tro.2017.2705103

[6]

Stühmer, J., Gumhold, S., and Cremers, D. (2010). Real-time dense geometry from a handheld camera. Joint Pattern Recognition Symposium, Springer. 10.1007/978-3-642-15986-2_2

[7]

Engel, J., Schöps, T., and Cremers, D. (2014, January 6–12). LSD-SLAM: Large-scale direct monocular SLAM. Proceedings of the European Conference on Computer Vision, Zurich, Switzerland. 10.1007/978-3-319-10605-2_54

[8]

Cadena "Past, Present, and Future of Simultaneous Localization and Mapping: Toward the Robust-Perception Age" IEEE Trans. Robot. (2016) 10.1109/tro.2016.2624754

[9]

Visual SLAM algorithms: a survey from 2010 to 2016

Takafumi Taketomi, Hideaki Uchiyama, Sei Ikeda

IPSJ Transactions on Computer Vision and Applicati... 10.1186/s41074-017-0027-2

[10]

ORB: An efficient alternative to SIFT or SURF

Ethan Rublee, Vincent Rabaud, Kurt Konolige et al.

2011 International Conference on Computer Vision 10.1109/iccv.2011.6126544

[11]

Fan "Dynamic objects elimination in SLAM based on image fusion" Pattern Recognit. Lett. (2019) 10.1016/j.patrec.2018.10.024

[12]

Sun "Motion removal for reliable RGB-D SLAM in dynamic environments" Robot. Auton. Syst. (2018) 10.1016/j.robot.2018.07.002

[13]

Sedaghat "Uniform Robust Scale-Invariant Feature Matching for Optical Remote Sensing Images" IEEE Trans. Geosci. Remote. Sens. (2011) 10.1109/tgrs.2011.2144607

[14]

Paul "Remote Sensing Optical Image Registration Using Modified Uniform Robust SIFT" IEEE Geosci. Remote. Sens. Lett. (2016) 10.1109/lgrs.2016.2582528

[15]

Bescos "DynaSLAM: Tracking, Mapping, and Inpainting in Dynamic Scenes" IEEE Robot. Autom. Lett. (2018) 10.1109/lra.2018.2860039

[16]

Yazdi "New trends on moving object detection in video images captured by a moving camera: A survey" Comput. Sci. Rev. (2018) 10.1016/j.cosrev.2018.03.001

[17]

Wangsiripitak, S., and Murray, D.W. (2009, January 12–17). Avoiding moving outliers in visual SLAM by tracking moving objects. Proceedings of the 2009 IEEE International Conference on Robotics and Automation, Kobe, Japan. 10.1109/robot.2009.5152290

[18]

Moo Yi, K., Yun, K., Wan Kim, S., Jin Chang, H., and Young Choi, J. (2013, January 23–28). Detection of moving objects with non-stationary cameras in 5.8 ms: Bringing motion detection to your mobile device. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, Portland, OR, USA. 10.1109/cvprw.2013.9

[19]

Sun "Improving RGB-D SLAM in dynamic environments: A motion removal approach" Robot. Auton. Syst. (2017) 10.1016/j.robot.2016.11.012

[20]

Kim "Effective background model-based RGB-D dense visual odometry in a dynamic environment" IEEE Trans. Robot. (2016) 10.1109/tro.2016.2609395

[21]

Zhao "A Compatible Framework for RGB-D SLAM in Dynamic Scenes" IEEE Access (2019) 10.1109/access.2019.2922733

[22]

Li "RGB-D SLAM in Dynamic Environments Using Static Point Weighting" IEEE Robot. Autom. Lett. (2017) 10.1109/lra.2017.2724759

[23]

You Only Look Once: Unified, Real-Time Object Detection

Joseph Redmon, Santosh Divvala, Ross Girshick et al.

2016 IEEE Conference on Computer Vision and Patter... 10.1109/cvpr.2016.91

[24]

SSD: Single Shot MultiBox Detector

Wei Liu, Dragomir Anguelov, Dumitru Erhan et al.

Lecture Notes in Computer Science 10.1007/978-3-319-46448-0_2

[25]

SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation

Vijay Badrinarayanan, Alex Kendall, Roberto Cipolla

IEEE Transactions on Pattern Analysis and Machine... 2017 10.1109/tpami.2016.2644615

[26]

Mask R-CNN

Kaiming He, Georgia Gkioxari, Piotr Dollar et al.

2017 IEEE International Conference on Computer Vis... 10.1109/iccv.2017.322

[27]

Zhang "Semantic SLAM Based on Object Detection and Improved Octomap" IEEE Access (2018) 10.1109/access.2018.2873617

[28]

Li, P., Zhang, G., Zhou, J., Yao, R., and Zhang, X. (2019, January 26–28). Study on Slam Algorithm Based on Object Detection in Dynamic Scene. Proceedings of the 2019 International Conference on Advanced Mechatronic Systems (ICAMechS), Shiga, Japan. 10.1109/icamechs.2019.8861669

[29]

Zhong, F., Wang, S., Zhang, Z., Chen, C., and Wang, Y. (2018, January 12–15). Detect-SLAM: Making Object Detection and SLAM Mutually Beneficial. Proceedings of the 2018 IEEE Winter Conference on Applications of Computer Vision (WACV), Tahoe, NV, USA. 10.1109/wacv.2018.00115

[30]

Yu, C., Liu, Z., Liu, X., Xie, F., Yang, Y., Wei, Q., and Fei, Q. (2018, January 1–5). DS-SLAM: A Semantic Visual SLAM towards Dynamic Environments. Proceedings of the 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Madrid, Spain. 10.1109/iros.2018.8593691

[31]

Han "Dynamic Scene Semantics SLAM Based on Semantic Segmentation" IEEE Access (2020) 10.1109/access.2020.2977684

[32]

Kovacs "Improved Harris Feature Point Set for Orientation-Sensitive Urban-Area Detection in Aerial Images" IEEE Geosci. Remote. Sens. Lett. (2012) 10.1109/lgrs.2012.2224315

[33]

Rosten "Faster and Better: A Machine Learning Approach to Corner Detection" IEEE Trans. Pattern Anal. Mach. Intell. (2008) 10.1109/tpami.2008.275

[34]

Xie "Moving target detection algorithm based on LK optical flow and three-frame difference method" Appl. Sci. Technol. (2016)

[35]

The Pascal Visual Object Classes (VOC) Challenge

Mark Everingham, Luc Van Gool, Christopher K. I. Williams et al.

International Journal of Computer Vision 2009 10.1007/s11263-009-0275-4

[36]

Sturm, J., Engelhard, N., Endres, F., Burgard, W., and Cremers, D. (2012, January 7–12). A benchmark for the evaluation of RGB-D SLAM systems. Proceedings of the 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, Vilamoura, Algarve. 10.1109/iros.2012.6385773

[37]

Kerl, C., Sturm, J., and Cremers, D. (2013, January 6–10). Robust odometry estimation for RGB-D cameras. Proceedings of the 2013 IEEE International Conference on Robotics and Automation, Karlsruhe, Germany. 10.1109/icra.2013.6631104

[38]

Chum, O., Matas, J., and Kittler, J. (2003, January 10–12). Locally Optimized RANSAC. Proceedings of the Joint Pattern Recognition Symposium, Magdeburg, Germany. 10.1007/978-3-540-45243-0_31

Cited By

33

An Overview on Visual SLAM: From Tradition to Semantic

Weifeng Chen, Guangtao Shang · 2022

Remote Sensing

Metrics

33

Citations

38

References

Details

Published: Apr 24, 2020
Vol/Issue: 20(8)
Pages: 2432
License: View

Authors

G

Guohao Fan

School of Mechanical and Precision Instrument Engineering, Xi’an University of Technology, Xi’an 710048, China

C

Cheng Zhao

School of Mechanical and Precision Instrument Engineering, Xi’an University of Technology, Xi’an 710048, China

D

Dexin Li

School of Mechanical and Precision Instrument Engineering, Xi’an University of Technology, Xi’an 710048, China

Funding

National Natural Science Foundation of China Award: 51475365

Cite This Article

Guohao Fan, Cheng Zhao, Dexin Li (2020). SGC-VSLAM: A Semantic and Geometric Constraints VSLAM for Dynamic Indoor Environments. Sensors, 20(8), 2432. https://doi.org/10.3390/s20082432

SGC-VSLAM: A Semantic and Geometric Constraints VSLAM for Dynamic Indoor Environments

You May Also Like