Wavelength-based Attributed Deep Neural Network for Underwater Image Restoration

Prasen Sharma; Ira Bisht; Arijit Sur

doi:10.1145/3511021

journal article Jan 05, 2023

Wavelength-based Attributed Deep Neural Network for Underwater Image Restoration

Prasen Sharma

Ira Bisht

Arijit Sur

ACM Transactions on Multimedia Computing, Communications, and Applications Vol. 19 No. 1 pp. 1-23 · Association for Computing Machinery (ACM)

View at Publisher Save 10.1145/3511021

Abstract

Background:
Underwater images, in general, suffer from low contrast and high color distortions due to the non-uniform attenuation of the light as it propagates through the water. In addition, the degree of attenuation varies with the wavelength, resulting in the asymmetric traversing of colors. Despite the prolific works for
underwater image restoration
(UIR) using deep learning, the above asymmetricity has not been addressed in the respective network engineering.

Contributions:
As the first novelty, this article shows that attributing the right receptive field size (
context
) based on the traversing range of the color channel may lead to a substantial performance gain for the task of UIR. Further, it is important to suppress the irrelevant multi-contextual features and increase the representational power of the model. Therefore, as a second novelty, we have incorporated an attentive skip mechanism to adaptively refine the learned multi-contextual features. The proposed framework, called
Deep WaveNet
, is optimized using the traditional pixel-wise and feature-based cost functions. An extensive set of experiments have been carried out to show the efficacy of the proposed scheme over existing best-published literature on benchmark datasets. More importantly, we have demonstrated a comprehensive validation of enhanced images across various high-level vision tasks, e.g., underwater image semantic segmentation and diver’s 2D pose estimation. A sample video to exhibit our real-world performance is available at
https://tinyurl.com/yzcrup9n
. Also, we have open-sourced our framework at
https://github.com/pksvision/Deep-WaveNet-Underwater-Image-Restoration
.

Topics

No keywords indexed for this article. Browse by subject →

References

90

[1]

10.1016/j.neunet.2013.01.008

[2]

C. Ancuti, C. O. Ancuti, T. Haber, and P. Bekaert. 2012. Enhancing underwater images and videos by fusion. In 2012 IEEE Conference on Computer Vision and Pattern Recognition. 81–88. DOI:10.1109/CVPR.2012.6247661 10.1109/cvpr.2012.6247661

[3]

Color Balance and Fusion for Underwater Image Enhancement

Codruta O. Ancuti, Cosmin Ancuti, Christophe De Vleeschouwer et al.

IEEE Transactions on Image Processing 10.1109/tip.2017.2759252

[4]

Anonymous. 2022. Patches are All you need? In Submitted to the 10th International Conference on Learning Representations. https://openreview.net/forum?id=TVHS5Y4dNvM. Under review.

[5]

Underwater Image Enhancement Based on Global and Local Equalization of Histogram and Dual-Image Multi-Scale Fusion

Linfeng Bai, Weidong Zhang, Xipeng Pan et al.

IEEE Access 10.1109/access.2020.3009161

[6]

10.1109/tpami.2020.2977624

[7]

Z. Cao, G. Hidalgo Martinez, T. Simon, S. Wei, and Y. A. Sheikh. 2019. OpenPose: Realtime multi-person 2D pose estimation using part affinity fields. IEEE Transactions on Pattern Analysis and Machine Intelligence (2019).

[8]

Zhe Cao, Tomas Simon, Shih-En Wei, and Yaser Sheikh. 2017. Realtime multi-person 2d pose estimation using part affinity fields. In CVPR.

[9]

10.1109/tcsvt.2020.3035108

[10]

10.1109/access.2020.3004141

[11]

10.1109/tip.2011.2179666

[12]

ImageNet: A large-scale hierarchical image database

Jia Deng, Wei Dong, Richard Socher et al.

2009 IEEE Conference on Computer Vision and Patter... 2009 10.1109/cvpr.2009.5206848

[13]

Nithish Divakar and R. Venkatesh Babu. 2017. Image denoising via CNNs: An adversarial approach. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops.

[14]

Chao Dong, Yubin Deng, Chen Change Loy, and Xiaoou Tang. 2015. Compression artifacts reduction by a deep convolutional network. In 2015 IEEE International Conference on Computer Vision (ICCV’15). 576–584. DOI:10.1109/ICCV.2015.73 10.1109/iccv.2015.73

[15]

Image Super-Resolution Using Deep Convolutional Networks

Chao Dong, Chen Change Loy, Kaiming He et al.

IEEE Transactions on Pattern Analysis and Machine... 10.1109/tpami.2015.2439281

[16]

10.1109/lsp.2020.2988590

[17]

Cameron Fabbri, Md Jahidul Islam, and Junaed Sattar. 2018. Enhancing underwater imagery using generative adversarial networks. In 2018 IEEE International Conference on Robotics and Automation (ICRA’18). DOI:10.1109/ICRA.2018.8460552

[18]

Xueyang Fu, Peixian Zhuang, Yue Huang, Yinghao Liao, Xiao-Ping Zhang, and Xinghao Ding. 2014. A retinex-based enhancing approach for single underwater image. In 2014 IEEE International Conference on Image Processing (ICIP’14). DOI:10.1109/ICIP.2014.7025927

[19]

Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In Advances in Neural Information Processing Systems, Z. Ghahramani, M. Welling, C. Cortes, N. Lawrence, and K. Q. Weinberger (Eds.). Curran Associates, Inc.

[20]

10.1109/joe.2019.2911447

[21]

10.1109/access.2020.3041280

[22]

Single Image Haze Removal Using Dark Channel Prior

Kaiming He, Jian Sun, Xiaoou Tang

IEEE Transactions on Pattern Analysis and Machine... 10.1109/tpami.2010.168

[23]

10.1109/access.2020.3006359

[24]

Sergey Ioffe and Christian Szegedy. 2015. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In Proceedings of the 32nd International Conference on International Conference on Machine Learning - Volume 37 (ICML’15). JMLR.org.

[25]

Md Jahidul Islam, Chelsey Edge, Yuyang Xiao, Peigen Luo, Muntaqim Mehtaz, Christopher Morse, Sadman Sakib Enan, and Junaed Sattar. 2020. Semantic segmentation of underwater imagery: Dataset and benchmark. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS’20). IEEE/RSJ.

[26]

10.15607/rss.2020.xvi.018

[27]

Md Jahidul Islam, Sadman Sakib Enan, Peigen Luo, and Junaed Sattar. 2020. Underwater image super-resolution using deep residual multipliers. In 2020 IEEE International Conference on Robotics and Automation (ICRA’20). DOI:10.1109/ICRA40945.2020.9197213

[28]

Fast Underwater Image Enhancement for Improved Visual Perception

Md Jahidul Islam, Youya Xia, Junaed Sattar

IEEE Robotics and Automation Letters 10.1109/lra.2020.2974710

[29]

Fast Underwater Image Enhancement for Improved Visual Perception

Md Jahidul Islam, Youya Xia, Junaed Sattar

IEEE Robotics and Automation Letters 10.1109/lra.2020.2974710

[30]

10.1109/tmm.2019.2938340

[31]

Justin Johnson, Alexandre Alahi, and Li Fei-Fei. 2016. Perceptual losses for real-time style transfer and super-resolution. In European Conference on Computer Vision.

[32]

Jiwon Kim, Jung Kwon Lee, and Kyoung Mu Lee. 2016. Accurate image super-resolution using very deep convolutional networks. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR Oral’16).

[33]

Diederik P. Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In 3rd International Conference on Learning Representations (ICLR’15), Conference Track Proceedings, Yoshua Bengio and Yann LeCun (Eds.).

[34]

Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. 2012. ImageNet classification with deep convolutional neural networks. In Proceedings of the 25th International Conference on Neural Information Processing Systems - Volume 1 (NIPS’12). 1097–1105.

[35]

Christian Ledig, Lucas Theis, Ferenc Huszar, Jose Caballero, Andrew Cunningham, Alejandro Acosta, Andrew Aitken, Alykhan Tejani, Johannes Totz, Zehan Wang, and Wenzhe Shi. 2017. Photo-realistic single image super-resolution using a generative adversarial network. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’17).

[36]

Underwater Image Enhancement via Medium Transmission-Guided Multi-Color Space Embedding

Chongyi Li, Saeed Anwar, Junhui Hou et al.

IEEE Transactions on Image Processing 10.1109/tip.2021.3076367

[37]

An Underwater Image Enhancement Benchmark Dataset and Beyond

Chongyi Li, Chunle Guo, Wenqi Ren et al.

IEEE Transactions on Image Processing 10.1109/tip.2019.2955241

[38]

10.1109/tip.2016.2612882

[39]

10.1109/lsp.2018.2792050

[40]

C. Li, J. Quo, Y. Pang, S. Chen, and J. Wang. 2016. Single underwater image restoration by blue-green channels dehazing and red channel correction. In 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP’16). 1731–1735. DOI:10.1109/ICASSP.2016.7471973 10.1109/icassp.2016.7471973

[41]

10.1109/tip.2018.2810539

[42]

10.1109/access.2019.2925209

[43]

10.1109/tip.2019.2908802

[44]

Ding Liu, Bihan Wen, Xianming Liu, Zhangyang Wang, and Thomas S. Huang. 2018. When image denoising meets high-level vision tasks: A deep learning approach. In Proceedings of the 27th International Joint Conference on Artificial Intelligence (IJCAI’18). AAAI Press, 842–848.

[45]

10.1109/access.2019.2928976

[46]

Pengju Liu, Hongzhi Zhang, Kai Zhang, Liang Lin, and Wangmeng Zuo. 2018. Multi-level wavelet-CNN for image restoration. In 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW’18). DOI:10.1109/CVPRW.2018.00121

[47]

10.1109/lgrs.2019.2950056

[48]

Fully convolutional networks for semantic segmentation

Jonathan Long, Evan Shelhamer, Trevor Darrell

2015 IEEE Conference on Computer Vision and Patter... 2015 10.1109/cvpr.2015.7298965

[49]

10.1109/access.2017.2648845

[50]

10.1142/s0218001418540186

Showing 50 of 90 references

Metrics

154

Citations

90

References

Details

Published: Jan 05, 2023
Vol/Issue: 19(1)
Pages: 1-23
License: View

Authors

P

Prasen Sharma

Indian Institute of Technology Guwahati, Assam, India

I

Ira Bisht

Indian Institute of Technology Guwahati, Assam, India

A

Arijit Sur

Indian Institute of Technology Guwahati, Assam, India

Funding

Department of Science and Technology, India Award: DST/NMICPS/TIH12/IITG/2020

Department of Biotechnology, Govt. of India Award: BT/COE/34/SP28408/2018

IITG Technology Innovation and Development Foundation

Cite This Article

Prasen Sharma, Ira Bisht, Arijit Sur (2023). Wavelength-based Attributed Deep Neural Network for Underwater Image Restoration. ACM Transactions on Multimedia Computing, Communications, and Applications, 19(1), 1-23. https://doi.org/10.1145/3511021

Wavelength-based Attributed Deep Neural Network for Underwater Image Restoration

You May Also Like