Research PaperNo Access

Blind Image Inpainting Using Low-Dimensional Manifold Regularization

Mei Gao

School of Information Science and Technology, Northwest University, Xi’an 710127, P. R. China

School of Freshmen, Xi’an Technological University, Xi’an 710021, P. R. China

E-mail Address: gaomei@xatu.edu.cn

Search for more papers by this author

and

Baosheng Kang

School of Information Science and Technology, Northwest University, Xi’an 710127, P. R. China

E-mail Address: bskang@nwu.edu.cn

Corresponding author.

Search for more papers by this author

https://doi.org/10.1142/S0218126622502115Cited by:1 (Source: Crossref)

Abstract

In this paper, we present a novel method for blind image inpainting, which can restore images with missing or corrupted pixels, or images where the location of the damaged pixels is unknown. The method applies weighted nonlocal Laplacian to address the problem of blind image inpainting using low-dimensional manifold model (LDMM) regularization, and uses semi-local blocks instead of point integrals to implement constraints in LDMM. This solves the problem of low solution efficiency caused by the asymmetry of the linear equations solved by point integration, and the problem of the high iteration count to get good restoration effect. Experiments show that our method is competitive with latest methods in terms of both repairing images with large missing pixels rate and inpainting speed.

This paper was recommended by Regional Editor Takuro Sato.

Keywords:

References

1. A. C. Bovik , Handbook of Image and Video Processing (Elsevier Academic Press, New York, 2010). Google Scholar
2. R. Karkare, R. Paffenroth and G. Mahindre, Blind image denoising and inpainting using robust hadamard autoencoders, preprint (2021), arXiv:2101.10876. Google Scholar
3. Y. Wang, Y. C. Chen, X. Tao and J. Jia , VCNet: A robust approach to blind image inpainting, Computer Vision– ECCV 2020:16th European Conf. (Springer, Cham, 2020), pp. 752–768. Crossref, Google Scholar
4. X. Li, R. Yu, H. Shen and M. Yu , Blind image inpainting using pyramid GAN on thyroid ultrasound images, 2019 IEEE Int. Conf. Bioinformatics and Biomedicine (BIBM), San Diego, CA, November 2019, pp. 678–683. Google Scholar
5. Y. Liu, J. Pan and Z. Su , Deep blind image inpainting, Int. Conf. Intelligent Science and Big Data Engineering (Springer, Cham, 2019), pp. 128–141. Crossref, Google Scholar
6. F. Wu, Y. Kong, W. Dong and Y. Wu , Gradient-aware blind face inpainting for deep face verification, Neurocomputing 331 (2019) 301–311. Crossref, Web of Science, Google Scholar
7. N. Cai, Z. Su, Z. Lin, H. Wang, Z. Yang and B. W. K. Ling , Blind inpainting using the fully convolutional neural network, Vis. Comput. 33 (2017) 249–261. Crossref, Web of Science, Google Scholar
8. F. Dong and N. Li , Multiplicative noise removal and blind inpainting of ultrasound images based on a new variational framework, Mach. Vis. Appl. 32 (2021) 86. Crossref, Web of Science, Google Scholar
9. M. Gao, B. Kang, X. Feng, L. Cao and W. Zhang , Low dimensional manifold regularization based blind image inpainting and non-uniform impulse noise recovery, IEEE Access 8 (2020) 200551–200560. Crossref, Web of Science, Google Scholar
10. M. Derle and G. Gagare , Sliding window based blind image inpainting to remove impulse noise from image, Int. J. Innov. Res. Comput. Commun. Eng. 4 (2016) 17120–17127. Crossref, Google Scholar
11. X. Y. Wang and L. J. Deng , Blind image inpainting based on TV model and edge detection, Applied Mechanics, Mechatronics and Intelligent Systems (World Scientific, Singapore, 2015), pp. 385–392. Google Scholar
12. M. V. Afonso and J. M. R. Sanches , Blind inpainting using $l_{0}$ $l_{0}$ and total variation regularization, IEEE Trans. Image Process. 24 (2015) 2239–2253. Crossref, Web of Science, Google Scholar
13. B. Dong, H. Ji, J. Li, Z. Shen and Y. Xu , Wavelet frame based blind image inpainting, Appl. Comput. Harmon. Anal. 32 (2012) 268–279. Crossref, Web of Science, Google Scholar
14. K. Yu, L. Tan, M. Aloqaily, H. Yang and Y. Jararweh , Blockchain-enhanced data sharing with traceable and direct revocation in IIoT, IEEE Trans. Ind. Inf. 17 (2021) 7669–7678. Crossref, Web of Science, Google Scholar
15. H. Li, Q. Zheng, X. Qi, W. Yan, Z. Wen, N. Li and C. Tang , Neural network-based mapping mining of image style transfer in big data systems, Comput. Intell. Neurosci. 2021 (2021) 8387382. Web of Science, Google Scholar
16. L. Tan, K. Yu, N. Shi, C. Yang, W. Wei and H. Lu , Towards secure and privacy-preserving data sharing for COVID-19 medical records: A blockchain-empowered approach, IEEE Trans. Netw. Sci. Eng. 9 (2022) 271–281. Crossref, Web of Science, Google Scholar
17. K. Yu et al., Securing critical infrastructures: Deep-learning-based threat detection in IIoT, IEEE Commun. Mag.59 (2021) 76–82. Crossref, Web of Science, Google Scholar
18. K. Yu, L. Tan, L. Lin, X. Cheng, Z. Yi and T. Sato , Deep-learning-empowered breast cancer auxiliary diagnosis for 5GB remote e-health, IEEE Wirel. Commun. 28 (2021) 54–61. Crossref, Web of Science, Google Scholar
19. C. Feng et al., Attribute-based encryption with parallel outsourced decryption for edge intelligent IoV, IEEE Trans. Veh. Technol. 69 (2020) 13784–13795. Crossref, Web of Science, Google Scholar
20. K. Yu, Z. Guo, Y. Shen, W. Wang, J. C. Lin and T. Sato , Secure artificial intelligence of things for implicit group recommendations, IEEE Internet Things J. 9 (2022) 2698–2707. Crossref, Web of Science, Google Scholar
21. L. Tan, K. Yu, F. Ming, X. Cheng and G. Srivastava , Secure and resilient artificial intelligence of things: A HoneyNet approach for threat detection and situational awareness, IEEE Consum. Electron. Mag. (2021), https://doi.org/10.1109/MCE.2021.3081874. Crossref, Web of Science, Google Scholar
22. K. Yu, M. Arifuzzaman, Z. Wen, D. Zhang and T. Sato , A key management scheme for secure communications of information centric advanced metering infrastructure in smart grid, IEEE Trans. Instrum. Meas. 64 (2015) 2072–2085. Crossref, Web of Science, Google Scholar
23. H. Li, K. Yu, B. Liu, C. Feng, Z. Qin and G. Srivastava , An efficient ciphertext-policy weighted attribute-based encryption for the internet of health things, IEEE J. Biomed. Health Inf. (2021), https://doi.org/10.1109/JBHI.2021.3075995. Web of Science, Google Scholar
24. C. Feng, B. Liu, K. Yu, S. K. Goudos and S. Wan , Blockchain-empowered decentralized horizontal federated learning for 5G-enabled UAVs, IEEE Trans. Ind. Inf. 18 (2022) 3582–3592. Crossref, Web of Science, Google Scholar
25. L. Tan et al., Speech emotion recognition enhanced traffic efficiency solution for autonomous vehicles in a 5G-enabled space-air-ground integrated intelligent transportation system, IEEE Trans. Intell. Transp. Syst. 23 (2021) 2830–2842. Crossref, Web of Science, Google Scholar
26. C. Feng, B. Liu, Z. Guo, K. Yu, Z. Qin and K. K. R. Choo , Blockchain-based cross-domain authentication for intelligent 5G-enabled internet of drones, IEEE Internet Things J. 9 (2021) 6224–6238. Crossref, Web of Science, Google Scholar
27. S. Osher, Z. Shi and W. Zhu , Low dimensional manifold model for image processing, SIAM J. Imaging Sci. 10 (2017) 1669–1690. Crossref, Web of Science, Google Scholar
28. H. Li, Q. Zheng, J. Zhang, Z. Du, Z. Li and B. Kang , Pix2Pix-based grayscale image coloring method, J. Comput.-Aided Des. Comput. Graph. 33 (2021) 929–938. Google Scholar
29. W. Cong, G. Wang, Q. Yang, J. Li, J. Hsieh and R. Lai , CT image reconstruction in a low dimensional manifold, Inverse Probl. Imaging 13 (2019) 449–460. Crossref, Web of Science, Google Scholar
30. W. Zhu, Z. Shi and S. Osher , Low dimensional manifold model in hyperspectral image reconstruction, Hyperspectral Image Analysis (Springer, Cham, 2020). Crossref, Google Scholar
31. H. Li, Q. Zheng, W. Yan, R. Tao, X. Qi and Z. Wen , Image super-resolution reconstruction for secure data transmission in internet of things environment, Math. Biosci. Eng. 18 (2021) 6652–6672. Crossref, Web of Science, Google Scholar
32. H. Li, M. Zhang, Z. Yu, Z. Li and N. Li , An improved pix2pix model based on Gabor filter for robust color image rendering, Math. Biosci. Eng. 19 (2022) 86–101. Crossref, Web of Science, Google Scholar
33. Z. Shi, S. Osher and W. Zhu , Generalization of the weighted nonlocal Laplacian in low dimensional manifold model, J. Sci. Comput. 75 (2018) 638–656. Crossref, Web of Science, Google Scholar
34. T. Goldstein and S. Osher , The split Bregman method for L1-regularized problems, SIAM J. Imaging Sci. 2 (2009) 323–343. Crossref, Web of Science, Google Scholar
35. M. V. Afonso, J. M. Bioucas-Dias and M. A. T. Figueiredo , Fast image recovery using variable splitting and constrained optimization, IEEE Trans. Image Process. 19 (2010) 2345–2356. Crossref, Web of Science, Google Scholar
36. Z. Li and Z. Shi , A convergent point integral method for isotropic elliptic equations on a point cloud, Multiscale Model. Simul. 14 (2016) 874–905. Crossref, Web of Science, Google Scholar
37. Z. Shi and J. Sun, Convergence of the point integral method for Poisson equation on point cloud, preprint (2014), arXiv:1403.2141v4. Google Scholar
38. K. Yu et al., A blockchain-based Shamir’s threshold cryptography scheme for data protection in industrial internet of things settings, IEEE Internet Things J. (2021), https://doi.org/10.1109/JIOT.2021.3125190. Web of Science, Google Scholar
39. H. Li, M. Zhang, K. Yu, X. Qi and J. Tong , A displacement estimated method for real time tissue ultrasound elastography, Mobile Netw. Appl. 26 (2021) 2014–2023. Crossref, Web of Science, Google Scholar
40. R. C. Gonzalez and R. E. Woods , Digital Image Processing (Pearson Education, Upper Saddle River, 2008). Google Scholar
41. Y. Wang, A. Szlam and G. Lerman , Robust locally linear analysis with applications to image denoising and blind inpainting, SIAM J. Imaging Sci. 6 (2013) 526–562. Crossref, Web of Science, Google Scholar
42. http://www.math.duke.edu/yjwang/kals.htm. Google Scholar
43. https://github.com/manyaafonso/Blind-Inpainting-l0-TV. Google Scholar
44. http://sipi.usc.edu/services/data-base/index.html. Google Scholar
45. S. Durand, J. Fadili and M. Nikolova , Multiplicative noise removal using $ℓ$ $ℓ$ 1 fidelity on frame coefficients, J. Math. Imaging Vis. 36 (2010) 201–226. Crossref, Web of Science, Google Scholar
46. Z. Wang, A. C. Bovik, H. R. Sheikh and E. P. Simoncelli , Image quality assessment: From error visibility to structural similarity, IEEE Trans. Image Process. 13 (2004) 600–612. Crossref, Web of Science, Google Scholar