No Access

Improvement in CNN-Based Multifocus Image Fusion Algorithm with Triangulated Fuzzy Filter

Meenu Manchanda

Vaish College of Engineering, Rohtak, Haryana, India

E-mail Address: meenumanchanda73@gmail.com

Search for more papers by this author

and

Deepak Gambhir

Galgotias College of Engineering and Technology, K.P. - II, Greater Noida, Uttar Pradesh, India

E-mail Address: gambhir.deepak@gmail.com

Corresponding author.

Search for more papers by this author

https://doi.org/10.1142/S0219467821500200Cited by:3 (Source: Crossref)

Abstract

Multifocus image fusion is a demanding research field due to the utilization of modern imaging devices. Generally, the scene to be captured contains objects at different distances from these devices and so a set of multifocus images of the scene is captured with different objects in-focus. However, to improve the situational awareness of the captured scene, these sets of images are required to be fused together. Therefore, a multifocus image fusion algorithm based on Convolutional Neural Network (CNN) and triangulated fuzzy filter is proposed. A CNN is used to extract information regarding focused pixels of input images and the same is used as fusion rule for fusing the input images. The focused information so extracted may still need to be refined near the boundaries. Therefore, asymmetrical triangular fuzzy filter with the median center (ATMED) is employed to correctly classify the pixels near the boundary. The advantage of using this filter is to rely on precise detection results since any misdetection may considerably degrade the fusion quality. The performance of the proposed algorithm is compared with the state-of-art image fusion algorithms, both subjectively and objectively. Various parameters such as edge strength ( $Q$ ), fusion loss (FL), fusion artifacts (FA), entropy ( $H$ ), standard deviation (SD), spatial frequency (SF), structural similarity index measure (SSIM) and feature similarity index measure (FSIM) are used to evaluate the performance of the proposed algorithm. Experimental results proved that the proposed fusion algorithm produces a fused image that contains all-in-one focused pixels and is better than those obtained using other popular and latest image fusion works.

Keywords:

Remember to check out the Check out our Most Cited Articles!
Check out these titles on Image Analysis

References

1. S. Paul, I. S. Sevcenco and P. Agathoklis, “Multi-exposure and multi-focus image fusion in gradient domain,” J. Circuits, Syst. Comput. 25(10), 1650123 (2016). Link, Web of Science, Google Scholar
2. K. Zhan, J. Teng, Q. Li and J. Shi, “A novel explicit multi-focus image fusion method,” J. Inf. Hiding Multimedia Signal Process. 6(3), 600–612 (2015). Google Scholar
3. S. Li, X. Kang and J. Hu, “Image fusion with guided filtering,” IEEE Trans. Image Process. 22(7), 2864–2875 (2013). Crossref, Web of Science, Google Scholar
4. M. Manchanda and D. Gambhir, “Multifocus image fusion based on waveatom transform,” Sadhana 44, 2 (2019). Crossref, Web of Science, Google Scholar
5. M. Manchanda and R. Sharma, “Fuzzy transform-based fusion of multiple images,” Int. J. Image Graphic. 17(2), 1750008 (2017). Link, Web of Science, Google Scholar
6. Y. Liu, X. Chen, Z. Wang, Z. J. Wang, X. Ward and R. K. Wang, “Deep learning for pixel-level image fusion: Recent advances and future prospects,” Inf. Fusion 42, 158–173 (2018). Crossref, Web of Science, Google Scholar
7. J. Li, G. Yuan and H. Fan, “Multifocus image fusion using wavelet-domain-based deep CNN,” Computational Intelligence and Neuroscience (2019), p. 2019. Web of Science, Google Scholar
8. S. Li, X. Kang, J. Hu and B. Yang, “Image matting for fusion of multi-focus images in dynamic scenes,” Inf. Fusion 14(2), 147–162 (2013). Crossref, Web of Science, Google Scholar
9. J. Tian and L. Chen, “Adaptive multi-focus image fusion using a wavelet-based statistical sharpness measure,” Signal Process. 92(9), 2137–2146 (2012). Crossref, Web of Science, Google Scholar
10. M. Manchanda, D. Gambhir and S. K. Singh, “An improved multifocus image fusion algorithm using deep learning and adaptive fuzzy filter,” in 2020 Int. Conf. Contemporary Computing and Applications (IC3A), 2020, pp. 70–75. Crossref, Google Scholar
11. J. Tian, L. Chen, L. Ma and W. Yu, “Multi-focus image fusion using a bilateral gradient-based sharpness criterion,” Opt. Commun. 284(1), 80–87 (2011). Crossref, Web of Science, Google Scholar
12. B. K. Shreyamsha Kumar, “Image fusion based on pixel significance using cross bilateral filter,” Signal Image Video Process. 9(5), 1193–1204 (2015). Crossref, Web of Science, Google Scholar
13. V. P. S. Naidu and B. Elias, “A novel image fusion technique using DCT based laplacian pyramid,” Int. J. Invent. Eng. Sci. (IJIES) 5(1), 2319–9598 (2013). Google Scholar
14. B. K. Shreyamsha Kumar, “Multifocus and multispectral image fusion based on pixel significance using discrete cosine harmonic wavelet transform,” Signal Image Video Process. 7(6), 1125–1143 (2013). Crossref, Web of Science, Google Scholar
15. R. Singh and A. Khare, “Fusion of multimodal medical images using daubechies complex wavelet transform — a multiresolution approach,” Inf. Fusion 19, 49–60 (2014). Crossref, Web of Science, Google Scholar
16. J. Yang, Zh. M. Zhao et al., “Multifocus image fusion method based on curvelet transform,” Opto-Electron. Eng. 6(34), 67–71 (2007). Google Scholar
17. M. Qiguang and B. Wang, “A novel image fusion method using contourlet transform,” in 2006 Int. Conf. Commun. Circuits Syst. 1, 548–552 (2006). Crossref, Google Scholar
18. Q. Zhang and B. L. Guo, “Multifocus image fusion using the nonsubsampled contourlet transform,” Signal Process. 89(7), 1334–1346 (2009). Crossref, Web of Science, Google Scholar
19. M. Browne and S. S. Ghidary, “Convolutional neural networks for image processing: An application in robot vision,” in Australasian Joint Conf. Artific. Intell. 2003, pp. 641–652. Crossref, Google Scholar
20. H. K. Kwan, Fuzzy Filters for Noise Reduction in Images, Vol. 122 (2003), pp. 25–53. Crossref, Google Scholar
21. Z. Shu-Long, “Image fusion using wavelet transform,” Int. Archiv. Photogram. Remote Sens. Spatial Inf. Sci. 34(4), 552–556 (2002). Google Scholar
22. S. S. Kumar and S Muttan, “PCA based image fusion,” International Society for Optics and Photonics, Vol. 6233 (2006) p. 62331T. Google Scholar
23. D. Gambhir and M. Manchanda, “Fusion of color images based on fuzzy transform and spatial frequency,” Int. J. Comput. Intell. Appl. 17(1), 1850004 (2018). Link, Web of Science, Google Scholar
24. D. Gambhir and M. Manchanda, “Waveatom transform-based multimodal medical image fusion,” Signal Image Video Process. 13, 10 (2018). Web of Science, Google Scholar
25. V. P. S. Naidu, “Hybrid DDCT-PCA based multi sensor image fusion,” J. Optic. 43(1), 48–61 (2014). Crossref, Google Scholar
26. C. S. Xydeas and V. Petrovic, “Objective image fusion performance measure,” Electron. Lett. 36(4), 308–309 (2000). Crossref, Web of Science, Google Scholar
27. M. Manchanda and R. Sharma, “An improved multimodal medical image fusion algorithm based on fuzzy transform,” J. Visual Commun. Image Represent. 51, 76–94 (2018). Crossref, Web of Science, Google Scholar
28. M. Manchanda and R. Sharma, “A novel method of multimodal medical image fusion using fuzzy transform,” J. Visual Commun. Image Represent. 40(6), 197–217 (2016). Crossref, Web of Science, Google Scholar
29. L. Zhang, D. Zhang and X. Mou, “FSIM: A feature similarity index for image quality assessment,” IEEE Trans. Image Process. 20(8), 2378–2386 (2011). Crossref, Web of Science, Google Scholar