No Access

Detection and Correction of Multi-Warping Document Image

Marian Wagdy

Department of Information Technology, Faculty of Computers and Information, Menofia University, Egypt

E-mail Address: labeeb@yahoo.com

Corresponding author.

Search for more papers by this author

Khaild Amin

Department of Information Technology, Faculty of Computers and Information, Menofia University, Egypt

E-mail Address: kh.amin.0.0@gmail.com

Search for more papers by this author

, and

Mina Ibrahim

Department of Information Technology, Faculty of Computers and Information, Menofia University, Egypt

E-mail Address: Mina.ibrahim@ci.menofia.edu.eg

Search for more papers by this author

https://doi.org/10.1142/S0219467822500346Cited by:2 (Source: Crossref)

Abstract

In this work, we aim to solve the multi-warping document image problems. We can overcome the limitations of the previous dewarping algorithms to recover the shape of the document. The proposed method is based on a well-defined pattern to simulate the distorted and undistorted connected component of document images. Some pairs of control points are selected for each connected component and its ground truth pattern to define the mapping function between them. The dewarping process transforms the warping connected component according to the geometric transformation defined by the calculated mapping function. Results on document dewarping dataset CBDAR demonstrate the effectiveness of our method. OCR error metrics are also used to evaluate the performance of the proposed approach.

Keywords:

Remember to check out the Check out our Most Cited Articles!
Check out these titles on Image Analysis

References

1. M. Wagdy, I. Faye and D. Rohaya , “Border noise removal and clean up based ons retinex theory, ” in DaEng, Lecture Notes in Electrical Engineering, Vol. 285 (Springer, 2014), pp. 345–352. Crossref, Google Scholar
2. O. Boudraa, W. K. Hidouci and D. Michelucci , “An efficient cooperative smearing technique for degraded historical document image segmentation,” Int. J. Image Graph. 21, 2150012 (2020). Link, Web of Science, Google Scholar
3. R. Keefer and N. Bourbakis , “A survey on document image processing methods useful for assistive technology for the blind,” Int. J. Image Graph. 15(1), 1550005 (2015). Link, Google Scholar
4. R. Rani et al., “Script identification for printed and handwritten Indian documents: An empirical study of different feature classifier combinations,” Int. J. Image Graph. 2140011 (2021). Web of Science, Google Scholar
5. M. Wagdy, I. Faye and D. Rohaya , “Fast and efficient document image clean up and binarization based on retinex theory,” in IEEE 9th Int. Colloquium on Signal Processing and its Applications, CSPA (IEEE, Malaysia, 2013), pp. 58–62. Crossref, Google Scholar
6. M. Wagdy, K. Amin and M. Ibrahim , “Dewarping document image techniques: Survey and comparative study,” Int. J. Image Graph. 21(1), (2021). Google Scholar
7. H. C. Vinod and S. K. Niranjan , “Dewarping of camera captured document images,” in IEEE Int. Symp. Consumer Electronics (ISCE) (Malasyia, 2017), pp. 13–18. Google Scholar
8. D. Schneider, M. Block and R. Rojas , “Robust document with interpolated vector fields,” in Ninth Int. Conf. Document Analysis and Recognition (ICDAR 2007), Vol. 1 (IEEE, 2007), pp. 113–117. Crossref, Google Scholar
9. B. S. Kim, H. I. Koo and N. I. Cho , “Document dewarping via text-line based optimization,” Pattern Recognit. 48(11), 3600–3614 (2015). Crossref, Web of Science, Google Scholar
10. T. Kil et al., “Robust document image dewarping method using text-lines and line segments,” in 2017 14th IAPR Int. Conf. Document Analysis and Recognition (ICDAR), Vol. 1 (IEEE, 2017), pp. 865–870. Crossref, Google Scholar
11. A. Masalovitch and L. Mestetskiy , “Usage of continuous skeletal image representation for document images de-warping,” in 2nd Int. Workshop on Camera-Based Document Analysis and Recognition, Curitiba, Brazil, September, 2007, pp. 45–53. Google Scholar
12. B. Fu et al., “A model-based book dewarping method using text line detection,” in Proc. 2nd Int. Workshop on Camera Based Document Analysis and Recognition, Curitiba, Barazil, 2007, pp. 63–70. Google Scholar
13. B. Gatos, I. Pratikakis and K. Ntirogiannis , “Segmentation based recovery of arbitrarily warped document images,” in IEEE Ninth Int. Conf. Document Analysis and Recognition (ICDAR 2007), Vol. 2, 2007, pp. 989–993. Crossref, Google Scholar
14. S. S. Bukhari, F. Shafait and T. M. Breuel , “Dewarping of document images using coupled-snakes,” in Proc. Third Int. Workshop on Camera-Based Document Analysis and Recognition, Barcelona, Spain, 2009, pp. 34–41. Google Scholar
15. Y. Zhang , “Arbitrary warped document image restoration based on segmentation and thin-plate splines,” in IEEE 19th Int. Conf. Pattern Recognition, USA, 2008, pp. 1–4. Crossref, Google Scholar
16. S. Lu, B. M. Chen and C. C. Ko , “A partition approach for the restoration of camera images of planar and curled document,” Image Vis. Comput. 24(8), 837–848 (2006). Crossref, Web of Science, Google Scholar
17. A. Ulges, C. H. Lampert and T. M. Breuel , “Document image dewarping using robust estimation of curled text lines,” in Eighth Int. Conf. Document Analysis and Recognition (ICDAR’05) (IEEE, 2005), pp. 1001–1005. Crossref, Google Scholar
18. F. Bolelli , “Indexing of historical document images: Ad hoc dewarping technique for handwritten text,” in Italian Research Conf. Digital Libraries (Springer, Cham, 2017), pp. 45–55. Crossref, Google Scholar
19. M. Tappen, W. T. Freeman and E. Adelson . “Recovering intrinsic images from a single image,” in Advances in Neural Information Processing Systems (NIPS), Vol. 15 (MIT Press, 2003), pp. 20–35. Google Scholar
20. M. Shamqoli and H. Khosravi , “Warped document restoration by recovering shape of the surface,” in 2013 8th Iranian Conf. Machine Vision and Image Processing (MVIP) (IEEE, 2013), pp. 262–265. Crossref, Google Scholar
21. N. Stamatopoulos , “A two-step dewarping of camera document images,” 2008 The Eighth IAPR Int. Workshop on Document Analysis Systems (IEEE, 2008), pp. 209–216. Crossref, Google Scholar
22. S. Sruthy and S. Suresh Babu , “Dewarping on camera document images”, Int. J. Pure Appl. Math. 119(16), 34–41 (2018). Google Scholar
23. C. Flagg and O. Frieder , “Searching document repositories using 3D model reconstruction,” in Proc. ACM Symp. Document Engineering 2019 (ACM, 2019), pp. 1–10. Crossref, Google Scholar
24. Y. Lin and W. Seales , “Opaque document imaging: Building images of inaccessible texts,” in Proc. Tenth IEEE Int. Conf. Computer Vision (ICCV05), Vol. 1, China, 2005, pp. 662–669. Google Scholar
25. L. Zhang, Y. Zhang and C. Tan , “An improved physically-based method for geometric restoration of distorted document images,” IEEE Trans. Pattern Anal. Mach. Intell. 30(4), 728–734 (2008). Crossref, Web of Science, Google Scholar
26. L. Galarza and H. Martin , “Integrating low-resolution depth maps to high-resolution images in the development of a book reader design for persons with visual impairment and blindness,” Int. J. Innov. Comput., Inf. Control 14(3), 797–816 (2018). Web of Science, Google Scholar
27. Y. Y. Tang and C. Y. Suen , “Image transformation approach to nonlinear shape restoration,” IEEE Trans. Syst. Man Cybern. 1(23), 155–171 (1993). Crossref, Web of Science, Google Scholar
28. K. B. Chua, L. Zhang, Y. Zhang and C. L. Tan , “A fast and stable approach for restoration of warped document images,” in IAPR Int. Conf. Document Analysis and Recognition, Vol. 1, Korea, 2005, pp. 384–388. Crossref, Google Scholar
29. T. Amano, T. Abe, T. Iyoda, O. Nishikawa and Y. Sato , “Camera-based document image mosaicing,” Proc. SPIE 4669, 250–258 (2002). Google Scholar
30. M. Brown and C. Pisula , “Conformal deskewing of non-planar documents,” in Proc. 2005 IEEE Computer Society Conf. Computer Vision and Pattern Recognition (CVPR05), Vol. 1, 2005, pp. 998–1004. Crossref, Google Scholar
31. W. Seales and Y. Lin , “Digital restoration using volumetric scanning,” in Joint ACM/IEEE Conf. Digital Libraries, Vol. 1, 2004, pp. 117–124. Crossref, Google Scholar
32. Y. Lin and W. Seales , “Opaque document imaging: Building images of inaccessible texts,” in Proc. Tenth IEEE Int. Conf. Computer Vision (ICCV05), Vol. 1, 2005, pp. 662–669. Google Scholar
33. L. Zhang and C. L. Tan , “Warped document image restoration using shape from-shading and physically-based modeling,” in 8th IEEE Workshop on Applications of Computer Vision (WACV), Austin, Texas, USA, 20–21, February, 2007, p. 29. Crossref, Google Scholar
34. L. Zhang, A. M. Yip, M. S. Brown and C. L. Tan , “A unified framework for document restoration using inpainting and shape-from-shading,” Pattern Recognit. 41, 2961–2978 (2009). Crossref, Web of Science, Google Scholar
35. L. Zhang, A. M. Yip and C. L. Tan , “Photometric and geometric restoration of document images using inpainting and shape-from-shading,” in Proc. National Conf. Artificial Intelligence, 22(1) (Menlo Park, CA; Cambridge, MA; London; AAAI Press; MIT Press, 1999), 2007. Google Scholar
36. L. Zhang, A. M. Yip and C. L. Tan , “Removing shading distortions in camera-based document images using inpainting and surface fitting with radial basis functions,” in Ninth Int. Conf. Document Analysis and Recognition (ICDAR 2007), Vol. 2 (IEEE, 2007). Crossref, Google Scholar
37. T. Wada, H. Ukida and T. Matsuyama , “Shape from shading with interreflections under proximal light source,” in Int. Conf. Computer Vision, 1995, pp. 66–71. Google Scholar
38. Z. Zhang and C. L. Tan , “Restortation of images scanned from thick bound documents,” in Int. Conf. Image Processing ICIP ’01, Thessaloniki, Greece, October, 2001, pp. 1074–1077. Google Scholar
39. L. Zhang, A. Yip and C. Tan , “Shape-from-shading based on Lax–Friedrichs fast sweeping and regularization techniques with applications to document image restoration,” in IEEE Conf. Computer Vision and Pattern Recognition, 2007, pp. 1–8. Crossref, Google Scholar
40. S. Bukhari, V. Kumar and B. Ramanna , “Document image dewarping using deep learning”, in 8th Int. Conf. Pattern Recognition Applications and Methods ICPRAM, 2019, pp. 524–531. Google Scholar
41. Z. Wang, A. C. Bovik, H. R. Sheikh and E. P. Simoncelli , “Image quality assessment: From error visibility to structural similarity,” IEEE Trans. Image Process. 13(4), 600–612 (2004). Crossref, Web of Science, Google Scholar
42. P. Isola, J.-Y. Zhu, T. Zhou and A. Efros, “A. image-to-image translation with conditional adversarial networks,” preprint (2017), pp. 1125–1134. Google Scholar
43. M. Wagdy, I. Faye and D. Rohaya , “Document image binarization based on retinex theory,” Electron. Lett. Comput. Vis. Image Anal. 14(1), (2015). Google Scholar
44. M. Wagdy, I. Faye and D. Rohaya , “Degradation enhancement for the captured document image using retinex theory,” in IEEE Information Technology and Multimedia ICIM μ, Malaysia, 2014. Crossref, Google Scholar
45. M. Wagdy, M. Ibrahim and K. Amin , “Document skew detection and correction using ellipse shape,” Int. J. Imaging Robot. 19(4), 2019. Google Scholar
46. M. Wagdy, I. Faye and D. Rohaya , “Border noise removal from the document image using $x$ - $y$ cut and filtering technique based on morphological operation,” Int. J. Imaging Robot. 15(3), (2015). Google Scholar
47. N. Paniphi and S. Tripathy , “Image registration using polynomial affine transformation,” Def. Sci. J. 52(3), 253 (2002). Crossref, Web of Science, Google Scholar
48. J. Qin et al., “Efficient query autocompletion with edit distance-based error tolerance,” VLDB J. 1–25 (2019). Web of Science, Google Scholar
49. M. M. M. Fuad and P.-F. Marteau , “The extended edit distance metric,” in 2008 Int. Workshop on Content-Based Multimedia Indexing (IEEE, 2008), pp. 242–248. Crossref, Google Scholar
50. F. Shafait and T. M. Breuel , “Document image dewarping contest,” in 2nd Int. Workshop on Camera-Based Document Analysis and Recognition, Curitiba, Brazil, 2007, pp. 181–188. Google Scholar