No Access

Sparse Optical Flow Computation Using Wave Equation-Based Energy

Departamento de Ciência da Computação, Universidade Federal de Juiz de Fora, Rua Lourenço Kelmer, Juiz de Fora, Minas Gerais 36036-330, Brazil

E-mail Address: luiz.maurilio@ice.ufjf.br

Corresponding author.

Search for more papers by this author

and

Marcelo Bernardes Vieira

Departamento de Ciência da Computação, Universidade Federal de Juiz de Fora, Rua Lourenço Kelmer, Juiz de Fora, Minas Gerais 36036-330, Brazil

E-mail Address: marcelo.bernardes@ufjf.edu.br

Search for more papers by this author

https://doi.org/10.1142/S0219467820500278Cited by:0 (Source: Crossref)

Abstract

Identification of motion in videos is a fundamental task for several computer vision problems. One of the main tools for motion identification is optical flow, which estimates the projection of the 3D velocity of the objects onto the plane of the camera. In this work, we propose a differential optical flow method based on the wave equation. The optical flow is computed by minimizing a functional energy composed by two terms: a data term based on brightness constancy and a regularization term based on energy of the wave. Flow is determined by solving a system of linear equations. The decoupling of the pixels in the solution allows solving the system by a direct or iterative approach and makes the method suitable for parallelization. We present the convergence conditions for our method since it does not converge for all the image points. For comparison purposes, we create a global video descriptor based on histograms of optical flow for the problem of action recognition. Despite its sparsity, results show that our method improves the average motion estimation, compared with classical methods. We also evaluate optical flow error measures in image sequences of a classical dataset for method comparison.

Keywords:

Remember to check out the Check out our Most Cited Articles!
Check out these titles on Image Analysis

References

1. S. Baker, D. Scharstein, J. P. Lewis, S. Roth, M. J. Black and R. Szeliski, “A database and evaluation methodology for optical flow,” Proc. IEEE International Conference on Computer Vision (Rio de Janeiro, Brazil, 2007). Crossref, Google Scholar
2. S. Baker, D. Scharstein, J. P. Lewis, S. Roth, M. J. Black and R. Szeliski, “A database and evaluation methodology for optical flow,” International Journal of Computer Vision, 92(1), 1 (2011). Crossref, Web of Science, Google Scholar
3. B. K. P. Horn and B. G. Schunck, “Determining optical flow,” Artificial Intelligence, 17(1–3), 185 (1981). Crossref, Web of Science, Google Scholar
4. A. Bruhn, J. Weickert and C. Schnörr, “Lucas/kanade meets horn/schunck: Combining local and global optic flow methods,” International Journal of Computer Vision, 61, 211 (2005). Crossref, Web of Science, Google Scholar
5. T. Brox, A. Bruhn, N. Papenberg and J. Weickert, “High accuracy optical flow estimation based on a theory for warping,” in European Conf. Computer Vision, Lecture Notes in Computer Science, Vol. 3024 (Springer, 2004). Crossref, Google Scholar
6. H. A. Rashwan, M. A. García and D. Puig, “Variational optical flow estimation based on stick tensor voting,” IEEE Transaction on Image Processing 22(7), 2589 (2013). Crossref, Web of Science, Google Scholar
7. D. D. Nguyen and J. W. Jeon, “Tuning optical flow estimation with image-driven functions,” IEEE International Conf. Robotics and Automation (Shanghai, China, 2011). Crossref, Google Scholar
8. D. J. Butler, J. Wulff, G. B. Stanley and M. J. Black, “A naturalistic open source movie for optical flow evaluation,” in European Conf. on Computer Vision, Part IV, Lecture Notes in Computer Science, Vol. 7577 (Springer-Verlag, 2012). Crossref, Google Scholar
9. M. Menze, C. Heipke and A. Geiger, “Joint 3d estimation of vehicles and scene flow,” ISPRS Workshop on Image Sequence Analysis (Santiago, Chile, 2015). Crossref, Google Scholar
10. H. Wang, A. Klaser, C. Schmid and C.-L. Liu, “Action recognition by dense trajectories,” in Proc. 2011 IEEE Conference on Computer Vision and Pattern Recognition, CVPR ’11, IEEE Computer Society, Washington, DC, USA (2011). Crossref, Google Scholar
11. I. Laptev, M. Marszałek, C. Schmid and B. Rozenfeld, “Learning realistic human actions from movies,” in Conf. Computer Vision and Pattern Recognition (2008). Crossref, Google Scholar
12. B. D. Lucas and T. Kanade, “An iterative image registration technique with an application to stereo vision,” in Proc. 7th Int. Joint Conf. Artificial Intelligence, IJCAI’81, Vol. 2 (Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 1981). Google Scholar
13. T. Myint-U and L. Debnath, Linear Partial Differential Equations for Scientists and Engineers (Birkhäuser, 2007). Google Scholar
14. F. Girosi, A. Verri and V. Torre, “Constraints for the computation of optical flow,” in Proc. Workshop on Visual Motion (1989). Crossref, Google Scholar
15. J.-Y. Bouguet, “Pyramidal implementation of the Lucas Kanade feature tracker: Description of the algorithm,” Technical report, Intel Corporation Microprocessor Research Labs (2000). Google Scholar
16. S. H. Hwang and S. U. Lee, “A hierarchical optical flow estimation algorithm based on the interlevel motion smoothness constraint,” Pattern Recognit. 26(6), 939 (1993). Crossref, Web of Science, Google Scholar
17. T. Brox and J. Malik, “Large displacement optical flow: Descriptor matching in variational motion estimation,” IEEE Trans. Pattern Anal. Mach. Intell. 33(3), 500 (2011). Crossref, Web of Science, Google Scholar
18. Z. Tu, N. Van Der Aa, C. Van Gemeren and R. C. Veltkamp, “A combined post-filtering method to improve accuracy of variational optical flow estimation,” Pattern Recogn. 47(5), 1926 (2014). Crossref, Web of Science, Google Scholar
19. Z. Tu, R. Poppe and R. C. Veltkamp, “Weighted local intensity fusion method for variational optical flow estimation,” Pattern Recognit. 50, 223 (2016). Crossref, Web of Science, Google Scholar
20. H. W. Haussecker and D. J. Fleet, “Computing optical flow with physical models of brightness variation,” IEEE Trans. Pattern Anal. Mach. Intell. 23(6), 661 (2001). Crossref, Web of Science, Google Scholar
21. H. Sakaino, “Fluid motion estimation method based on physical properties of waves,” in Conf. Computer Vision and Pattern Recognition (Anchorage, Alaska, USA, 2008). Google Scholar
22. S. T. Barnard and W. B. Thompson, “Disparity analysis of images,” IEEE Trans. Pattern Anal. Mach. Intell. 2(4), 333 (1980). Crossref, Web of Science, Google Scholar
23. D. J. Fleet and A. D. Jepson, “Computation of component image velocity from local phase information,” International Journal of Computer Vision, 5(1), 77 (1990). Crossref, Web of Science, Google Scholar
24. D. Fortun, P. Bouthemy and C. Kervrann, “Optical flow modeling and computation: A survey,” Computer Vision and Image Understanding, 134, 1 (2015). Crossref, Web of Science, Google Scholar
25. D. Sun, X. Yang, M.-Y. Liu and J. Kautz, “Pwc-net: Cnns for optical flow using pyramid, warping, and cost volume,” in Proc. IEEE Conf. Computer Vision and Pattern Recognition (2018). Crossref, Google Scholar
26. T.-W. Hui, X. Tang and C. C. Loy, “Liteflownet: A lightweight convolutional neural network for optical flow estimation,” in Proc. IEEE Conf. Computer Vision and Pattern Recognition (2018). Crossref, Google Scholar
27. J. Chen, Z. Cai, J. Lai and X. Xie, “A filtering based framework for optical flow estimation,” IEEE Transactions on Circuit and Systems for Video Technology, 29(5), 1350 (2018). Crossref, Web of Science, Google Scholar
28. C. Zhang, Z. Chen, M. Wang, M. Li and S. Jiang, “Robust non-local tv-l1 optical flow estimation with occlusion detection,” IEEE Transactions on Image Processing, 26(8), 4055 (2017). Crossref, Web of Science, Google Scholar
29. D. Giancoli, Physics for Scientists and Engineers with Modern Physics (Prentice Hall, 1989). Google Scholar
30. J. Latter, “Tsunamis of volcanic origin: Summary of causes, with particular reference to Krakatoa, 1883,” Bulletin of Volcanology, 44(3), 467 (1981). Crossref, Google Scholar
31. J. l. R. d’Alembert, “Recherches sur la courbe que forme une corde tendue mise en vibration” (1747). Google Scholar
32. S. A. Van Duyne and J. O. Smith, “The 2-D digital waveguide mesh,” in Proc. IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (New York, USA, 1993). Crossref, Google Scholar
33. F. Fontana and D. Rocchesso, “A new formulation of the 2D-waveguide mesh for percussion instruments,” in Proc. XI Colloquium on Musical Informatics (Bologna, Italy, 1995). Google Scholar
34. C. Lanczos, The Variational Principles of Mechanics, Dover Books on Physics and chemistry (Dover Publications, 2012). Google Scholar
35. V. F. Mota, E. A. Perez, M. B. Vieira, L. M. Maciel, F. Precioso and P. H. Gosselin, “A tensor based on optical flow for global description of motion in videos,” SIBGRAPI - Conf. Graphics, Patterns and Images (Ouro Preto, Brazil, 2012). Crossref, Google Scholar
36. V. Mota, E. Perez, L. Maciel, M. Vieira and P. Gosselin, “A tensor motion descriptor based on histograms of gradients and optical flow,” Pattern Recognition Letters 39, 85 (2014). Crossref, Web of Science, Google Scholar
37. A. A. Efros, A. C. Berg, E. C. Berg, G. Mori and J. Malik, “Recognizing action at a distance,” in Int. Conf. Computer Vision (Nice, France, 2003). Crossref, Google Scholar
38. E. A. Perez, V. F. Mota, L. M. Maciel, D. O. Sad and M. B. Vieira, “Combining gradient histograms using orientation tensors for human action recognition,” in Proc. Int. Conf. Pattern Recognition (Tsukuba, Japan, 2012). Google Scholar
39. C. Schuldt, I. Laptev and B. Caputo, “Recognizing human actions: A local svm approach,” in Proc. Int. Conf. Pattern Recognition (Cambridge, UK, 2004). Crossref, Google Scholar
40. M. Marszałek, I. Laptev and C. Schmid, “Actions in context,” in Conf. Computer Vision & Pattern Recognition (Miami, USA, 2009). Crossref, Google Scholar