No Access

FSLLE: A Fast $K$ $K$ Selection Algorithm for Locally Linear Embedding

Jin-Hang Liu

School of Computer Science Technology, Wuhan University of Technology, Wuhan, Hubei 430000, China

E-mail Address: liujinhang@hotmail.com

Search for more papers by this author

Tao Peng

School of Mathematical and Computer, Wuhan Textile University, Wuhan, Hubei 430073, China

E-mail Address: pt@wtu.edu.cn

Corresponding author.

Search for more papers by this author

Xiaogang Zhao

International School of Software, Wuhan University, Wuhan, Hubei 430079, China

E-mail Address: zxgang302@whu.edu.cn

Search for more papers by this author

Kunfang Song

School of Mathematical and Computer, Wuhan Textile University, Wuhan, Hubei 430073, China

E-mail Address: skf@wtu.edu.cn

Search for more papers by this author

Minghua Jiang

School of Mathematical and Computer, Wuhan Textile University, Wuhan, Hubei 430073, China

E-mail Address: jmh@wtu.edu.cn

Search for more papers by this author

Ming Hu

School of Mathematical and Computer, Wuhan Textile University, Wuhan, Hubei 430073, China

E-mail Address: hm@wtu.edu.cn

Search for more papers by this author

XinRong Hu

School of Mathematical and Computer, Wuhan Textile University, Wuhan, Hubei 430073, China

E-mail Address: 1997004@wtu.edu.cn

Search for more papers by this author

, and

Xiao Qin

Department of Computer Science and Software Engineering, Shelby Center for Engineering, Technology, Samuel Ginn College of Engineering, Auburn University, AL 36849-5347, USA

E-mail Address: xqin@auburn.edu

Search for more papers by this author

https://doi.org/10.1142/S1469026818500037Cited by:0 (Source: Crossref)

Abstract

Data in a high-dimensional data space may reside in a low-dimensional manifold embedded within the high-dimensional space. Manifold learning discovers intrinsic manifold data structures to facilitate dimensionality reductions. We propose a novel manifold learning technique called fast $K$ $K$ selection for locally linear embedding or FSLLE, which judiciously chooses an appropriate number (i.e., parameter $K$ $K$ ) of neighboring points where the local geometric properties are maintained by the locally linear embedding (LLE) criterion. To measure the spatial distribution of a group of neighboring points, FSLLE relies on relative variance and mean difference to form a spatial correlation index characterizing the neighbors’ data distribution. The goal of FSLLE is to quickly identify the optimal value of parameter $K$ $K$ , which aims at minimizing the spatial correlation index. FSLLE optimizes parameter $K$ $K$ by making use of the spatial correlation index to discover intrinsic structures of a data point’s neighbors. After implementing FSLLE, we conduct extensive experiments to validate the correctness and evaluate the performance of FSLLE. Our experimental results show that FSLLE outperforms the existing solutions (i.e., LLE and ISOMAP) in manifold learning and dimension reduction. We apply FSLLE to face recognition in which FSLLE achieves higher accuracy than the state-of-the-art face recognition algorithms. FSLLE is superior to the face recognition algorithms, because FSLLE makes a good tradeoff between classification precision and performance.

Keywords:

Remember to check out the Most Cited Articles!
Check out these titles in artificial intelligence!

References

1. I. T. Jollffe, Principal Component Analysis, Vol. 2. (Springer-Verlag, New York, NY, USA, 2002). Google Scholar
2. J. De Leeuw, Multidimensional scaling, in Handbook of Statistics (North-Holland Publishing Company, 2001), pp. 285–316. Crossref, Google Scholar
3. J. B. Tenenbaum, Mapping a manifold of perceptual observations, in Advances in Neural Information Processing Systems 10 (MIT Press, 1998), pp. 682–688. Google Scholar
4. J. B. Tenenbaum, V. de Silva and J. C. Langford, A golobal geometric framework for nonlinear dimensionality reduction, Science 290 (5500) (2000) 2319–2323. Crossref, Google Scholar
5. S. T. Roweis and L. K. Saul, Nonlinear dimensionality reduction by locally linear embedding, Science 290 (5500) (2000) 2323–2326. Crossref, Google Scholar
6. L. K. Saul and S. T. Roweis, Think globally, fit locally: Unsupervised learning of low dimensional manifolds, J. Mach. Learn. Res. 4 (2003) 115–155. Google Scholar
7. M. Belkin and P. Niyogi, Laplacian eigenmaps and spectral techniques for embedding and clustering, in Advances in Neural Information Processing Systems 14. (MIT Press, Cambridge, MA, USA, 2002). Crossref, Google Scholar
8. M. Belkin and P. Niyogi, Laplacian eigenmaps for dimensionality reduction and data representation, Advances in Neural Information Processing Systems 15 (6) (2003) 1373–1396. Google Scholar
9. D. Demers and G. C. Y, Non-linear dimensionality reduction, in Advances in Neural Information Processing Systems 5, (Morgan Kaufmann, 1993), pp. 580–587. Google Scholar
10. M. Brand, Continuous nonlinear dimensionality reduction by kernel eigenmaps, in Int. Joint Conf. Artificial Intelligent, 2003, pp. 547–554. Google Scholar
11. D. de Ridder and R. P.W. Duin, Locally linear embedding for classification (2002). Google Scholar
12. V. De Silva and J. B. Tenenbaum, Global versus local methods in nonlinear dimensionality reduction, in Advances in Neural Information Processing Systems 15 (MIT Press, 2003), pp. 705–712. Google Scholar
13. D. DeCoste, Visualizing mercer kernel feature spaces via kernelized locally-linear embeddings, in Proceedings of International Conference on Neural Information Processing, 2001, pp. 14–18. Google Scholar
14. M. Polito and P. Perona, Grouping and dimensionality reduction by locally linear embedding, in Advances in Neural Information Processing Systems 14 (MIT Press, 2001), pp. 1255–1262. Google Scholar
15. Y. W. Teh and S. Roweis, Automatic alignment of local representations, in Advances in Neural Information Processing Systems 15, (MIT Press, 2003), pp. 841–848. Google Scholar
16. J. J. Verbeek, N. Vlassis and B. Krse, Fast nonlinear dimensionality reduction with topology preserving networks, in Proc. 10th European Symp. on Artificial Neural Networks (Publications, 2002), pp. 193–198. Google Scholar
17. M. Vlachos, C. Domeniconi, D. Gunopulos, G. Kollios and G. Kollios, Non-linear dimensionality reduction techniques for classification and visualization, in Proc. 8th ACM SIGKDD Int. Conf. Knowledge Discovery and Data Mining, 2002, pp. 645–651. Google Scholar
18. H. Zha and Z. Zhang, Isometric embedding and continuum isomap, in Proc. 20th Int. Conf. Machine Learning, 2003, pp. 864–871. Google Scholar
19. D. de Ridder and V. Franc, Robust manifold learning, Technical Report, Department of Cybernetics, Czech Technical University (2003). Google Scholar
20. D. Peel and G. J. Mclachlan, Robust mixture modelling using the $t$ $t$ -distribution, Stat. Comput. 10 (2000) 334–348. Crossref, Google Scholar
21. Z. Zhang and H. Zha, Local linear smoothing for nonlinear manifold learning, Technical Report, Department of Computer Science and Engineering, Pennsylvania State University, University Park (2003). Google Scholar
22. A. Hadid and M. Pietikinen, Efficient locally linear embeddings of imperfect manifolds, in Proc. 3rd Int. Conf. Machine Learning and Data Mining in Pattern Recognition, 2003, pp. 188–201. Google Scholar
23. O. Kouropteva, O. Okun and M. Pietikinen, Selection of the optimal parameter value for the locally linear embedding algorithm, in 1st Int. Conf. Fuzzy Systems and Knowledge Discovery, 2002, pp. 359–363. Google Scholar
24. G. Daza-Santacoloma, A. L. Meza and J. Valencia-Aguirre, Global and local choice of the number of nearest neighbors in locally linear embeddings, Pattern Recognit. Lett. 32 (2011) 2171–2177. Crossref, Google Scholar
25. G. Castellanos-Domnguez, G. Daza-Santacoloma and C. D. Acosta-Medina, Regularization parameter choice in locally linear embedding, Neurocomputing 73 (10–12) (2010) 1595–1605. Crossref, Google Scholar