ArticlesNo Access

STIEFEL-MANIFOLD LEARNING BY IMPROVED RIGID-BODY THEORY APPLIED TO ICA

Faculty of Engineering, University of Perugia, Polo Didattico e Scientifico del Ternano, Loc. Pentima bassa, 21, I-05100 Terni, Italy

Search for more papers by this author

and

ROBERTO ROSSI

Faculty of Engineering, University of Perugia, Polo Didattico e Scientifico del Ternano, Loc. Pentima bassa, 21, I-05100 Terni, Italy

Search for more papers by this author

https://doi.org/10.1142/S0129065703001625Cited by:3 (Source: Crossref)

Abstract

In previous contributions we presented a new class of algorithms for orthonormal learning of a linear neural network with p inputs and m outputs, based on the equations describing the dynamics of a massive rigid frame in a submanifold of ℛ^p. While exhibiting interesting features, such as intrinsic numerical stability, strongly binding to the orthonormal submanifolds, and good controllability of the learning dynamics, tested on principal/independent component analysis, the proposed algorithms were not completely satisfactory from a computational-complexity point of view. The main drawback was the need to repeatedly evaluate a matrix exponential map. With the aim to lessen the computational efforts pertaining to these algorithms, we propose here an improvement based on the closed-form Rodriguez formula for the exponential map. Such formula is available in the p=3 and m=3 case, which is discussed with details here. In particular, experimental results on independent component analysis (ICA), carried out with both synthetic and real-world data, help confirming the computational gain due to the proposed improvement.

Keywords:

References

S.-I. Amari, Neural Computation 10, 251 (1998), DOI: 10.1162/089976698300017746. Crossref, Web of Science, Google Scholar
S.-I. Amari , A. Cichocki and H. H. Yang , Advances in Neural Information Processing Systems 8 ( MIT Press , 1993 ) . Google Scholar
J. Angeles, Springer Tracts in Natural Philosophy 34, (1989). Google Scholar
E. Bayro-Corrochano, IEEE Trans. Neural Networks 12(5), 968 (2001), DOI: 10.1109/72.950128. Crossref, Medline, Web of Science, Google Scholar
E. Bayro-Corrochano and G. Sobczyk (eds.) , Advances in Geometric Algebra with Applications in Science and Engineering. Automatic Theorem Proving, Computer Vision, Quantum and Neural Computing and Robotics ( Birkhauser , New York , 2001 ) . Google Scholar
A. J. Bell and T. J. Sejnowski, Neural Computation 7(6), 1129 (1995), DOI: 10.1162/neco.1995.7.6.1129. Crossref, Medline, Web of Science, Google Scholar
J. Blitz , Electrical and Magnetic Methods of Non-Destructive Testing , 2nd edn. ( Chapman and Hall , 1997 ) . Crossref, Google Scholar
J. M. Bowler, Int. Journal of Applied Electromagnetics and Mechanics 8, 3 (1997). Web of Science, Google Scholar
G. E. Bredon , Topology and Geometry ( Springer-Verlag , New-York , 1995 ) . Google Scholar
R. W. Brockett, Linear Algebra and Its Applications 146, 79 (1991), DOI: 10.1016/0024-3795(91)90021-N. Crossref, Web of Science, Google Scholar
J. F. Cardoso and B. Laheld, IEEE Trans. Signal Processing 44(12), 3017 (1996), DOI: 10.1109/78.553476. Crossref, Web of Science, Google Scholar
E. Celledoni and B. Owren, Numerical Algorithms 32(2–4), 163 (2003), DOI: 10.1023/A:1024079724094. Crossref, Web of Science, Google Scholar
P. Comon, Signal Processing 36, 287 (1994), DOI: 10.1016/0165-1684(94)90029-9. Crossref, Web of Science, Google Scholar
P. Comon and E. Moreau, Improved contrast dedicated to blind separation in communications, Proc. Int. Conf. Acoust., Speech and Signal Processing (ICASSP) (1997) pp. 3453–3456. Google Scholar
S. Costa and S. Fiori, Image and Vision Computing Journal 19(9–10), 649 (2001), DOI: 10.1016/S0262-8856(01)00042-7. Crossref, Web of Science, Google Scholar
L. Dieci and E. Van Vleck, Numerical Mathematics 83, 599 (1999). Crossref, Web of Science, Google Scholar
A. Edelman, T. A. Arias and S. T. Smith, SIAM Journal on Matrix Analysis Applications 20(2), 303 (1998), DOI: 10.1137/S0895479895290954. Crossref, Web of Science, Google Scholar
Y. Ephraim and L. Van Trees, IEEE Trans. on Speech and Audio Processing 3(4), 251 (1995), DOI: 10.1109/89.397090. Crossref, Web of Science, Google Scholar
S. Fiori, Network: Computation in Neural Systems 10(2), 171 (1999), DOI: 10.1088/0954-898X/10/2/304. Crossref, Medline, Web of Science, Google Scholar
S. Fiori, Neurocomputing 34(1–4), 239 (2000), DOI: 10.1016/S0925-2312(00)00161-2. Crossref, Web of Science, Google Scholar
S. Fiori, Neural Networks 13(6), 597 (2000), DOI: 10.1016/S0893-6080(00)00044-7. Crossref, Medline, Web of Science, Google Scholar
S. Fiori, Neural Computation 13(7), 1625 (2001), DOI: 10.1162/089976601750265036. Crossref, Web of Science, Google Scholar
S. Fiori, IEEE Trans. Neural Networks 13(3), 521 (2002), DOI: 10.1109/TNN.2002.1000121. Crossref, Medline, Web of Science, Google Scholar
S. Fiori, Int. J. Neural Systems 12(3–4), 219 (2002), DOI: 10.1142/S012906570200114X. Link, Google Scholar
A. Fujiwara and S.-I. Amari, Physica D 80, 317 (1995), DOI: 10.1016/0167-2789(94)00175-P. Crossref, Web of Science, Google Scholar
K. Gao, M. O. Ahmed and M. N. Swamy, IEEE Trans. Circuits Syst.-II 41(11), 718 (1994). Web of Science, Google Scholar
A. Hyvärinen and E. Oja, Signal Processing 64(3), 301 (1998). Crossref, Web of Science, Google Scholar
J. Karhunen, Neural approaches to independent component analysis and source separation, Fourth European Symposium on Artificial Neural Networks (ESANN'96) (1996) pp. 249–266. Google Scholar
A. Kern, D. Blank and R. Stoop, An optimal noise cleaning by local manifold projection, Proc. of Second International ICSC Symposium on Neural Computation (NC) (2000) pp. 399–404. Google Scholar
J. Kivinen and M. Warmuth, Information and Computation 132, 1 (1997), DOI: 10.1006/inco.1996.2612. Crossref, Web of Science, Google Scholar
R.-W. Liu, Blind signal processing: An introduction, Proc. of International Symposium on Circuits and Systems (IEEE-ISCAS)2 (1996) pp. 81–84. Google Scholar
Manodet project, Data pertaining to the 'Manodet Project', hosted by the Hungarian Academy of Sciences. Mirrored on the web page: http://neurolab.ing.unirc.it . Google Scholar
B. C. Moore, IEEE Trans. Automatic Control AC-26(1), 17 (1981). Web of Science, Google Scholar
H. Niemann and J.-K. Wu, IEEE Trans. Neural Networks 4(4), 615 (1993), DOI: 10.1109/72.238316. Crossref, Medline, Web of Science, Google Scholar
Y. Nishimori, Learning algorithm for ICA by geodesic flows on orthogonal group, Proc. Int. Joint Conf. Neural Networks (IJCNN'99)2 (1999) pp. 1625–1647. Google Scholar
E. Oja, Int. J. Neural Systems 1, 61 (1989), DOI: 10.1142/S0129065789000475. Link, Web of Science, Google Scholar
E. Oja, A. Hyvärinen and P. Hoyer, Pattern Analysis and Applications Journal 2(2), 104 (1999), DOI: 10.1007/s100440050021. Crossref, Web of Science, Google Scholar
P. Saisanet al., Dynamic texture recognition, Proc. of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition2 (2001) pp. 58–63. Google Scholar
G. Simone and F. C. Morabito, ICA-NN based data fusion approach in ECT signal restoration, Proc. Int. Joint Conf. Neural NetworksV (2000) pp. 59–64. Google Scholar
D. Sona, A. Sperduti and A. Starita, Neural Computation 12(6), 1355 (2000), DOI: 10.1162/089976600300015402. Crossref, Medline, Web of Science, Google Scholar
L. Xu, E. Oja and C. Y. Suen, Neural Networks 5, 393 (1992), DOI: 10.1016/0893-6080(92)90006-5. Google Scholar
B. Yang, IEEE Trans. Signal Processing 43(1), 1247 (1995). Google Scholar
H. H. Yang and S.-I. Amari, Neural Computation 9, 1457 (1997), DOI: 10.1162/neco.1997.9.7.1457. Crossref, Web of Science, Google Scholar
K. Zhang and T. J. Sejnowski, Journal of Neuroscience 19(8), 3122 (1999). Crossref, Medline, Web of Science, Google Scholar