No Access

A VECTOR MATRIX REAL TIME RECURSIVE BACKPROPAGATION ALGORITHM FOR RECURRENT NEURAL NETWORKS THAT APPROXIMATE MULTI-VALUED PERIODIC FUNCTIONS

PETER STUBBERUD

Department of Electrical and Computer Engineering, University of Nevada Las Vegas, 4505 Maryland Parkway, Las Vegas, NV 89154-4026, USA

Search for more papers by this author

https://doi.org/10.1142/S1469026809002667Cited by:5 (Source: Crossref)

Abstract

Unlike feedforward neural networks (FFNN) which can act as universal function approximators, recursive, or recurrent, neural networks can act as universal approximators for multi-valued functions. In this paper, a real time recursive backpropagation (RTRBP) algorithm in a vector matrix form is developed for a two-layer globally recursive neural network that has multiple delays in its feedback path. This algorithm has been evaluated on two GRNNs that approximate both an analytic and nonanalytic periodic multi-valued function that a feedforward neural network is not capable of approximating.

Keywords:

Remember to check out the Most Cited Articles!
Check out these titles in artificial intelligence!

References

K. Hornik, M. Stinchcombe and H. White, Neural Networks 2, 359 (1989), DOI: 10.1016/0893-6080(89)90020-8. Crossref, Google Scholar
A. Logar, E. Corwin and W. Oldham, A comparison of recurrent neural network learning algorithms, IEEE Conf. Neural Networks (1993) pp. 1129–1134. Google Scholar
D. E. Rumelhart , G. E. Hinton and R. J. William , Learning Internal Representations by Error Propagation ( MIT Press , Cambridge , 1986 ) . Google Scholar
P. Werbos, Proc. IEEE 78, 1550 (1990), DOI: 10.1109/5.58337. Crossref, Google Scholar
P. Werbos, Neural Networks 339 (1988), DOI: 10.1016/0893-6080(88)90007-X. Google Scholar
S. W. Piché, IEEE Trans. Neural Networks 5, 198 (1994). Crossref, Google Scholar
O. Jesus and M. Hagan, IEEE Trans. Neural Networks 18, 14 (2007). Crossref, Google Scholar
K. Narendra and L. McBride Jr., IEEE Trans. Automat. Cont. 9, 31 (1964), DOI: 10.1109/TAC.1964.1105638. Crossref, Google Scholar
F. J. Pineda, Neural Information Processing Systems, ed. D. Z. Anderson (American Institute of Physics, 1988) pp. 602–611. Google Scholar
K. S. Narendra and K. Parthasarathy, IEEE Trans. Neural Networks 1, 4 (1990), DOI: 10.1109/72.80202. Crossref, Google Scholar
R. Williams and D. Zipser, Neural Comput. 1, 270 (1989), DOI: 10.1162/neco.1989.1.2.270. Crossref, Google Scholar
O. Nerrandet al., IEEE Trans. Neural Networks 5, 178 (1994), DOI: 10.1109/72.279183. Crossref, Google Scholar
J. Barhen, N. Toomarian and S. Gulati, Appl. Math. Lett. 3, 13 (1990), DOI: 10.1016/0893-9659(90)90127-W. Crossref, Google Scholar
J. Barhen, N. Toomarian and S. Gulati, Adv. Neural Inform. Process. Syst. 2, 498 (1990). Google Scholar
N. Toomarian and J. Barhen, Neural Networks 5, 473 (1992). Crossref, Google Scholar
D. Rumelhart and J. McClelland , Parallel Distributed Processing ( MIT Press , 1987 ) . Crossref, Google Scholar
B. Pearlmutter, IEEE Trans. Neural Networks 6, 1212 (1995), DOI: 10.1109/72.410363. Crossref, Google Scholar
O. Olurotimi, IEEE Trans. Neural Networks 5, 185 (1994), DOI: 10.1109/72.279184. Crossref, Google Scholar
A. Atiya and A. Parlos, IEEE Trans. Neural Networks 11, 697 (2000), DOI: 10.1109/72.846741. Crossref, Google Scholar
G. Sun, H. Chen and Y. Lee, Advances in Neural Information Processing Systems (Morgan Kaufmann, 1992) pp. 333–340. Google Scholar
Q. Song, Y. Wu and Y. Soh, IEEE Trans. Neural Networks 19, 1841 (2008). Crossref, Google Scholar
S. Hochreiter and J. Schmidhuber, Neural Comput. 9, 1735 (1997), DOI: 10.1162/neco.1997.9.8.1735. Crossref, Google Scholar
J. Schmidhuber, Neural Comput. 4, 243 (1992), DOI: 10.1162/neco.1992.4.2.243. Crossref, Google Scholar
P. Stubberud and S. Dionisi, Adaptive image restoration using a two layer globally recurrent neural network trained by a recursive LMS algorithm, COMCON 5th Int. Conf. Communications and Control with Emphasis on Telecommunications/Signal Processing in the Multimedia Era pp. 319–330. Google Scholar