PARTIALLY PRE-CALCULATED WEIGHTS FOR BACKPROPAGATION TRAINING OF RAM-BASED SIGMA-PI NETS

The chapter outlines recent research that enables one to train digital "Higher Order" sigma-pi artificial neural networks using pre-calculated constrained look-up tables of Backpropagation delta changes. By utilising these digital units that have sets of quantised sitevalues (i.e. weights) one may also quantise the sigmoidal activation-output function and then the output function may also be pre-calculated. The research presented shows that by utilising weights quantised to 128 levels these units can achieve accuracy's of better than one percent for target output functions in the range Y ∈ [0,1]. This is equivalent to an average Mean Square Error (MSE) over all training vectors of 0.0001 or an error modulus of 0.01. The sigma-pi are RAM based and as such are hardware realisable units which may be implemented in Microelectronic technology. The article present a development of a sigma-pi node which enables one to provide high accuracy outputs utilising the cubic node's methodology of storing quantised weights (site-values) in locations that are stored in RAM-based units. The networks presented are trained with the Backpropagation training regime that may be implemented on- line in hardware. One of the novelties of this work is that it shows how one may utilise the bounded quantised site-values (weights) of sigma-pi nodes to enable training of these Neurocom-puting systems to be relatively simple and very fast.