A MODULAR APPROACH TO STORAGE CAPACITY

Modularity is a valuable principle in analysing and synthesising large systems. This chapter gives an overview on how to apply such principle to the assessment of the storage capacity of RAM-based neural networks. The storage capacity of a network is a function of the storage capacity of its component neurons and subnetworks. This modular approach allows the independent treatment of storage capacity at different levels — a model for the single neuron and a model for each architecture. The technique is illustrated in the major architectures with limited storage capacity in use nowadays — general neural unit (GNU) and pyramid — and in a composition of them. The results fit well both with the existing architecture-dependent theories and with the experimental data currently available in the literature with the advantages of simplicity and flexibility for the modularity. This approach is based on collision of information during training taken as a probabilistic process.