where a and b are weight vectors, the same as w in the previous notations, u is the excitatory input vector, and v is the inhibitory input vector. There are a total of 41,399 such neurons in this version of the neocognitron.
Figure 7.3-5 illustrates in summary form the architecture of this neocognitron. Not only is the system hierarchical and multilayered, but it also uses complex feedback connections. Its design was motivated by known behavior of single units in the vertebrate visual cortex, and in the author's opinion, a good deal of ANN art.
' Converging or diverging connections (between two groups of neurons)
One-to-one connection (between two neurons)
I ^ Gain control inputs
* Threshold control inputs
O Fixed weights —Variable weights
FIGURE 7.3-5 Block diagram of the neocognitron. This ANN uses over 41,000 neurons of the type shown in Figure 7.3-4. This is a very complex ANN. See text for description.
For the reader to appreciate Fukushima's design rationale, his 1988b paper section, "Physiology," is quoted:
In the visual area of the cerebrum, neurons respond selectively to local features of a visual pattern, such as lines and edges in particular orientations. In the area higher than the visual cortex, cells exist that respond selectively to certain figures like circles, triangles, squares or even human faces [SMFs ?]. Accordingly, the visual system seems to have a hierarchical structure, in which simple features are first extracted from a stimulus pattern, then integrated into more complicated ones. In this hierarchy, a cell in a higher stage generally receives signals from a wider area of the retina and is more insensitive to the position of the stimulus.
Within the hierarchical structure of the visual system are forward (afferent or bottom-up) and backward (efferent or top-down) signal flows. For example, anatomical observations show that the major visual areas of the cerebrum interconnect in a precise topographical and reciprocal fashion.
Such neural networks in the brain are not always complete at birth. They develop gradually, neurons extending branches and making connections with many other neurons, adapting flexibly to circumstances after birth.
This kind of evidence suggests a network structure for the model."
Unlike the binary neurons found in perceptrons and MADALINEs, the neocog-nitron uses analog-type neurons of the form shown in Figure 7.3-4. The neuron's non-negative analog output, y', corresponds to the instantaneous firing frequency of the neuron. Figure 7.3-5 shows that some of the interconnecting weights are fixed, while others are variable and are subject to training or optimization. The system operates in the following manner:
A stimulus pattern is presented to the lowest stage of the forward paths, the input layer, which consists of a two-dimensional array of receptor cells. The highest stage of the forward path is the recognition layer. After the process of learning ends, the final result of the pattern recognition shows in the response of the cells of the highest stage. In other words, cells of the recognition layer work as gnostic cells (or grandmother cells); usually one cell is activated, corresponding to the category of the specific stimulus pattern. Pattern recognition by the network occurs on the basis of similarity of shape among patterns, [it is] unaffected by deformation, changes in size, and shifts in the position of the input patterns.
Clearly, of all the ANN architectures to date, Fukushima's neocognitron is the most biological. Like the vertebrate brain, its complexity defies analysis. Its development followed neurophysiologically inspired heuristics and led to a working system.
Was this article helpful?
This guide will help millions of people understand this condition so that they can take control of their lives and make informed decisions. The ebook covers information on a vast number of different types of neuropathy. In addition, it will be a useful resource for their families, caregivers, and health care providers.