Fukushimas Neocognitron

There has been an explosion of effort and innovation in the field of ANNs since the original work of Rosenblatt and Widrow. It is not the purpose here to describe the evolution of modern ANNs or the details of their operation. Instead, this section will examine the general properties of one of the more biologically inspired ANN systems, the neocognitron.

Fukushima, whose model of a static pattern recognition system based on the vertebrate retina was introduced in Section 7.1.3, went on to develop an amazingly complex ANN that he called the neocognitron (Fukushima, 1980, 1984, 1988a,b). The neocognitron is an hierarchical ANN; it has strong roots from the retina in its design. Hecht-Nielsen (1990) calls it "the largest and most complicated neural network yet developed."

The purpose of the neocognitron was to recognize members of a set of low-resolution, binary, alphanumeric character images such as the numerals 0 to 9. One neocognitron has an amazing eight layers of nodes (Fukushima, 1988a). In the layers, there are 156 "slabs" or processing subunits. For example, there is a 19 x 19 element receptor array followed by the first (US1) layer, which has 12, 19 x 19 element slabs in it. The second (UC1) layer has 8, 21 x 21 element slabs in it; the third (US2) layer has 38, 21 x 21 element slabs in it; the fourth (UC2) layer has 19, 13 x 13 element slabs in it; the fifth (US3) layer has 35, 13 x 13 element slabs in it; the sixth (UC3) layer has 23, 7 x 7 element slabs in it; the seventh layer (US4) has 11, 3 x 3 element slabs in it, and finally, the eighth layer has 10 1-element slabs in it. (Element is synonymous with neuron or node.) There is a total of 34,980 nodes in the eight layers, and over 14 million connections with weights; Figure 7.3-3 illustrates the basic architecture of the eight-layer neocognitron and its slab subunits. Clearly, space does not permit description of the operation and training of this neocognitron in detail here. (The interested reader should read Hecht-Nielsen's, 1990, section 6.3 for a clear summary description of how the neocognitron works.)

recall segmentation

Recognition

Recognition

Crohn Disease Segmentation

recall segmentation

Selective attention

FIGURE 7.3-3 The eight layers with subunits of Fukushima's vastly complex ANN, the Neocognitron. The heritage of the neocognitron can be traced to Fukushima's early multilayer, static feature extractors, which were based on retinal neurophysiology.

Selective attention

FIGURE 7.3-3 The eight layers with subunits of Fukushima's vastly complex ANN, the Neocognitron. The heritage of the neocognitron can be traced to Fukushima's early multilayer, static feature extractors, which were based on retinal neurophysiology.

Fukushima (1988b) also described a six-layer neocognitron, one of whose "neurons," or cells, is shown in Figure 7.3-4. N excitatory inputs from the previous layer are summed to form the parameter, e. One or more inhibitory inputs are summed to form h. e and h are combined according to the function, y(e, h):

Neocognitron Fukushima

FIGURE 7.3-4 Block diagram of a basic "neuron" used in Fukushima's neocognitron. e, summed excitatory inputs (e S 0); h, summed inhibitory inputs (h 3 0). Note that for |h| > e, y < 0. Since y represents an instantaneous frequency of a neuron, it must be non-negative, hence the actual output, y', is the rectified y.

FIGURE 7.3-4 Block diagram of a basic "neuron" used in Fukushima's neocognitron. e, summed excitatory inputs (e S 0); h, summed inhibitory inputs (h 3 0). Note that for |h| > e, y < 0. Since y represents an instantaneous frequency of a neuron, it must be non-negative, hence the actual output, y', is the rectified y.

As in the earlier (1969) Fukushima feature extractor models, y is half-wave-rectified to form the neuron output, y'. That is, y' = y, y S 0

Also,

Was this article helpful?

0 0
Peripheral Neuropathy Natural Treatment Options

Peripheral Neuropathy Natural Treatment Options

This guide will help millions of people understand this condition so that they can take control of their lives and make informed decisions. The ebook covers information on a vast number of different types of neuropathy. In addition, it will be a useful resource for their families, caregivers, and health care providers.

Get My Free Ebook


Post a comment