In a layered feet-forward network the error surface with respect to a desired goal function completely describes the potential of that network to carry out a particular task. Such an error surface can be obtained by plotting the maximum error or the sum of the squares of the errors, where the individual errors are the deviation of the actual output of the network from the desired goal function. For a network with
W individual (synaptic) weights, these error surfaces are
W-dimensional surfaces embedded in a
W + 1-dimensional space and defined over the
W-dimensional domain determined by the ranges of the
W weights.
We have studied in detail the error surfaces of a few small networks that we believe exhibit some of the typical characteristics also found in large networks. For the chosen networks we have determined the inherent symmetries of their associated error surfaces, the shape of the region around the origin, and the numbers and types of local minima. For each type of local minimum, we examined its multiplicity, the fractional coverage of the total domain space by its collection zone, and the approximate shape of the valleys leading into it. This study of small three-layered networks performing the Exclusive-OR function of two or three inputs provides some insight into the general structures of the error surfaces and thus the capabilities of more complicated networks.
Title
Analysis of Simple Neural Networks
Published
1988-12-20
Full Collection Name
Electrical Engineering & Computer Sciences Technical Reports
Other Identifiers
CSD-88-482
Type
Text
Extent
62 p
Archive
The Engineering Library
Usage Statement
Researchers may make free and open use of the UC Berkeley Library’s digitized public domain materials. However, some materials in our online collections may be protected by U.S. copyright law (Title 17, U.S.C.). Use or reproduction of materials protected by copyright beyond that allowed by fair use (Title 17, U.S.C. § 107) requires permission from the copyright owners. The use or reproduction of some materials may also be restricted by terms of University of California gift or purchase agreements, privacy and publicity rights, or trademark law. Responsibility for determining rights status and permissibility of any use or reproduction rests exclusively with the researcher. To learn more or make inquiries, please see our permissions policies (https://www.lib.berkeley.edu/about/permissions-policies).
