Abstract

Automatic feature extraction by adaptive networks is an attractive approach to pattern recognition for many visual and image processing tasks where the underlying features are unknown. These networks, composed of units that perform nonlinear memoryless transformation on the sum of their inputs, can be viewed as a particular implementation of adaptive pattern recognition schemes. For a given problem, the network is usually trained by repeated presentation of examples from a training set and modified until a solution is reached as defined by a criterion performance. As in any trainable pattern recognition scheme, the fundamental problem concerns the networks’ capabilities to respond appropriately to novel patterns. This ability is referred to as desirable generalization or induction. First, it is demonstrated that the specific generalizations depend on particular solutions obtained to the training set and that the number of possible solutions is likely to be very large. Consequently, even the simplest networks that can solve a given problem are usually too unconstrained to produce desirable generalizations. Thus the network pattern recognition problems are frequently ill-posed due to the fact that the power or ability to solve an arbitrary problem conflicts with constraints required for good generalization. For the networks to generalize in a desirable manner it is necessary to add additional constraints in the same manner as regularization is used for other ill-posed problems.

© 1988 Optical Society of America