Artificial neural network for supervised learning based on residual analysis

This paper proposes a new artificial neural network for pattern classification. The network is able to learn from a training set of preclassified objects (or patterns) to obtain a classifier. Each object (pattern) in the set is described by a fixed number of attributes whose values may be subject to noise perturbation. The proposed learning procedure is based on residual analysis, a statistical technique often used to determine how well a set of data fits a certain model. By computing the residuals for each combination of attribute value and class label and comparing them against the model that assumes independence between attribute values and class labels, the network identifies attribute values that are important in the characterization of various classes of objects. Using an information theoretic measure, the network then establishes the weighted connections between those processing elements which represent the classes and those which represent the important attribute values. The proposed training method has the advantage of being non-iterative and therefore very efficient computationally. It is also guaranteed to converge. Furthermore, since the topology of the network is deterministic, it is possible for the heuristic function of each element to be precisely recognized and for the internal associations directly analyzed. This is in contrast to many popular learning procedures, such as back propagation networks, in which the role each element performs is arbitrarily determined during training. The proposed network has been tested with data obtained from several real-world applications. The training time is of orders of magnitude less than that for back-propagation learning, yet its classification accuracy is superior to both back- propagation and the symbolic decision tree-based (ID3) approach to supervised learning.