Constructing low-order discriminant neural networks using statistical feature selection

The selection of relevant inputs and determining an appropriate network topology are two critical issues faced when applying neural networks to classification problems. This paper presents an algorithm called Pair Attribute Learning (PAL) for addressing both input selection and the determination of network topology. The PAL algorithm uses a preprocessing stage to search for features derived from pairs of training instances. A statistical rank is used to select a good set of features, and these features are then used to drive the construction of a single hidden layer neural network. Only inputs relevant within the context of a feature are used in constructing the network. This results in a sparsely connected hidden layer and lower-order discriminants. The results on nine learning problems demonstrate that on average PAL-constructed networks are 70% less complex than networks built using other constructive techniques, without a significant loss of predictive accuracy. In addition, the PAL algorithm does not use iterative construction or suffer from bias mismatch. Because it addresses both input selection and network topology, the PAL algorithm provides an end-to-end solution for applying neural networks to classification problems.