Abstract
Empirical modeling methods based on linear projection include linear methods such as, ordinary least squares regression, partial least squares regression, principal components regression, and nonlinear methods such as, backpropagation networks with a single hidden layer, projection pursuit regression, nonlinear partial least squares regression, and nonlinear principal components regression. In this paper, these popular modeling techniques are unified to yield a single method called nonlinear continuum regression (NLCR). This unification is based on the common framework for empirical modeling methods developed in Part I (Bakshi and Utojo, 1997). The unification of methods based on linear projection by NLCR is achieved by basis functions that adapt to the measured data and are determined by univariate smoothing in the projected input-output space, a common optimization criterion for finding the projection directions that subsumes all methods based on linear projection, and a hierarchical training methodology that allows efficient modeling. The NLCR optimization criterion contains a new adjustable parameter that controls the degree of overfitting or bias of the model, and spans the continuum of methods from projection pursuit regression or backpropagation networks to nonlinear principal components regression. Consequently, NLCR results in models that are usually more general and compact than those obtained by existing methods based on linear projection, while eliminating the need for arbitrary selection of an empirical modeling method based on linear projection for a given task. The improved modeling ability of NLCR and its performance on different types of training data are illustrated by examples based on simulated and industrial data.