Collective classifier learning strategy based upon competition in the coexistence regime

The purpose of this research is to create a new learning strategy for collective classifiers aimed at approximating the Bayesian optimal classification rule. A collective classifier is an ensemble of simple elements, each characterized by a specific response function and free of internal dynamics or variable parameters. Learning is achieved by targeted reshaping the composition of the ensemble (quantities of elements of specific types). A formerly known strategy was based on winnertake-all competitive dynamics, which limited dramatically the capabilities of the learning strategy e.g. in application to classes with bimodal probability distribution. Methods. The population dynamics of the learning strategy was modified to ensure a globally stable coexistence regime. The system evolution is described by a discrete stochastic selection algorithm for the ensemble elements and in the form of ordinary differential equations as a continuous approximation. An analytic expression for the stable equilibrium is available, thus allowing targeted algorithm design for achieving a prescribed learning outcome. A learning algorithm designed for approximating the Bayesian optimal classification rule is described in the paper. Results. By numerical integration of differential equations and by simulating the selection algorithm for a sample classification task with a bimodal probability distribution we confirm all theoretical statements, including the mutual agreement between the discrete and continuous descriptions, the coexistence of elements with different response functions in the trained ensemble, and achieving a summary ensemble output approximating the Bayesian classifier. The simulation suggests that relative fluctuations of the discrete variables (quantities of elements of specific types) may be lowered by increasing the total ensemble size. Conclusion. The collective classifier principle implies its possible use in creating classifiers from ensembles of simple elements, such as "smart dust" which means miniature extremely simplified sensor devices, or genetically reprogrammed living cell populations acting as biosensors. Achieving the coexistence regime in the learning strategy complicates its implementation, but broadens the range of amenable classification tasks.