Some approaches to the solution of optimization problems in supervised learning

There are some optimization problems that arise when highly accurate recognition algorithms are developed. One of them is to determine an optimal feasible (consistent) subsystem in a given system of linear inequalities. The optimality is defined by a number of constraints imposed on the subsystem, which can vary. Various approaches to the solution of this problem are proposed. Solution methods based on the search through the set of nodal subsystems of the given system of linear inequalities are developed. This can be exhaustive search or partial guided search that finds an approximate solution. A drastically different approximate method based on geometric considerations is proposed.