An Adaptive Algorithm for Determining Variations of the Characteristics of an Observed Random Process

An adaptive algorithm is proposed for the determination of the variations of the characteristics of an observed random process. It is postulated that the indicated variations behave according to the law of an unobservable homogeneous Markov chain with unknown transition probabilities. The number of states of the Markov chain and the conditional distribution functions for the observed variables are presumed to be known. At each instant (discrete time) the a posteriori distribution with respect to the unobservable states of that chain is computed. An algorithm converging to the true values is described for estimating the unknown transition probabilities of the chain. An example is given of the operation of the formulated adaptive algorithm in the probabilistic model of a Markov chain with observables having a binomial distribution function.