Controlled markov chains - some stability problems

Abstract

"Along this work we study some stability-related problems in the context of Controlled Markov chains. As a first problem, we consider a division of the state space, and the goal is to construct a control policy such that the chain stabilize as much as possible in each portlon of the division without leaving it. As a second problem, we characterize the domain of attraction and escape set of a controlled Markov chain via a function v, which happens to be the solution of a Bellman's equation. The interpretation of v as the solutlon of a Bellman's equation also provides a way to calculate such function via a linear program. Finally, under some assumptions, we find a policy that maximize the probability of reaching certain set A. Our approach uses certain cost functions and dynamical programming, so it can be solved using a linear program."