
Paper Code 

Title 
Discretetime Markov chains with twotime scales and a countable state space: limit results and queueing applications 
Authors 
Zhan Hanqin 
Corresponding Author 

Year 
2008 
Title of Journal 

Volume 
80 
Number 
4 
Page 
339369 
Abstract 
Abstract
This work is concerned with discretetime Markov chains having countable state spaces and twotimescale structures. We examine two classes of Markov chains. In the first class, the state space of the Markov chain is nearly decomposable into a finite number of subspaces, each of which has countably many states, whereas in the second class, the state space is nearly decomposable into infinitely many subspaces each of which has finitely many states. Singular perturbation methods and twotime scales are used to alleviate the computational complexity. Under appropriate conditions, for the first class of models, we show that the formulation is 'equivalent' to a continuoustime Markov chain with a finite state space resulting in a substantial reduction of computational burden; for the second class of models, a similar 'equivalence' is established. These results are obtained using asymptotic expansions of probability vectors and transition matrices, and properties of aggregated processes. Moreover, we prove that suitably scaled sequences of occupation measures converge weakly to switching diffusion processes. An application to queuing networks is also presented. 
Full Text 

Full Text Link 

Others: 
Abstract
This work is concerned with discretetime Markov chains having countable state spaces and twotimescale structures. We examine two classes of Markov chains. In the first class, the state space of the Markov chain is nearly decomposable into a finite number of subspaces, each of which has countably many states, whereas in the second class, the state space is nearly decomposable into infinitely many subspaces each of which has finitely many states. Singular perturbation methods and twotime scales are used to alleviate the computational complexity. Under appropriate conditions, for the first class of models, we show that the formulation is 'equivalent' to a continuoustime Markov chain with a finite state space resulting in a substantial reduction of computational burden; for the second class of models, a similar 'equivalence' is established. These results are obtained using asymptotic expansions of probability vectors and transition matrices, and properties of aggregated processes. Moreover, we prove that suitably scaled sequences of occupation measures converge weakly to switching diffusion processes. An application to queuing networks is also presented. 
Classification: 

Source: 

