Convergence of a second order Markov chain

Shenglong Hu, Liqun Qi

Research output: Journal article publicationJournal articleAcademic researchpeer-review

13 Citations (Scopus)

Abstract

In this paper, we consider convergence properties of a second order Markov chain. Similar to a column stochastic matrix being associated to a Markov chain, a transition probability tensor P of order 3 and dimension n is associated to a second order Markov chain with n states. For this P, defineFPasFP(x) =Px2on the n-1 dimensional standard simplexΔn. If 1 is not an eigenvalue ofFPonΔnand P is irreducible, then there exists a unique fixed point ofFPonΔn. In particular, if every entry of P is greater than 12n, then 1 is not an eigenvalue ofFPonΔn. Under the latter condition, we further show that the second order power method for finding the unique fixed point ofFPonΔnis globally linearly convergent and the corresponding second order Markov process is globally R-linearly convergent.
Original languageEnglish
Pages (from-to)183-192
Number of pages10
JournalApplied Mathematics and Computation
Volume241
DOIs
Publication statusPublished - 15 Aug 2014

Keywords

  • Nonnegative tensor
  • Second order Markov chain
  • Transition probability tensor

ASJC Scopus subject areas

  • Computational Mathematics
  • Applied Mathematics

Cite this