Semi-Markov process
In the mathematical theory of stochastic processes, a semi-Markov process Z, also known as a Markov renewal process, is associated with and can be constructed from a pair of processes <math>W=(X,Y)</math>, where <math>X</math> is a Markov chain with state space <math>S</math> and transition probability matrix <math>P</math>, whereas <math>Y</math> is a process for which <math>Y(n)</math> depends only on <math>r=X(n-1)</math> and <math>s=X(n)</math>, and whose distribution function is <math>F_{rs}</math>.
The semi-Markov process <math>Z</math> is then the process that chooses its sites (on <math>S</math>) according to <math>X(n)</math>, and that chooses the transition time from <math>X(n-1)</math> to <math>X(n)</math> according to <math>Y(n)</math>.
Since the properties of <math>Y</math> (such as mean transition time) may depend on which site <math>X</math> chooses next, the process <math>Z</math> is in general not a Markov process. Yet, the associated process <math>W(n)=(X(n),Y(n))</math> is a Markov process. Hence the name semi-Markov.
See also
- Markov process
- Stochastic process
- Renewal theory
- Variable-order Markov model