Let E₁, E₂, …, E_j(j = 0, 1, 2, …) represent the exhaustive and mutually exclusive outcomes (states) of a system at any time. Initially, at time t₀, the system may be in any of these states. Let
a_j⁽⁰⁾(j = 1, 2, …) be the absolute probability that the system is in state E_j at t₀. Assume further that the system is Markovian.
Define the following as the one-step transition probability of going from state i at t_n-1 to state j and t_n and assume that these probabilities are stationary over time. The transition probabilities from state E₁ to state E_j can be more conveniently arranged in a matrix form as follows.