This example shows how to compute and visualize state redistributions, which show the evolution of the deterministic state distributions over time from an initial distribution.
Consider this theoretical, right-stochastic transition matrix of a stochastic process.
Create the Markov chain that is characterized by the transition matrix P.
P = [ 0 0 1/2 1/4 1/4 0 0 ; 0 0 1/3 0 2/3 0 0 ; 0 0 0 0 0 1/3 2/3; 0 0 0 0 0 1/2 1/2; 0 0 0 0 0 3/4 1/4; 1/2 1/2 0 0 0 0 0 ; 1/4 3/4 0 0 0 0 0 ]; mc = dtmc(P);
Plot a directed graph of the Markov chain and identify classes using node colors and markers.
mc represents a single recurrent class with a period of 3.
Suppose that the initial state distribution is uniform. Compute the evolution of the distribution for 20 time steps.
numSteps = 20; X = redistribute(mc,numSteps);
X is a 21-by-7 matrix. Row t contains the evolved state distribution at time step t.
Visualize the redistributions in a heat map.
The periodicity of the chain is apparent.
Remove the periodicity from the Markov chain by transforming it to a lazy chain. Plot the transition matrix of the lazy chain as a heatmap.
lc = lazy(mc); figure; imagesc(lc.P); colormap('jet'); axis square; colorbar; title('Theoretical Lazy Chain Transition Matrix')
lc is a
lazy creates the lazy chain by adding weight to the probability of persistence, that is,
lazy enforces self-loops.
Compute the evolution of the distribution in the lazy chain for 20 time steps. Plot the redistributions in a heatmap.
X1 = redistribute(lc,numSteps); figure; distplot(lc,X1);
View the evolution of the state distribution as an animated histogram. Specify a frame rate of 1 second.
Compute the stationary distribution of the lazy chain. Compare it to the final redistribution in the animated histogram.
xFix = asymptotics(lc)
xFix = 1×7 0.1300 0.2034 0.1328 0.0325 0.1681 0.1866 0.1468
The stationary distribution and the final redistribution are nearly identical.