Clipboard interview question

Stochastic - Markov Chain problem