A Markov chain is a mathematical system that undergoes transitions from one state to another on a state space. It is essentially a kind of random process without any memory. This last statement, emphasizes the idea behind this process: “The future is independent from the past given the present”. In short, we could say that, the next step of our random process depends only on the very last step occurred. (Note that we are operating in discrete time in this case).
Let’s say that we would like to build a statistical model to forecast the weather. In this case, our state space, for the sake of simplicity, will contain only 2 states: bad weather (cloudy) and good weather (sunny). Let’s suppose that we have made some calculations and found out that tomorrow’s weather somehow relies on today’s weather, according to the matrix below. Note that P(AB) is the probability of A given B.
Therefore, if today’s weather is sunny, there is a P(SuSu) chance that tomorrow will also be sunny, and a P(CSu) chance that it will be Cloudy. Note that the two probabilities must add to 1.
Let’s code this system in Python:
Obviously the real weather forecast models are much more complicated than this one, however Markov chains are used in a very large variety of areas and weather forecast is one on them. Other real world applications include:
Machine learning (in general)
Speech recognition and completion
Algorithmic music composition
Stock market and Economics and Finance in general
For more information on Markov chains, check out the Wikipedia page.
If you are interested in Markov chains, I suggest you to check these two video series on YouTube which are (in my opinion) good explanations of the subject.
Brandon Foltz’s Finite Math playlist, very clear explanation with real world examples and the math used is fairly simple. You just need to know a bit of matrices, operations on matrices and probability (but if you are here I guess you have no problems on this)
Mathematicalmonk’s playlist on Machine Learning, where a more technical (formal) explanation is given in the videos on Markov chains, starting from here.
Hope this was interesting and useful.

Your blog is amazing. It is really worth reading. Hope you would like to read Markov Chains and Markov Chain applications
ReplyDeleteHi, Thanks for your comment! I'm glad you like the blog!
ReplyDelete