As I see myself

For the people, who are unaware of , there are two approaches to Probability. I studied about these in the math courses in my Undergrad, but was never told of these being two different approaches to probability theory. Alas!!

1) The Classical Approach (also called the frequentist view)

2) The Bayesian Approach (also called the subjectivist view)

I shall be talking about the latter.. (as it is the one used in Machine Learning! and also my personal favourite )

The essence of the Bayesian approach to Probability:

You believe that you like a person. You don’t know how much you like him/her. The actions of the person towards you affect your liking or the belief (your belief that you like the person) you have towards him/her. This is common-sense. The beauty of the Bayesian principle lies in, encoding/representing the above common-sense fact using math.

An important thing one should be keeping in mind is that the Bayesian approach to Probability talks about ‘your’ beliefs on a subject. My belief about the Tajmahal is great. Another person’s might not be the same. This is the main difference when compared with the classical approach.

A simple example to explain the difference between the classical and the bayesian view is: Let us take the usual coin-tossing experiment. I toss a coin a 100 times, and then set out to estimate the probability of getting a heads when i toss it the 101th time.

1) Classical approach: You take a note of the number of times heads has appeared in all these 100 flips and then find the probability of getting heads by using: (No.of heads appeared)/(Total times the coin has tossed). You use this value for your future experiments saying that, in my first 100 coin tosses, i got heads 60 times (say), so the 101th time when I toss a coin, there is chance of getting a head than a tail!

2) Bayesian approach: Your aim of this experiment is not to know the probability of a coin showing heads in your first 100 tosses, but to know what might the coin show on the 101th toss or on the tosses later. Well, in this case the probability as obtained by these 2 methods might be the same, but your approach of seeing things is what that matters most.

Say, you have a belief that the coin shows tails. Now with every toss of the coin showing tails, you increase this belief or yours, and with every occurrence of a head, you decrease your belief. (You increase your likeliness about a person, for every nice thing he/she does towards you…isn’t it?) This is the Bayesian view of seeing things.

P(coin = tails|E) = P(E|coin = tails) * P(coin = tails)/P(E) <– the bayes theorem.

P(coin = tails) — Your initial belief that the coin shows tails.

P(E|coin = tails) — Now, given you believe that the coin shows tails, you test that against the Experiment, E. If in this test of yours, if you notice that the coin is infact showing tails, then your belief, P(coin = tails|E) gets increased.

(because, P(coin=tails | E) is proportional to P(E|coin = tails)*P(coin = tails) , from the above equation.)

You obtain the final value after completing the experiment. Now, based on this value you might take decisions on the future tosses.

Remember the word ‘belief’ is synonymous with ‘probability’ when u see things in bayesian perspective.

The coin-tossing experiment might not be a very good example to differentiate the bayesian and classical approaches.. I’ll try to give a better insight into this issue sometime later when it is being discussed with some machine learning problem. Just know that there are different views of seeing things, and the latter is the convenient one. It is convenient because it represents a common-sense/reality fact.

The Bayesian view is very very useful when dealing with Machine Learning stuff. and I’m a Bayesian.

#A good explanation on Bayesian philosophy. And there are many more!

(Abridged contents from "The Bayesian" by Vinay Gryffindor)