The Bayesian Approach
Aug 31, 2007
For the people, who are unaware of , there are two approaches to Probability. I studied about these in the math courses in my Undergrad, but was never told of these being two different approaches to probability theory. Alas!!
1) The Classical Approach (also called the frequentist view)
2) The Bayesian Approach (also called the subjectivist view)
I shall be talking about the latter.. (as it is the one used in Machine Learning! and also my personal favourite 
)
The essence of the Bayesian approach to Probability:
You believe that you like a person. You don’t know how much you like him/her. The actions of the person towards you affect your liking or the belief (your belief that you like the person) you have towards him/her. This is common-sense. The beauty of the Bayesian principle lies in, encoding/representing the above common-sense fact using math.
An important thing one should be keeping in mind is that the Bayesian approach to Probability talks about ‘your’ beliefs on a subject. My belief about the Tajmahal is great. Another person’s might not be the same. This is the main difference when compared with the classical approach.
A simple example to explain the difference between the classical and the bayesian view is: Let us take the usual coin-tossing experiment. I toss a coin a 100 times, and then set out to estimate the probability of getting a heads when i toss it the 101th time.
1) Classical approach: You take a note of the number of times heads has appeared in all these 100 flips and then find the probability of getting heads by using: (No.of heads appeared)/(Total times the coin has tossed). You use this value for your future experiments saying that, in my first 100 coin tosses, i got heads 60 times (say), so the 101th time when I toss a coin, there is chance of getting a head than a tail!
2) Bayesian approach: Your aim of this experiment is not to know the probability of a coin showing heads in your first 100 tosses, but to know what might the coin show on the 101th toss or on the tosses later. Well, in this case the probability as obtained by these 2 methods might be the same, but your approach of seeing things is what that matters most.
Say, you have a belief that the coin shows tails. Now with every toss of the coin showing tails, you increase this belief or yours, and with every occurrence of a head, you decrease your belief. (You increase your likeliness about a person, for every nice thing he/she does towards you…isn’t it?) This is the Bayesian view of seeing things.
P(coin = tails|E) = P(E|coin = tails) * P(coin = tails)/P(E) <– the bayes theorem.
P(coin = tails) — Your initial belief that the coin shows tails.
P(E|coin = tails) — Now, given you believe that the coin shows tails, you test that against the Experiment, E. If in this test of yours, if you notice that the coin is infact showing tails, then your belief, P(coin = tails|E) gets increased.
(because, P(coin=tails | E) is proportional to P(E|coin = tails)*P(coin = tails) , from the above equation.)
You obtain the final value after completing the experiment. Now, based on this value you might take decisions on the future tosses.
Remember the word ‘belief’ is synonymous with ‘probability’ when u see things in bayesian perspective.
The coin-tossing experiment might not be a very good example to differentiate the bayesian and classical approaches.. I’ll try to give a better insight into this issue sometime later when it is being discussed with some machine learning problem. Just know that there are different views of seeing things, and the latter is the convenient one. It is convenient because it represents a common-sense/reality fact.
The Bayesian view is very very useful when dealing with Machine Learning stuff. and I’m a Bayesian.
#A good explanation on Bayesian philosophy. And there are many more!
(Abridged contents from "The Bayesian" by Vinay Gryffindor)Posted bySandeep Sekharamantri at 1:48 PM 0 comments
Labels: bayes theorem, machine learning, maths, statistics
Random Rambling
Aug 27, 2007
Life seems to be on the run. Just a month ago I used hog like a horse and slumber like a sloth all through the lovely week and go out to explore the countryside (or whatever was left of it in coimbatore) on the weekends. Now I lead a friday to monday life. The rest of the time just flies by.
Wake up on a monday morning with those legendary monday blues at about 8am. Get ready and go through the rush hour to office. The next thing worth mentioning is the beer bash on friday. The missing pieces in between are filled by me sitting in front of a computer, me drinking coffee and me banging my head all the time trying to figure out how in pluto's name do these software professionals survive, let alone make millions by just kutting all day long!!!
I had had enough of the software life. One full year filled with copied codes, reproduced assignments and sloppy professors. I expected the industry to be a lot better considering that we Indians make a lot of dollars doing IT development. And the industry was an awesome experience, at least in the beginning. I walked into NetApp to a great welcome. My manager walked with me upto my cubicle which already had my name plate on it. I had a free laptop with internet. New found friends took me to the pantry where food was free. I was told that on fridays beer was served, free ofcourse. The software industry employs some of the best looking girls too. And for a rough and hardened mechite like me this was paradise. I totally freaked out the first few days utilising every freebie that I could get. But suddenly I realised that I missed something amidst the partying. THE WORK!. What exactly do these software guys do to live that way? Well, they just code and code and code and more code!
In BITS I heard my computer science 'engineering' friends talk about development work happening in the MNC's in the nation and felt that they would be working at the fore-front of cutting edge computer science technology someday soon. They also talked about something called "testing" in a much demeaning way and I got the impression that it was just people putting in x=1 and x=0 to test whether a code is working or not.
Indeed even within the industry I found 'engineers' talk the same way about development and testing jobs. But soon I realised that these so called engineers do the same things, whether its development or testing, whether he's a fresher or a manager with 30years experience. They all do the same things. They all code and code and code! The interface might change, the exterior might change, the pay package might change, their timings can change, but their core work does not. They just code.
Well, 1 month into my internship and I have firmly decided never to go back to a pure software industry and work with software 'engineers' (they love that term in these companies!)
Engineering, in my opinion, is the useful manipulation of the laws of nature. Engineering has physics as its most fundamental building block. And that is how we were taught in school. If you are good in physics you can become an engineer. A computer science problem, even in its most primitive basic format cannot be remotely related to physics and hence it cannot be related to engineering.
I got into a little trouble when I started discussing this angle with my manager and asked him why he was not called a computer scientist instead of a software engineer!!
I am happy that I made a job with a core manufacturing firm where they have real engineers who practice engineering in the real sense of the term.
The trouble with living life from friday to monday is that you get these niggling seconds in between your orkut and free juices where you let your thoughts ramble randomly. And when you sit down to write a blog at the first chance you get, you are lost and those random sentiments desist to form a chorus. What results is this aphrodisiac poop that you have just read.
Posted bySandeep Sekharamantri at 10:49 PM 4 comments
Labels: fun, job, life, random, realisation, software, work
