Probability

• Objective versus Subjective view
• Objective view generally deals with frequency oriented interpretation
  • You assume or visualize an experiment
  • Coin that is tossed. The actual experiment may or may not be conducted
  • If you assume randomness, the probability of head is 0.5.
  • P(head)=0.5
  • P(head,head)=0.5*0.5=0.25
  • P(head,tail)=0.25
  • P(tail,head)=0.25
  • P(tail,tail)=0.25
  • P(a head and a tail)=P(head,tail)+P(tail,head)=0.5
• P(x or y) = P(x) + P(y) provided x and y are mutually exclusive.
• P(x or y) = P(x) + P(y) - P(x and y)
• The previous rule can be derived from the simple addition axiom.
• Let us use the concept of random variable for simplicity assume that these variables take true or false values.
• the variable X1,X2,X3,....,Xn can take values x1,x2,x3,x4,....xn or -x1,-x2,-x3,....,-xn
• P(X1,X2)=P(X2|X1)*P(X1)
• P(X1,...Xn)=P(X2,.....,Xn|X1)*P(X1)
• =P(Xn|X1,....Xn-1)*P(X1,...,Xn-1)
• =P(Xn|X1...Xn-1)*P(Xn-1|X1...Xn-2)*....*P(X2|X1)*P(X1)
• See the example when n = 5. The graph with no independence assumption is quite complicated
• Total number of probabilities required is O(2^n)
• Conditional independence assumptions are made to reduce the number of probability values that need to be obtained
• You should be able to look at a belief network and construct a formula for joint probability and vice versa
• You should be able to explain the advantage of belief networks for a short descriptive question
• In the belief networks, the probabilities of evidence are propagated through the network using conditional probabilities.
• Local computations in a parallel computer are possible using belief networks.
• Most of the probabilities used in belief networks are essentially subjective in nature. We have to stretch our imagination to visualize the experiments
  • Coin toss: easy to visualize the experiment
  • How do you visualize the experiment for Mary called given that there was an alarm. How many times has this type of situation has occurred or likely to occur to obtain reliable frequency.
  • We are using our subjective judgment. Do laws of probability really hold for these subjective probabilities? They don't.
  • 1920-1950 people came up with a betting scenario, concept of rationality, utility.
  • A rational person will not make a book to lose money. Make assumptions about rational behaviour and show that with those assumptions the laws of probability hold.
  • Is betting the right scenario for reasoning uncertainty? It is not.
  • There are several objections to the theory of probability.
• Fuzzy sets use this criticism of probability, the debate continues
• You can come up with many different reasoning schemes but they have to be mathematically sound.
• One mathematically sound theory is theory of belief functions.
• Bel(A)+Bel(-A)<=1.
• More generally, Bel(A or B) >= Bel(A) + Bel(B) assuming A and B are mutually exclusive.
• Bel(rain) = 0.56 this was based on evidence in support of rain.
• Bel(-rain) if you never did any calculations or there was no evidence to make the calculations to calculate this then you should be allowed say that this value is zero.
• Bel(rain or -rain) = 1.
• It is snowing, thermometer is saying 0. The probability that therm. is working is 0.95. If therm is working the streets can't be slippery there is a lot of traffic out there
• Bel(-slip)=0.95
• the prob. that therm is broken is 0.05. but if the thermometer is broken the temp. could really be anything there is no need to assume that the temp is well below 0. So we can't say Bel(slip) =0.05 we have to say Bel(slip)=0 because we have no evidence to conclude slip.
• But Bel(slip or -slip)=1.
• Bel(slip or -slip)>Bel(slip)+Bel(-slip)
• The super additivity axiom is in fact generalized for non-mutually exclusive propositions similar to the probability. That's why belief functions are called generalizations of the probability functions.
• The evidential rule is not as sophisticated as the Bayes rule.
• There are Markov trees for belief functions similar to the belief nets for probability.