Bayes' theorem in predictive coding
Chapter 5 endnote 51, from Lisa Feldman Barrett.
Some context is:
...if you are on vacation in Paris and you perceive a stranger frowning at you in a subway car, you might not have any past experience with that stranger or that subway, and you might not have visited Paris before, but your brain does have past experiences of other frowning people in unfamiliar places. Your brain can then construct a sample of concepts, based on past experience and probability, to use as predictions. [...] Using Bayesian rules of probability.
Bayesian inferences plays a part in the theory of constructed emotion because, as I describe in chapter 4, the brain works like a scientist, making and testing hypotheses. Each categorization it constructs is a hypothesis that has some probability of being the best explanation for the latest sensory input from the body and the world. The brain is thought to estimate this probability, before it knows anything about the evidence, based on probabilities from previous times that it used same categorization. This situation sets the stage, mathematically speaking, to apply Bayes' theorem:
- p(h/e) = [p(h) x p(e/h)] / p(e)
where h = hypothesis about the state of the world (or prediction) and e = sensory evidence. The individual terms are:
- p(h/e) = posterior probability
- What is the probability that the prediction is correct given the incoming sensory inputs? This tells us how strongly a prediction is supported by the sensory evidence; if this number is highest, then this prediction will become the categorization.
- p(h) = prior probability
- What is the probability that the hypothesis was true on prior occasions in similar contexts? This probability is estimated using past instances of posterior probabilities in similar contexts.
- p(e/h) = likelihood
- What is the probability that this evidence will occur given that the prediction is true? This tells us how likely the sensory evidence is in the world where the prediction is true.
- What is the probability of that this sensory evidence will occur regardless of the the prediction?
p(e/h) and p(e) together reveal whether the sensory evidence is diagnostic for the prediction. If the sensory evidence is a reliable indicator that the hypothesis is correct, then it is possible to have confidence in the evidence to confirm or disconfirm the prediction.
History of Bayesian probability
Bayes theorem was developed by Reverend Thomas Bayes in the 1700s but languished in obscurity until after his death. It was taught and widely used in the 1800s, but was attacked by the evolutionary biologist Ronald Fisher, who is responsible for many of the statistical tests (based on frequencies) that have been mainstays in psychology for almost a century. Bayesian inference has been revived several times since then, even helping to break the code of the Nazi’s Enigma machine in World War II, and is now a very big deal in psychology and neuroscience.
Notes on the Notes
- Perfors, Amy, Joshua B. Tenenbaum, Thomas L. Griffiths, and Fei Xu. 2011. “A Tutorial Introduction to Bayesian Models of Cognitive Development.” Cognition 120 (3): 302–321.