Why much of what you think you know about learning is likely to be wrong
Ever since Ivan Pavlov (an endocrinologist with a particular interest in dog drool) made his famous observations about bells and dog-food, we have known that animals can be conditioned to respond to pairings of cues and events. Ring a bell every time you feed your dog, and you’ll soon be seeing some disappointed drooling whenever the doorbell chimes.
Pavlov, and other early students of learning, assumed that because a dog learns to associate a bell with its dinner after the two have been paired together, then this kind of learning was the product of a simple process of tracking and forming associations. The dog sees or hears one thing, then sees or hears another thing, it notices that they “go together” and, voila! Learning occurs.
Unfortunately, not only is does it turn out that this common understanding of how animals — or, indeed humans — learn associations is completely wrong, but to the consternation of Robert Rescorla, who did a lot of the important work in figuring out how animals actually do learn form to form associations, half a century after the simple associative account of learning was shown to be utterly incorrect, most psychologists and neuroscientists are still taught to believe that this is how animals — and, indeed humans — learn. In the light of this, our recent work showing that scientific accounts of cognitive ageing have spectacularly failed to consider the impact on learning on cognitive processing has a grim inevitability about it. How can scientists control for learning if their training ensures that they fail to understand what it is?
The problem runs deep. As a number of ingenious experiments by Rescorla and other researchers’ studies have helped make clear, most people’s — and, what is worse, most scientists’ — understanding of the way associative learning works is pretty much the opposite of our best estimation of the truth.
In my last post, I promised to provide more in-depth coverage of our aging work. On reflection, I realized that unless I made some kind of effort to explain what our best understanding of how our brains learn is, then our explanations of what actually happens to our minds and memories as we age would be difficult, if not impossible to grasp. This post — and my next — are intended to bridge this gap, by helping you to understand why it is that learning almost certainly doesn’t work the way you think it does, and by helping you to understand how it actually does work. Today, I’m mainly going to focus on the first part of the equation: I’m going to explain why, if you think that learning associations is about putting two and two together, then a lot of that stuff you think you know about learning is wrong.
One of the key findings that helped to completely undermine traditional ideas about learning as a simple process of associations was first discovered by Robert Rescorla. The experiments that led to Rescorla’s discovery are at once deceptively simple and insidious.
In its first phase, a bunch of rats are put into a cage. They then repeatedly here some tones, and after each tone they are shocked.
The interesting part comes in the second phase, where Rescorla looked at what would happen if he held the number of tones that were followed by shocks (the association rate) constant, while varying the background rate of tones that weren’t followed by shocks:
Rescorla discovered that as he increased the number of background tones, the degree to which rats froze upon hearing the test tone decreased. Even though the exact same number of tones were paired with shocks across all of his experiments, he found that as the number of unconnected tones increases, the rats freeze less and less upon hearing the subsequent test tones.
This result makes intuitive sense. Of course the rats will pay more or less attention to the tones depending on the amount of information that they convey about the shocks. Yet this totally intuitive finding undermines two stubborn myths about learning that many people find equally intuitive. The first myth it undermines is that learning in animals happens in response to rewards and punishments.
Think about it: The number of tones that lead to shocks (punishment) is identical in both conditions. The only difference is that lots of tones lead to nothing in the second. Given that the rats learn something different in the second experiment, and given that the rate of shocks is the same in both experiments, it follows the rats in the second experiment must be learning something from what they experience after the no-shock tones… which is nothing.
If the rats are learning from situations where nothing happens, it follows that rats can’t just be learning in response to rewards and punishments. (In fact, it turns out that the rewards and punishments used in experiments just help make animal behavior interpretable: If a dog salivates on hearing a bell, we can tell that it has learned something, and can even hazard a guess as to what.)
The second myth that Rescorla’s findings overturn is the idea that learning is a process of attending to associations. The association-rate — the number of tones followed by shocks — which provides positive evidence about the information provided by cues, is identical in both experiments. All that changes is the background rate of tones that don’t precede shocks. The tones that don’t lead to shocks provide negative evidence about the value of tones as a useful cue to events in a rat’s life. And it turns out that the usefulness of a cue can be estimated from the information these two forms of evidence provide: If a cue to an event – tones and shocks, or thunder and lightening – has a high association rate and a low background rate, it is going to be informative about that event; If it has a low association rate and a high background rate, it will not be.
What scientists have figured out from experiments like this one is that learning has evolved to do more than simply help rats learn “associations.” Rather, it helps them figure out the informativity of the various cues they encounter in the world, so that they can use them to predict the various events that will influence their day to day ratty existence.
It turns out that the best way to understand how way rats learn to balance association rates and background rates is to suppose that the first time an event – say, a shock – occurs, the rat records its current sensory inputs as being positive evidence that the event will occur when those inputs occur. The interesting, and slightly counterintuitive part of learning occurs when any of those sensory inputs are present and that event doesn’t occur. Say, a tone is heard, and the shock doesn’t materialize. In this case, the rats’ brain treats this faulty evidence as an error, and downgrades its record of the relationship between the tone and the shock that failed to materialize in memory.
In other words, learning appears to have evolved so as to help animals’ reduce their uncertainty about the world in a flexible, adaptive way. Thus, for instance, scientists have discovered that if a rat has already learned to expect that shocks will follow tones, the introduction of a new cue – say an accompanying bell – does not produce much learning. If a rat is trained on tones and shocks, then later introduced to tones/bells then shocks, and then later still only the bell is rung, while the rat might flinch a little, it may well do nothing at all. However, if the tone is played, the same rat will freeze like it always did. The fact that the shocks are already predicted by the tones inhibits learning of the bell as a new cue. It would seem to be because, given the rats’ prior experience, the bell is uninformative: Evolution appears to have tuned rat learning to respond only when there is uncertainty in the world.
These apparently simple processes produce a very subtle and powerful mechanism for learning. It is also a mechanism that is the opposite of that supposed by most naïve theories of associative learning. Rats and dogs don’t learn to “associate things.” Rather, at any given point, a rat’s brain (an information processing device absurdly more powerful than the computer on which I’m typing) is weighing up the information in every scrap of evidence its senses are providing, assessing the information in that sensory data, and updating its expectations about the world accordingly.
The cues provided by a rats’ sensory inputs compete for relevance in learning as it moves through time and space, and given the nature of the world, most of these data turn out to be uninformative. This means that as a rat acquires more experience, most of the sensory data it experiences will lose out in the competition for relevance that is created by the learning mechanism I have described: The background rates of many features of the rat’s world will continually increase, and its expectations will become ever more refined, which will mean that many of its sensory experiences will be uninformative, and go unrecorded in its memory. As its experience increases, the rat will learn to get better at discriminating against uninformative sensory information and in favor of informative sensory information. And what this means is that, despite the fact that people talk about “associative learning,” rat learning is ultimately discriminative.
Instead of being the simple process of tracking “associations” Pavlov imagined, it turns out that the way animals learn “associations” is an evolutionary twist on Sherlock Holmes’ dictum: “When you have eliminated the impossible, whatever remains, however improbable, must be the truth.” A rat does not simply learn to associate a tone with a shock, but rather the association between the tone and the shock is what is left over after the rat’s brain has weighed every other potential source of information and found it wanting.
This last point is somewhat heartbreakingly illustrated by experiments of the kind I described earlier: As the number of background tones continues to increase, then eventually, rats may learn to ignore them altogether. In the sparse environment of an experimental cage, the rats’ brains come to learn that the best predictor of the shocks is the cage itself. At this point, the rats simply freeze on entering the cage, exhibiting the most pitifully bamboozled form of learned helplessness.
Rescorla, R. (1988). Pavlovian conditioning: It’s not what you think it is. American Psychologist, 43 (3), 151-160 DOI: 10.1037/0003-066X.43.3.151