Why bad science is as much of a threat to the elderly as age itself
A couple of years ago, I moved from California to a small town Southern Germany. Determined to experience the ancient wonders of my new hometown to the fullest, I spent my first two months living in a cupboard-sized hotel room, patiently waiting for my dream apartment — in a converted fifteenth century nunnery — to become available.
When the time came to leave my cupboard and move in, I realized I had neglected a very important piece of due diligence. The apartment is a loft in the attic of the nunnery, and its many windows stare directly at the clock tower of a church. ‘Handy view of the clock!’ I had thought at the time I fell in love with the place. What did not occur to me then, but was soon to be brutally revealed, is that a ‘clock tower’ can also be a ‘bell tower.’ (Had I thought of this, I might have noticed that by sticking my head out of the window, I could enjoy a glorious, uninterrupted view of the clock tower of another church, which dominates the town centre.)
Because I failed to make the clock tower – bell tower association, I settled down to sleep that first night in complete ignorance of the plans that had been made to greet me. On the stroke of each quarter-hour I was to be welcomed to the neighborhood by locals only too happy to pass the time of day: One deafening bong for the quarter hour; two deafening bongs for the half-hour, three deafening bongs for the three-quarter-hour, and as many deafening bongs as it took to enumerate the hour (plus a few helpful peals to discriminate the hour-bongs from all the other bongs). And although this alone would have been perfectly sufficient to crush my hopes of a good night’s sleep, there was more: Each heavy metal communiqué from the bell tower opposite my apartment was accompanied by another, slightly offset message from the other church; a subtle reminder that just as they differ slightly in their conceptions of the divine, so each demurs at the other’s idea of the exact state of time.
Less than three years ago I despaired of every sleeping more than an hour or so in my new home, yet now I barely notice the bells. Why?
Have I gone deaf, my mandibles blasted by one call to prayer too many?
Perhaps it’s my age?
No again, for the same reason: If the bells ring unexpectedly, as they are wont to do for weddings, or to mark the religious origins of many mysteriously-timed local holidays, I hear them just fine. All too fine, in fact. I think of holidays and Saturdays — when many couples tie the knot — as good times to enjoy a few extra hours of sleep in the morning. The bells think otherwise.
The reason I am now oblivious to the bells, and why I notice them only if I fail to anticipate their ringing — or consciously attend to it — is that my brain has learned to actively ignore the predictable peals. It has noted them, learned that they are uninformative, and decided they aren’t worth wasting my attention on. Somewhere, back in my brain, the bells are still detected, but my brain and I have learned that the messages they impart are not news. So we ignore them.
Deciding what to ignore is an essential part of the process we call ‘learning.’ The details of this process can sometimes seem counter-intuitive, but they needn’t be so baffling. Consider the problem of trying to hear what your friend is saying in a busy room. You could ask your friend to speak up, of course, but this is likely to cause everyone else to pipe up too. Nor, given the way our senses work at the input level, is it obvious how you might ‘turn up’ the volume of your friend’s speech relative to everyone else’s. That is, of course, unless we reframe the idea of ‘turning up’ a desired signal into that of ignoring the noise around it. If our brains can find some predictable regularities in the signal we wish to attend to, or in the noise we want to filter, they can learn to discriminate the signal from the noise. By ignoring the latter, our minds can amplify the former.
Quite how this works in any given case is complicated. Your brain comprises many systems that learn to discriminate the informative stuff in this way, and the rate at which these subsystems do this and the effect of their learning on one’s subjective perception of the world varies. While it took my brain many months to figure out that I’d be better off if it ignored the bells, learned adaptation is a far more rapid process in other contexts (such as hearing speech at a party) and in other sensory domains.
A nice example of the latter can be found in a fabulous study which recorded brain activity as people performed the apparently trivial task of reporting whether puffs of isoamyl acetate (a pungent odor) had been blown under their noses. Underlining the point that learning cares only about what is informative about the world, after only a few seconds of repeated exposure to the isoamyl acetate, subjects became incapable of saying whether the odor had been presented or not.
Yet fascinatingly, although the subjects ‘themselves’ appeared clueless as to whether pungent odors were being whiffed under their noses or not, the recordings from their brains revealed that ‘someone’ was still paying attention. As each odor puff was released, it was accompanied by a signature pattern of electronic activity in each subject’s brain. The intensity of this activity declined across the experiment, but it was still there, faithfully responding to each dose of isoamyl acetate, long after the subjects’ conscious responses showed that ‘they’ had no idea what was going on.
Learning, aging, and the myth of cognitive decline
What, you might ask, have bells, smells and churches got to do with the aging mind? The answer, as I will endeavor to show you, is everything. These stories remind us that the way we actually learn about the world is the result of a process, something that is all too often ignored when researchers contemplate the effects of age and experience on our minds.
To show why this matters, I’m going to take a close look at a test that is commonly used to measure people’s day-to-day learning ability over the lifespan, Paired Associate Learning (PAL). The relationship between learning and testing in PAL tests is obvious, and what is more, the results of these tests appear to provide some of the most robust evidence for age-related cognitive decline. What I will show you is that the implications of these results look very different once the processes of learning are brought into the picture.
PAL tests are remarkably simple. In their commonest form, subjects hear a series of word pairs, such as perspire–green, chicken–egg, etc., and are then asked to recall the second word of each pair when cued with the first (hence PAL tests are also called ‘cued-recall tests’). The graph below plots PAL recall on the vertical axis and age on the horizontal axis, and as you can see, average scores on these tests decline significantly across the lifespan:
This data is taken from a “normative” study that was designed to document adults’ performance at various ages on the PAL subtest of Wechsler’ s Memory Scale, one of the most commonly used of all PAL tests. The data was gathered from patients who had been hospitalized for non-neuropsychiatric conditions (which neatly avoided the problem of testing people in conditions that might bias responses in favor of one age-group or another), and it provides an assessment of how hard the test is for healthy adults in any given age-group.
For me, the data also contains something of a surprise: Over the lifespan, the biggest drop in PAL performance does not come sometime after 65, as I might have imagined, but rather in the decade after people turn thirty:
Does our ability to learn really decline as early as this graph suggests?
Who knows? The one thing I’m certain of is that we cannot conclude much from this data. This is because the averages plotted in the graph above equate people’s “ability to learn” with their ability to recall one word given another word after hearing them together once. There is a huge, rather obvious problem with this: Just as it is impossible to assess your ability to smell an odor from a single trial without controlling for your previous exposure to that odor, and just as it is impossible to assess your ability to notice a bell without considering your prior experience of bells, we have known for at least fifty years that it is impossible to measure associative learning like this: How well someone learns to pair word 2 with word 1 in a PAL test will ultimately depend on what they already know about the words before they take the test.
The mismeasurement of mind
To help explain why this matters and how it works, here’s another view of this data, showing the average score for the twenty- and thirty-somethings on each of the PAL word pairs:
As you can see, average performance on the individual word-pairs varies a lot. At the risk of stating the obvious, the reason it varies is because what people already know about words like lead and pencil and cabbage and pen is influencing their ability to learn each word-pairing.
Another important thing to notice is that it isn’t the case that the difference between the twenty- and thirty-somethings’ scores is the same across all of the items. Rather, what happens between age 20 and 39 is that the “hard” items (at the bottom of the graph) get harder — the gap between the blue and the red bars gets bigger. Rather than PAL learning ability “declining,” it appears that the difference in people’s ability to learn items that belong together versus items that don’t has grown.
What is really going on?
How the mismeasurement of mind sustains the myth of “healthy cognitive decline”
As I mentioned in previous posts, Robert Rescorla has spent many years highlighting the incompetent way in which learning theory has been taught to successive generations of students in the brain and cognitive sciences. Ignorance about how learning actually works is widespread, and this has reinforced many myths regarding “healthy cognitive decline.” To explain just how bad this mess is, it is first worth recapping the scientific legacy of Rescorla and his peers, who mapped out much of what we know about how learning works in animals and humans (for a fuller account, go here).
Rescorla systematically investigated the effects of background information on learning, showing that how often a dog hears a bell in the absence of food (the bell’s background rate) has as much influence on how much it learns to associate bells and food as how often that same dog hears a bell in the presence of food (the pair’s association rate). This finding suggests that it is the informativity of bells about food that drives learning, not mere association. This idea was further confirmed when Leon Kamin discovered blocking: Even if one controls for background rates, if a dog already expects his dinner because it has been trained to associate a flashing light with food, then this prior knowledge will block its learning about a bell if it is now rung as the light flashes. Because the bell adds no new information, to all intents and purposes the dog’s brain ignores it when it comes to learning, just as my brain now ignores the bells that once seemed to be my nemesis.
These and other findings have revealed learning to be a discriminative process that serves to reduce a learner’s uncertainty about future events. Learning tunes in to the features of the environment that help anticipate future events by detuning – actively filtering – those features that are less informative. From this perspective, the fickle nature of our perception of bells and smells is not a bug, but rather very much a feature.
Because of the way learning actually works, the process of forming “simple associations” is actually far more complicated than people tend to imagine. To illustrate this, think about what you might expect to happen if I asked you to learn to associate the following word pairs (and for good measure, let’s pretend you have never seen or heard any of the individual words before), e.g.:
Naively, you might expect that each pair would be equally easy to learn. Yet the processes that best explain learning predict otherwise: American and perspire occur only once in this little test-set, which means that they have a lower background-rate than obey. Meanwhile, eagle occurs twice as an outcome, which means that as a learner’s experience grows, obey will be in competition with American to see which is most informative about eagle. An important factor in this competition is going to be the fact that obey is also paired with rules, whereas American occurs only with eagle .
The plot above shows the value a learning model assigns to the predictive relationship between obey and Eagle in this situation. Note that it is negative. This is because American is more reliably predictive of Eagle than obey, and studies have shown that this will make learning to associate obey and eagle harder.
Let’s now see what happens when we hold the association-rates of obey-eagle and perspire-eagle at 1, but incrementally increase the association-rates of obey-rules and American-eagle up to 50, to reflect the fact that people typically hear meaningful pairs more often than meaningless pairs:
Increasing the frequency of obey-rules and American-eagle has a negative impact on the model’s assessment of how easy it will be to learn to associate obey with eagle: Even though the association-rate of obey-eagle remains constant, as obey-rules and American-eagle are learned better, the model predicts that obey–eagle will become harder to learn.
A large body of empirical research has established the influence of association-rates, background-rates, and blocking on learning as being the closest thing to a set of scientific facts that psychological science has to offer; The models developed to formalize these principles explain and predict more data than any other models in psychological science; And these models have in turn given rise to some of the more compelling findings in behavioral neuroscience.
At every level, from the behavioral results in animals and humans, to the models, to the neuroscience, all of this work takes for granted the fact that associative learning cannot be explained solely in terms of association rates. Yet despite this, all studies of ageing that use PAL tests simply measure people’s average performance based on a fixed association rate in training. And all of these studies then interpret any differences that they see in between-age-group performance as providing evidence of a decline in the brain’s associative learning capability.
Our everyday experiences with bells and smells, along with everything we have discovered about learning in the past 50 years, shows this to be wrong.
Just how wrong is current thinking about “ordinary, healthy cognitive decline”?
This graph plots the average performance of the 20-29-year-olds for each PAL item in the normative data, ordered from easy (top) to hard (bottom):
As you can see, while twenty-somethings learn to associate up with down almost all the time, they recall that dig goes with guilty only half of the time. As I noted earlier, their prior experience with these words clearly makes it easier for them to learn up-down than dig-guilty.
It follows that if we were to test one half of these twenty-somethings on the easiest PAL pairs and the other half on the hardest, we might expect to see results like this:
If we were to interpret this as showing that the people in Group 1 are worse at associative learning than the people in Group 2, this would clearly be wrong. All these results can tell us is that if we ask different groups of people to memorize word pairs that differ in what they already know about them, we are likely to find differences. What we need in order to draw conclusions from this kind of data is, of course, an estimate of the extent to which we ought to expect people to recall a particular word-pair in the light of the experience we might expect them to have had with those particular words.
Learning theory allows us to make this kind of estimate. Either by simulating learning “experience-by-experience,” or by estimating what we might expect people to learn based on a numerical description of that experience. To keep things simple , I’m going to take the latter course here.
I mentioned above that three main factors influence associative learning: association rates, background rates, and blocking. Thanks to the development of huge databases of linguistic material, we can now count how often a word appears — and the company it keeps — in millions, even billions of words of text and speech. This means that if we want to estimate an association rate for word pairs like up-down or dig-guilty, we can do so by simply counting how often up-down or dig-guilty actually appear together in these databases; We can estimate the background rates of the cue words by counting how often up or dig appear; And, if we assume that more frequent words are generally more predictable than less frequent words, we can estimate blocking – the finding that events that are well-predicted support less learning – by dividing our count of how often down appears by our count of how often up appears, to get an idea of how predictable down is compared to up (or guilty compared to dig).
We can then see how well these estimates account for people’s actual PAL performance using a regression analysis. This is a statistical method that estimates how much of the variation in a target variable is systematically explained by our predictor variables. It returns a value called R-squared which varies from 0 to 1, where a value of 1 indicates that the variation in the target variable is exactly what we should expect to see given the predictors, and 0 means there is no association between the predictors and the target. (Of course in the real world, R-squared is very rarely 0 or 1, since it is very hard to find a statistical model that is that bad or that good.)
In this case, the R-squared value was 0.857, indicating that my word frequency estimates of the association rates, background rates, and blocking for the PAL items account for over 85% of the variation in how hard or easy it was for the twenty-somethings to learn each pair.
Regression analyses also return a number called a “beta” for each predictor. These (roughly) indicate the extent to which a high predictor value is associated with a high value in the target — in which case the beta value is positive — or a low value in the target (in which case the beta is negative). Reassuringly, the beta for the association rate, which we would expect to promote learning, is positive, whereas the betas for background rates and blocking (which we might expect to inhibit learning) are negative.
In other words, this analysis of 20-29 year old adults PAL learning indicates that 85% of the difference one sees in the learnability of the individual items is explained by factors that have been shown again and again to explain “associative learning.” The factors that we might have expected to explain why items are “hard” or “easy” to learn in the first place.
If this is not entirely surprising, then it is reassuring. Moreover, now that we have a way of estimating the learnability of each word-pair based on what we might expect people to have experienced, we can look at the results of the 30-39 year olds to see if the changes in their scores really do show evidence of “cognitive decline.”
Repeating the analyses above on the thirty-something data reveals that once again the association rate again has a positive beta, while the background rate and blocking betas are negative. Moreover, all the beta values in the thirty-something regression model are greater (or else more negative) than those in the twenty-something model (indicating that the associations between these values are stronger), while the regression value is essentially identical to the one above: R-squared = 0.85.
There is a simple and very straightforward way of interpreting this result.
If we assume:
- That PAL tests measure associative learning;
- That the people taking these tests learn to associate words in the way that we should expect them learn to associate them;
- That the average thirty-something has more linguistic experience than the average twenty-something;
Then the patterns of difference we see in the scores of the twenty- and thirty-somethings are exactly what we ought to expect to see.
Which also means that the differences in performance between these two groups do not reflect any decline in the thirty-somethings’ ability to learn, or to think. In fact, the data suggest that the opposite is true: Recall how the gap between the blue and the red bars is biggest at the bottom of the graph (for the nonsense pairings), and smallest at the top, for the meaningful pairings? This analysis indicates that for the 30-39 year-olds, not only has learning ability not declined, but rather continued learning has increased the difference in the degree to which meaningful and meaningless pairs of words are associated in memory. Just as learning to ignore irrelevant chatter at a cocktail party “amplifies” the voice of the person you are trying to listen to, learning has amplified the meaning of the older group’s meaningful pairings by increasingly ignoring meaningless pairings.
As many grown-ups may have noticed, the average twenty-year-old is more likely to produce half-formed, barely intelligible sentences than the average thirty-year-old, whereas the average thirty-year-old is more likely to be able to clearly articulate an idea than the average twenty-year-old. What these results actually show is that the thirty-somethings’ linguistic knowledge is much better discriminated than that of twenty-somethings. Which goes some way to explaining why these things are so.
(If any twenty-year-olds feel I am doing them an injustice here, I apologize: But please take comfort from the fact I’m telling you that it isn’t all down hill from here; If that still doesn’t work, then come talk to me in ten years.)
“But no-one says that thirty-year-olds are in decline”
The skeptical reader may object that I have focussed too much on under-forties, rather then genuine old age. “No-one says that thirty-year-olds are in decline,” goes the line. Except, of course, plenty of people do. They do so, despite the obvious absurdity of this notion to anyone who has ever actually been inside a twenty- and thirty-year-old-mind, because the same faulty interpretations of test results I just laid bare drive the whole idea of “healthy cognitive decline.”
These data show that PAL test scores drop considerably between 20-29 and 30-39, plateau, and then decrease linearly with age after age 50. It might be tempting to think that the change between 20-29 and 30-39 is an anomaly, and the change after 50 is the real evidence for “healthy cognitive decline.” There are two good reasons why you shouldn’t think this:
- Because this inference is based on looking at averages, and as we have already seen, these averages are meaningless;
- Because if the things that support your beliefs are the only things you count as data, you aren’t doing science. You are just confirming your prejudices.
Instead, here’s what an analysis of the average data above looks like when we again control for learning. This plot was taken from one of our recent papers, and uses a slightly more sophisticated regression technique that is a better choice for comparing all of the data at once:
The vertical axis plots this technique’s equivalent of the betas I described in the regression models above, and the horizontal axis plots age. As you can see, performance on the individual PAL items becomes increasingly associated with the main predictors of associative learning as people’s age and experience increases.
Or in other words, as we get older, the degree to which we find words easy or hard to put together comes to ever more accurately reflect the degree to which words in English do and don’t meaningfully go together. Far from showing evidence of cognitive decline, this data in fact provides us with powerful evidence that our capacity for learning is retained, and remains engaged, throughout our adult lives.
Newton and the naturalized mind
This next graph plots the average performance of a group of people suffering from Alzheimer’s disease (AD) against a group of age-matched controls on three repeated tests of the same set of PAL word-pairs:
As you can see, while the healthy controls get better at learning the PAL pairs with repeated training and testing, the AD group show no sign of learning at all. However many — perhaps even most — people erroneously believe that all ageing minds exist somewhere on a continuum from “healthy cognitive decline” to neurocognitive diseases like AD: Mentally, the idea goes, we are all on the way down; some people simply hit the bottom faster. The graph above belies this nonsense: AD involves serious neuronal degeneration that is simply not evident in healthy brains.
Herein lies the rub: There are two ways of deciding if an elderly brain is healthy or not. The best, autopsy, comes a little late for the people it matters most to. The other, a diagnosis of ill-health, is currently something of a blunt instrument: In the words of Ronald Petersen, a leading expert on AD and a common AD precursor called Mild Cognitive Impairment (MCI), “Whereas clear diagnostic criteria are now available in international disease classification systems, the notion of normal aging is less well understood.”
Although there are no useful treatments for AD and MCI, older adults can reduce their risk of these genuine neurocognitive disorders. To take care of their brains, they should first and foremost look after their bodies. In Petersen’s words, “Regular physical exercise is probably the best means we have of preventing Alzheimer’s disease today, better than medications, better than intellectual activity, better than supplements and diet.”
How older adults see ageing has an impact on how they look after themselves: a positive disposition towards the future is associated with, among other things, better health and exercise, better workplace performance, even better memory. Indeed, positive perceptions of ageing even seem to promote longevity: Over-fifties who took a positive attitude to statements like, “Things keep getting worse as I get older,” “I have as much pep as I did last year,” and “As you get older, you are less useful,” later lived an average of seven and a half years longer than their more negatively disposed peers.
Yet despite this, almost every scientific and medical resource available to older adults informs them in confident terms that declining cognitive performance and failing memories are an inevitable part of healthy ageing: “You are on the downslope,” we tell the elderly, “And there ain’t no getting off.” The effect this misinformation has on the motivation and health of older adults can only be guessed at, but the simple fact is that the scientific basis for these claims is wretched: the way PAL data is misanalyzed and misinterpreted across the lifespan isn’t controversial or debatable, it is just wrong.
Newton’s third law states that for every action, there is an equal and opposite reaction. Yet the naive assumptions that underpin current theories of “healthy cognitive decline” suppose that some changes – especially learning – do not incur costs, such that some powerful actions – like the acquisition of knowledge – ought to have no reaction. Our minds and brains do not flout Newton’s laws: As our formal understanding of learning has grown, so too has our understanding of its costs as well as its benefits. PAL tests show both that learning to pair obey and eagle is harder for older adults than younger adults, and that our internal models of the information structure of language become more refined and accurate as we age. The benefit of a greater mastery of language comes at the cost of learning to pair obey and eagle, a trade-off that is a product of the processes by which our minds learn.
The understanding of processes lies at the very heart of science. In 1871, Charles Darwin published The Descent of Man. It laid out his theory of natural selection, explaining how things as varied as bugs, bears, and begonias are the products of a single underlying mechanism, paving the way to an understanding of biological processes that has revolutionized our physical lives. A year later, Darwin’s Expression of the Emotions in Man and Animals rolled off the presses. This less well-known masterpiece described how the ideas that naturalized our physical selves might eventually help naturalize our minds.
A hundred and fifty years later, the first part of Darwin’s vision is all but realized. The insights gained from animal models have led to the development of innumerable cures and vaccines, and even the idea of transplanting animal-parts into our bodies has traversed the spectrum from shocking to mundane. As for the second part, while ethers and phlogiston have been banished from the most of the natural sciences, in the sciences of the mind their equivalents still abound. Far too much of what passes for theory in the brain and cognitive sciences is little more than superstition and wishful-thinking.
We can and should be doing better. Naturalizing the mind is more than an intellectual challenge. Explaining the natural basis of mind, and reconciling people to the idea that minds and brains are just different ways of describing the same thing, are likely to be key to helping make people’s ever-longer lives as happy, healthy, and productive as they can be.
LOWNDES, G., SALING, M., AMES, D., CHIU, E., GONZALEZ, L., & SAVAGE, G. (2008). Recall and recognition of verbal paired associates in early Alzheimer’s disease Journal of the International Neuropsychological Society, 14 (04) DOI: 10.1017/S1355617708080806
Rescorla, R. (1988). Pavlovian conditioning: It’s not what you think it is. American Psychologist, 43 (3), 151-160 DOI: 10.1037/0003-066X.43.3.151
Wang, L. (2002). The correlation between physiological and psychological responses to odour stimulation in human subjects Clinical Neurophysiology, 113 (4), 542-551 DOI: 10.1016/S1388-2457(02)00029-9