Regression to the Mean

While working full-time at IBM Research, I was also a Fellow at the Institute for Rational-Emotive Therapy in Manhattan. I wrote an article in 1978 for their Journal, Rational Living. The title was: “Why Do I Self-Down? Because I’m an Idiot?” Indeed, many people put themselves down and it is not helpful. I hypothesized several different causes for this kind of self-slamming behavior. Most of these causes you could probably figure out on your own. But one in particular is subtle and non-intuitive. It is based on a statistical phenomenon which few people know about despite the fact that it is extremely pervasive. This phenomenon is called “Regression to the Mean.” 

I want to define this term by explaining some examples. Imagine that you have a new soft drink which contains a combination of herbs that will purportedly make you smarter; e.g., gingko and bacopa. (There is some evidence these may actually work but let’s assume that they don’t or that your tea has too little to be effective). Here’s what you do to “prove” that it works anyway. You give an IQ test to 10,000 people and choose the 50 who score the lowest on the test and have them drink your tea for the next six months. At the end of that time, you give those 50 people an IQ test again and — Voila! The average (or mean) of the IQ scores has almost certainly gone up. Yay! It works! 

Or does it? One of your competitors is not too happy about your study. In fact, they aren’t even happy you put your tea on the market. They decide to prove that your tea is not only ineffective but that it makes people less smart. So what do they do? They give an IQ test to 10,000 people and they pick the 50 who score the highest. They have them drink your tea for six months and at the end of that time, they have them take another IQ test. In this case, the mean (average) score is lower than the first time! Ouch! They say your tea causes brain damage! 

How can the same tea make people smarter and make them dumber? In this case, it does neither. What is going on? Here’s what is going on. When you measure something, there is always some error. Whether you are measuring your weight, your height, your blood pressure, or your IQ, the measurement is never exactly perfect. Your weight may vary slightly because of atmospheric pressure and more so because of water retention. If you take an IQ test, your score will partly reflect how well you do on such tests in general, but it will partly depend on luck. You may have felt particularly good that day, or a few of the questions might have been on topics you just heard about on TV the day before, or you may have made some lucky guesses. Or, you may have been unlucky on a particular day. You might have had a cold or misread one of the questions or forgotten your morning coffee. On any given day, some people will be a little lucky and some people will be a little unlucky. These things tend to balance out in a large group and if you tested all 10,000 people after six months, then assuming the tea has no real effect, no effect will be shown in the data. 

However, if you select the very best scores, you are partly picking smart people, of course, but you are also picking the people who were lucky that day. When you test just those people six months later, they will generally be just as smart but there is no reason to suppose they will be lucky again. Some will be lucky both times, most will not be particularly lucky or unlucky and a few will be unlucky. The average score will be lower. Conversely, if you choose the lowest scoring people, you will partly be choosing people who don’t do well on such tests in general. But you will also be choosing people who were tired, sick, guessed wrong or were otherwise unlucky that day. When you retest, those people will still tend to be people who do poorly on such tests, but they won’t necessarily all be unlucky again. Some will. Some won’t. On average, the scores will be higher than they were the first time. 

The phenomenon of “Regression to the Mean” was first noted by Francis Galton in the 1880’s. Tversky and Kahneman, so far as I know, were the first to note that this phenomenon could easily cause managers, coaches, and parents to end up being unnecessarily negative. Here’s how it works. Let’s say you are learning to hit tennis serve. Although you will likely improve in general, over time, there will also be a lot of variation in your performance. Sometimes, everything will work well together and you’ll hit an excellent serve, one that is above your average level. At first, the coach’s natural inclination will be to praise this by saying, “Wow! Great serve!” or something like that. Unfortunately, your next serve, due to regression to the mean is very likely not to be quite as good as that one was. Your coach’s praising behavior was thereby punished. On the other hand, if you hit a particularly poor serve for your level, your coach might say, “Oh, come on. You can do better than that!” If they choose to say such things only on your very worst performances, then, due to regression to the mean, your next serve is likely to be somewhat better. In other words, their slamming you will be rewarded by your doing better the next time. The same general tendencies will apply to managers and parents as well.  

The same applies to you! Whatever you are doing, your performance will vary somewhat over time. If you begin by praising yourself internally whenever you hit a particularly great shot, your next shot will most likely be not so great. On the other hand, if you put yourself down when you find your performance particularly bad, “You idiot! How could you miss that!?” Your next shot will tend to be somewhat better. Over time, your positive self-talk will tend to be punished and your negative self-talk will tend to be rewarded. 

It’s no wonder then that many managers, coaches, and parents end up saying very negative things about their charges. It’s also no wonder that many people say (or more likely think) many more negative things about themselves than they say positive things.

 

Is there anything to be done? First, simply be aware of this phenomenon. That is step one. If you are running a study, you need to be careful in selecting. The study about your tea could be fixed by re-testing the entire population; by selecting a random group of 50 rather than the best or worst; or by using a control group who did not drink tea but was retested anyway. When praising or punishing someone’s performance, do not bother with trying to reward or punish outcomes based on one trial. That’s actually a pretty poor way to coach yourself or others in any case. See The Winning Weekend Warrior for more on this. Also watch out for this when you read about various conclusions of other studies. Did the investigators select either the “best” or the “worst” for their study? If they did such a selection, did they talk about the bias this introduces? Did they have a control group? 

Meanwhile, treat your mistakes as opportunities to learn, not as opportunities to put yourself down. There’s really no point in self-downing. But if you do find yourself self-downing, remember that it’s common; relax; smile at this human foible; then quit doing it. At least give yourself a break for the holidays. 

————————————-

Tversky, A., & Kahneman, D. Judgment under uncertainty: Heuristics and biases. Science, 1974, 185, 1124-1131. 

Author Page on Amazon

My Cousin Bobby

The Update Problem

The Stopping Rule

Finding the Mustard

What about the Butter Dish?

The Self-Made Man

Labelism

Wednesday

Where does your loyalty lie?

After All

Roar, Ocean, Roar

Plans for US; some GRUesome

The Walkabout Diaries: Sunset

The First Ring of Empathy

Stoned Soup

The Tale of the Three Blind Mice

Fifteen Properties

A Pattern Language for Cooperation

Travels with Sadie: Cooperation