When grappling with life fluctuations, we often seek explanations and remedies for our ups and downs. Consider the case of a man suffering from back pain that fluctuates dramatically. On some days, he felt capable of conquering mountains; on others, even simple movements were excruciating. During the most painful episodes, his wife would drive him to a chiropractor, after which he would experience significant relief and enthusiastically recommend the therapist to everyone he knew.
In a different scenario, a young man with a respectable golf handicap of 12 was similarly fervent about his golf instructor. After particularly dismal performances on the course, he would book a session with the pro, only to find his game markedly improved in subsequent rounds.
Then there was an investment adviser, who, in a rather peculiar ritual, performed a ‘rain dance’ in the restroom every time his stocks underperformed. Despite the absurdity of the practice, he was convinced that it somehow influenced a turnaround in his investment returns.
What links these three stories is a common cognitive fallacy: the regression-to-mean delusion. This concept illustrates that extreme experiences are likely to be followed by more moderate ones. Understanding this can help explain why certain interventions or rituals might seem effective when, in reality, the observed changes are part of a natural statistical fluctuation.
The Case of the Back Pain
Back pain is a common affliction that many people experience at some point in their lives. For some, the pain is chronic, fluctuating between manageable discomfort and intense, debilitating episodes. Take the case of a man who experiences this kind of variation in his back pain. On some days, he wakes up feeling almost no discomfort, capable of doing his usual activities and feeling confident and energetic. On other days, the pain is so intense that even simple tasks, like bending down to tie his shoes or sitting at his desk, become difficult or impossible. These flare-ups can be discouraging, leading him to seek medical intervention.
When the pain reaches its peak, he visits a chiropractor. After a session, he feels much better, often amazed at the rapid improvement. The following day, he can move much more easily, and the relief is so significant that he praises the chiropractor to everyone he meets, recommending them without hesitation. However, the improvement he experiences might not be solely attributed to the chiropractor’s treatment. Pain, particularly chronic pain like back issues, naturally fluctuates. Just as the weather can experience a cold snap or a heatwave, physical conditions swing between extreme highs and lows. The chiropractor’s visit may have provided some relief, but the pain would likely have improved without it.
What’s at play here is the concept of regression to the mean. In simple terms, it’s the idea that after an extreme event—whether it’s pain or a particularly low point—things naturally tend to return to a more typical state. His back pain, which was particularly severe on certain days, would have naturally decreased as part of this process. Therefore, although the chiropractor’s intervention might have felt like the turning point, it was more likely the natural ebb and flow of the pain returned to a baseline that contributed to the improvement.
The Golf Handicap Paradox
Golf, a game of both physical skill and mental resilience, is full of fluctuations. A golfer may have an exceptional round one day, hitting every shot precisely, only to struggle in the next. Take the example of a man with a handicap of 12, which is respectable for most golfers. This handicap reflects his typical performance, but like all golfers, his game is subject to variability. On some rounds, he plays fantastically, while others leave him frustrated and dissatisfied with his performance.
The man books a lesson with his golf instructor on particularly bad days. After the lesson, he usually finds that his game improves the next time he plays, often leading to a better score than expected based on his recent struggles. It’s tempting to think that the lesson caused the improvement. After all, the timing matches up perfectly. However, what’s happening is a natural tendency for performance to return to its average.
This phenomenon is known as regression to the mean. On the days when he was struggling, his game was likely performing below its usual standard, driven by factors such as poor form, bad luck, or external distractions. After the lesson, his game improved, but it was likely more due to the fact that extreme performance—either good or bad—rarely lasts. His handicap of 12 is an average, and it’s statistically likely that his next round would return to a performance level near this average, regardless of the lesson. The lesson may have helped, but the improvement can be largely attributed to the natural variation in his performance, a return to his typical skill level, rather than a magic fix from the instructor.
The Investment Adviser’s Rain Dance
In the world of finance, stock markets are notorious for their volatility. Investments rise and fall constantly, sometimes with no clear pattern. For one investment adviser at a large bank, the rollercoaster of stock performance caused him great frustration. His investments would fall drastically in value whenever the stock market took a sharp downturn. In these moments, he felt powerless to control the outcome, and that’s when his peculiar ritual came into play: a “rain dance.” The adviser would perform this ritual, which he believed had a positive impact on the market. As strange as it sounds, every time he completed his rain dance, the stock market seemed to recover, and his investments would bounce back.
While the adviser was convinced that his actions were influencing the market, the reality is that the market’s behavior was following a natural pattern. Stock prices often experience extreme fluctuations, dipping far below average during periods of uncertainty or economic stress. However, over time, the market tends to recover and return to normal levels, even without external intervention. This is the essence of regression to the mean. After a significant drop, the market was naturally more likely to experience a rebound, and the adviser’s ritual coincidentally took place during that recovery. His belief that the ritual was responsible for the change was a classic case of misattribution.
Much like the back pain or the golf game, the stock market operates within a cycle of highs and lows. While individual stock performance may vary, the overall market tends to revert to its average, making extreme dips often followed by recoveries. Though the adviser’s actions provided him with a sense of control, they had no actual bearing on the market’s natural fluctuations.
Natural Fluctuations in Performance
Whether in sports, business, or daily life, it’s common for performance to fluctuate. Athletes, for instance, experience both peaks and valleys in their performances. A sprinter may run a race in record time one day only to come in last place the next. These variations are not necessarily due to lack of effort or ability but rather a natural fluctuation in performance. Over time, even the best athletes experience inconsistency in their performances. This is true for athletics and any competitive field, including business, education, and personal goals.
Understanding regression to the mean is critical to recognizing that extreme performances are often outliers. For example, an entrepreneur might have an exceptionally successful product launch that exceeds expectations, only to see sales fall back to more typical levels in subsequent months. This isn’t a sign of failure or a problem with the business but rather a return to a normal state after a temporary spike. Similarly, a stockbroker might achieve extraordinary returns during a bull market but see those returns drop once the market corrects. The key takeaway is that success and failure are rarely permanent, and more average results often follow extreme outcomes.
Athletes know that their most impressive achievements often cannot be repeated indefinitely, and they don’t place too much stock in the short-term fluctuations of their performance. Instead, they focus on consistency over time. It’s important to understand that extreme results—whether in sports, business, or life—tend to regress to the mean. This knowledge can help us set more realistic expectations and avoid making overconfident decisions based on fleeting successes or failures.
Workplace Morale and Regression to Mean
Managers often look for ways to improve employee morale, especially in organizations where motivation can fluctuate. One strategy is sending the least motivated employees to special training or motivational courses, hoping this will ignite a spark of engagement. Let’s consider a division manager who sends the bottom 3% of his team to such a course. These individuals are identified as the least motivated, and the manager hopes that their attitudes and performance will improve by providing them with training or motivation.
However, when the manager reassesses motivation levels after the course, he might find that others have replaced the employees who were once at the bottom of the list. This is a natural consequence of regression to the mean. The employees initially at the bottom of the motivation scale might have been going through a temporary phase of low engagement, and their performance was likely an outlier. With time, they would have naturally improved, and other employees who were not initially at the bottom might now exhibit low motivation. The manager might attribute the change to the course, but the shift is more likely due to the natural ebb and flow of employee morale.
This is why managers need to recognize that workplace performance often fluctuates, and interventions may not always yield the long-term results they expect. By understanding regression to the mean, managers can avoid making hasty decisions based on short-term improvements and focus on creating consistent, long-term strategies for boosting morale.
The Depressed Patient’s Recovery
In healthcare, particularly in mental health treatment, patients often experience periods of improvement that may not necessarily be due to the interventions they receive. Consider a person diagnosed with depression who is hospitalized for treatment. After a few days of care, they feel significantly better, with improved energy and a more positive outlook. This improvement is often attributed to the treatment they receive, whether it’s medication, therapy, or support from medical professionals.
However, depression is a condition marked by natural fluctuations in mood and energy. Patients with depression often go through periods of feeling worse, followed by periods of feeling better, even without treatment. This is due to the cyclical nature of the illness. People may feel better after a few days in the hospital simply because they’ve passed through a phase of intense symptoms. Just as someone might recover from a cold or the flu over time, someone with depression can also experience improvement naturally, without any intervention.
This doesn’t mean that treatment doesn’t work; however, it’s important to consider that some portion of the recovery might simply result from the body or mind returning to a more typical state. By recognizing the natural course of depression, healthcare providers can better evaluate the effectiveness of treatments and avoid over-attributing success to short-term interventions.
Misinterpreting Regression to Mean in Education
In the educational sphere, regression to the mean often affects how students are perceived and how interventions are evaluated. Consider the case of a teacher who, after a major exam, decides to praise the highest-scoring students and criticize the lowest-scoring ones. The assumption is that praise motivates students to continue performing well, while criticism will push those at the bottom to improve. However, the next time an exam is taken, the highest and lowest performers may be different students altogether.
This is a textbook example of regression to the mean. Like golf handicaps or stock performance, extreme performances—whether good or bad—tend to return to a more typical level. The students who performed poorly on the first exam might have simply been experiencing an unusually bad day or a temporary lapse in focus. Their performance will likely improve naturally on the next exam, regardless of the teacher’s actions. Similarly, the highest-scoring students might not replicate their top performance, and the next round of assessments will likely see a new set of students achieving the highest and lowest scores.
This can lead to misunderstandings in educational settings. Teachers may believe that their strategies of praise or criticism are directly responsible for changes in student performance when in reality, those changes are simply part of the natural fluctuation in student outcomes. By recognizing the role of regression to the mean, educators can make more informed decisions and focus on long-term strategies that encourage consistent growth rather than placing undue weight on short-term results.
Consequences of Ignoring Regression to Mean
Ignoring regression to the mean can have significant consequences in personal decision-making and larger organizational strategies. When we fail to recognize that more typical outcomes often follow extreme events, we risk making misguided decisions based on faulty assumptions. In business, this can lead to unnecessary interventions or misguided strategies, like overestimating the impact of a motivational training course or a short-term management change.
In healthcare, failing to recognize the natural course of illness can lead to patients attributing their recovery to the wrong factors, potentially leading to unnecessary treatments or interventions. In education, misunderstanding regression to the mean can cause teachers to adopt ineffective strategies, such as punishing students who are underperforming or temporarily over-rewarding those who excel.
Ultimately, understanding regression to the mean can help us make better, more informed decisions. Whether in business, healthcare, or education, recognizing that extremes are often followed by more typical results helps temper expectations and avoid overreacting to short-term fluctuations. This understanding empowers us to focus on long-term improvements and avoid attributing success or failure to transient factors.
Conclusion
In summary, understanding regression to mean is crucial for accurately interpreting changes in performance or outcomes. When faced with stories of miraculous recoveries or sudden improvements after certain interventions, it’s important to consider the possibility of natural statistical fluctuations. Recognizing this fallacy can help make more informed decisions and avoid the pitfalls of attributing success or failure to specific actions without acknowledging the role of natural variation.
This article is a part of The Cognitive Bias Series based on The Art of Thinking Clearly by Rolf Dobelli.