You probably have a boss. A manager, an executive, the head honcho. How did they get that position? Was it their skills and knowledge? Or were they just lucky? How about you? Are you where you are because of your expertise, or because of luck? I have no doubt that you deserve your role, but luck may have played a greater role than you think. Especially if you work in a competitive environment.
It is very easy to perceive luck as merit. When I think of how I ended up as a researcher at Harvard, it is intuitive to attribute it to long evenings of study, resilience after rejections, and the ability to consume a superhuman amount of coffee. To me, this makes sense. I remember my choices and sacrifices clearly (egocentric bias), such as going abroad for a PhD. In hindsight, all these choices seem causally related to the outcome (hindsight bias). Yet I miss the fact that thousands of people with the same expertise have not had that same outcome: classmates who worked equally hard in school, or equally diligent women that never got the chance to attend school at all. This is survivorship bias. If you are only looking at the people who made it to the top, and not those that were unsuccessful, you can easily come to the wrong conclusions. If I lived my life a million times – the same me but in different places and times – how many of us would end up at Harvard?
Considering these million “alternative histories” is a tactic to battle the biases that cause us to confuse luck and merit. The mathematical statistician Nassim Nicholas Taleb describes the power of alternative histories in his book Fooled by Randomness.1 To determine the contribution of luck and merit to someone’s success, he looks at whose lives are more “resistant to randomness”. A dentist is not so dependent on luck: if she lived her working life a million times, she would earn a good living in most of them, occasionally becoming a millionaire, and occasionally becoming unemployed. The same cannot be said of a gambler putting everything on one number at the roulette table. If she happens to win big, you still would not recommend it as an investment strategy. That is because, in most of her alternative histories, she would end up losing everything.
Alternative histories highlight that luck plays a bigger role in the life of stars – the massively successful musicians, vloggers, influencers – than in the lives of successful dentists. Still, those musicians and vloggers tend to receive more praise and media attention. This is survivorship bias in action: we pay attention to successful “survivors”, forgetting how many people with the same characteristics did not achieve the same success. In the media, the life stories of survivors, such as famous college dropouts, reach large audiences. Yet when we glorify Steve Jobs and other billionaires that never got their degree, we ignore all the college dropouts that did not do so well.
Are professors just a lucky lot?
If there are many college dropouts, or gamblers or vloggers, it is likely that one of them will have an extraordinary outcome, just by chance, Taleb explains. The greater the competition and the more successful someone is, the more likely that luck played a role. This knowledge sheds a new light on our role models, such as university professors. It raises questions about their merit: are they indeed the cleverest people around, or are they just a lucky lot? The current academic landscape is competitive indeed: only 0.45% of science PhD students will go on to become a professor.2 If Taleb is right, professors may not necessarily exhibit more skill, talent or perseverance than their fellow PhDs who did not obtain that title.
With a thought experiment to reproduce the selection of successful professors from a group of science PhDs, we can explore exactly how lucky professors are. As academia is a meritocratic system (or aspires to be), this selection is based on students’ excellence, not on their wealth or social class. Excellence is difficult to measure directly. What gets measured in the selection for professorships is probably a combination of merit (good skills, great research, experience, perseverance) and a little bit of luck (having had stimulating supervisors, winning a travel bursary, making a discovery). Merit should be more important than luck. Let’s assume, for now, that merit accounts for 95% and luck for 5% of a selection decision.
To make this thought experiment more tangible, I ran a simulation. I generated “merit scores” and “luck scores” for 100,000 PhD students. Every PhD student was assigned two numbers between 0 and 100, one for merit, one for luck. I then calculated their “excellence”: the weighted mean of merit (accounting for 95%) and luck (accounting for 5%), which ends up between 0 and 100 too. I then selected the 450 PhD students (0.45%: one in 222) with the highest excellence score: those are our professors. This sample of professors can answer many questions: is their merit higher than the average PhD student? And what about their luck? Would they have been chosen if selection were based on merit alone? And how many PhD students were unlucky, having the same merit but not being selected to become a professor?
My simulation shows that those who get picked to become professors have a high merit score (see Figure 1). Over 20 simulations, their mean merit is 92/100, higher than the mean merit of all PhD students (75/100). But professors’ luck scores are also higher: 69/100, compared to 50/100 for the average PhD student. To become a professor, you need to be good, but you also need to be lucky, even if luck only accounts for 5%. Of our 450 professors, 120 are what I would call “a lucky lot”: they would not have been chosen if selection were based on merit alone. It illustrates Taleb’s warning about perceiving luck as merit: even if merit accounts for 95%, a quarter of successful people in our simulation are successful because they are luckier than their peers with similar merit.
Figure 1: From a sample of 100,000 PhD students, 450 will go on to become professors. PhD students are assigned a merit score between 0 and 100 (mean: 75/100) and a random luck score between 0 and 100 (see left). Professors will be selected based on excellence, the weighted mean of their merit score (weight: 95%) and their luck score (weight: 5%). The simulation shows that professors have above-average merit (92/100), but also above-average luck score (69/100; see right).
The assumptions: normally-distributed merit and random luck
This experiment is an adaptation of one done by vlogger Derek Muller for his YouTube video “Is Success Luck or Hard Work?”. He and I differ slightly in our assumptions. He talks about astronauts, selected based on their skill. His measure of skill is uniform, randomly selecting between 0 and 100, each skill score having an equal chance of being selected. I prefer talking about merit – a combination of skill, talent and hard work – and assume that it is normally-distributed.3 Most PhD students have an average merit, with fewer PhD students at either extreme. I used a mean of 75 rather than the middle of the scale. That is because I assumed that to get admitted to a PhD program, one has to already display quite some level of skill, talent and perseverance. With a standard deviation of 6, the merit scores lie between 0 and 100.
For luck, I took a random number between 0 and 100, so each luck score is equally likely to be assigned to a PhD student. The real distribution of “total amount of luck during one’s lifetime” may not be uniform, of course. But the saying “lucky at cards, unlucky in love” suggests to me that luck presents itself disproportionately in singular aspects of life. So, with that saying in mind, a uniform distribution of luck as a random number between 0 and 100 should be a decent assumption for the purposes of the experiment.
Intensifying the competition: football stars
By changing the number of competitors and the success rate, I can model the role of merit and luck in even more competitive environments. The English Football Association reports that 11 million people in the UK play football. Only 800 of them make it to the top-level leagues: the Premier League and the Women’s Super League. Of those top-level players, around 50 will be selected for the national teams to compete at the World Cup. For simplicity, I disregard the fact that the 11 million people are not all English nationals, and thus may not qualify for the English national team. I also disregard all English citizens who play football abroad.
We start off with 11 million football players, with both mean merit and luck being 50, and a standard deviation of merit of 11. The 800 players selected for the top-level league are very skilled and talented indeed: they have a mean merit score of just under 99. They are also lucky: their mean luck score is 66, and there are over 2,000 footballers with equal merit who did not make the cut. Players in the national team, meanwhile, have a higher mean merit score (100/100), and a much higher luck score (93/100). In total, I could fill 9 more national women’s and men’s teams with footballers who were just as talented, but not quite so lucky. So, if your favorite footballer does not make it to the final selection for this year’s World Cup, that may be lack of luck rather than lack of merit (see Figure 2).
Figure 2: Average merit (number shown in football) and average luck (number shown in lucky clover) for all 11 million football players (left), 800 footballers selected for the top-level league (middle), and 50 for the national teams (right).
Because the number of competitors at football is large, many people do not get selected despite having the highest merit score possible. By adapting the parameters of the experiment, we can dive deeper into the numbers. In Figure 3A, success rate is fixed at 1% and success is based on 95% merit, 5% luck. By increasing sample size (number of competitors), the number of successful people who would not have been selected based on talent alone grows (the “lucky lot”). So too does the number of people who don’t get selected in spite of having “equal merit”. In Figure 3B, the number of competitors is fixed to 100,000, but we adjust the success rate. Again, we see the numbers of the “lucky lot” and those of “equal merit” increase – but, at the same time, the mean merit and luck scores of the successful competitors decrease: you can get selected at lower levels of merit and luck (not shown in the figure).
A valid criticism of my simulations would be that my weighted average of merit and luck (95% vs. 5%) is unrealistic. So, in Figure 3C we fix sample size to 100,000 and hold success rate at 1%, but change the weight of luck. Interestingly, this does not affect the mean merit of the successful people. The main change is to the mean luck score of the successful people: it goes up from 50 (weight of luck 0.1%) to 73 (weight of luck 10%). Increasing the weight of luck also pushes up the number of people of equal merit who do not get selected.
Figure 3: Number of people that are part of the “lucky lot” (would not have been successful based on merit score alone) or of “equal merit” (not selected, in spite of having equal merit to someone successful) under different assumptions: (A) with different sample sizes, (B) with different number of selected competitors, and (C) with different weights of luck. All parameters but one (sample size in A; success rate in B; relative weight of luck in C) were kept constant at 100,000 competitors, 1% success rate, and success being based on the weighted mean of merit (weight 95%) and luck (weight 5%).
Lessons for our lives and our meritocracies
Although my simulation is quite simple, more complex analyses support its conclusions. Alessandro Pluchino and colleagues developed an agent-based mathematical model that simulated the evolution of careers over people’s lifetimes.3 Every simulated person (“agent”) was assigned a certain amount of talent, normally distributed. At the start of their life, all agents had the same amount of capital. Then, they simulated 40 years of life with randomly occurring lucky events (doubling an agent’s capital) and unlucky events (halving an agent’s capital). Their simulations showed that the most talented people almost never achieved the highest success. Instead, the most successful agents were mediocre in terms of talent, but extraordinary in terms of luck. Although talented agents were more likely to become rich than averagely talented agents, those average agents were more likely to become extremely rich.
This finding, that luck has a large role in extreme success, casts doubt on our implementations of meritocracy. Academia aspires to be meritocratic, as opposed to class-based or wealth-based. However, grants are usually awarded based on an academic’s past successes, which are easier to measure than an individual’s abilities. Yet the people that reach the highest (past) success level, are often the luckiest, rather than the most skilled. Pluchino and colleagues mention that these “meritocratic” strategies lead to a snowball effect, where success leads to more success. This is also called the “Matthew effect”, after the apostle who wondered why the rich only get richer, while the poor get poorer. A system where grants are assigned randomly, or randomly after a first quality check, would help against the Matthew effect. Although random grant assignments feel unfair, they would be much more effective in ensuring that the most talented people achieve some level of success.3
For our personal lives, we should not conclude that luck is everything. The simulations show that merit – talent, skills, and perseverance – matter. Our professors and football stars alike had high merit scores. But while merit is necessary, it alone is not always sufficient for success. Luck explains more about success than we think. So, if and when you are successful, be open to the possibility that you were fortunate to be in the right place at the right time, rather than being more deserving than your less successful peers. Because, as Pluchino says: “Even a great talent becomes useless against the fury of misfortune.”
About the author
Dr. Anna Beukenhorst is an expert in health data science and biotechnology. She holds a PhD in digital epidemiology and data science from the University of Manchester and worked in digital phenotyping at Harvard University. Currently, she is senior scientist and academic liaison at a startup.
Disclosure statement
The author declares no competing interests.
Note
This article was a finalist for the 2021 Statistical Excellence Award for Early Career Writing. The 2022 writing award is open for submissions until 31 May 2022. See significancemagazine.com/writingcomp for details.
References
- Taleb, N. (2005) Fooled by Randomness: The Hidden Role of Chance in Life and in the Markets. Vol 1. Random House Incorporated.
- Royal Society (2010). The Scientific Century: Securing Our Future Prosperity. Royal Society.
- Pluchino, A., Biondo, A. E. and Rapisarda, A. (2018) Talent versus luck: The role of randomness in success and failure. Advances in Complex Systems, 21(03n04), 1850014.
