Survivorship Bias: Are running injuries far more prevalent and damaging than we realise?

Of all the medical research papers that I have perused over time it would appear that the consensus view of running injuries is that every runner is injured at least once, possibly twice, every year. This figure varies depending on the paper you read.

But it struck me that all of these investigations into running injuries were conducted amongst those actually claiming to run, even if temporarily injured. This indeed is the first possible hint that we may be dealing with a “survivorship bias”,  a bias in the sampling that could seriously distort our understanding of the true incidence of running injuries.

So what is a survivorship bias, you may ask? The Wikipedia definition is:  

Survivorship bias is the logical error of concentrating on the people or things that "survived" some process and inadvertently overlooking those that did not because of their lack of visibility. This can lead to false conclusions in several different ways. Survivorship bias can lead to overly optimistic beliefs because failures are ignored, such as when companies that no longer exist are excluded from analyses of financial performance. It can also lead to the false belief that the successes in a group have some special property, rather than just coincidence.

The research of running injuries typically looks only at those currently running (even if they're temporarily side-lined with injury).  These are the "survivors". We need to ask about those that have given up. A true reflection of running injuries therefore should include an analysis of past runners - those that used to run. It should not be limited to the survivors that are "active" runners.

Let me take this hypothesis one step further. Below is a very interesting chart showing the cumulative number of Comrades runners (finishers and non-finishers) over the period 1984-2013. The number of subjects is very large making the data set fairly robust. At first glance a couple of very interesting observations arise from this chart. 1) At the age of 40 there is a sharp decline in Comrades runners. 2) The proportion of non-finishers appears to increase after the age of 40 and especially in the 45+ group where they account for about a third, or more, of starters. 

My first impression when I saw this chart was that it reveals the possibility of a “survivorship bias” in much of the running injury research I had read. My reason for this observation is that the drop-off in numbers after the age of 40 is very sharp and somewhat unexpected. One would expect this drop-off to be more gradual and perhaps closer to the blue line I have inserted in the chart. This raises the question: “Why were these runners quitting Comrades?”.  Of course there could be a thousand reasons but the obvious possibility is that they are injured. And my anecdotal experience would suggest that this is a distinct likelihood.

OK. So there are a lot of long distance runners that stop running. In this instance they stop running Comrades and they do this around the age of 40 and we suspect injury is a factor. So what? What does this imply, if anything?

Well it changes everything. It means that any injury research amongst runners, and especially those over 40, should look not only at runners but also at runners that have quit running. This is the survivorship bias we know about. When assessing the incidence and nature of running injuries I believe that we will uncover significantly different results if the samples of observations include the group we could call “lapsed runners”. My belief is that we could expect the rate of injury to be much higher than currently revealed because we focus only on survivors at present. The “survivors” that are still running and volunteering for medical research are far less likely to be injured than those that have quit.

As a counterpoint you may well posit that my hypothesis about quitters, injuries and survivors is a mere rumination and has no empirical foundation. And yes, you may be right. But my anecdotal experience tells me otherwise. I constantly meet people that “used to run” but due to deteriorating knees, hips, ankles and other damaged impact zones have given up. They quit running. They give up because it’s too painful to run or because they simply can’t run at all. Often their running demise is passed off glibly (and incorrectly) as “old age”.

The survivorship bias in the research and understanding of running injuries leads me to my next point, another hypothesis. It is that traditional running shoes as we know (heels, anti pronation, toe lift, etc) are far more dangerous than we assume. And it is these self-same shoes that tend to be one of the "constants" across this group of lapsed runners more so than running style, training habits and so on. My belief is that over-built running shoes can give a wonderful temporary illusion of prowess. They are hugely seductive at both a brand and tactile experiential level. But the key is that this “veneer” of prowess is temporary. Running shoes may enhance athletic appearance in the short term but in the longer term the insidious, gradual, granular and cumulative damage will eventually reduce many runners to non-runners. Based simply on the aforegoing data I would suggest that running shoes are pretty certain to limit your happy running years. It seems like 10-15 years may be a reasonable interval of active running after which the chances of being side-lined are greater than the chance of continuing to run. And so I believe that running shoes are not only highly likely to injure you but that this will manifest most strongly only after an extended period of time. And this simply makes it so much easier for the aging injured runners to pass off their dilemma to some other spurious cause … such as “getting old”!

It is because of the long term gradual deleterious effect of traditional running shoes (call it slow poison) and a survivorship bias that our current medical research fails to deliver good reliable information. In turn we merely perpetuate damaging choices and behaviour.

That is my comment on survivorship bias and running injuries. I have no empirical evidence and I have conducted no statistically significant research. But I do recognise a flaw in data and research methodology when I see it, and I do know what runners and ex runners say when I talk to them.