Phoebe, a 21 year old entrepreneur, is done with all forms of elitism! She has started two new clothing brands: Average Annie, and Average Aaron! The clothes are aimed at the average sizes for women and men, respectively. Pants all come in one waist size: 39 inches for men, and 35.8 inches for women. Why? Because these are the latest measurements published by the Australian Bureau of Statistics. Problem? Too few people fit those exact measurements!
This was a made up story, but similar ones do exist in real life. In a little while, you will be reading about how catering to the average cost pilot lives and possibly misdirected education funding.
Refresher: What is average?
We have seen before that the average is calculated from values that are usually either below it or above it. To remember how it is calculated and when it is useful, see this post. As a short example here, imagine four cousins who want to share, equally, the chocolates that they each have. They put all the chocolates in one bowl: Sheldon’s 3, Penny’s 8, Raj’s 5 and Bernadette’s 9 bars. This makes a total of 26 bars of chocolate. Shared among all four cousins results in each of them netting 6 and a half bars. 6.5 is the average number of chocolate bars per person.
When designing for average costs lives
In the 1940s, the US Air Force noticed that it was losing pilots to accidents that could not be explained by mechanical failure or enemy fire. In fact, the accidents weren’t even due to pilot errors. What was downing the airplanes and killing the pilots were the physical settings of the cockpit. Those were designed to fit the “average pilot”.
The Air Force decided that the average measurements it was using were out of date. They decided to re-measure their pilots, some 4,000 of them, and calculate new averages for height, arm span, chest circumference and so on. In all, 140 measurements were taken for each pilot!
To crunch the numbers, they appointed Lieutenant Gilbert S. Daniels. He had recently graduated from Harvard, where he had studied hand measurements.
Lt Daniels asked a different question: How many pilots are average in all of their measurements? He was happy to count anyone who was within 30% of the average. Yet, the number he found was Zero! For instance, if a pilot was within the average range for height and torso circumference, he may not be in armspan.
Instead of updating the average measurements and then tasking airplane manufacturers to design their cockpits to fit them, that is to fit no single pilot, the US airforce asked its suppliers to make adjustable seats, helmets and controls. This way, the planes would be functional enough for all of the pilots!
When calculating the average wastes resources
In education, as in medicine, we like to read the results of a research practice known as meta-analysis. In meta-analysis, an average of the “effect sizes” of different studies is calculated. This is good in the sense that it aggregates the results of many studies of the same phenomenon, such as a teaching technique. However, some notable meta-analyses have been published that have averaged out the results of dissimilar studies.
Take the idea of “feedback” as an example. Feedback can take many forms. Here is an informal list:
- Praise: “You did well”.
- Criticism: “That was an unacceptable mistake!”
- Task-level feedback: “This is what you did well, this is what you need to get better at”.
- Process-level feedback: “When solving a linear equation, first see if you can take a common factor on both sides of the equal sign”.
- Providing a solution sheet at the end of a task.
Averaging the results of such different approaches and telling me, as a teacher, that the net effect of feedback is positive, and that it is greater than the effect of, say, smaller class sizes, isn’t that useful. What I need to know is which forms of feedback have the most positive effect.
Similarly, if the average effect size turns out to be negative, but one form of feedback was shown to be extraordinarily useful, then I would want to know about that form and not shun feedback altogether.
Such meta-analyses direct governments and non-governmental school systems to spend money on certain interventions over others. Being “sloppy” with statistics, namely by calculating an average effect from dissimilar phenomena, will misdirect resources.
Conclusion
Having already covered the mathematical operation for calculating feedback and described how it can be helpful to us in understanding the world, I thought it important to provide cautionary tales of the misapplication of the concept of average. Like all useful statistical tools, we must consider the context and ask ourselves whether it is the right tool to use.
This was the third in a series of posts about statistics and the pitfalls of using statistical measures in a way that contradicts their intended applications. The first two posts were:
- Sunrises Impossible without Roosters: Studies Show, in which I discuss the difference between establishing an association between two things (rooster crows, sun rises) and establishing cause and effect.
- Sampling and the Survivorship Bias, in which I discuss the complexity of considering a representative sample of the population when we want to draw valid conclusions.
Leave a Reply