The certainty trap – Why data may not be wrong but not right either

Are women more likely to suffer from side effects and complications following the vaccination against Covid-19?

How is it that there are more men than women being involved in car accidents each year, yet women are more likely to be seriously injured in a car crash?

These and other questions seem to be an issue of gender when in fact they are an issue of data availability, aggregation, and interpretation. For the month of April, WiE (Women in Economics Initiative) is highlighting the “gender data gap” and we want to share how data can create an illusion of certainty, which often entails a gender bias.

We are used to data and numbers being accurate demonstrations of our world. They are clear, hard facts of the issues they are describing. People think of data and statistics as something definitive and clear and tend to be trusting towards it. If someone wants to support an argument it is not rare to support it by statistics and numbers. It gives the verbal argument a more scientific and certain foundation and makes it therefore more trustworthy.

However, data and statistics can in fact lead to false conclusions, illusionary certainties and bias the focus of what is at the core of an issue. This is one of the central issues of the gender data gap which is why we want to explore it in more detail.

So, what if “the numbers don’t lie,” however, they fail to paint the full picture?

AstraZeneca controversy

The Covid-19 pandemic has given rise to an unprecedented flood of data rolling over the average person. Newspaper articles, the TV news and the internet are full of statistics, graphs, and data models on the effect of wearing masks, infection rates, death rates and the vaccination progress. What seems like a clear relationship sometimes, needs in fact careful consideration and study, which can be seen best with regards to the AstraZeneca controversy in Europe.

When the AstraZeneca vaccine was first put on halt in several European countries news reports came out stating that the vaccines side-effects were especially pronounced in young women. However later reports stated that there was no gender difference in the severity of side-effects but only an age effect. How can this be? Had the data changed?

The short answer is no, the data from which such conclusions were drawn did not change substantially but the context of interpretation was a different one. It is in fact true that in absolute numbers there were more young women than young men affected following a vaccination, however this statistic does not tell the full story. The fact that there are more women affected in absolute numbers is attributed to more women being in the early priority groups since they work in health care and education in larger numbers, which are occupations eligible for early vaccination. Hence the clustering of side-effects observed in young women may be a statistical quirk of more young women than (young) men getting the vaccine early rather than a gender effect.

This example highlights how data and statistics need context to be understandable but a lack in accuracy or data literacy (or both) often cause shortcomings in exactly this regard which can create misunderstandings and false convictions for the broader public. Another problem arises where data is in fact incomplete and creates black boxes that are overlooked.

The car safety paradox

Have you ever struggled to adjust the car seat such that it fits your needs but could not find an optimal position and then went with the second best solution? I certainly have, my mom has and if you are a woman yourself you are more likely to have nodded while reading the question…

As highlighted in the introduction there is a paradoxical relationship in the gender of absolute number of individuals involved in car accidents each year and the likelihood to be seriously injured in a car crash. The fact that the majority of car injured individuals are men can be attributed to the fact that men drive on average more miles than women. However, the fact that women are still more likely to be severely injured in a car accident is caused by a data gap, or a lack of data on drivers who are women.

This is because cars are designed with an average man in mind, the dummies used to simulate a car crash are modeled after the physique of a men, not a woman. The man is the default driver in crash testing. This makes cars significantly less safe for women when being involved in a car accident since the seating positions vary and make severe injuries more likely for the average women.

It is a fact that women are less represented in the data collected car safety since the data on crash effects on car crash dummies use dummies which are representative of the average man, not the average woman. Our conviction that enough data has been collected on the safety of the car before it is released for sale creates an illusion of certainty and safety. The problem here is not the misinterpretation or the missing context but the sheer lack of data: a gender data gap.

“Certainty” in data

Data and statistics are a powerful tool, precise and clear to interpret if done cautiously. As much as data or its interpretation can be flawed and biased there is no better tool to get behind causal relationships to understand the problems of our world. In fact data driven policies and innovations can even enhance peoples trust into them. Data-driven arguments and carefully collected and analysed data can improve both the policy outcome of the issue at stake as well as the public’s perception of the very research and policy.

But disadvantages in data persist, not only because of statistical illiteracy but also because data is not always inclusive as the car dummy example illustrates. Instead of “going where the data is” concerted effort is needed to accumulate the data where the people are, to design policies and products with humans as the default rather than men.

The issue of illusionary certainty in statistics and data is especially important for economics as a social science. Economics concerns itself with issues about humans, society, and their interactions. Drawing conclusions on the small scale without keeping the bigger picture and context in mind or using data that is not inclusive of all people who are affected by the issue at stake can make for less inclusive research results and hence less inclusive policies and solutions. The issue of the “gender data gap” highlights this problem that has yet to be changed.

The views expressed in WiE opinion pieces are those of the authors and do not necessarily reflect the opinion of the organisation, its partners, other members, or any other affiliated people and organisations. As ever-learning, critical-thinking people, these opinions are subject to revision and adjustment at any time. WiE welcome constructive feedback in the comments section below and reserve the right to delete any comment deemed inappropriate, rude, irrelevant, or abusive. All posts are for informative purposes only and, while they are accurate and authentic to the best of our knowledge, WiE accept no liability for any errors or missing information.

Further links and reads: Check your statistical knowledge on gender equality, environmental protection, and education: Gapminder To learn about more examples of black boxes in statistics concerning gender, check out the book Invisible Women by Caroline Criado-Perez where our author first learned about such things as gender differences in car safety  

Leave a Reply