How to not be completely wrong about masks

Almost everything you read on this topic is nonsense in one direction or the other

Few topics have provoked more ire during the past 3 years than the use of facemasks. Despite much of the world now having largely dispensed of their use in the community the debates still rage on, most recently due to the publication of a prestigious Cochrane review article on the topic.

For those unfamiliar, Cochrane reviews apply a rigorous, systematic appraisal of evidence to a given intervention to assess the quality of evidence and likely estimates of effectiveness.

You would think this would be a good thing. However, the review seems to have been most successful in simply exposing how woefully (and in some cases wilfully) incapable many people are of objectively assessing evidence and it’s meaning.

What follows is what I believe to be the only sensible way of considering the evidence around the use of facemasks for Covid-19.

What did the review actually show?

I will not get into the nitty gritty of the methodology of systematic reviews and meta-analysis here, as it’s actually not very relevant. Cochrane have rigorous standards for this, and the review was done appropriately.

It brought together randomised trials of mask use for influenza (or influenza like illnesses) with the few which have been done for Covid-19, and found there was very little good quality evidence, with very wide estimate of effectiveness which range from very effective to ineffective or hazardous. To put it simply, there is not enough good RCT evidence to inform us of their effectiveness.

What it DOES NOT show is that masks are ineffective. It is also not “deeply flawed¹”, and cannot be dismissed. It tells us at least one very important thing - there it not high quality, clinical evidence that community masking reduces transmission of Covid-19 (or other respiratory viruses).

The take away is that we cannot be certain on the basis of high quality clinical evidence that community mask use is, or is not, effective at reducing transmission of Covid-19.

The only thing we can be sure of that there is not enough high quality evidence - full stop. That alone is a massive failing, which I have addressed previously.

How to think about evidence for masks

The statement “masks don’t work/work” is meaningless. We need to be specific about precisely what masks and in what setting we are discussing.

The lack of high quality clinical trials does not mean we know nothing at all about masks. The world of evidence is never black and white, and we have a suitable gradient of greys to ponder. We also have prior knowledge to incorporate into our thinking (this is using Bayesian probabilistic reasoning).

Many people like to talk the engineering, or physical/mechanistic evidence for masks (almost always in the context of high quality N95 respirators). Devices such as N95s undergo rigorous testing by regulators to ensure they do what they say under very fixed conditions. As such, we can be confident N95s remove 95% of particulate matter, under very fixed conditions. Particulate matter will include aerosols containing SARS-CoV-2, and so we can be confident these masks when worn appropriately will reduce exposure to aerosolised particles, under very fixed conditions.

That means if we take a cut and dry example under very fixed conditions, such as a health care worker briefly entering an isolation room of a patient with Covid-19, we can be confident that correctly wearing a fit tested N95 will reduce the likelihood of getting infected by a significant amount (precisely how much is difficult to quantify and I would be sceptical of any precise estimates).

Despite a lack of high quality clinical trials, our prior knowledge would heavily suggest this is likely to be an effective intervention.

That is one end of the spectrum, and it is the end of the spectrum which is least interesting to the general public, predominantly because of the requirement of very fixed conditions which do not apply in the real world.

At the other end of the spectrum, we have flimsy bits of cloth which people were advised to fashion themselves at home, worn intermittently, often under noses or chins, and were removed during the periods of time which are highest risk for transmission to occur (whilst at home, or whilst eating/drinking/socialising with close contacts). There is huge uncertainly about how effective implementing community mask mandates in this fashion could possibly be, because there are a million ways in which it could fail to work.

This is one reason why the existing RCTs of community mask wearing for Covid-19 have failed to show an enormous benefit. In several of the trials in low and middle income settings, most people didn’t even wear the masks. These trials are useful for understanding how effective these strategies might be in those settings (i.e. not very effective), but if we want to know how effective they are in settings where adherence was very high (e.g. US or Europe) we cannot generalise these findings.

What about observational evidence about masks?

Aside from the mechanistic evidence, most people refer to ecological and observational studies which find reductions in cases of Covid-19 in communities with higher mask use. I would advise almost completely ignoring these studies. The risk of bias is so large as to render them almost completely uninformative, and this is only reinforced by the inconsistent and often wildly implausible effect sizes observed (many are so huge they would be visible from space even in the few RCTs which have been conducted).

If that seems harsh, I will point out that from these forms of studies there is equally good evidence for hand washing at preventing Covid-19 transmission than mask use. Take from that what you will.

I’m sorry to say that the production and promotion of some of the lowest quality versions of these studies by public health agencies (not in the UK thankfully) has likely done serious damage to their reputation.

Why it matters

Even if we could all agree that masks do something, this is not very helpful when trying to make policy decisions. Despite many people reluctance to admit so, masking does carry significant social, economic and environmental costs - especially when applied to billions of people for years at a time.

To make adequate assessments of the cost: benefit analysis, we at least need to be able to roughly quantify the benefit. This will be different in different settings (e.g. health workers in a respiratory ICU vs toddlers attending day care), but as it is we are all left making rough guestimates informed by our own priors (and whether people care to admit it or not, their political leanings).

All we can really be sure of is quotes like, “masks are magnificent”, or, “masks could end the pandemic” are nonsense. Mask use in Sweden was almost non-existent, and their pandemic outcome is comparable or better than most places in Europe. This doesn’t prove masks do nothing, but it clearly demonstrates that whatever their effect, it is completely obliterated by other factors such as population health and behaviour, access to healthcare etc.

We could easily have generated evidence in time to inform this pandemic. Failing that, we could have tried to generate it to inform the next one. Sadly it now appears that ship has also sailed.

NB: there is a completely separate discussion to be had about the impact of low level NPIs in a setting where transmission is largely controlled by immunity, but that is for another time.

Summary

It is likely based on our understanding of the mode of SARS-CoV-2 transmission and the mechanism of action of facemasks, that wearing them could do something to reduce transmission. How much it actually does is largely unknown, and is highly dependent on the type of mask worn, and how well defined and controlled the period of exposure is. On one end of this spectrum, it’s likely they significantly reduce the risk of transmission. At the other end, it is highly uncertain whether they provide any meaningful benefit worthy of the associated social, economic and environmental costs.

The only thing truly certain is we didn’t generate enough high quality evidence to answer these questions. Be wary of people claiming otherwise.

1. If you read an unfortunate viral twitter thread claiming the Cochrane review is flawed, please see this excellent break down of it’s many fundamental errors.