Perhaps once some better quality evidence have been generated this might provide more of an incentive for people to look more closely at investing in their own practice!
In the senior living facilities I'm familiar with, I would guess the greatest risk of transmission is in the dining rooms and other commons areas where residents gather in numbers, often w/o masks (esp. while eating). Running filters in individual residents' rooms might reduce the risk of transmission while a resident is being visited there by caregivers. What did this study do to adjust for that exposure bias?
There should not be any exposure bias since the individuals were randomised (so the distribution of exposure within rooms in each group is random, enabling inference).
The issue of common areas is valid, however intervening in these would have meant needing to randomise at the level of the facility, not the individual. This would then turn the trial into a cluster randomised trial. Once you randomise clusters, your power is dependent on the number of clusters, not the number of individuals. This drastically reduces power, and you would need more like 20 facilities to detect the same effect size.
Right, I should not have called it a bias. My point remains that the likelihood of transmission is almost certainly much higher in the public areas, where residents in both arms of the study were getting exposed ~equally. Pick a number that you like for comparative risk in the two locations (private room vs. public spaces). I'll guess that infections are 3x more likely while in the public areas, in which case, the in-room filters are only (partially) preventing 1/4 of all transmissions to residents in the experimental group. The effect of public transmissions could easily be swamping the effect that the study was purporting to measure, and the data seen seems consistent with that. . .Agree with your point about randomizing at facility level. A heavy lift, but one that could have been (could still be) undertaken by chain operated LTCFs with many physical locations (more and more common in the US).
While having good quality air filtration may eventually be a factor more important in USA is providing these workers with actual paid grime off whensick and NOT the punitive policies in place. Lower wage workers do NOT earn great amounts of paid time off, cannot afford to be off work, rely on OT wages etc. why do NONE of these articles address the biggest elephant in room??
Yes the dichotomania inherent to NHST has a lot to answer for. Whilst it should help us remain appropriately sceptical, it's too easy to excuse "no definitive evidence of effectiveness" for, "evidence of no effectiveness".
Dr McDonald reported receiving grants from the National Health and Medical Research Council (NHMRC), Medical Research Future Fund, and GlaxoSmithKline; receiving personal fees from GlaxoSmithKline for participation on the advisory board and provision of education; and receiving personal fees from AstraZeneca, Boehringer Ingelheim, and Menarini for provision of education outside the submitted work.
"the study results are absolutely consistent with them being highly effective - they just also happen to be consistent with them being ineffective"
Not just ineffective, but actually detrimental. It's almost certain the effect is not zero.
I am not here to defend significance testing, but if it must be used the clearest way to say the data is compatible with both positive and negative effects.
i remain shocked at how many doctors offices and even hospitals did not improve air quality. Cost yes...
but even if we reduce the risk by 10% it is huge
Perhaps once some better quality evidence have been generated this might provide more of an incentive for people to look more closely at investing in their own practice!
In the senior living facilities I'm familiar with, I would guess the greatest risk of transmission is in the dining rooms and other commons areas where residents gather in numbers, often w/o masks (esp. while eating). Running filters in individual residents' rooms might reduce the risk of transmission while a resident is being visited there by caregivers. What did this study do to adjust for that exposure bias?
There should not be any exposure bias since the individuals were randomised (so the distribution of exposure within rooms in each group is random, enabling inference).
The issue of common areas is valid, however intervening in these would have meant needing to randomise at the level of the facility, not the individual. This would then turn the trial into a cluster randomised trial. Once you randomise clusters, your power is dependent on the number of clusters, not the number of individuals. This drastically reduces power, and you would need more like 20 facilities to detect the same effect size.
Right, I should not have called it a bias. My point remains that the likelihood of transmission is almost certainly much higher in the public areas, where residents in both arms of the study were getting exposed ~equally. Pick a number that you like for comparative risk in the two locations (private room vs. public spaces). I'll guess that infections are 3x more likely while in the public areas, in which case, the in-room filters are only (partially) preventing 1/4 of all transmissions to residents in the experimental group. The effect of public transmissions could easily be swamping the effect that the study was purporting to measure, and the data seen seems consistent with that. . .Agree with your point about randomizing at facility level. A heavy lift, but one that could have been (could still be) undertaken by chain operated LTCFs with many physical locations (more and more common in the US).
While having good quality air filtration may eventually be a factor more important in USA is providing these workers with actual paid grime off whensick and NOT the punitive policies in place. Lower wage workers do NOT earn great amounts of paid time off, cannot afford to be off work, rely on OT wages etc. why do NONE of these articles address the biggest elephant in room??
Interestingly this point is addressed by Adam Kucharski in his latest newsletter about the woeful norovirus
https://open.substack.com/pub/kucharski/p/what-would-you-do-to-avoid-norovirus?r=1fhhmw&utm_campaign=post&utm_medium=web
This study highlights one of the fundamental problems with frequentist inference in RCTs. I posted about it on Blue Sky
https://bsky.app/profile/paulpharoah.bsky.social/post/3lazjqj2v2k2i
Thanks Paul, great post-hoc analysis!
Yes the dichotomania inherent to NHST has a lot to answer for. Whilst it should help us remain appropriately sceptical, it's too easy to excuse "no definitive evidence of effectiveness" for, "evidence of no effectiveness".
Conflict of Interest Disclosures:
Dr McDonald reported receiving grants from the National Health and Medical Research Council (NHMRC), Medical Research Future Fund, and GlaxoSmithKline; receiving personal fees from GlaxoSmithKline for participation on the advisory board and provision of education; and receiving personal fees from AstraZeneca, Boehringer Ingelheim, and Menarini for provision of education outside the submitted work.
You can just say that the data is consistent with both an increase or a decrease in the risk of infection.
You could say that and it wouldn’t be wrong, but it would be incomplete.
The data is much more consistent with a decrease than an increase.
"the study results are absolutely consistent with them being highly effective - they just also happen to be consistent with them being ineffective"
Not just ineffective, but actually detrimental. It's almost certain the effect is not zero.
I am not here to defend significance testing, but if it must be used the clearest way to say the data is compatible with both positive and negative effects.