A new mask study is poised to affect Boston area children

by Vinay Prasad | Aug 14, 2022

Recently, a parent in the Boston area emailed me a pre-print (see below), asking for my thoughts. Spoiler alert: the paper is irredeemably flawed. But read to learn just why & some broader lessons about the need for skepticism & rivals in science.

The parent writes, this “study has already been highlighted by the Boston Globe as justifying the reimposition of mask mandates in the Boston area schools.”

“I am writing to you in the hope that you will apply your critical talents to this.” Thanks for that. So, let’s tackle the question: Is the study good or is it garbage? (Note: I will say, which I rarely do, the manuscript is well written)

The authors leverage something unique. In Massachussetts between Feb and June of this year, some districts lifted the “mandatory mask rule,” while 2 districts— Boston and Chelsea never did, and kept masks till June.

The authors looked at 72 districts in the Greater Boston area to see what happened to COVID19 cases after lifting the mask mandate vs. keeping it in place. The use a diff-in-diff analytic technique.

Strangely the authors’ main analysis does not adjust for community burden, considering it a mediator; I don’t agree with this. Schools are not a major driver of community spread. They write however “In sensitivity analyses, we found that the benefits of universal masking requirements persisted even after controlling for several measures of community COVID-19 incidence” Either way, we won’t have to argue this too much.

The authors find, “We estimate that lifting of school masking requirements was associated with an additional 44.9 (95% CI: 32.6, 57.1) COVID-19 cases per 1,000 students and staff.”

The authors conclude, “once masking requirements were lifted, we observed consistently higher case rates in those school districts compared to districts which continued masking requirements”

The authors believe their study makes the case for continued mask mandates.

I have 5 points, but the 5th is bonus. The second is so problematic the study falls entirely. Let’s address them

#1 That was then, this is now. Even if you believe these results were true between Feb and June 2022 (I don’t). That was then, and this is now. Now, nearly all kids have had COVID19. Every single kid could get a vaccine if they are worried. If parents want more data about vaccination, they should read our pro-con debate on Sensible Medicine. The last few kids without COVID will get COVID in the next few months or years (with or without vaccines or masks), and then they will get it again and again and again over their long lives.

The authors logic of continuing masking is like applying SPF 75 sunscreen over a sunburn when you are back from a beach vacation and lying in bed in your dark bedroom. It’s a moot pt.

#2 They test differently. At no point in the manuscript do the authors recognize that people are tested differently if schools mask vs. don’t mask. This limitation is disqualifying, and I won’t spend much more time looking for other errors. This error alone means the paper should never appear in a journal.

Worse, there is no mention of it in the manuscript, suggesting the authors were not aware of it. This is a classic research problem, and why it is always good to include skeptical voices on research teams (team of rivals), so that someone might have saved them the trouble. What is the error?

In Massachusetts, you don’t have to test if people wear masks; if they don’t, you do (see pic). This is a fundamental imbalance that invalidates the results. Rule #1 of research is you have to measure the endpoint similarly.

Forget about masking. Imagine if instead of cloth masks we believed in a different totem. Imagine we believed wearing a ‘Fauci locket’ would keep COVID away. Of course, in this world, Trump didn’t wear a Fauci locket, and it became polarized and of course, we never ran cluster RCTs.

In our world of Fauci locket wearing, the CDC and MA make a rule: if 2 people wear lockets, you don’t have to test. If one does not, you do. Then districts that drop the “Fauci locket” mandate will have more testing, and more cases, but we all know how useful “Fauci lockets” are in stopping an airborne virus.

Alasdair Munro puts it this way:

Alasdair Munro @apsmunro

@fakecarlhill @EpiEllie @cbpolis No, this renders the results totally invalid This is a clear systematic difference which cannot be accounted for

Strangely the authors kept arguing— that even if mask mandates don’t slow the spread, but solely affect test results—that is still good! But this logic is broken because as Dr. Cristea notes: a policy of no testing at all would be even better! (PS: this should be the policy, and is in several European nations. If kids feel fine, let them go to school. It will maximize their welfare.)

This is not a unique problem to this paper, elsewhere Tracy Beth Hoeg and I wrote about it for a Pediatric study.

Tracy Høeg, MD, PhD @TracyBethHoeg

The letter @VPrasadMDMPH, @smalltownpeddoc & I wrote about the Duke mask study being confounded by different contact tracing policies for masked students is published? The confounder makes transmission among masked appear lower Study should be retracted publications.aap.org/pediatrics/art…

#3 Equity is being misused. The paper is full of language about health equity, but the authors have it backwards. Equity is important, and poor, minority communities deserve the best school and education possible. In fact, prolonged school closure hurt these communities the most; that is, in part, why I opposed them,

The authors write, “…school districts that sustained masking requirements for longer periods tended to have school buildings in worse physical condition, more crowded classrooms, and a higher proportion of staff and students made vulnerable by historic and contemporary systems of oppression (e.g., racism, capitalism, xenophobia, and ableism)”.

But if masks don’t actually make students better off than this is yet another unfair imposition on these kids. They are yet again the victim of an oppressive policy that doesn’t work. The question is solely: Does masking help them?

We have better studies

Studies from Finland & Spain are much better; I have written about the latter here, and done videos on the former. These are regression discontinuity designs without ascertainment bias, and show no effect.

Rob Hughes @R_Hughes1

Dear #epitwitter and kids #MaskMandates Twitter, Please look at this ?graph of covid transmission risk amongst 15,000 kids within bubbles, and tell me: *What age group of children were mandated to wear masks at school/pre-school??*

Bonus material: FOR ADVANCED METHODOLOGISTS ONLY

This is a fun one; First, take a long look at this figure from the supplement, and think it through with me….

I want to leave this for subscribers because I am not sure I have it totally correct, so hoping you all can help me think through it.

As you see, one variable when added to the model drastically inflates the effect size; and that is test + rate; it blows the effect size up. But doesn’t that prove the testing bias is at play?

When you drop the mask mandate, you find more cases, but you run way more tests than cases you find. You change the test + rate on a different gradient than the case rate elevates, and as such adjustment for the test + variable exaggerates the effect size.

Put another way, dropping mask mandates increases cases, but adjusted for test + it really increases cases, b/c you really increased testing. Is that not circumstantial evidence that proves my point? Let me know your thoughts.