The threat posed by the virus, though, has made randomized trials extremely difficult to do. While researchers have randomly assigned macaque monkeys to receive either a vaccine or a placebo and then exposed them to the virus (those with vaccine-produced antibodies were less likely to become infected), it would be unethical to perform this experiment on humans, given the risk of severe illness and death. Instead, vaccine trials are enrolling thousands of people, randomly giving them a vaccine or a placebo and waiting months until, in the normal course of their lives, a small percentage of them become infected; at that point, they’ll learn which group they were in. This process is much less efficient and provides less detailed information about the circumstances and biology of each infection than an experiment in which participants are closely monitored in a controlled setting.
That makes cases like the Dynasty’s, where a high percentage of people under fairly uniform conditions were infected with the virus while others weren’t, potentially very useful. Testing during an outbreak that infected some 700 people on the Diamond Princess cruise ship in February helped reveal that many infections never produced noticeable symptoms. “We’re trying to learn with limited evidence,” says Emily Oster, an economist at Brown University. “Digging into, What are the particular details of an incident that we can learn from? I think there’s a lot of value there.”
Such digging can help generate hypotheses. The classic example is the study by John Snow, an English physician, of a cholera outbreak in London in 1854. Plotting cases of the disease on a map and interviewing residents — “It’s like the original contact tracing,” Oster says — Snow began to suspect that a pump, where many of those getting sick got their water, was the culprit. But Snow had also tested his theory, that cholera was waterborne, by getting maps of households that were (randomly) served by one of two rival water companies, with different water sources, and noting which of them experienced cholera deaths. When the death rate for one company proved much higher than the other, it was clear that its water was the cause. (Sewage contamination was to blame.)
Such “natural experiments,” in which some event or factor has randomized participants into experimental and control groups, have been especially hard to find during the pandemic. The urgent need to stop the spread of the virus has led policymakers to change many variables at once — school and business closures and reopenings, mask ordinances. This makes it difficult to separate their effects. To find out if closing schools lowered community infection rates early in the outbreak, for example, you might look at demographically similar areas where schools closed either in mid-March or early April and compare their infection rates in early May. “But the places that have been hit the hardest might be the ones that are pulling the trigger earlier,” says Joseph Doyle, an economist at the M.I.T. Sloan School of Management — which can make it seem that closing schools leads to high infection rates, when actually an anticipated rise in infections caused the schools to close. Randomizing them would mean finding ones that closed for reasons “unrelated to anything about the health of the community,” Doyle says. For instance, several schools in Tennessee were hit by tornadoes in March and shuttered early, while neighboring ones stayed open. Comparing their community infection rates weeks later could approximate a randomized trial — if the storms didn’t meaningfully affect other local interactions.