Tim Hoff: Welcome to another special edition of Ethics Talk, the American Medical Association Journal of Ethics podcast on ethics in health and health care. I'm your host, Tim Hoff. This episode is an audio version of a video interview conducted by the Journal's Editor in Chief, Dr. Audiey Kao, with Dr. Steven Goodman. Dr. Goodman is Associate Dean of Clinical and Translational Research and Professor of Medicine and of Epidemiology and Population Health at Stanford School of Medicine. He joined us to talk about the ethical and scientific implications of unblinding COVID-19 vaccine trials. To watch the full video interview, head to our site, JournalOfEthics.org, or visit our YouTube channel. [music fades out]
Dr. Audiey Kao: So, there's not a lot of public understanding about why the relationship between science and ethics matter in biomedical research. So, we're going to investigate a concrete example: the current COVID-19 vaccine studies, or Phase 3 clinical trials. Last December, the U.S. Food and Drug Administration, or FDA, granted emergency-use authorization for two COVID-19 vaccines. Despite this emergency authorization, Phase 3 clinical trials are ongoing for these vaccines. So, making a change to the research design of these trials have scientific and ethical implications. Pfizer-BioNTech and Moderna, developers of these two vaccines have accepted these implications and are revealing to people enrolled in these trials whether they receive the investigational vaccine or placebo. This process of revealing is called unblinding. And that's a big deal because blinded placebo-controlled studies are widely regarded as the gold standard in biomedical research. So, why would we ever not want the gold standard? Here to help us think this through is Dr. Steven Goodman, Associate Dean of Clinical and Translational Research and Professor of Medicine and of Epidemiology and Population Health at Stanford School of Medicine. Dr. Goodman, thanks for being a guest on Ethics Talk today.
Dr. Steven Goodman: Thanks very much for having me. I look forward to the conversation.
Kao: So, can you first tell our audience why blinded placebo-controlled studies are so important in clinical research?
Goodman: Well, for two reasons. First is, you obviously, the whole purpose of clinical research is to figure out the effect of an intervention. You do one thing to one group. You do another thing to the other group. And in the case of placebo-controlled trials, what you do to the other group is you give them something that mimics the intervention but is not the intervention. The reason that the blinding can be very, very important is because we want to isolate the effect of the intervention, and we want to make the groups as similar as possible. In the case—and I will talk of COVID—there are two things that blinding and blinded placebo achieves.
First, if we're interested in vaccine reactions, it prevents people from, obviously, knowing whether they got the vaccine or a saline placebo so they don't preferentially report or interpret the feelings that they have after they got the shot in terms of whether they knew they got the shot or not.
Goodman: If they thought they got the vaccine, they might be much more conscious about mild or even moderate symptoms like headache, dizziness, soreness, etc.
But there's another element in many studies where, and particularly in the COVID space, which is that you're comparing the rates of COVID in the two groups. And that depends not just on whether you received a vaccine or not, if it's effective, but how you behave: that is if you go out in public with or without a mask, if you congregate in small or large groups, with or without protection, these all have an impact on your risk of COVID completely independently of whether you got a vaccine or not.
So, the reason for the blinding is both to make sure that the reporting and also interpretation by doctors of any signs or symptoms of the vaccine or indeed of COVID itself are equal between the arms, but in particular, that the behavior that affects the risk of COVID is also equal between the arms.
Kao: Given what you've just said, why did these biopharmaceutical companies decide to unblind these trials, and what are the ethical and scientific implications of breaking this blind?
Goodman: Oh, that's a very complex question. I'll have to answer it in pieces, sort of unpack the issue. First of all, we have to understand that most clinical trials—and this trial, it was no less true of the COVID vaccine trials—have a monitoring aspect. That is, people are looking at the data as it's coming in, in sort of secret, that data monitoring committee, just to make sure of several things. First of all, that there aren't some safety concerns that evolve, so that we begin to realize that the vaccine might be dangerous in ways that might lead us to just stop giving the vaccine within the course of the trial. So, they're called data and safety monitoring committees. And they look at safety.
The other thing they look for is whether there is the difference in efficacy between the two arms. And it can go in both directions. It may be that the treatment arm—and I'm now just talking in general—does much worse than the placebo arm or the comparison arm. But it could also be that the treatment arm does much, much better. And the size of these trials is planned on the basis, partly, of how big a plausible treatment effect is.
And I can tell you that these trials were not planned with the idea that these vaccines would be 95 percent effective.
Goodman: Meaning that it was possible to come to a pretty good scientific conclusion, or at least a provisional conclusion, after viewing not all of the data.
Goodman: Then instantly, we're confronted with an ethical dilemma. [chuckles] That is—And this is not just for the people in the trial, but for the people outside of the trial. So, originally, these trials were planned, as many vaccine trials are, for two years of observation.
Goodman: But it looked like these had the tremendous potential to save lives outside of the trial, if these vaccines could be administered. And this is only a problem in the context of the pressure of a pandemic. I would say if this was a routine, more routine virus, there would not have been that much pressure.
Goodman: But obviously, people are dying every day, so now, the companies and the data safety monitoring board is confronted with, do we keep this a secret to get yet more information, get more and better information? There's two ways you can get more and better information. One is by just letting more patients into the trial so you learn more about subgroups, etc. But it's also to let it run longer so we can answer some absolutely critical questions. Which I will tell you, we still are going to confront, which is how long does this protection last?
Goodman: Is it greater in some groups than in another? Now, that in turn produces a dilemma for the trialists, because there's no such thing as a clinical trial that somebody can't walk away from. And so, once it's offered outside the trial—this is separate from any obligations that one might think that the investigators have to the patients in the trial. This is completely separate from that. We can talk about that separately if you wish—every participant has an opportunity at some point to leave the trial and get immunized outside.
Goodman: And that's the situation that the investigators face, and that we as a biomedical research committee, community, face in thinking about the ethics here, about what to do within the trial when every patient has an opportunity at some point—And I have to say at some point, because I can't go out and get a vaccine right now if I want to, and many of the people in the trials are not eligible for vaccine. So, that's not necessarily an immediate option for them, but for many it is. So, that's the pressure to break the blind.
Kao: So, let's explore that a little more. Because when the FDA's Vaccines and Related Biological Products Advisory Committee considered emergency-use approval of a COVID-19 vaccine, you suggested an alternative study design, a so-called crossover design that would preserve the blind.
Kao: So, can you explain why you suggested this? And what kind of response did you receive from the FDA committee and the companies themselves?
Goodman: Yes. So, first I'll, describe the design. So, as I said, we're confronted with a situation where we have apparently a highly efficacious vaccine, people in the trial are free to leave, and there's a tremendous push outside the trial to get as many people immunized as possible. So, it is natural to think that what we should simply do is find out what everybody had, and for those who were not immunized, immunize them. But as I said before, there's actually, this is a very early stage. We still haven't learned everything we need to learn.
Goodman: And so, if we did that for ethical purposes or for ethical reasons, I should say, there would be an alternative ethical pressure, which is this might be our last chance to test these vaccines in this way. And to leave critical things unlearned is an ethical problem itself, not just for the people, not so much for the people in the trial, but for the world.
Goodman: So, the question is, can we retain as much information as possible while splitting the difference in terms of the pressure to vaccinate? And the proposal, which did not come up with me. It was a biostatistician at the NIH who led a group that has proposed this—is what we might call a blinded crossover or actually better described as deferred vaccination design. Although at this point it would be a blinded crossover. And what the blinded crossover involves is not telling people in either arm which arm they were in, but rather simply at the point when, at whatever point it was decided, that they might be immunized—and that could be decided by the company to be today or could be, in a sense, decided by society—at the point where they became eligible outside the trial, where they could walk out and get it, that at that point everybody come back, that person comes back in. And of course, their records, it's known internally whether they got the vaccine or not. They don't know it, and their doctors don't know it. And if they got the vaccine originally, they would be administered a placebo shot. If they were in the placebo group, they would be administered a vaccine shot. So, and they would then come back three weeks or one month later to get the second shot. One would be a second placebo shot. In the other group, it would be the second booster shot of the vaccine.
Goodman: And this would be a way to sort of split the difference, to reduce the ethical tension by immunizing everybody, but maintaining a valid comparison group to allow us—and what's interesting, it's not maybe immediately obvious, but this is obvious when you look at the calculations—to learn some very important things, continue to learn some very important things. We would get an extra roughly six weeks of observation by doing this. Because even though everybody would now be assured that they are immunized, during the six weeks after they got either the placebo or the vaccine, they wouldn't know whether they were in the second phase, the deferred vaccination, or the original vaccination. So, they wouldn't know whether they were fully immunized yet.
Goodman: So, in that period, they wouldn't go out and act in ways that would incur risk of COVID. If you told me today that I had been in the original vaccinated group, I would know that I could probably go into public spaces with less care. So, this would give us another roughly four to six weeks of protected and equal activity behavior.
The other thing it would do, which I think we're going to find is very important, would give people the sense that they're still in an investigatory enterprise with their colleagues, with other people, and that they would stay part of the study. Because no matter what we do with these volunteers, no matter what we do, it's critical that we continue to observe them for a year or two. And this would sort of keep them within the confines of the trial, and there would be, with a sense that they're still contributing to science.
Goodman: If they were unblinded tomorrow and vaccinated tomorrow, there would be very much a sense, or could be, that it's over. So, that was the proposal: that everybody get immunized, so there's no question about leaving one group immunized or not immunized, but protecting the trial to some extent and allowing us to learn how long that protection lasts. And that's a critical part. By doing it this way, we still retain some ability or more ability to know how long this protection and how robust it's going to be. And that is maybe the most important question that faces us right now. And if we lose the opportunity to get as much information as possible, we may pay a price down the line.
Kao: Yeah. So, I think you make some excellent points, including this design maybe reinforcing the original motivation, altruism potentially, of the volunteers in the first place to stay in the trial to learn information that you can only learn in the long-term.
Goodman: Correct. Absolutely right. And most people go into trials with, at least in part, an altruistic purpose. People enrolling in clinical trials should know that their own benefit is not guaranteed. And of course, even if they got the vaccine, their own benefit wouldn't have been guaranteed because we didn't know at the beginning that it worked.
Goodman: They were taking a risk, and that needs to be appreciated. And some degree of reciprocity indeed is owed. And that's also part of the motivation for immunizing everybody through a crossover fashion. Now, it's not just that they could walk away. It's the sense that they invested something, and this is natural reciprocity. But there is limits to that reciprocity in clinical trials. We don't owe them everything.
Kao: Yeah. So, as we near the end of our conversation, I wanted to talk to you about the fact that public distrust in science seems to have only grown during this pandemic. And among your professional responsibilities, you co-direct the Meta-Research Innovation Center, as well as a new center called the Stanford Program for Research, Rigor and Reproducibility. Help our viewers understand what these centers do and why this work is important for promoting greater trust in science and the biomedical research enterprise.
Goodman: Yeah, thank you for asking about that. This is something I've invested quite a lot of my professional life in. The ability and inclination for the public to trust in science is subject to many forces beyond science. There's been a politicization of science and distrust, having very little to do with many of the facts. So, some of the solutions that, the natural thing we think of is, well, let's adopt processes that make science more reliable. And that is part of the picture. But there's a much broader political context when we're talking about public trust in science. And I would say they're part of a broad movement that's been happening in science for at least 10 to 15 years with greater concerns about the reproducibility of results. And reproducibility is a complex term: I'll just say whether the claims we make are true. And it isn't just the claims that something works or doesn't work, but how uncertain we are. The uncertainty is the critical element. It's not just the certainty, because all science is uncertain.
And it has become evident that—and evident to the journals, evident to the scientific leaders, evident to the NIH, to the funders—that a lot of science is conducted in a way where the, I will say, the efficiency is lower than it should be. And that is the efficiency is the number of correct claims per unit of investment, whether you measure the unit in dollars or in time. And there's a wide range of activities having to do with how data are handled, how rigorous the design of experiments is, whether it is open to scrutiny by others, how results are interpreted and analyzed, the proverbial p-hacking, changing your analyses to make it look, until you find something that's statistically significant. The number of practices really go on and on and on.
And these two centers, the METRICS, Media-Research Innovation Center, is devoted to, in a sense, to the study of, meta-science is the science of science. That is, looking at large numbers of studies to see patterns, to see practices, and see how they impinge on credibility. The other program, and that's more general. And it's been meta-science that has provoked a lot of the changes that we're beginning to see today, including an NIH requirement that all trainees—this was just in the last few months—be trained in issues around rigor and reproducibility, and that their mentors role model it.
The second group I'm head of is basically to make that real at Stanford. It's one thing to have high-minded principles and teach trainees, but if they then go out into the lab and they don't see these practices used, that's effectively unteaching them. Or if people are not rewarded through promotion and funding for doing things the right way, it's dead. So, the second program is to make this real. And that is maybe the most difficult part of this science reform movement: to do the right thing on the ground. And it's an enormous challenge, and we're at the early stages of that.
Kao: Yeah. Well, on that note, I want to thank Dr. Goodman for sharing his expertise and insights with our audience today. Steven, thanks again for being a guest on Ethics Talk.
Goodman: Thank you very much for having me.
Kao: For more COVID ethics resources, please visit the AMA Journal of Ethics at JournalOfEthics.org. And finally, to our viewing audience out there, be well and be safe. We'll see you next time on Ethics Talk. [bright theme music plays]
Credit Designation Statement: The American Medical Association designates this enduring material activity for a maximum of .25 AMA PRA Category 1 Credit™. Physicians should claim only the credit commensurate with the extent of their participation in the activity.
CME Disclosure Statement: Unless noted, all individuals in control of content reported no relevant financial relationships.
If applicable, all relevant financial relationships have been mitigated.