Technology Governance

Precaution can be consistent with support of science.

By: Gregory E. Kaebnick, Elizabeth Heitman, James P. Collins, Jason A.Delborne, Wayne G. Landis, Keegan Sawyer, Lisa A. Taneyhill, David E.Winickoff

Precautionary approaches to governance of emerging technology call for constraints on the use of technology whose outcomes include potential harms and are characterized by high levels of complexity and uncertainty.

Although articulated in a variety of ways, proponents of precaution often argue that its essential feature is to require more evaluation of a technology before it is put to use, which increases the burden of proof that its overall effect is likely to be beneficial.

Critics argue that precaution reflects irrational fears of unproven risks—“risk panics” (1)—and would paralyze development and use of beneficial new technologies(1, 2). Advocates give credence to this view when they suggest that precaution leads necessarily to moratoria (3). Progress in the debate over precaution is possible if we can reject the common assumption that precaution can be explained by a simple high-level principle and accept instead that what it requires must be worked out in particular contexts. The 2016 report from the U.S. National Academies of Science, Engineering, and Medicine (NASEM) on gene drive research (4) illustrates this position. The report shows both that precaution cannot be rejected out of hand as scaremongering and that meaningful precaution can be consistent with support for science. Gene drives are a form of preferential inheritance that occur naturally and can be constructed in the laboratory using new genetic editing tools such as CRISPR-Cas9. Constructed drives could be used to alter, reduce, or eliminate populations of organisms in the environment; in principle, an entire species could be modified. Gene drives have considerable potential benefit, especially in the control of vector-borne diseases and the protection and restoration of environments threatened by nonindigenous organisms. Gene drives also might have harmful effects, especially to the environment. A drive designed to eliminate a non-native mouse population from an island to protect native species might pose threats to related species, to populations of the mouse elsewhere in the world, or to other species on the island that depend on the mouse population. The range of effects due to hybridization, geographic dispersal, and predator-prey interactions, for example, would need to be studied and the probabilities quantified. These are a few possible harms for one hypothetical use; given the present state of knowledge for gene drives, the outcomes and their probabilities are not yet well understood.

INTERPRETING PRECAUTION

At least four common objections to precaution underlie critics’ claims that precaution is irrational and paralyzing. First, precaution is said to be too vague and ambiguous to provide useful guidance. In response, some advocates hold that precaution is not meant to provide a decision-making algorithm that is able to identify appropriate precautionary measures for each and every technology. Precaution is better described at a high level not as a principle but as an attitude or approach that consists in sharpening or broadening the scrutiny of a proposed project (5, 6). Deciding whether precautionary measures are appropriate, and then determining what they are, depends on examining details of the technology and its potential impacts (2). The NASEM report demonstrates this contextual approach to precaution. It starts from an understanding of the science, how the science might be used, and downstream effects of attempts to use it. Some general potential benefits and harms can be easily identified, but prospective uses involve multiple complex systems—genomic, environmental, and social—that add layers of uncertainty. For a given proposed gene drive, we may be uncertain about the probability of outcomes but also about how outcomes should be described and valued and about what all the possible outcomes are. The latter kinds of uncertainty can be distinguished from uncertainty about probability by referring to them as issues of ambiguity and ignorance (6). The potential benefits of research on gene drives make prohibiting the research unjustifiable. The potential harms and  uncertainties make a too-quick commitment to those uses unacceptable. The report therefore recommends research under four broad constraints:

  1. Any proposed release of a gene drive must be understood as raising issues related to values. Questions will arise about potential benefits and harms, the moral seriousness of the problem that the release is meant to address, the merits of other ways to address the problem, the distribution of benefits and harms, and control over decisions about the proposed release. Such questions can address economic, social, environmental, and health effects. Attention to them is itself an important aspect of precaution (5), and they frame discussion of other possible precautionary measures.
  2. Any proposed release requires engagement with relevant publics, fostering attention to the values questions, broadening control over decisions beyond the community of scientists and engineers, creating an additional layer of review, and improving scientists’ understanding of potential outcomes.
  3. Researchers should follow a phased testing approach that provides a step-bystep framework from initial development of a research plan through to post release monitoring, with predefined, study-specificcriteria for determining whether to transition to the next phase based on evidence regarding potential outcomes.
  4. Proposed releases require ecological risk assessment as a basis for examining the probability of immediate and long-term environmental and health effects. Missing from these recommendations are detailed substantive requirements—for example, performance goals for the evolutionary stability of a proposed drive or specific requirements for environmental or social outcomes. The constraints are instead process requirements: They offer strategies for thinking about proposed releases and identifying, characterizing, and evaluating outcomes. In carrying them out, substantive requirements specific to an application should emerge. For example, a high level of assurance that a drive will be geographically limited might be demanded. In effect, contextual development of precaution continues as researchers and oversight bodies examine specific applications. ITERATIVE APPROACH TO  UNCERTAINTY

Two other common objections to precaution are closely connected: Precaution sets epistemologically  impossible demands because uncertainty about outcomes can never be fully resolved, and therefore, precaution simply means giving up on technological innovation and its potential benefits (1, 2). However, precaution comes in more and less restrictive forms. Any precautionary position, whether developed as a high-level principle or with reference to a particular technology, has three components: a preliminary reason to suspect a possible harm, a preliminary reason to believe that there is uncertainty about the effects, and a recommendation (triggered by the first two components) for precautionary measures (7–9). The restrictiveness of precaution depends on how much potential harm and uncertainty are necessary to trigger precautionary measures and how severe the measures are. The charge of epistemological impossibility is plausible if the presence of any uncertainty is a triggering condition. When precaution is formulated as a high-level principle, the language is inevitably vague, and extreme demands are more easily read into it. The NASEM report, however, taking a contextual approach, identifies fairly specific, concrete, and manageable triggers, such as that gene drives could have unwanted effects in the organism’s genome,that a gene drive–modified organism may have unwanted effects on an ecosystem, and that what counts as “unwanted” may vary among different publics. The charge that precaution means giving up on technology depends both on the triggers and the precautionary recommendations. Those opposed to a technology sometimes use precaution to argue for barricades to technology development—for example, general moratoria pending global enactment of very stringent oversight measures (3). The NASEM report, however, calls for targeted but meaningful measures. These measures will be familiar to many scientists, but, taken together, they encourage a broader range of perspectives on and questions about the technology; ensure that any proposed release receives robust and iterative assessment that can incrementally reduce uncertainty surrounding its outcomes and probabilities; raise the bar for demonstrations of efficacy and safety that those proposing a gene drive release must provide; and ensure that risks are acceptable to the relevant publics and are reduced to the greatest extent possible. The constraints would almost certainly bring some proposed releases to a halt. Yet their purpose is not to halt research but to establish conditions under which it can be successful. They constitute a path toward possible release of gene drives—a route with flashing red lights, checkpoints, and off ramps rather than barricades.

SCIENCE AND VALUES

A fourth common objection to precaution, implicit in the idea of a risk panic (1), is that precaution is  grounded on emotion—fear of the unknown—rather than reason. This objection is not wholly mistaken. Risk panics might explain some precautionary policy positions. We should, however, seek input from a wide range of stakeholders, and although the values that stakeholders embrace should be subject to the critical examination of public deliberation, we should be wary at the outset about rejecting some as less rational than others (10, 11).

There is another possible emotional response to new technologies: instead of a risk panic, an “innovation thrill.” As Oppenheimer said, “When you see something that is technically sweet, you go ahead and do it and you argue about what to do about it only after you have had your technical success” (12). Although the primary rationale for releasing a gene drive would be human or environmental  benefit, the thrill of understanding and engineering biological systems would be part of the motivation to undertake the basic science. That thrill could also generate an impulse to move the science from the laboratory to the field. The NASEM report aims to counteract that impulse. An innovation thrill is no better than a risk panic as a basis for policy on when and how to use gene drives.

REFERENCES AND NOTES

  1. C. R. Sunstein, Laws of Fear: Beyond the Precautionary Principle (Cambridge Univ. Press, Cambridge, 2005).
  2. J. B. Weiner, in Human and Ecological Risk Assessment: Theory and Practice, D. D. Paustenbach, Ed. (Wiley,Hoboken, NJ, 2007), chap. 32.
  3. Friends of the Earth, International Center for Technology Assessment, and ETC Group, The Principles for the Oversight of Synthetic Biology (Friends of the Earth, Washington, DC, 2012).
  4. NASEM,Gene Drives on the Horizon: Advancing Science, Navigating Uncertainty, and Aligning Research with Public Values (NASEM Washington, DC, 2016).
  5. J. Wolff, Hast. Cent. Rep. 44 (suppl. 5), S27 (2014).
  6. A. Stirling, EMBO Rep. 8, 309 (2007).
  7. E. C. Parke, M.A. Bedau, inThe Ethics of Protocells: Moral and Social Implications of Creating Life in the Laboratory, M.A. Bedau, E. C. Parke, Eds. (MIT Press, Cambridge, MA, 2009), chapter 5.
  8. S. M. Gardiner, J. Polit. Philos.14, 33 (2006).
  9. D. A.Kysar, J. Land Use Environ. Law22, 1 (2006).
  10. D. M. Kahan, P. Slovic, D. Braman, J. Gastil, Harv. Law Rev. 119, 1071 (2006).
  11. A. Finkel, Ann. N.Y. Acad. Sci.1128, 121 (2008).
  12. Atomic Energy Commission, “In the matter of J. Robert Oppenheimer: Transcripts of hearing before Personnel Security Board” (Government Printing Office, Washington, DC, 1954).

ACKNOWLEDGMENTS

The authors are some of the members and staff of the committee that authored the report discussed in this paper (4). The opinions in this essay are those of the authors and may not reflect the view of NASEM or all members of the National Academies committee on gene drive research in nonhuman organisms. G.E.K. was supported by NSF grant 1353433.

710 11 NOVEMBER 2016 • VOL 354 ISSUE 6313

Published byAAAS on November 12, 2016 http://science.sciencemag.org/

Featured photo: Antipodes Island landscape. Credit: Jason Zito/Island Conservation