Review included multiple steps, opportunities for feedback
A peer review process was a critical component in developing the final EPA Bristol Bay Watershed Assessment. As Pebble Watch wrote in our December 2011 newsletter: For EPA, peer review “is a process for enhancing a scientific or technical work product so that the decision or position taken by the Agency, based on that product, has a sound, credible basis.”
Here, we review what that peer review process was like for the Bristol Bay Watershed Assessment and recap some basics: Who were the peer reviewers? What were their main concerns? How well were their comments addressed? Read on for our summary, and follow the links to read the original materials.
Selection of panelists
Using independent contractor Versar, EPA first obtained public input to build a pool of 68 possible panelists. They then selected 12 independent scientists from this pool to serve on the peer review panel. The reviewers, all PhD scientists from universities and agencies around the nation, were selected based on factors including areas of expertise and a lack of conflicts of interest. (See a complete list of panelists below.)
The process
1st draft: Panelists initially provided overall impressions of the May 2012 draft assessment, answers to 14 “charge questions” asked by EPA, and specific observations. In August 2012, they met in Anchorage, listened to concerns from the public, and held both open and closed deliberations.
2nd draft: EPA took several months to incorporate revisions, publishing the second draft of the assessment in April 2013. The agency also provided a written response explaining how comments from the public and panelists had been addressed. Although EPA had not intended to open the document back up to public review, public interest was high enough to warrant a second public input period. (Both public input periods garnered more than one million comments, landing the draft versions of the Bristol Bay Watershed Assessment in the “Top 10” of most-commented on EPA documents.) The same 12-panel review team also reviewed the second draft.
Final version: EPA considered public and peer review comments in completing the final assessment. Reviewer comments and EPA’s responses to them are presented in a 400-page “Response to Peer Review Comments.” (A similar document responding to more than 5,000 public comments is forthcoming.)
Comments
The panelists’ comments were extensive and ranged from very general to highly specific. EPA addressed each one, and in most cases made substantive changes. In the “Response to Peer Review Comments” reviewers appeared generally satisfied with the changes.
Following are a few examples of the many key recommendations made by reviewers, along with associated EPA responses:
Reviewer Comment | EPA response | Reviewer counter-response |
| The assessment should be framed in terms of not just mine failures caused by catastrophic events, but by day-to-day operations as well. | Made many substantive changes, reorganized the document and added specific language to clarify that the assessment includes both catastrophic events and normal operations in their evaluation of impacts. | Satisfied |
| Use a broader range of mine sizes to describe potential impacts. | Added a third mine size scenario to represent world-wide median-sized porphyry copper mine. | Satisfied |
| Include evaluation of impacts due to exploratory processes. | Added some information about exploration, but asserted that it is outside the scope of the assessment | Some disagreed with this assertion, others satisfied. |
| Include transportation corridor in cumulative risks. | Expanded cumulative risk section to include transportation corridor. | Satisfied |
| Discuss other fish and aquatic life besides salmonids, including those that are integral to salmon food webs. | Included more information, but explained that a comprehensive discussion of other species is outside the scope of the document. | Satisfied. |
| Explain what is the acceptable level of uncertainty. | Not possible because the assessment is intended only for scientific background, not for decision making. | Accepted. |
Panel members, Affiliations & Expertise
Mr. David Atkins, Watershed Environmental, LLC (mining and hydrology)
Mr. Steve Buckley, WHPacific (mining and seismology)
Dr. Courtney Carothers (indigenous Alaskan cultures)
Dr. Dennis Dauble, Washington State University (fisheries biology and wildlife ecology)
Dr. Gordon Reeves, USDA Pacific Northwest Research Station (fisheries biology and aquatic biology)
Dr. Charles Slaughter, University of Idaho (hydrology)
Dr. John Stednick, Colorado State University (hydrology and biogeochemistry)
Dr. Roy Stein, Ohio State University (fisheries and aquatic biology)
Dr. William Stubblefield, Oregon State University (aquatic biology and ecotoxicology)
Dr. Dirk van Zyl, University of British Columbia (mining and biogeochemistry)
Dr. Phyllis Weber Scannel (aquatic ecology and ecotoxicology)
Dr. Paul Whitney (wildlife ecology and ecotoxicology)
Read more
Pebble Watch coverage of the peer review panel meetings in Anchorage, 2012