            2014 Compliance Recertification Application (CRA-2014) 
       Compliance Application Review Document (CARD for Section 194.26)
                                Expert Judgment

26.0	Background

 	The requirements of Section 194.26 apply to expert judgment elicitation, which is a process for obtaining data directly from experts in response to a technical problem.  Expert judgment is typically used to elicit two types of information: numerical values for parameters that are measurable only by experiments that cannot be conducted due to limitations of time, money, and physical situations; and essentially unknowable information, such as which features should be incorporated into passive institutional controls to deter human intrusion into the repository.  The U.S. Environmental Protection Agency (EPA or Agency) prohibits expert judgment from being used in place of experimental data, unless the U.S. Department of Energy (DOE or Department) can justify why the necessary experiments cannot be conducted.  Expert judgment may be used to support a compliance application, provided that it does not substitute for information that could reasonably be obtained through data collection or experimentation.  Expert judgment may substitute for experimental data in those instances in which limitations of time, resources, or physical settings preclude the successful and timely collection of data.  EPA evaluates compliance with Section 194.26 by ensuring that all the steps and requirements, as described in Section 194.26, have been followed in obtaining and relying upon expert judgment for the Waste Isolation Pilot Plant (WIPP).

26.1  Requirements

	(a) "Expert judgment, by an individual expert or panel of experts, may be used to support any compliance application, provided that expert judgment does not substitute for information that could reasonably be obtained through data collection or experimentation."

	(b)  "Any compliance application shall:
	
		(1)  Identify any expert judgments used to support the application and shall 			identify experts (by name and employer) involved in any expert judgment 				elicitation processes used to support the application.

		(2)  Describe the process of eliciting expert judgment, and document the results of 		expert judgment elicitation processes and the reasoning behind those results.  			Documentation of interviews used to elicit judgments from experts, the questions 			or issues presented for elicitation of expert judgment, background information 			provided to experts, and deliberations and formal interactions among experts shall 		be provided.  The opinions of all experts involved in each elicitation process shall 			be provided whether the opinions are used to support compliance applications or 			not.

		(3)  Provide documentation that the following restrictions and guidelines have 			been applied to any selection of individuals used to elicit expert judgments:
	
			(i) Individuals who are members of the team of investigators requesting 				the judgment or the team of investigators who will use the judgment were 				not selected; and
                  
			(ii) Individuals who maintain, at any organizational level, a 						supervisory role or who are supervised by those who will utilize 					the judgment were not selected.

		(4)  Provide information which demonstrates that:
	
			(i) The expertise of any individual involved in expert judgment elicitation 				comports with the level of knowledge required by the questions or issues 				presented to that individual; and
                  
			(ii) The expertise of any expert panel, as a whole, involved in expert 				judgment elicitation comports with the level and variety of 						knowledge required by the questions or issues presented to that 					panel.

		(5)  Explain the relationship among the information and issues presented to 			experts prior to the elicitation process, the elicited judgment of any expert panel 			or individual, and the purpose for which the expert judgment is being used in 			compliance applications(s).

		(6)  Provide documentation that the initial purpose for which expert judgment was 		intended, as presented to the expert panel, is consistent with the purpose for which 		this judgment was used in compliance application(s).

		(7)  Provide documentation that the following restrictions and guidelines have 			been applied in eliciting expert judgment: 
	
			(i) At least five individuals shall be used in any expert elicitation process, 				unless there is a lack or unavailability of experts and a documented 				rationale is provided that explains why fewer than five individuals were 				selected. 
                   
			(ii) At least two-thirds of the experts involved in an elicitation shall 				consist of individuals who are not employed directly by the 						Department or	by the Department's contractors, unless the 						Department can demonstrate and document that there is a lack or 					unavailability of qualified independent experts.  If so 						demonstrated, at least one-third of the experts involved in an 					elicitation shall consist of individuals who are not employed 					directly by the Department or by the Department's contractors."

	(c)  "The public shall be afforded a reasonable opportunity to present its scientific and technical views to expert panels as input to any expert elicitation process."

26.2  1998 Certification Decision

	To meet the requirements of 194.26, EPA expected DOE to identify places in the Compliance Certification Application (CCA) where expert judgment was used and to describe why it was being used.  EPA expected DOE to thoroughly document the expert judgment panel process and participants. 

	In the CCA, DOE did not identify any formal expert judgment activities.  However, during EPA's review of the performance assessment parameters EPA required DOE to apply the expert judgment process to obtain a value for the waste particle size distribution parameter.   DOE conducted and documented the expert judgment elicitation for waste particle size distribution in May 1997.

	EPA observed the expert judgment elicitation, conducted an audit of the supporting documentation, and considered public comments.  EPA concluded that DOE complied with the requirements of 194.26 in conducting the required expert elicitation. 

	A complete description of EPA's 1998 Certification Decision for Section 194.26 can be obtained from EPA Air Docket, A-93-02, Items V-A-1 and V-B-2.

26.3  Changes In the 2004 Compliance Recertification Application (CRA-2004)

	The CRA-2004 did not identify any expert judgment activities that were conducted since the 1998 Certification Decision.

26.3.1  Evaluation of 2004 Compliance for Recertification

	EPA's evaluation of the CRA-2004 did not identify any new areas where expert judgment was or should have been used in demonstrating compliance.

	EPA did not receive any public comments on DOE's continued compliance with the expert judgment requirements of Section 194.26.

26.3.2  2004 Recertification Decision

	Based on a review of the CRA-2004, and supplemental information provided by DOE (FDMS Docket ID No. EPA-HQ-OAR-2004-0025, Air Docket A-98-49), EPA determined that DOE continued to comply with the requirements for Section 194.26.

26.4  Changes In the 2009 Compliance Recertification Application (CRA-2009)

	The CRA-2009 did not identify any expert judgment activities that were conducted since the 1998 Certification Decision and the 2004 recertification.

26.4.1  Evaluation of 2009 Compliance for Recertification

	EPA's evaluation of the CRA-2009 did not identify any new areas where expert judgment was or should have been used in demonstrating compliance.

	EPA did not receive any public comments on DOE's continued compliance with the expert judgment requirements of Section 194.26.

26.4.2  2009 Recertification Decision

	Based on a review of the CRA-2009 documents, and supplemental information provided by (FDMS Docket ID No. EPA-HQ-OAR-2009-0330, Air Docket A-98-49), EPA determined that DOE continued to comply with the requirements for Section 194.26.

26.5  Changes In the 2014 Compliance Recertification Application (CRA-2014)

	The CRA-2014 did not identify any expert judgment activities that were conducted since the 2009 CRA.

26.5.1  Evaluation of 2014 Compliance for Recertification

	EPA's evaluation of the CRA-2014 did not identify any new areas where expert judgment was or should have been used in demonstrating compliance.

	EPA did not receive any public comments on DOE's continued compliance with the expert judgment requirements of Section 194.26.

26.5.2  2014 Recertification Decision

	Based on a review and evaluation of the CRA-2014 documents (i.e. DOE Section 26), and supplemental information provided by (Docket ID No. EPA-HQ-OAR-2016-0609), EPA determines that DOE continues to comply with the requirements for Section 194.26.

