
External Peer Review for EPA's
Draft Definition of Solid Waste Final Rule Environmental Justice Analysis


Contract No.: EP-W-10-055
Task Order EP-G11S-00004
                                       
                                       
                                       
                                       
Submitted To:
Tracy Atagi
Office of Solid Waste and Emergency Response
U.S. Environmental Protection Agency
1200 Pennsylvania Avenue
Washington D.C. 20460





Submitted by:
Eastern Research Group, Inc.
110 Hartwell Avenue
Lexington, MA  02421-3136



February 9, 2011
                                       
                                       
                                       
                                       
Printed on Recycled Paper


Contents
Introduction	1
Background	1
Peer Review Process	2
Overview of Reviewers' Comments	3
Responses to Charge Questions	5
General	7
Ga.	Please state your overall assessment of the content, readability and utility of this report, including any changes needed.	7
Gb.	Is this information accurately and clearly presented? How might the document be improved?	9
Gc.	Does the document adequately review, present, analyze and summarize the available data? If not, what can be improved?	10
Executive Summary	11
ESa.	Is the Executive Summary clear and easy to read?	11
ESb.	Does the Executive Summary provide an accurate view of the contents, findings, and conclusions of the analysis? If not, please specify how it could be improved.	12
1.	Introduction	13
1a.	Does the Introduction clearly lay out the contents of the analysis? If not, please describe how it could be improved.	13
2.	Hazard Characterization	14
2a.	Do the potential scenarios under the definition of solid waste final rule clearly describe how EPA thinks facilities may react to the rule? If not, please suggest where improvements should be made.	14
2b. 	Are these scenarios plausible, and are there any other likely scenarios? Please explain.	15
2c.	Is EPA's analysis of the types and quantities of hazards from the definition of solid waste final rule thorough and accurate? Please explain.	16
2d.	Is EPA's analysis of the differences between hazardous waste regulations and the regulations in the definition of solid waste accurate? If not, please explain.	17
3. 	Identification of Potentially Affected Communities	19
3a.	Has EPA correctly identified the facilities that may be likely to begin recycling hazardous secondary materials under the definition of solid waste exclusions? If not, what facilities are missing?	19
3b.	Is EPA's use of a threshold of 25 tons of recyclable hazardous secondary material generated annually an appropriate way of identifying generators that might choose to recycle onsite?  If not, what threshold would be more appropriate?	20
3c.	Has EPA appropriately identified the communities and populations that are potentially affected by these facilities?  In particular is the 3-km circle the appropriate area of consideration?  Please explain.	20
4.	 Analysis of Demographics of Potentially Affected Communities	21
4a.	Are EPA's data sources for its demographic analysis appropriate? Should EPA consider any additional data sources for this information? Please explain.	21
4b. 	Did EPA select appropriate characteristics for analysis of environmental justice concerns in communities?	22
4c.	Was the choice of the areal apportionment method to determine the characteristics of the communities surrounding each facility appropriate and properly conducted?	23
4d. 	Is a statistical analysis of these results to determine how the demographic characteristics compare to those on a county, state, and national level, in both urban and rural communities, a scientifically appropriate way to assess the existence of environmental justice concerns in connection with this rulemaking?  Are there additional methods that should be considered for analyzing the environmental justice concerns with this rule?	24
4e. 	Is the statistical analysis in this section properly done?	25
4f. 	Are the assumptions in the analysis presented accurately and thoroughly?	26
4g. 	Are the uncertainties in the data input into the analysis appropriately accounted for and discussed?	26
4h.	Based on internal EPA review of the document, the Agency is considering additional approaches for analyzing the demographic data. The decision to conduct additional analyses will depend in part on timing and resource concerns. Which of the following analyses are most likely to yield new and/or improved information, and how would that information inform EPA's decision-making process?	27
1.	For the category of facilities from the damage case study, analyze the demographics for the timeframe the damages occurred.	27
2.	Compare potentially affected areas to non-affected areas. Compare the demographics in the 3-km concentric areas surrounding potentially affected facilities to demographics of areas in the state not containing facilities, rather than to averages for the entire state.	27
3.	Add comparison to host metropolitan areas for facilities situated in metropolitan areas.	28
5.	Other Factors that Affect Vulnerability in Communities	28
5a. 	Has EPA accurately described the other factors that may affect vulnerability in communities and their relevance to this analysis?	28
5b.	EPA has included a discussion of five broad categories of other factors that affect vulnerability in communities. Based on available information, only two of those appear to be specifically associated with the 2008 DSW rule. Is the broader discussion helpful, or would it be better to focus more specifically on the factors associated with the rule?  Please explain.	29
5c.	Are there any additional factors that should be described? If so, please describe.	30
6. 	Conclusions	30
6a.	Do the results of this analysis support EPA's conclusions?	30
6b.	Has EPA clearly and accurately explained the difference between its community-level analysis and its population-level analysis?	31
6c.	Please identify any additional information that would make a significant impact on the conclusions of this analysis.	31
Appendix A:  Charge to Reviewers	39
Appendix B: Individual Reviewer Comments	45
Dorothy M. Daley, Ph.D.	47
Deeohn Ferris, J.D.	59
Diane S. Henshel, Ph.D.	73



Introduction
Background
On October 30, 2008, the U.S. Environmental Protection Agency (EPA) published final revisions to the definition of solid waste that excludes certain hazardous secondary materials from regulation under Subtitle C of the Resource Conservation and Recovery Act (RCRA), as amended (73 FR 64663). Specifically, EPA amended 40 CFR Part 261 to provide that hazardous secondary materials (HSMs) reclaimed under the control of the generator are not solid wastes if specified requirements are met. EPA also amended Part 261 to provide that HSMs that are generated and then transferred to another person for the purpose of reclamation, including exports, are not solid waste, provided that specified conditions are met. In addition, EPA finalized other amendments to address particular issues (e.g., established standards in Part 260 to enable a person to apply to EPA for a formal determination that a HSM is not discarded [i.e., non-waste] and therefore not a solid waste). 
On January 29, 2009, the Sierra Club submitted an administrative petition to EPA requesting that the definition of solid waste (DSW) rule be repealed. The petition argued that the revised regulations are unlawful and that they increase threats to public health and the environment without producing compensatory benefits and, therefore, should be repealed. In addition, the petition disagreed with the Agency's findings that the rule would have no adverse environmental impacts, including no adverse impact to minority or low-income communities.
Under the 2008 DSW final rule, EPA determined that the conditions of the rule would prevent any increase in risk, and therefore there would be no disproportionate adverse impacts to minority or low-income communities. However, the 2008 analysis did not take into account whether the conditions of the rule would operate as effectively in the real world as the more detailed requirements of the RCRA hazardous waste regulations. 
In response to comments by Sierra Club and other stakeholders, the Agency published the Draft Environmental Justice Methodology for the Definition of Solid Waste Final Rule in January of 2010. The methodology adheres to existing Federal and EPA-specific guidance on how to incorporate EJ into the Agency's process for developing regulations and how to conduct analyses that incorporate EJ considerations. This includes, in particular, EPA's Action Development Process and Interim Guidance on Considering Environmental Justice During the Development of an Action. The methodology includes six steps, as described in the following table.
                         Summary of DSW EJ Methodology
Step 1: Hazard characterization

Includes two phases: (1) identifying potential hazards that could pose risks to human health and the environment from the recycling of hazardous secondary materials, including accidental releases of hazardous constituents; and (2) analyzing the likelihood of such hazards occurring under the requirements of the DSW exclusions, as compared to the pre-2008 DSW hazardous waste regulations.
Step 2:	Identification of potentially affected communities
Modeling the locations of facilities (including potential new facilities) that are likely to choose to take advantage of the 2008 DSW final rule.
Step 3:	Demographics of potentially affected communities
Mapping the location of the facilities modeled in Step 2 and identifying the demographics (e.g., minority population and income level) of the surrounding communities.
Step 4: Identifying other factors that affect vulnerability in potentially affected communities 
Identifying important vulnerability factors. These include factors that may increase the likelihood of "damages," the likelihood that a facility is sited within a community, or the likelihood of health risks in the event of releases. Examples include the presence of other pollution sources and any information about the public health of the surrounding population.
Step 5: Information synthesis: assessment of disproportional impact
Synthesizing all the information to characterize whether the DSW rule will facilitate the occurrence of any adverse impacts and whether some population groups (e.g., minority or low income populations) would be overrepresented in the impacted communities.
Step 6: Identification of potential preventive and mitigation strategies
Identifying potential strategies to prevent non-compliance and releases to the environment and also strategies to mitigate any impacts identified under step 5.

After receiving the petition and communicating with stakeholders, EPA committed to conducting an expanded analysis of the environmental justice (EJ) impacts of the 2008 DSW final rule. This draft analysis is presented in EPA's Preliminary Draft Definition of Solid Waste Final Rule Environmental Justice Analysis, which is the subject of this peer review.
Peer Review Process
From December 2010 to January 2011, Eastern Research Group, Inc. (ERG), an EPA contractor, organized and facilitated an independent letter peer review of  EPA's Preliminary Draft Definition of Solid Waste Final Rule Environmental Justice Analysis, following the guidelines in EPA's Peer Review Handbook, Third Edition (http://www.epa.gov/peerreview/). 
In the first stage of this process, ERG searched for and identified qualified candidates who, collectively, met the following technical selection criteria provided by EPA and who had no conflict of interest in performing the review: 
   * Knowledge, expertise, and experience with conducting Environmental Justice analyses
   * Familiarity with RCRA
   * Expertise in statistics
   * Familiarity with EPA's Solid Waste Rule
After EPA confirmed that the credentials of the pool of candidate reviewers proposed by ERG fulfilled EPA's selection criteria, ERG then selected three experts from this pool to conduct the review:
   * Dr. Dorothy M. Daley, University of Kansas
   * Dr. Deeohn Ferris, Sustainable Community Development Group, Inc.
   * Dr. Diane S. Henshel, Henshel EnviroComm Consulting and Indiana University
ERG provided the reviewers with a charge prepared by EPA (Appendix A), which asked for their individual comments on the various aspects of the document, and with review instructions prepared by ERG. Reviewers were also provided with EPA's Interim Guidance on Considering Environmental Justice During the Development of an Action, as well as a link to EPA's web site (http://www.epa.gov/epawaste/hazard/dsw/rulemaking.htm) for additional background information.
Reviewers then worked independently over a four-week period to prepare individual written comments, which they submitted to ERG. ERG checked the reviewers' written comments to ensure that each reviewer had provided a substantive response to each charge question within their area of expertise. ERG did not edit the reviewers' comments in any way, but did format reviewers' comments as needed for consistency in this final peer review summary report. 
Reviewer comments are compiled by charge question in the next section and are included in this report, as received, in Appendix B. These comments reflect the individual opinions of the reviewers.
Overview of Reviewers' Comments
The reviewers thought that the Draft Definition of Solid Waste Final Rule Environmental Justice Analysis adequately reviews, presents, analyzes, and summarizes the available data. 

All reviewers offered suggestions for improving the readability of the document, including adding a glossary and list of abbreviations and acronyms, making the transition from different sections more clear, and better explaining the figures. Two of the three reviewers said that the potential regulatory scenarios were plausible. The third reviewer noted that the "unlikely" scenario of a generator switching from managing hazardous secondary material within the United States to exporting it for recycling under the DSW rule was not necessarily unlikely and thought that the potential impacts in foreign countries should be discussed. The reviewer also offered two additional scenarios for EPA to consider: (1) a generator that consistently functions in violation of the regulations and (2) a generator that goes out of business after accumulating hazardous secondary materials. 

One reviewer noted that the document did not include any discussion or analysis of the implications of each assumption for total exposure and estimated changes in risk values for the potential environmental justice communities. Another reviewer suggested taking the example situations and quantifying the potential impact on the neighboring community. 

The reviewers offered suggestions for improvements, but generally agreed with EPA's analysis of demographics of potentially affected communities. Two reviewers said that comparing potentially affected areas to non-affected areas would be useful. Two reviewers said that comparing host metropolitan areas for facilities would be useful. One reviewer suggested that at least one other stressor, such as TRI facilities or road density, should be added to the analysis. One reviewer suggested assessing the environmental justice impacts of the baseline (RCRA's approach to HSM recycling and disposal).









Responses to Charge Questions


General
Ga.	Please state your overall assessment of the content, readability and utility of this report, including any changes needed. 
Reviewer
Comments
Daley
My overall assessment is that the report manages to present a very complex decision making process and analysis in a fairly clear and accessible fashion. The discussion of potential hazards and exposure pathways seems clear and complete. The eight scenarios are clearly laid out. There report is quite detailed. The main points seem to be that the 2008 Draft Solid Waste Final Rule (DSW) enables generators to store more hazardous secondary material (HSM) for a longer period of time, and provides no prescriptive standards to ensure safe containment. Transporters of waste and intermediate and reclamation facilities are also impacted in ways not necessarily analyzed in the initial consideration of the rule change. Decreased transparency and public participation provisions combined with deregulation of hazardous secondary material to encourage reclamation is likely to have differential impacts across the county. The findings from this report present the EPA with a challenge. Recycling hazardous secondary material presents significant health and environmental hazards, and the 2008 rule change is not uniformly health protective. According to this analysis, some communities will be more impacted than others, and this differential impact has direct environmental justice implications. 
To fully understand the differential impact of the 2008 DSW final rule on low income and minority communities and populations, it would be helpful to have a meaningful baseline analysis. What if continuing to regulate HSM under RCRA has even greater environmental justice impacts?  The report and analysis do not provide that information. So, there is no way to identify if the 2008 DSW rule  -  while likely to have disproportional impacts  -  is an improvement over current reclamation and disposal practices under RCRA. I recognize this analysis would entail considerable work, but if HSM regulated under RCRA currently has even greater environmental justice impacts, then this would be very important to know this as decision making proceeds. 
Ferris
Generally, the DSW Rule is read-able and clear.
Henshel
   a.    Content: The environmental justice assessment process undergone was a reasonably thorough approach to assessing environmental justice concerns associated with the 2008 DSW rule.  However, there was a lot of repetition that seemed to involve just copying and pasting the same statements and phrases over and over again, and rarely elaborated in depth.  In addition, the analysis never included taking the example situations (places where greater exposure by the neighbors could occur, for example) and quantifying the potential impact on the neighboring community (maybe within the 3 km diameter defined "proximity to site" area).  Using the basic approaches for predictive risk assessment, one would take the real situation, make assumptions for the average and high end of the range for the increased exposure situation, calculate the potential increased exposure to the chemicals, and then calculate the potential for increased risk, with average and high end assumptions.  At least one such scenario calculated for each situation identified as potentially increasing exposure under the new regulations would be helpful in determining just how critical revising the regulations would be.  It would have been even better to then recalculate the exposure and risk inherent in at least a few scenarios with the assumptions made under the recommendations for revision of the 2008 DSW rule, weak and non-specific as those recommendations were. 
   b.    Readability.  Unfortunately, this document is pretty standard for an EPA document.  If the audience is only technical, the document is usable and readable. If the audience is intended to be the EJ communities and include a non-technical audience, this document is dense and difficult to read.  There is also a lot of "spaghetti" logic  -  explanations that are internally referencing without sufficient definition of terms or acronyms.  The document is missing some key components that would make it more readable.  These include: 
   * A Glossary
   * A list of abbreviations and acronyms
   * Remembering to define every acronym the first time it is used in every chapter, and in the legend for every table.
   * A few diagrammatic algorithms that could help explain two complexities:
      * The regulations that apply under each condition that increases exposure under the new 2008 DSW.  This algorithm could be bilaterally symmetrical  -  taking the reader through the process, and when there is a change that occurs due to the regulations, the algorithm can identify the regulations in play and not in play for each set of exposure conditions (as in longer stored waste materials, transport exposure, etc).  One side of the symmetrical algorithm would identify regulations in play before the new 2008 DSW regulations, and the other side would identify the regulations in play after the new 2008 DSW regulations.
      * A second algorithm that would be useful would be to outline the framework, the process used to analyze the environmental justice concerns about the 2008 DSW rules.  This algorithm could then tersely identify each step in the process and the implications for the outcomes of each step.
   c.    Utility.  This analysis and report does demonstrate that there already is a bias in the location of the facilities that already produce hazardous waste, and this includes a bias in the facilities that might utilize this rule to increase storage time and volume of exempted otherwise hazardous materials.  The environmental justice analysis could have been more complete.  Further discussion of other approaches are included below. The report does make reasonably recommendations for some of the ways that the environmental justice concerns might by reduced, although the discussion of these abbreviated comments was both vague and needed elaboration. 
Gb.	Is this information accurately and clearly presented? How might the document be improved?
Reviewer
Comments
Daley
For the most part, the information in the report is clearly presented. There are, however, some areas that could be improved. While Figure 1 is informative, the other figures in the report make little sense to me. They either need additional explanation, or they should be removed. Figure 4.41.1 is an example of the type of bar chart that can be confusing. Perhaps if the base category of Whit/Non-Hispanic was added, the figure would make more sense. As it is now, these figures detract rather than add to the report. 
Why are urban and rural comparison omitted for the damage case facilities?  Consistent analysis across categories is important. If the analysis cannot be performed for some reason, an explanation in the text would be helpful. 
A list of acronyms would be extremely helpful. Also, there are also several typos and minor mistakes that should be addressed. These include: 
   (1) Page iv  -  under the hazardous waste baseline, second row, "recycles" should be changed to "recycling"
   (2) Page 2, sec 1.3, bullet 1  -  "HSMHSMs" should be changed to "HSM"
   (3) Page 10, top of the page: "as a focus" is written twice. One occurrence should be deleted. 
   (4) Page 81, section 4.2.3  -  "area" should be added after "a 3 km"
   (5) Page 91, section 4.42  -  lists 219 damage case facilities. There are only 217 reported in the analysis. Either explain the missing cases, or fix a typo. 
   (6) Attachment A,  page A-2, section 1. Toward the end of the first paragraph, should read "regulation if" not "regulation is". 
Ferris
Generally, information in the DSW Rule presented is clear and appears to be accurate.
Henshel
   a.    Needed improvements for clarity: As mentioned above, it would be very helpful for the general reader to provide a Table of Abbreviations/Acronyms and a Glossary, in addition to remembering to follow some other basic rules of clear communication as outlined above (Readability).   In addition, the excessive use of acronyms makes the document very hard for any lay reader to read.  It makes the document hard to read for even an expert, as one must constantly look for the meaning of the new acronyms that pop up pages after they have been defined.  It would be helpful, should the acronyms stay in such a density, to provide a list of acronyms used on each page in a footnote at the bottom of each page.
   b.    The Wind Dispersal category would be better (and less confusingly) labeled as "Particulate [or Dust] Wind Dispersal."
   c.    There is no need to repeat the exact same phrases multiple times without additional elaboration.
   d.    In no scenario was there a consideration of generators who consistently function in violation of the regulations once they start handling waste under the 2008 DSW rule.  As there are such companies, this would seem to be a scenario to include, or at least discuss, in an EJ analysis. 
   e.    No scenario considered the possibility of the impact on the community if a generator acting under the 2008 DSW rules goes out of business with accumulated, non-processed, possibly not appropriately contained excluded hazardous material waste on the property.  Yet this situation would only happen (legally) under the 2008 regulations, not under the previous HSM regulations. 
   f.    The methods for analysis of environmental justice impacts (as in biases in the siting) are not  as sensitive as might be obtained with using regression analysis and grouping the data into maybe quartiles rather than doing all analyses on data that has only been divided into two categories  (ie equivalent of yes/no data).
Gc.	Does the document adequately review, present, analyze and summarize the available data? If not, what can be improved?
Reviewer
Comments
Daley
It seems like the document is quite thorough. As noted in my response to question Ga, the document would be improved by a assessing the Environmental Justice impacts of the baseline  -  or RCRA's approach to HSM recycling and disposal. 
Ferris
The DSW Rule is an extensive document with significant analysis and extensive data.
Henshel
   a.    This is a nice review of the data.  This is a nice first analysis.  Improvements can be made.  This question is a perfect example of the redundancy that could be avoided.


Executive Summary
   ESa.	Is the Executive Summary clear and easy to read?
Reviewer
Comments
Daley
Considering the complexity of the document, the Executive Summary is relatively clear and easy to read. The "Summary of Potential Impacts" is very helpful. It is important to note, however, that the findings in the executive summary are not properly numbered: the executive summary lists findings 1, 3, and 4, omitting 2. The table and paragraph describing "Community Level Analysis of Potential Disproportionate Impacts" is difficult to follow. It would be helpful if the Environmental Protection Agency (EPA) could identify and describe the four types of facilities in this analysis more consistently. For example, the text on page vii could be edited to include the following language connecting the text to the community analysis tables: 
	   The second step of the methodology identified facilities that can represent the facilities that are likely to take advantage of the 2008 DSW final rule. These facilities are grouped into four different categories: (1) facilities that have already notified under the 2008 DSW final rule[Notification facilities], (2) hazardous waste facilities that are likely to recycle under the rule (including hazardous waste generators producing more than a truckload (25 tons) of recyclable hazardous secondary materials annually, and hazardous waste recyclers) to use the rule [Hazardous Waste Facilities], (3) facilities from the environmental problems study (many of which operated under exclusions or reduced regulations)[Damage Facilities], and (4) facilities currently recycling non-hazardous industrial waste (e.g., antifreeze) that could most easily switch or expand to recycling under the 2008 DSW final rule [Non-Industrial Hazardous Waste Facilities].
It is difficult to understand where the categories in the Table on page X come from. They should be clearly labeled in the text and - table categories: the ordering in the text and the table should be consistent. Also, it would be helpful to define "damage facilities". 
Ferris
The Executive Summary is very clear and readable. 
The material is complex and dense which is to be expected in an agency rulemaking. 
Readers reviewing the synopsis of the Sierra Club petition in this summary (or, in the relevant section below) would be informed by an abbreviated specific list of issues identified by the Club (instead of the more generalized ""raised a number of issues about the protectiveness of the rule...' -- especially since the language of the rule emphasizes that a number of commenters "echoed"...concern..."
Summary of DSW EJ Methodology is clear and concise with a couple of exceptions. The term "damages" in Step 4 could be explicitly defined. Presumably, the language that follows the term "damages" is explanatory but that is somewhat unclear. 
Does "Information" in Step 5 include data?  Perhaps strengthening the language to "Data and Information..." is more supportive of the Agency's rationale.
Henshel
   a.    The executive summary is readable.  The executive summary needs to include definitions of terms used (for example, "notification facilities," "damage case facilities," etc).  In addition, the last sentence on page ix (starts "In addition,...") is unclear and needs  rewriting.  Similarly the last sentence on page xi needs additional clarity.  The summary of Finding 4 should include some discussion of the other three categories not discussed in the executive summary (example, susceptible populations).  You might include leaking septics in the list on the top of page xiii (under multiple and cumulative effects), as leaking septics are more likely to be addressed more slowly, and thus pose a higher health risk, in low income communities.
   ESb.	Does the Executive Summary provide an accurate view of the contents, findings, and conclusions of the analysis? If not, please specify how it could be improved.
Reviewer
Comments
Daley
Yes, the Executive Summary provides an accurate summary of the more detailed material presented in the report. 
Ferris
The "Summary of Potential Impacts...Under Different Recycling Scenarios" is clear however the accuracy of the presumptions is challenged by the language on its face. Pursuant to Hazard Characterization, EPA demonstrates and discusses the highly hazardous nature of the 2 main sources of reclaimed hazardous secondary materials (hereafter HSM) upon which much of the Draft DSW Rule analysis is based in addition to the storage hazards including leaks, fires and explosions. Container standards do not appear to compare with the highly hazardous nature of the 2 main sources upon which much of the Draft DSW Rule analysis is based. 
The Scenarios chart shows several instances involving movement or activity related to HSM where it appears that various historic RCRA safeguards are not required see e.g., certain scenarios under Scenario 3; Scenario 5; Scenario 6; Scenario 7; Scenario 8) without, it could be said, on balance, assuring a regulatory atmosphere that influences the regulated community to handle HSW in ways that prevent hazards. Comparatively, streamlined notice and recordkeeping would both facilitate emergency planning and response and protect human health and the environment. 
The benefits (per language in the Scenarios) don't balance the potential negative impacts. This could be more clearly set forth or demonstrated. Although the reviewer does not express agreement that any benefits would offset the potential hazards or actual hazards, more clarity or quantification of benefits would strengthen the analysis.
Does EPA plan to establish a definition or explanation of what constitutes legitimate recycling and/or recycler-facility?  This could serve as notice to generators, receivers, recyclers that the exemptions and exclusions are accompanied by affirmative duties that include compliance with statutes and regulations?
Regarding DSW Rule controversies about disproportionate negative impacts and exposures, would these communities feel more protected (versus adding to the burden of already cumulative pollution) if HSW were handled similar to those substances categorized under Universal Wastes?
Henshel
   a.    See above comments.  But yes, the Executive Summary does provide a reasonably accurate view of the findings and conclusions of this analysis.
1.	Introduction
1a.	Does the Introduction clearly lay out the contents of the analysis? If not, please describe how it could be improved.	
Reviewer
Comments
Daley
Yes, the Introduction is clear. It is easy to follow and well organized.
Ferris
Summary of DSW EJ Methodology is clear and concise with a couple of exceptions. 
The term "damages" in Step 4 could be explicitly defined. Presumably, the language that follows the term "damages" is explanatory but that is somewhat unclear. 
Does "Information" in Step 5 include data?  Perhaps strengthening the language to "Data and Information..." is more supportive of the Agency's rationale.
Readers reviewing the synopsis of the Sierra Club petition would be informed by an abbreviated specific list of issues identified by the Club (instead of "raised a number of issues about the protectiveness of the rule..." 
1.3  minor typo -- see "HSMHSMs"
In view of the complexity of what is explained, Scenarios are clear.
In addition to the advantage of adhering to RCRA's decades old cradle-to-grave protective framework, consider whether requiring a streamlined notice and record-keeping process for generators, receivers and recyclers of HSM benefit EPA and states e.g., monitoring, as well as data that shows the environmental and economic benefits of HSW exclusions?
The threat of post facto CERCLA liability is not interchangeable with up-front environmental protection standards and requirements. This reviewer does not presume that the entire regulated community is villainous; however, any vigor/rigor that a generator, receiver, recycler might invest in environmental protection could wane as oversight and compliance requirements are reduced or eliminated. 
Henshel
   a.    No complaints.
2.	Hazard Characterization
	
	Question 2 Introductory Remarks

Reviewer
Comments
Ferris
The transition to this section should be clear.  It's not clear that this section is distinct from the prior rule discussion (the "Introduction").  This is a graphics issue.
The discussion of hazards associated with mainly the 2 selected vectors is clear. Hazards are extensive in contrast to reducing the levels of regulatory oversight provided by RCRA. Should all 4 legitimacy factors should be mandatory in view of the hazards?  What about release reporting?  Should releases be grounds for losing exclusion and HSW status?  How is a significant release determined?
In view of the stated hazards, could managing HSW as a valuable commodity influence the generator, receiver, recycler and affect compliance?  This goes to the concept of legitimacy. Would the mandating a showing of economic value militate in favor of safeguards against releases, spillage?
If wastes that are going to be recycled as hazardous as wastes that are not recycled, are both equally likely to be mishandled?  Are the regulatory and enforcement tools provided by the DSW Rule commensurate with the task of ensuring compliance?
2a.	Do the potential scenarios under the definition of solid waste final rule clearly describe how EPA thinks facilities may react to the rule? If not, please suggest where improvements should be made. 
Reviewer
Comments
Daley
Yes, the scenarios clearly define how EPA thinks facilities may react to the rule. Table 2.1 is particularly useful in outlining the baseline and potential changes based on the DSW Exclusions. I would however, like to know how the EPA developed these scenarios. Are these based on professional experience, or did the agency talk with the regulated community to get a sense of potential reactions to the DSW Exclusions?   If it is the latter, could the report provide more detailed information regarding the feedback from the regulated community?  If it is the former, it would add value to outline some of the ways in which that experience shaped the discussion / selection of the scenarios. 
Ferris
The description of potential scenarios is clear. 
The chart situated in the "Overview" section, which describes the 3 relevant exclusions is clear and concise. The applicability of RCRA (or not) is readily discernible from the scenarios.
Henshel
   a.    What is lacking overall from the analysis, from the Hazard Characterization step onward, is any discussion or analysis of the implications (i.e secondary consequences) of each of these assumptions (about the possibility of increased exposure) for total exposure and estimated changes in risk values for the potential EJ communities (ie those communities within 3 km of each facility, according to this report).
2b. 	Are these scenarios plausible, and are there any other likely scenarios? Please explain. 
Reviewer
Comments
Daley
These scenarios seem plausible. They are described in clear and logical detail. 
Ferris
The Scenarios are plausible. 
Is there a significant possibility that small businesses could be more vulnerable to sham recycling if many won't have the capacity and staffing to thoroughly assess recyclers?
The DSW Rule could inadvertently encourage mismanagement while transitioning materials into the recycling stream. Does the Rule apply where HSW is recycled into even more toxic products? 
Should notice be the hazard quid pro quo for the exclusions which provide the benefit of different (reclamation), streamlined treatment of HSW?
Without adequate record-keeping and environmental protection safeguards (streamlined requirements would not be objectionable), the international export exclusion/scenario is troubling re potential releases, impacts, whether or not the recipient nation has genuine knowledge of the contents and consents to the receiving the export; see e.g., Circle of Poison and pesticide regulation. 
Henshel
   a.    "Unlikely" scenario 1: If the cost of recycling off-shore (Africa, China) are low enough, these lower costs will offset the additional costs of transportation to get the now excluded hazardous waste (which costs less also as an excluded waste, and one for which the same expensive containers do not have to be bought).  This is a reasonable scenario (not unlikely) unless the EPA provides calculations to prove that the costs of transportation would never be overcome by the reduced costs of recycling in Africa or China.  It is also disturbing that there is insufficient discussion and analysis of EJ concerns for the locations of the recycling plant and no discussion and analysis of EJ concerns for the locations where recycling is carried out in another country.  (This last sentence applies as well to the whole report.)
   b.    Two other possible scenarios to address are listed above under "general comments".
2c.	Is EPA's analysis of the types and quantities of hazards from the definition of solid waste final rule thorough and accurate? Please explain.
Reviewer
Comments
Daley
Yes, EPA's analysis of the types and quantities of hazardous secondary material is clear. Figure 1 provides a clear justification for the focus on solvents and electric arc furnace dust. As mentioned earlier, it would be helpful if the report clarified the definition of a damage case or damage facility. As noted on page 10, the analysis highlights solvent wastes because of their prevalence in damage cases. A definition and some references to RCRA damage cases would be helpful, even if provided in a footnote. 
Ferris
The selection of electric arc furnace dust and solvent wastes as national HSM proxies appears sound based on the analysis. Excluding used oil, non-ferrous metals and scrap metals from the hazard analysis, although associated with "damages cases?" 
How and when does enforcement after a "significant" release occur?  How is the determination made as to what constitutes a significant release?  Which party has the burden of proof?  It would seem that placing the burden of proof on the party that causes or is responsible for the release should bear the burden of proof opposed to, for example, the governmental entity. 
Intervening prior to a release is clearly preferable  -  the possibility of enforcement action prior to a release would be protective and more likely in the event that there are clear regulations e.g., storage limitations, labeling requirements and requirements related to the integrity of storage containers and areas.
Does worker training (e.g., protection standards, labeling, handling) or the lack of it at relevant/specified facilities factor into the EPA hazard analysis?
Does emergency response and planning, the existence or lack of these programs, factor into the EPA hazard analysis?  Similarly, does eliminating the requirement for a Manifest (or some version of a tracking mechanism) factor into possible increases or reductions in the emergency response preparedness and planning, enforcement or compliance monitoring?  
Would streamlined, but standardized HSW reporting on a Manifest or comparable document be feasible, e.g., for state and local government, the industry?
Compliance with financial assurance can be a high hurdle given conditions imposed by the environmental insurance market. All parties under this rule should be required to comply with financial assurance. Would requiring management standards for reclamation facilities that don't have RCRA permits strengthen the DSW Rule by increasing or reducing the potential for environmental releases and/or disproportionate negative impacts?
Does the EPA hazard analysis factor imposition (or not) of container integrity and container management standards into the determination of the potential or actual hazards associated with higher storage volumes, and/or higher concentrations of HSW?  What constitutes a safe "container?"  Does the analysis factor in whether making storage subject to controls, environmental monitoring and inspections increases or reduces community protection?
Intermediate facilities and off-site HSM self policing may not be effective as a means to ensure protection of health and the environment in communities whether disproportionately affected or not.
Henshel
   a.    Toxicity information: When applicable, endocrine disruption toxicity is not even mentioned.  Developmental toxicity of heavy metals would also seem to be an important, very sensitive effects indictor to mention.
   b.    p 19 top of the page. There isn't mention of fugitive emissions (leaks at valves, escape at points of transfer) as part of the way in which hazardous materials/waste can be released.
2d.	Is EPA's analysis of the differences between hazardous waste regulations and the regulations in the definition of solid waste accurate? If not, please explain. 
Reviewer
Comments
Daley
It seems as if EPA has noted many of the differences between hazardous waste regulations and the changes stemming from the 2008 Draft Solid Waste (DSW) Final Rule. Table 2.3 is a very detailed description of the eight likely scenarios and their impact on generators, transporters, and intermediate reclamation facilities. This material clearly communicates that there is likely to be substantial changes in the accumulation and storage of hazardous secondary material (HSM). 
Ferris
The threat of post facto CERCLA liability is not interchangeable with up-front environmental protection standards, notice and record-keeping requirements. Streamlined record, keeping, tracking and monitoring would contribute to ensuring that toxic wastes are not dumped illegally dumped in disproportionately affected communities and would contribute to ensuring that HSW is safely delivered to the recycling facility.
The DSW Rule could be characterized, to some extent, as industry self-certification on HSW, which contrasts sharply with RCRA's historic cradle-to-grave framework and preventative approach to hazardous waste management and disposal.
Generators and third party reclaimers must file notices  -  they should be required or the HSW should be treated as hazardous waste.
Audits of reclaimers are not the same as or superior to periodic inspections or other agency oversight. 
Henshel
Detailed Comments to Improve Writing:
   a.    Page 3 Inset.  Typo or residual undeleted "("  in explanation for Generator-Controlled Exclusion.
   b.    Section 2.3 p 30, third paragraph down  -  This paragraph needs rewriting to make it clearer.  A particularly difficult sentence is sentence 3, and the logic between sentences 3 and 4 may be faulty.  If a material's reporting quantity is lower when it is characterized as hazardous waste then if it is characterized as non-waste, as written in sentence 3, then a smaller quantity would be needed before it exceeds the reportable quantity as a hazardous waste compared to as a non-waste, not a larger quantity (as currently written in sentence 3).  But this does mean that if the implementation of the DSW results in what would otherwise be considered hazardous waste being now considered non-waste, then indeed more of the same material (now classified as "non-waste") would be required to trigger reporting requirements then would have been true if the material were classified as hazardous waste.
   c.    P 32, end of second full paragraph.  A "reasonable" effort is not defined or discussed.  Without such discussion or clarification, a "reasonable" effort remains so open to interpretation that a phone call to the foreign facility owner and asking for assurances of safety would more than satisfy that requirement, and yet not provide any real assurance that the community and workers in the foreign facility would be protected from hazardous exposures.
   d.    p 32 end of the second bullet there's a typo, a misplaced period.
   e.    As an example of a need to define terms and abbreviations, K061 is used in the footnote on page 27 (footnote 25) without any explanation of the meaning of the term.
   f.    When requirements or criteria are referred to, it is important to specify and list those requirements or criteria, in the text, a footnote, or an inserted box.  An example of such a referral to insufficiently explained and detailed requirements are on the top of page 28.  The details about the "SQG" requirements are lacking.
   g.    The discussion about reportable quantities (second full paragraph down on page 30) needs to be written more clearly.  The third sentence in that paragraph is particularly problematic.


3. 	Identification of Potentially Affected Communities

	Question 3 Introductory Remarks

Reviewer
Comments
Ferris
The transition to this section should be clear. It's not clear that this section is distinct from the prior rule discussion. This is a graphics issue.
Does the large number of damages cases support granting the transfer-based exclusion?
Diversion of materials from the waste stream is beneficial (although on it face, without safeguards, reclamation falls short of environmental justice) and should be accompanied by adequate protections for health and the environment.
3a.	Has EPA correctly identified the facilities that may be likely to begin recycling hazardous secondary materials under the definition of solid waste exclusions? If not, what facilities are missing?
Reviewer
Comments
Daley
EPA's approach to identifying facilities that are likely to begin recycling hazardous secondary material seems reasonable. I cannot think of another category to include. Out of the categories presented (already notified; damage facilities; currently generating hazardous secondary material, and facilities that may easily expand or switch to reclamation because of the DSW final rule), the last three categories seem more useful to understanding likely environmental justice impacts. With only 40 facilities providing notification at the time of the report, it is likely that "street level" implementation will be driven by the larger population of facilities rather than early adopters. 
Ferris
Are the facilities that gave notice located; in disproportionately affected communities or not?  The recycling exclusions should not exacerbate impacts or exposures in these communities. A troubling compliance history could signal future endangerment of the health and the environment.
Henshel
no comment, outside this reviewer's area of expertise

3b.	Is EPA's use of a threshold of 25 tons of recyclable hazardous secondary material generated annually an appropriate way of identifying generators that might choose to recycle onsite?  If not, what threshold would be more appropriate?
Reviewer
Comments
Daley
I cannot comment directly on whether or not the annual marker of 25 tons of recyclable material is appropriate. The report provided limited justification for this cut off point, and while it seems reasonable, someone with more applied experience in waste management and reclamation may think otherwise. 
Ferris
Is the 25-ton threshold empirical?
Henshel
no comment, outside this reviewer's area of expertise
3c.	Has EPA appropriately identified the communities and populations that are potentially affected by these facilities?  In particular is the 3-km circle the appropriate area of consideration?  Please explain.
Reviewer
Comments
Daley
EPA is defining a community as a 3 kilometer circle or buffer around one of the facilities included in this analysis. The agency then uses census information to apportion socio-economic and demographic ratios within this radius. This method is superior to simply using standard census divisions such as census tracts, blocks, or even zip code aggregation. The operational definition of "a community" is challenging and the environmental justice literature has numerous empirical studies at different levels of analysis, sometimes providing conflicting and competing results. Thus, using a measure of community with stronger measurement validity is important. Like all measurements, this is not a perfect representation of "a community", but it is a much improved measure over more traditional approaches, i.e., studies that rely upon preexisting census designations to ascertain community characteristics. Increasingly, a 3 kilometer radius is becoming a standard approach to improving the precision of community measures in environmental justice and public health research. The report however, could be edited to more clearly explain this approach. The text on Page 80 and 81 should be streamlined. First, the agency should suggest that the 3 km buffer is reasonable given the potential for acute exposure from accidental releases. Then, this should be followed by a discussion of the area apportionment method. As it reads now, it is a bit confusing. 
Ferris
EPA synchronizes with Professor Paul Mohai and Professor Robin Saha who are doing cutting edge demographic analysis. Their work shows increasing disproportionality, for example, nationally, people of color now make up the majority of those living within 3-km of such facilities and that where facilities cluster they make up over two-thirds. These are the highest concentrations of people of color ever reported from a national study of the distribution of hazardous waste facilities. Furthermore, when applying their recommended statistical analysis, it was found that racial disparities persist even when controlling for socioeconomic factors such as income, housing values, education, and employment status.
Henshel
a.	Other environmental justice analyses have used other diameters to define the proximate neighborhoods.  On studies of Indianapolis, especially, we have tested multiple diameters (all smaller than 3 km  -  0.5, 1 and 2 km) and typically found 2 km to be a good cut-off to define proximate communities.  It would seem that this question should be answered in the study, that is, that the study design would test multiple possible diameters (or donuts) around the facilities and determine which diameter provides the most sensitive distinction. It is quite possible that different types of facilities would have different cut-off diameters, based on differences in potential for dispersal/distribution of (excluded) hazardous waste (i.e. as liquid solvent waste or dust).
b.	Other problems:
In conducting similar EJ analyses, using TRI facilities, we discovered that we had to truthed the facility addresses to the GPS coordinates by on the ground determination of whether the address listed for the facility was actually the address of the releasing facility or the address of company offices.  Even when the two buildings were on the same campus, and not separated by miles, the difference between the buildings (between the addresses) could be a significant distance (on a mile long campus, they could be a mile apart), and thus could affect which neighborhoods were included in the proximate circle.  From the information provided in chapter 3 about the methods used for obtaining GPS coordinates, it appears that this truthing was never done or even attempted for this analysis.  No efforts were apparently made to ascertain whether the listed address referred to the hazardous materials processing location or to the main offices.
4.	 Analysis of Demographics of Potentially Affected Communities

	Question 4 Introductory Remarks

Reviewer
Comments
Ferris
The transition to this section should be clear. It's not clear that this section is distinct from the prior rule discussion. This is a graphics issue.
4a.	Are EPA's data sources for its demographic analysis appropriate? Should EPA consider any additional data sources for this information? Please explain.
Reviewer
Comments
Daley
The census data that the EPA uses for this analysis is standard. Improvements would include relying upon the 2010 census numbers to more accurately reflect community composition. If possible, it would be ideal to be able to rely upon the American Community Survey (ACS) to obtain more detailed and temporal information on community change. However, I am not sure that the sample size for the ACS are adequate for this facility based approach. It may be that by drilling down to the facility level and relying upon survey rather than census data, introduces too much variability. 
Ferris
EPA synchronizes with Professor Paul Mohai and Professor Robin Saha who are doing cutting edge demographic analysis. Their work shows increasing disproportionality, for example, nationally, people of color now make up the majority of those living within 3-km of such facilities and that where facilities cluster they make up over two-thirds. These are the highest concentrations of people of color ever reported from a national study of the distribution of hazardous waste facilities. Furthermore, when applying their recommended statistical analysis, it was found that racial disparities persist even when controlling for socioeconomic factors such as income, housing values, education, and employment status.
Henshel
a.	Using census data is standard protocol for many EJ analyses.
4b. 	Did EPA select appropriate characteristics for analysis of environmental justice concerns in communities? 
Reviewer
Comments
Daley
Yes. The EPA has selected appropriate characteristics to analyze in this report. I think it would be a stronger analysis if along with "children under five", "adults over 65" were included in the detailed analysis in chapter 4. Given the demographic transitions of the baby boomers into an elderly population, it will become more critical to understand differential exposure for susceptible  populations at both ends of the spectrum. 
Ferris
Is the preventative approach a reasonable starting point when evaluating harm to the environment and public health.
Without RCRA status or Treatment, Storage and Disposal (TSD) protections, communities will many facilities are not required to get a permit, provide notice and opportunity to comment, address public input, monitor or be subject to protective container standards or storage time limits. These are baseline historic protections that, while they do exist, as regards most environmental statutes, they haven't been updated to reflect the "real world" since original passage. 
Furthermore, taking advantage of existing public participation mandates commonly requires a massive investment of usually volunteer community resources to engage in a process that is time consuming and expensive and in which, comparatively, the private sector and government stakeholders have extensive resources and dedicated staff. 
Have either the absence of these protections or the "real world" limitations on a community's ability to publicly participate been evaluated as an impact upon low income communities and communities of color?
Notice requirements inform communities, the public and municipalities about how HSW is being managed and recycled. How would the public learn about releases, spillage without notice, monitoring, recordkeeping; how would they advocate safeguards or alternatives?  Streamlined requirements would be preferable to none.
In disproportionately affected communities, the issue is not whether there will be impacts; rather the issue is what the full extent of the hazard impacts will be. 
Should recyclers be required to demonstrate a history of compliance prior to benefiting from the exclusions?  Are the facilities that gave notice disproportionately located in low income and communities of color?
Potential impacts associated with transportation of HSW warrant a closer look.
Henshel
   a.    The same methods and break outs should have been carried out for all analyses.  There should not have been different protocols and analyses used for some types of facilities and not for others.  There should have been a break out of asian, black, other minority, and possibly immigrant populations for all facilities.
4c.	Was the choice of the areal apportionment method to determine the characteristics of the communities surrounding each facility appropriate and properly conducted? 
Reviewer
Comments
Daley
Yes, it is fine. It would be beneficial to have some discussion of the assumption behind this method. As noted on Page 80, the areal apportionment method assumes all populations are equally distributed in the block or block group. This assumption will not hold in practice. Therefore, some indication of how this assumption influences the analysis would be useful. Given the population distribution in the nation, how might this assumption bias the analysis?  I am not suggesting that the agency change their approach, only that they articulate the implication of this assumption. 
Ferris
EPA synchronizes with Professor Paul Mohai and Professor Robin Saha who are doing cutting edge demographic analysis. Their work shows increasing disproportionality, for example, nationally, people of color now make up the majority of those living within 3-km of such facilities and that where facilities cluster they make up over two-thirds. These are the highest concentrations of people of color ever reported from a national study of the distribution of hazardous waste facilities. Furthermore, when applying their recommended statistical analysis, it was found that racial disparities persist even when controlling for socioeconomic factors such as income, housing values, education, and employment status.
Henshel
   a.    It appears, yes.
4d. 	Is a statistical analysis of these results to determine how the demographic characteristics compare to those on a county, state, and national level, in both urban and rural communities, a scientifically appropriate way to assess the existence of environmental justice concerns in connection with this rulemaking?  Are there additional methods that should be considered for analyzing the environmental justice concerns with this rule?
Reviewer
Comments
Daley
Yes. A statistical analysis of these results is an appropriate method to understand patterns in administrative and community level data. These patterns can reflect social and environmental disparities and the EPA can consider these patterns for future decision making. Note however, that the analysis presented in Chapter 4 did not include a comparison between community (3 km buffer area) and county level characteristics. State, national, and urban and rural were examined, but county information was not analyzed in this section of the report. County level information and analysis would have added value to this report. 
Ferris
EPA synchronizes with Professor Paul Mohai and Professor Robin Saha who are doing cutting edge demographic analysis. Their work shows increasing disproportionality, for example, nationally, people of color now make up the majority of those living within 3-km of hazardous waste facilities and that where facilities cluster they make up over two-thirds. These are the highest concentrations of people of color ever reported from a national study of the distribution of hazardous waste facilities. Furthermore, when applying their recommended statistical analysis, it was found that racial disparities persist even when controlling for socioeconomic factors such as income, housing values, education, and employment status.
Henshel
   a.   Yes, statistical methods are appropriate in general.  However, a more sensitive analyses seems to be to break the demographics up (quartiles, quintiles) into some binned division of the total range, create donuts around the facility, and run the analyses as a regression of the demographics against proximity.  You are able to detect (significantly) more subtle shifting of demographics over time.
   b.   It is also appropriate to use the more commonly used cut-off of 2x the poverty line, since even families at 2x the poverty line are struggling just to provide food and shelter these days.  The census determined poverty line (that is, above or below the threshold) does not include childcare costs, as one example of the kind of critical expenditures that enable both parents to work.  Thus it would be appropriate to test poverty threshold versus 2x*poverty threshold as the criteria for determining bias in representation of low income populations with regard to proximity to the HSM or DSW facilities.
   c.   Given the neighborhoods in which these facilities are placed, it would seem to be appropriate to at least consider other toxic stressors, such as TRI facilities or road density as part of the EJ analysis looking at cumulative exposure potential (based on permit data for the HSM/potential DSW facilities for maximal exposure estimates).
4e. 	Is the statistical analysis in this section properly done?
Reviewer
Comments
Daley
The statistical analysis seems to be properly done. As far as I can tell, these are conditional probability calculations. It is a relatively straightforward and simple approach. While I am not a statistician, it seems like a reasonable way to approach the analysis. It provides the agency with two types of information: the chances that a facility exists in a buffer area given the demographic and socio-economic composition (threshold risk),  and the probability of a particular demographic or socioeconomic composition given the likelihood of facilities (the demographic ratio). 
Ferris
EPA synchronizes with Professor Paul Mohai and Professor Robin Saha who are doing cutting edge demographic analysis. Their work shows increasing disproportionality, for example, nationally, people of color now make up the majority of those living within 3-km of hazardous waste facilities and that where facilities cluster they make up over two-thirds. These are the highest concentrations of people of color ever reported from a national study of the distribution of hazardous waste facilities. Furthermore, when applying their recommended statistical analysis, it was found that racial disparities persist even when controlling for socioeconomic factors such as income, housing values, education, and employment status.
Henshel
   a.    There is a lot of detail provided for some of the analyses.  Other analyses (producing some of the results listed in the tables) do not have such clear or extensive or sometimes even any clear statement of how some calculations were made.  At the very least, a clear, jargonless description of how each column in each table was determined should be evident in the text in proximity to the tables.
   b.    There needs to be an explanation of why some Fisher's analytical results are listed as N/A.
   c.    Why are the same charts repeated twice?  Is this necessary?
   d.    The analysis should be broken out by state for all the analyses, and similarly, the notification facility results could be summed across states, as was done for the other types of facilities, even though the states covered are small in number compared to the total number of states represented in the analyses for the other facilities.  That discrepancy should be pointed out in the legend.  Yet there needs to be some way to compare across analyses.  Only summing across states for most, and separating out one subset of data makes it hard to compare across results.
   e.    The same set of demographics should be tested for all types of facilities for all types of analyses.  There is no justification for cutting back to 4 categories of analysis for some analyses, and none is given.
   f.    The first set of analyses for each facility type would be better and more clearly represented by box and whisker plots.  When using box and whisker plots, it is important to explain (for the lay audience) what data is represented in each part of the plots at some point in the chapter.  With that explanation, the box and whisker plots provide more information for the comparisons, and provide a visual sense of variability and bias around the means.
4f. 	Are the assumptions in the analysis presented accurately and thoroughly?
Reviewer
Comments
Daley
Yes, I think the assumptions are accurately presented. Footnotes 34  -  37 are important components in the analysis. The report very clearly outlines data sources and how the information is aggregated together. As already noted, the distributional assumptions of the areal apportionment method should be more fully explained.
Ferris
Assumptions appear accurate; however at this reading, this reviewer cannot determine whether they are thorough. 
Would assessment of an additional number of damages case affect the assumptions? 
Henshel
[No response provided]
4g. 	Are the uncertainties in the data input into the analysis appropriately accounted for and discussed?  
Reviewer
Comments
Daley
There is considerable selection bias or selection effect in the "notification" facilities and likely also in the damage facilities. It is likely that these early adopters (notification facilities) are significantly different (and non-randomly distributed) compared to the three remaining categories. Damage facilities are also likely to be systematically different. I am not sure exactly how to address this, but it seems like something that the agency needs to consider. Perhaps analyzing a comparison group of randomly selected and/or "matched" facilities could be useful, in addition to the broader state and national comparisons. 
Ferris
Uncertainties in the data input in the analysis appear to be appropriate.
Henshel
No  -  see comments above about problems with addresses linked to facilities that have corporate offices away from the processing facilities, even when the two buildings are on the same campus.
4h.	Based on internal EPA review of the document, the Agency is considering additional approaches for analyzing the demographic data. The decision to conduct additional analyses will depend in part on timing and resource concerns. Which of the following analyses are most likely to yield new and/or improved information, and how would that information inform EPA's decision-making process?
For the category of facilities from the damage case study, analyze the demographics for the timeframe the damages occurred.
Reviewer
Comments
Daley
See 2. below.
Ferris
The reviewer is uncertain if this analysis will yield additional useful information without an assessment of additional cases.
Henshel
Your meaning is not fully clear in this question, and it is not clear what analyses you would do to address this question, and how you would break up the data.  I am also concerned that the number of facilities in each group might then be so small as to really decrease power to detect significant differences.
Compare potentially affected areas to non-affected areas. Compare the demographics in the 3-km concentric areas surrounding potentially affected facilities to demographics of areas in the state not containing facilities, rather than to averages for the entire state. 
Reviewer
Comments
Daley
Option 2 has the strongest potential to add to the ability of the agency to draw inferences from their data sources. If statistical analysis indicates systematic socioeconomic and demographic differences in area with and without facilities, then this provides even stronger evidence of the distributional affects of decision making. In addition to this, it would be ideal if the agency could model cumulative environmental stressors in communities in a more integrated fashion. Rather than simply noting multiple stressors in the notification areas, provide some indication of cumulative stressors in the other categories as well (damage case facilities, hazardous waste, and non-hazardous industrial waste). 
Ferris
The reviewer is uncertain if this analysis will yield additional useful information other than another take on areas where disproportionate impacts occur.
Henshel
Yes  -  this provides a better comparison group  -  a counterfactual.
Add comparison to host metropolitan areas for facilities situated in metropolitan areas. 
Reviewer
Comments
Daley
See 2. above.
Ferris
This comparison may yield information about metropolitan areas where disproportionate impacts exist.
Additional comments for question 4:
Has EPA evaluated additional damages cases?  Is the pool of 218 adequate to reveal the widest reasonable range of damages, potential damages and types of comparisons?
Do host metropolitan areas factor in rural areas where capacity, including staffing, to oversee HSM facilities may be more limited compared to urban and suburban areas.
Henshel
Yes, this also would be a very appropriate comparison to include.
5.	Other Factors that Affect Vulnerability in Communities

	Question 5 Introductory Remarks

Reviewer
Comments
Ferris
The transition to this section should be clear. It's not clear that this section is distinct from the prior rule discussion. This is a graphics issue.
5a. 	Has EPA accurately described the other factors that may affect vulnerability in communities and their relevance to this analysis? 
Daley
Yes. The agency has identified a wide range of stressors that can shape individual and community vulnerability. Additional resources that the agency could consider  - particularly if they embark on a more detailed county level comparison  -  include the county health rankings (http://www.countyhealthrankings.org/). 
Ferris
The description of other factors that may affect vulnerability is accurate. 
The DSW Rule contains sound baseline vulnerability factors. 
Where does immigrant status factor into vulnerability factors?
Should whether the generator is a small business factor into community vulnerability?  For example, some small businesses could be less likely due to capacity to conduct in-depth due diligence or examinations of HSW facilities or how their wastes are recycled.
What is the universe of industrial facilities that may be generators of HSW and how do they factor into this analysis?  How do industrial facilities factor into the potential impacts analysis?
Henshel
[No response provided]
5b.	EPA has included a discussion of five broad categories of other factors that affect vulnerability in communities. Based on available information, only two of those appear to be specifically associated with the 2008 DSW rule. Is the broader discussion helpful, or would it be better to focus more specifically on the factors associated with the rule?  Please explain.
Daley
The broader discussion is helpful. The report is complex, removing the broader discussion will not simplify the report tremendously. The broader discussion, however adds thoughtful detail and clearly communicates the challenges confronting public and private decision makers. 
Ferris
What are "structural" reasons as stated in section 5.1.4?  
Ability to participate in the decision making process is a stated environmental justice vulnerability; however, this section states explicitly (in clear contradiction) that community input is not required of facilities or regulators.
The section is clear that lower participation levels can translate into experiencing greater negative impacts since "their input has not been considered fully, particularly if competing interests are set forth more effectively."  
Does the importance and/or benefit of recycling override the value of public participation especially where disproportionality is an issue?  Additionally, the studies show that facilities which have notified already show multiple facilities, higher disease incidence and a dearth of hospitals in the 3-km area.
Henshel
   a.    It is better to keep the discussion broader.  In this kind of document, it is important to include the negative results as well as the positive results.
5c.	Are there any additional factors that should be described? If so, please describe.
Daley
If the country level examination is pursued, the agency might be able to rely upon data compiled under the country health rankings. Otherwise, the data presented in the report seems adequate. 
Ferris
Immigrant status, particularly new immigrants and undocumented workers and residents constitute an important additional factor.
Unfunded mandates would affect EPA ability to ensure compliance if enforcement is left to states that don't have the budget to inspect, monitor, notify the public or implement public involvement.
Henshel
   a.    Other health endpoints to consider include: low birth weight as a measure of embryonic development; some measure of endocrine disruption including, and particularly, thyroid disruption, although other endocrine function indices should also be considered, such as diabetes; and some measure of IQ in children, if this is available.
6. 	Conclusions

	Question 6 Introductory Remarks

Ferris
The transition to this section should be clear. This is a graphics issue.
6a.	Do the results of this analysis support EPA's conclusions? 
Daley
Yes, the conclusions seem to stem directly from the analysis presented in the report. 
Ferris
The analysis and conclusions are extensive.
Arguably, DSW Rule conclusions supporting exclusion of HSW from RCRA protections are not thoroughly supported by the analysis as it pertains, in particular, to multiple (disproportionate?) impacts, potential hazards and public participation.
Henshel
a.	Yes, the results support EPA's conclusions, but likely not as well, currently, as they might with more extensive EJ analysis carried out on the data.
6b.	Has EPA clearly and accurately explained the difference between its community-level analysis and its population-level analysis? 
Daley
Yes, I think the agency has clearly differentiated between the community level analysis  -  comparing the 3 km buffer area to state, national and urban and rural areas, and the population level analysis  -  exploring the threshold risk ratio and the demographic ratio for a larger set of categories (minority, poverty, American Indian and Alaska Native, and children under 5). 
Ferris
The differences between community level and population level analysis are clear.
Henshel
a.	I think so, but I am not fully sure that my answer comes from the readability and clarity of the explanations  -  I suspect my familiarity with the information is affecting my comfort with the descriptions.
6c.	Please identify any additional information that would make a significant impact on the conclusions of this analysis. 
Daley
An environmental justice analysis of current regulatory practices for those states and facilities not adopting the 2008 DSW Final Rule would be helpful and may provide guidance if regulatory changes to the DSW rule are likely. The assumption seems to be that the status quo before the rule is neutral for communities of concern. Given the weight of the evidence in the environmental justice literature, this is not likely a reasonable assumption. 
In addition, it would be helpful to have county level comparisons for the community analysis. I am less certain that this addition would significantly change the conclusions, but it may strengthen the recommendations. 
Finally, the report would also benefit from a clear assessment of compliance under RCRA and the DSW final rule. If there are no prescriptive standards for storing waste while waiting to reclaim/recycle, are facilities more likely to have accidents?  
Ferris
Specific and explicit storage timeframes and container requirements would facilitate safer handling by reducing the prospects for potential explosions, spills and other uncontrolled releases of HSM.
Off-site reclamation  -  regulatory safeguards that ensure proper management of HSW while on-site and during shipment are important.
Is there sufficient analysis of the impacts of storing HSW in land-based units such as pits, ponds and lagoons. Should standard RCRA regulatory controls apply where HSW is stored in such land-based units?
Does the DSW Rule consider the affects of tracking, monitoring and enforcement across multiple state lines or, nationally, whether HSW transport will operate smoothly across multiple state programs?  Is there favorable or negative impact on interstate commerce?
What constitutes "reasonable effort" is not clear. This may not be a sufficiently high bar to deter mismanagement of HSW and could encourage high risk conduct.
Should record-keeping include quantities of HSW generated?  This data would be useful for emergency planning and response as well as helpful to determining economic and environmental benefit.
Has EPA assessed profitability and the viability of end-use markets for various HSW and facilities?  Would lack of markets affect, encourage or discourage a facility from taking steps and investing in protecting health and the environment?
Does the DSW Rule facilitate broad adoption by the states and state implementation?
Tribes, including Inuits, will need additional resources to staff and implement the DSW Rule. Should more cases on Native lands be included in the damages cases analysis?
Encouraging recycling should be possible while retaining a reasonable assurance of regulatory protection for health and the environment. 
Henshel
      a.   Table 5.1 would allow for a very nice small cluster analysis (or principal component analysis) to look at which emissions and types of facilities link with which endpoints, and (using maybe MANOVA) incorporating transportation and health care availability would be interesting as confounding variables.
      b.   The chapter analyses seem to be ignoring the risk ratio data. Since the "full" set of analyses (ie Fishers Exact Test, for example) did not actually carry out the full set of analyses (separating out Black and Asian, for example), it seems that the results might have been stronger with the additional analyses.
      c.   Table numbers are not provided.
      d.   Population level analysis table results seem to show that there is a numbers (sample size) bias in the level of statistical significance.  Once the notification facilities used in the analysis are pooled, or incorporate facilities from other states as will, to increase sample size, chances are that these results will also be significant in a pooled state analysis.  Thus the need to do both pooled state, and divided state, to assess how much of the sensitivity might be sample size.  It is also important to note that in regions with few Blacks, or few minorities, the minority population comparisons are always less of a driver than the low income analysis, as in these regions, are larger percentage of the low income community is white.  This is true in Indianapolis, where there is a significantly large percentage of poor whites.  In Northwest Indiana, however, there are more minorities, and minority population comparisons drive the EJ analysis results as strongly, or more strongly, than the low income analysis.
      e.   P121  -  7[th] line down in paragraph  -  typo  -  higher total population than.... I believe was intended.
      f.   P 123 Multiple and Cumulative Effects  - note the numbers with % above 70% (most) and above 60% (all but a few!!!)  -  and a cumulative frequency chart might make a nice and very visually effective summary of these observations.
      g.   The proposed regulatory and implementation measures section needs solid expansion.  It's truly pitiful right now.  It should also be noted that without regulatory clarification of some of the weaknesses in the 2008 DSW rule, enforcement is only as strong as the environmental consciousness of the administration.  Even with revised regulation, enforcement and oversight is susceptible to political influence, just somewhat less so. Monitoring is not as much a part of the 2008 DSW rule, so how could monitoring data be used to help ensure facilities are not creating more of an exposure hazard for their communities under the new rule?
      h.   Comments on appendices:
Appendix A  -  Clarification is needed.  
   1. page A-4  -  second paragraph  -  last line  -  "typically 120 days following promulgation" of what? 
   2. Page A-4 still  -  last paragraph  -  5 lines down  -  please add examples here to each type of industry not subject to CAA regulation under the 2008 DSW rule.
   3. Page A-6 third bullet  -  clarify if you mean leaks to only ground, to surface water, or also leaching to ground water  -  ie clarify
   4. Page A-7 last bullet  -  capitalize OSH.
   5. Page A-8 first bullet  -  a great example of spaghetti writing.  Spaghetti writing makes the report more difficult to read for everyone, especially the lay person.
   6. Page A-9 top of the page.  A mention of the differences in monitoring would be appropriate here.
   7. Page A-10  -  second full paragraph (starts... "Further...") -  This paragraph needs some clarification, and needs rewriting.
   8. Page A-10 section 3.9 second two paragraphs.  These paragraphs are very unclear, and also seem to be internally inconsistent, as written.  Clarify in these paragraphs what is included in each type of exclusion.  A good example of a clearly written paragraph is the next paragraph (first paragraph of 3.10).  This paragraph includes a short but clear definition which enables the reader to comprehend the information provided more readily.  The reader does not have to go searching for the definition, which helps reduce reader confusion.
   9. Page A-10 last paragraph, first sentence.  Add clarifying phrases  -  what kind of characteristics, and where are these listed.
   10. Page A-11. First paragraph needs a sentence inserted clarifying what could be exempted.  Second paragraph needs clarification in the middle sentence  -  specify briefly or provide example (in parentheses) for conditions under which listed waste may not be determined to be hazardous waste.
   11. Table A-11 page A-13 middle column, middle section  -  This is too dense for the average reader referring only to CFR citations and statutes (in inconsistent style no less).  Provide details or names for these sections and their overall regulation titles in clear, jargon-free writing.
   12. Table A-11  -  Overall  -  this table (and the report) lacks any discussion of the implication of these changes due to the adoption of the 2008 DSW rule.  An example might be to evaluate the implication of leaking containers, which (under the current vagueness of the 2008 DSW ruling language) might not be caught for up to 2 years, or maybe longer.  What kinds of exposures might be increased to what diameter population under this scenario?
   13. Page A -17  -  why the changed font size in column 1?
   14. Page A-18.  Last paragraph in column 1.  Include a list of the 140 toxic and flammable regulated chemicals referred to here, and elsewhere in the report.  The easiest way to handle this might be a footnote, or adding the list to the appendices, and referring to the inserted list (footnote, endnote, or appendix) here and whenever else this list is referenced.
   15. Page A-21 first column, second line  -  missing an article ("the" or "a") between "above" and "specified quantity".
   16. Page A-26  -  last column, first paragraph  -  this section (and its text tie-in) is just calling out for a discussion of the implications of lack of required training on personnel at the facility if exposures indeed rise.
   17. Page A-29 first column  -  need a space between paragraphs.
   18. Page A-30 third column, first word "these" needs clarification  -  to which regulations is the writer referring.  Do not expect the lay audience to know for sure.  Same point for page A-32 third column, last paragraph, second line and again page A-33 second line.
   19. Page A-34 third column second paragraph.  This very statement again seems to call out for a discussion of the implications of the lack of record keeping and exception reporting.  It is NOT sufficient to point out that the 2008 DSW rule changes the recording and record keeping requirements.
   20. Page A-35 first column middle paragraph  -  clarify, provide more detail.  Similarly third column second paragraph  -  specify which excluded materials may not qualify as a DOT hazardous material, and provide some discussion of the potential implications of this from an EJ perspective.
   21. Page A-36 third column second paragraph  -  this again needs further discussion here and in the text on the possible implication of this change due to the 2008 DSW rules.  Specifically, would any residues from reclamation ever be potentially hazardous and NOT get identified as such through the 2008 DSW process.
   22. Page A-37 GCE: is something missing here?  Seems like it.
   23. Page A-49 right hand column, second paragraph  -  use the full citation, not the abbreviated citation.  The full citation (ie using the same citation format throughout) is more understandable to the lay person.
   24. Page A-52 right column, top line "an SPCC" fix typo
   25. Page A-61 middle column, top paragraph.  Keep citation format consistent and summarize the legitimacy criteria in this paragraph, a footnote, or the next paragraph.
   26. Page A-63 right column.  Provide examples of potential problematic emissions that don't meet threshold criteria and don't meet other applicability criteria.  It would be great if the implications of the resultant exposure (were this to occur) be determined quantitatively, maybe at the mean/lower assumption (9.5 TPY single type HAP) and high assumption (24.95 TPY mixed HAP) levels.
   27. Page A-64  -  right hand column  -  No one in this report is discussed how much might be released before anything would trigger the need to determine if a cleanup is needed.  Surely this would help explore the potential for a significant disproportionate impact occurring? 
   28. Page A-68, right column, last sentence.  Where is the "note below"?  Probably better to put the note in a footnote on the same page or in the right hand column immediately after this paragraph.
   29. Page A-82 on  -  It is great to have the summary of the Federal Regulations here.  However, the formatting and depth of each discussion is not consistent and needs to be made so.  For example, the most readable formats start with a sentence or two that talks about the relevant regulation(s) more broadly, and then includes a sentence that explains the relevance of the regulation to this report/study.
   30. Page A-83 Second paragraph under "Legitimate recycling"  -  include a brief summary of the six criteria referred to here (and elsewhere) and not clearly defined.  The listing could be in the paragraph, in an offset list, in a separated inset box, or in a footnote.
   31. Page A-84  Top of the page  -  provide details about standards in 262.34 (d) and (e).  They are not clearly described anywhere.  Also maintain a consistent formatting for citations.
   32. Page A-94 Exports first paragraph last two sentences  -  a) remember to keep citations in consistent format.  B) To make this document most readable, summarize these requirements briefly.
   33. Page A-96  -  No where is there a discussion of at what point in a chain of non-compliance would the chain of permission stop and oversight identify a problem that needs to be addressed.
   34. Page A-99 bullets under 5.3.3  In each bullet, first identify the key factors that determine whether processes fit as Program 1, 2 or 3 (i.e. defined for the appropriate bullet).  These three was to characterize a process are never clearly defined throughout this whole document.  This would seem to be the place to put those definitions.
   35. Page A-106 5.8.2 line 3  -  TSDFs is an acronym that is never defined.
   36. Appendix B is good to have, as is Appendix C and D.  Note that in Appendix B, and in a few other places, the writer uses the personal "we" instead of a noun.  This is formal writing and should not include any first person (personal) pronouns anywhere in the document.  Search to find the "we"s and rewrite to replace the personal pronoun with a clearer sentence using the impersonal 3[rd] person or noun.
   37. Page B-2 first paragraph.  Need to add a sentence that explains why it is important to understand what a "wastewater" versus a "nonwastewater" is.
   38. Page B-3 table bottom:  POTW and NPDES is not defined here or elsewhere in the document.  Acronyms are best spelled out the first time they are used in any given chapter, and still it is kind to the read to provide a Table of Acronyms and Abbreviations, and spelling out of all acronyms in footnotes for any table in which they are used.
   39. Appendix C is written like an abbreviated technical reference manual, but without the definitions usually included in such a manual.  Most of this section reads like a jargonfest, and is not accessible to the lay reader.  So, 
      a. Identify all acronyms and abbreviations.  There's two in the second paragraph.
      b. In the introductory section, explain by the K, F, U, P and D codes represent, how they are used, and how understanding the coding was needed for the environmental justice analyses.
      c. Explain all codes that have no other referent to explain them.  For example, on page C-5 top line, F037-F038 codes are referred to without placing them in any context or defining what these codes cover.
      d. Table C-1 needs some additional explanation before it will be easily read by the lay reader.
   40. Appendix D is written in a way that is essentially understandable.  However in two places, where groups are moved from the logical grouping to a not as obvious grouping, explanation is needed for why the move was made.  These points are on Page D-3 bottom of the page, and on Page D-5 top of the page.
   41. Appendix E needs some clarifying verbiage written in easy to understand English.  The two ratios could, for example, be easily explained by one or two sentences.  The description of the tests are more theoretical in places.  Specific places that clarification is needed includes:
      a. Page E-9 bottom of the last Fisher Exact Test paragraph, there is no explanation for why or when the minus sign is removed, nor where the minus sign is likely to show up and why.
      b. Page E-10  -  it is not clear what the tables are for the various states.  Add clarifying language.
      c. Since these tests are not always run (but should be) the verbiage should include an explanation of why they are not run in some situations in the analysis and are run in other, apparently similar situation.


Appendix A:  Charge to Reviewers
                  Technical Charge to External Peer Reviewers
                                       
                           Contract No. EP-W-10-055
                         Task Order No. EP-G11S-00004
                               December 21, 2010
                                       
                                       
   External Review for EPA's Draft "Definition of Solid Waste Final Rule 
                        Environmental Justice Analysis"
                                       
                                       
BACKGROUND INFORMATION

In October 2008, EPA published a final rule revising the definition of solid waste under the Resource Conservation and Recovery Act (RCRA) for certain types of hazardous secondary materials being recycled. The intent of the rulemaking was to encourage safe recycling of hazardous waste by excluding certain materials from full regulation as hazardous waste if they were recycled under certain conditions. During a June 2009 public meeting to discuss a petition asking EPA to reconsider the final rule, concerns were expressed that the Agency had not paid adequate attention to environmental justice during the rulemaking process. EPA developed the draft "Definition of Solid Waste Final Rule Environmental Justice Analysis" (DSW Rule) in response to these concerns.

When analyzing environmental justice concerns during the 2008 rulemaking, EPA assumed that the conditions it had included as part of the exclusion would be sufficient to prevent all releases to the environment. As stated in its environmental justice discussion, EPA reasoned that with no releases, there could be no disproportional impact on disadvantaged communities. In the draft document that is the subject of this peer review, EPA has revisited those assumptions and performed a more complete environmental justice analysis.

The draft revised analysis starts by identifying what potential hazards may result from the definition of solid waste final rule and which communities may potentially be affected by these hazards. The analysis then examines the demographics of the potentially affected communities by looking at the numbers of members of racial minorities, children under 5, and people with income below the poverty level within a 3-km radius around each facility. These demographics are then compared to those for the county and state in which the facility is located, and also for the nation as a whole. The analysis then identifies what other factors in the community may also affect any vulnerability in these communities to the potential hazards from the final rule. 

Finally, the draft analysis uses all this information to make a determination about whether some population groups (e.g., minority or low-income populations) would be over-represented in the impacted communities. 

This draft analysis will be used to inform EPA and the general public about the potential impacts of the 2008 definition of solid waste final rule on communities in which the relevant facilities are sited or may be sited in the future. It will also be used to inform EPA and the general public about potential approaches to mitigating those impacts in the future, including potential regulatory changes, as well as non-regulatory measures. 

The draft analysis has been reviewed internally at EPA, and the current draft reflects response to the internal comments. EPA will consider and respond to comments from this external peer review as it prepares the final version of this analysis. 

CHARGE QUESTIONS

Please provide a written response to all questions, either as a specialist if the topic is within your area of expertise, or as a general reader if it is not. If no response is possible because the question lies entirely outside your area of expertise, please indicate that. 

Be sure to provide sufficient explanation in each of your responses so that EPA can understand the basis for them. As you note areas needing improvement, provide specific suggestions for improvement, where possible. 

As you prepare your responses, please be sure to follow the formatting instructions at the end of this charge.

General
Ga.	Please state your overall assessment of the content, readability and utility of this report, including any changes needed. 
Gb.	Is this information accurately and clearly presented?  How might the document be improved?
Gc.	Does the document adequately review, present, analyze and summarize the available data? If not, what can be improved?

Executive Summary
ESa.	Is the Executive Summary clear and easy to read?
ESb.	Does the Executive Summary provide an accurate view of the contents, findings, and conclusions of the analysis? If not, please specify how it could be improved.

1.	Introduction
1a.	Does the Introduction clearly lay out the contents of the analysis? If not, please describe how it could be improved.	

2.	Hazard Characterization
2a.	Do the potential scenarios under the definition of solid waste final rule clearly describe how EPA thinks facilities may react to the rule? If not, please suggest where improvements should be made. 
2b. 	Are these scenarios plausible, and are there any other likely scenarios? Please explain. 
2c.	Is EPA's analysis of the types and quantities of hazards from the definition of solid waste final rule thorough and accurate? Please explain.
2d.	Is EPA's analysis of the differences between hazardous waste regulations and the regulations in the definition of solid waste accurate? If not, please explain. 


3.   Identification of Potentially Affected Communities
3a.	Has EPA correctly identified the facilities that may be likely to begin recycling hazardous secondary materials under the definition of solid waste exclusions? If not, what facilities are missing?
3b.	Is EPA's use of a threshold of 25 tons of recyclable hazardous secondary material generated annually an appropriate way of identifying generators that might choose to recycle onsite?  If not, what threshold would be more appropriate?
3c.	Has EPA appropriately identified the communities and populations that are potentially affected by these facilities?  In particular is the 3-km circle the appropriate area of consideration?  Please explain.

4.	 Analysis of Demographics of Potentially Affected Communities
4a.	Are EPA's data sources for its demographic analysis appropriate? Should EPA consider any additional data sources for this information? Please explain.
4b. 	Did EPA select appropriate characteristics for analysis of environmental justice concerns in communities? 
4c.	Was the choice of the areal apportionment method to determine the characteristics of the communities surrounding each facility appropriate and properly conducted? 
4d. 	Is a statistical analysis of these results to determine how the demographic characteristics compare to those on a county, state, and national level, in both urban and rural communities, a scientifically appropriate way to assess the existence of environmental justice concerns in connection with this rulemaking?  Are there additional methods that should be considered for analyzing the environmental justice concerns with this rule?
4e. 	Is the statistical analysis in this section properly done?
4f. 	Are the assumptions in the analysis presented accurately and thoroughly?
4g. 	Are the uncertainties in the data input into the analysis appropriately accounted for and discussed?  
	4h.	Based on internal EPA review of the document, the Agency is considering additional approaches for analyzing the demographic data. The decision to conduct additional analyses will depend in part on timing and resource concerns. Which of the following analyses are most likely to yield new and/or improved information, and how would that information inform EPA's decision-making process?
        1. For the category of facilities from the damage case study, analyze the demographics for the timeframe the damages occurred.
        2. Compare potentially affected areas to non-affected areas. Compare the demographics in the 3-km concentric areas surrounding potentially affected facilities to demographics of areas in the state not containing facilities, rather than to averages for the entire state. 
        3. Add comparison to host metropolitan areas for facilities situated in metropolitan areas. 
5.	Other Factors that Affect Vulnerability in Communities
5a. 	Has EPA accurately described the other factors that may affect vulnerability in communities and their relevance to this analysis? 
5b.	EPA has included a discussion of five broad categories of other factors that affect vulnerability in communities. Based on available information, only two of those appear to be specifically associated with the 2008 DSW rule. Is the broader discussion helpful, or would it be better to focus more specifically on the factors associated with the rule?  Please explain.
5c.	Are there any additional factors that should be described? If so, please describe.
6. 	Conclusions
6a.	Do the results of this analysis support EPA's conclusions? 
6b.	Has EPA clearly and accurately explained the difference between its community-level analysis and its population-level analysis? 
6c.	Please identify any additional information that would make a significant impact on the conclusions of this analysis. 



Appendix B: Individual Reviewer Comments












                             COMMENTS SUBMITTED BY

Dorothy M. Daley, Ph.D.
Associate Professor
Department of Political Science & Environmental Studies Program
University of Kansas
504 Blake Hall, 1541 Lilac Lane
Lawrence, KS 66044
785-864-9839
Email: daley@ku.edu 



External Peer Review for EPA's Draft Definition of Solid Waste Final Rule
Environmental Justice Analysis
                                       
             Response to Charge Questions by Dr. Dorothy M. Daley
                                       

General
Ga.	Please state your overall assessment of the content, readability and utility of this report, including any changes needed. 
My overall assessment is that the report manages to present a very complex decision making process and analysis in a fairly clear and accessible fashion. The discussion of potential hazards and exposure pathways seems clear and complete. The eight scenarios are clearly laid out. There report is quite detailed. The main points seem to be that the 2008 Draft Solid Waste Final Rule (DSW) enables generators to store more hazardous secondary material (HSM) for a longer period of time, and provides no prescriptive standards to ensure safe containment. Transporters of waste and intermediate and reclamation facilities are also impacted in ways not necessarily analyzed in the initial consideration of the rule change. Decreased transparency and public participation provisions combined with deregulation of hazardous secondary material to encourage reclamation is likely to have differential impacts across the county.  The findings from this report present the EPA with a challenge. Recycling hazardous secondary material presents significant health and environmental hazards, and the 2008 rule change is not uniformly health protective. According to this analysis, some communities will be more impacted than others, and this differential impact has direct environmental justice implications. 
To fully understand the differential impact of the 2008 DSW final rule on low income and minority communities and populations, it would be helpful to have a meaningful baseline analysis. What if continuing to regulate HSM under RCRA has even greater environmental justice impacts?  The report and analysis do not provide that information. So, there is no way to identify if the 2008 DSW rule  -  while likely to have disproportional impacts  -  is an improvement over current reclamation and disposal practices under RCRA. I recognize this analysis would entail considerable work, but if HSM regulated under RCRA currently has even greater environmental justice impacts, then this would be very important to know this as decision making proceeds. 
Gb.	Is this information accurately and clearly presented?  How might the document be improved?
For the most part, the information in the report is clearly presented. There are, however, some areas that could be improved.  While Figure 1 is informative, the other figures in the report make little sense to me. They either need additional explanation, or they should be removed. Figure 4.41.1 is an example of the type of bar chart that can be confusing. Perhaps if the base category of Whit/Non-Hispanic was added, the figure would make more sense.  As it is now, these figures detract rather than add to the report. 
Why are urban and rural comparison omitted for the damage case facilities?  Consistent analysis across categories is important. If the analysis cannot be performed for some reason, an explanation in the text would be helpful. 
A list of acronyms would be extremely helpful. Also, there are also several typos and minor mistakes that should be addressed.  These include: 
      (1)    Page iv  -  under the hazardous waste baseline, second row, "recycles" should be changed to "recycling"
      (2)    Page 2, sec 1.3, bullet 1  -  "HSMHSMs" should be changed to "HSM"
      (3)    Page 10, top of the page: "as a focus" is written twice. One occurrence should be deleted. 
      (4)    Page 81, section 4.2.3  -  "area" should be added after "a 3 km"
      (5)    Page 91, section 4.42  -  lists 219 damage case facilities. There are only 217 reported in the analysis. Either explain the missing cases, or fix a typo. 
      (6)    Attachment A,  page A-2, section 1. Toward the end of the first paragraph, should read "regulation if" not "regulation is". 
Gc.	Does the document adequately review, present, analyze and summarize the available data? If not, what can be improved?
It seems like the document is quite thorough. As noted in my response to question Ga, the document would be improved by a assessing the Environmental Justice impacts of the baseline  -  or RCRA's approach to HSM recycling and disposal. 

Executive Summary
ESa.	Is the Executive Summary clear and easy to read?
Considering the complexity of the document, the Executive Summary is relatively clear and easy to read. The "Summary of Potential Impacts" is very helpful.  It is important to note, however, that the findings in the executive summary are not properly numbered: the executive summary lists findings 1, 3, and 4, omitting 2.  The table and paragraph describing "Community Level Analysis of Potential Disproportionate Impacts" is difficult to follow. It would be helpful if the Environmental Protection Agency (EPA) could identify and describe the four types of facilities in this analysis more consistently. For example, the text on page vii could be edited to include the following language connecting the text to the community analysis tables: 
      The second step of the methodology identified facilities that can represent the facilities that are likely to take advantage of the 2008 DSW final rule. These facilities are grouped into four different categories: (1) facilities that have already notified under the 2008 DSW final rule[Notification facilities], (2) hazardous waste facilities that are likely to recycle under the rule (including hazardous waste generators producing more than a truckload (25 tons) of recyclable hazardous secondary materials annually, and hazardous waste recyclers) to use the rule [Hazardous Waste Facilities], (3) facilities from the environmental problems study (many of which operated under exclusions or reduced regulations)[Damage Facilities], and (4) facilities currently recycling non-hazardous industrial waste (e.g., antifreeze) that could most easily switch or expand to recycling under the 2008 DSW final rule [Non-Industrial Hazardous Waste Facilities].

It is difficult to understand where the categories in the Table on page X come from. They should be clearly labeled in the text and - table categories: the ordering in the text and the table should be consistent. Also, it would be helpful to define "damage facilities". 
ESb.	Does the Executive Summary provide an accurate view of the contents, findings, and conclusions of the analysis? If not, please specify how it could be improved.
Yes, the Executive Summary provides an accurate summary of the more detailed material presented in the report. 

1.	Introduction
1a.	Does the Introduction clearly lay out the contents of the analysis? If not, please describe how it could be improved.	
Yes, the Introduction is clear. It is easy to follow and well organized.

2.	Hazard Characterization
2a.	Do the potential scenarios under the definition of solid waste final rule clearly describe how EPA thinks facilities may react to the rule? If not, please suggest where improvements should be made. 
Yes, the scenarios clearly define how EPA thinks facilities may react to the rule. Table 2.1 is particularly useful in outlining the baseline and potential changes based on the DSW Exclusions. I would however, like to know how the EPA developed these scenarios. Are these based on professional experience, or did the agency talk with the regulated community to get a sense of potential reactions to the DSW Exclusions?   If it is the latter, could the report provide more detailed information regarding the feedback from the regulated community?  If it is the former, it would add value to outline some of the ways in which that experience shaped the discussion / selection of the scenarios. 
2b. 	Are these scenarios plausible, and are there any other likely scenarios? Please explain. 
These scenarios seem plausible. They are described in clear and logical detail. 
2c.	Is EPA's analysis of the types and quantities of hazards from the definition of solid waste final rule thorough and accurate? Please explain.
Yes, EPA's analysis of the types and quantities of hazardous secondary material is clear. Figure 1 provides a clear justification for the focus on solvents and electric arc furnace dust.  As mentioned earlier, it would be helpful if the report clarified the definition of a damage case or damage facility. As noted on page 10, the analysis highlights solvent wastes because of their prevalence in damage cases. A definition and some references to RCRA damage cases would be helpful, even if provided in a footnote. 
2d.	Is EPA's analysis of the differences between hazardous waste regulations and the regulations in the definition of solid waste accurate? If not, please explain. 
It seems as if EPA has noted many of the differences between hazardous waste regulations and the changes stemming from the 2008 Draft Solid Waste (DSW) Final Rule. Table 2.3 is a very detailed description of the eight likely scenarios and their impact on generators, transporters, and intermediate reclamation facilities. This material clearly communicates that there is likely to be substantial changes in the accumulation and storage of hazardous secondary material (HSM).  

3. 	Identification of Potentially Affected Communities
3a.	Has EPA correctly identified the facilities that may be likely to begin recycling hazardous secondary materials under the definition of solid waste exclusions? If not, what facilities are missing?
EPA's approach to identifying facilities that are likely to begin recycling hazardous secondary material seems reasonable. I cannot think of another category to include. Out of the categories presented (already notified; damage facilities; currently generating hazardous secondary material, and facilities that may easily expand or switch to reclamation because of the DSW final rule), the last three categories seem more useful to understanding likely environmental justice impacts. With only 40 facilities providing notification at the time of the report, it is likely that "street level" implementation will be driven by the larger population of facilities rather than early adopters. 
3b.	Is EPA's use of a threshold of 25 tons of recyclable hazardous secondary material generated annually an appropriate way of identifying generators that might choose to recycle onsite?  If not, what threshold would be more appropriate?
I cannot comment directly on whether or not the annual marker of 25 tons of recyclable material is appropriate. The report provided limited justification for this cut off point, and while it seems reasonable, someone with more applied experience in waste management and reclamation may think otherwise. 
3c.	Has EPA appropriately identified the communities and populations that are potentially affected by these facilities?  In particular is the 3-km circle the appropriate area of consideration?  Please explain.
EPA is defining a community as a 3 kilometer circle or buffer around one of the facilities included in this analysis. The agency then uses census information to apportion socio-economic and demographic ratios within this radius. This method is superior to simply using standard census divisions such as census tracts, blocks, or even zip code aggregation. The operational definition of "a community" is challenging and the environmental justice literature has numerous empirical studies at different levels of analysis, sometimes providing conflicting and competing results. Thus, using a measure of community with stronger measurement validity is important. Like all measurements, this is not a perfect representation of "a community", but it is a much improved measure over more traditional approaches, i.e., studies that rely upon preexisting census designations to ascertain community characteristics. Increasingly, a 3 kilometer radius is becoming a standard approach to improving the precision of community measures in environmental justice and public health research. The report however, could be edited to more clearly explain this approach. The text on Page 80 and 81 should be streamlined. First, the agency should suggest that the 3 km buffer is reasonable given the potential for acute exposure from accidental releases. Then, this should be followed by a discussion of the area apportionment method. As it reads now, it is a bit confusing. 

4.	Analysis of Demographics of Potentially Affected Communities
4a.	Are EPA's data sources for its demographic analysis appropriate? Should EPA consider any additional data sources for this information? Please explain.
The census data that the EPA uses for this analysis is standard. Improvements would include 
relying upon the 2010 census numbers to more accurately reflect community composition. If possible, it would be ideal to be able to rely upon the American Community Survey (ACS) to obtain more detailed and temporal information on community change. However, I am not sure that the sample size for the ACS are adequate for this facility based approach. It may be that by drilling down to the facility level and relying upon survey rather than census data, introduces too much variability. 
4b. 	Did EPA select appropriate characteristics for analysis of environmental justice concerns in communities? 
Yes. The EPA has selected appropriate characteristics to analyze in this report. I think it would be a stronger analysis if along with "children under five", "adults over 65" were included in the detailed analysis in chapter 4. Given the demographic transitions of the baby boomers into an elderly population, it will become more critical to understand differential exposure for susceptible  populations at both ends of the spectrum. 
4c.	Was the choice of the areal apportionment method to determine the characteristics of the communities surrounding each facility appropriate and properly conducted? 
Yes, it is fine. It would be beneficial to have some discussion of the assumption behind this method. As noted on Page 80, the areal apportionment method assumes all populations are equally distributed in the block or block group. This assumption will not hold in practice. Therefore, some indication of how this assumption influences the analysis would be useful. Given the population distribution in the nation, how might this assumption bias the analysis?  I am not suggesting that the agency change their approach, only that they articulate the implication of this assumption. 
4d. 	Is a statistical analysis of these results to determine how the demographic characteristics compare to those on a county, state, and national level, in both urban and rural communities, a scientifically appropriate way to assess the existence of environmental justice concerns in connection with this rulemaking?  Are there additional methods that should be considered for analyzing the environmental justice concerns with this rule?
Yes. A statistical analysis of these results is an appropriate method to understand patterns in administrative and community level data. These patterns can reflect social and environmental disparities and the EPA can consider these patterns for future decision making. Note however, that the analysis presented in Chapter 4 did not include a comparison between community (3 km buffer area) and county level characteristics. State, national, and urban and rural were examined, but county information was not analyzed in this section of the report. County level information and analysis would have added value to this report. 
4e. 	Is the statistical analysis in this section properly done?
The statistical analysis seems to be properly done. As far as I can tell, these are conditional probability calculations. It is a relatively straightforward and simple approach.  While I am not a statistician, it seems like a reasonable way to approach the analysis. It provides the agency with two types of information: the chances that a facility exists in a buffer area given the demographic and socio-economic composition (threshold risk),  and the probability of a particular demographic or socioeconomic composition given the likelihood of facilities (the demographic ratio). 
4f. 	Are the assumptions in the analysis presented accurately and thoroughly?
Yes, I think the assumptions are accurately presented. Footnotes 34  -  37 are important components in the analysis. The report very clearly outlines data sources and how the information is aggregated together. As already noted, the distributional assumptions of the areal apportionment method should be more fully explained.
4g. 	Are the uncertainties in the data input into the analysis appropriately accounted for and discussed?  
There is considerable selection bias or selection effect in the "notification" facilities and likely also in the damage facilities. It is likely that these early adopters (notification facilities) are significantly different (and non-randomly distributed) compared to the three remaining categories. Damage facilities are also likely to be systematically different. I am not sure exactly how to address this, but it seems like something that the agency needs to consider. Perhaps analyzing a comparison group of randomly selected and/or "matched" facilities could be useful, in addition to the broader state and national comparisons. 
4h.	Based on internal EPA review of the document, the Agency is considering additional approaches for analyzing the demographic data. The decision to conduct additional analyses will depend in part on timing and resource concerns. Which of the following analyses are most likely to yield new and/or improved information, and how would that information inform EPA's decision-making process?
For the category of facilities from the damage case study, analyze the demographics for the timeframe the damages occurred.
Compare potentially affected areas to non-affected areas. Compare the demographics in the 3-km concentric areas surrounding potentially affected facilities to demographics of areas in the state not containing facilities, rather than to averages for the entire state. 
Add comparison to host metropolitan areas for facilities situated in metropolitan areas. 
Option 2 has the strongest potential to add to the ability of the agency to draw inferences from their data sources. If statistical analysis indicates systematic socioeconomic and demographic differences in area with and without facilities, then this provides even stronger evidence of the distributional affects of decision making. In addition to this, it would be ideal if the agency could model cumulative environmental stressors in communities in a more integrated fashion. Rather than simply noting multiple stressors in the notification areas, provide some indication of cumulative stressors in the other categories as well (damage case facilities, hazardous waste, and non-hazardous industrial waste). 

5.	Other Factors that Affect Vulnerability in Communities
5a. 	Has EPA accurately described the other factors that may affect vulnerability in communities and their relevance to this analysis? 
Yes. The agency has identified a wide range of stressors that can shape individual and community vulnerability. Additional resources that the agency could consider  - particularly if they embark on a more detailed county level comparison  -  include the county health rankings (http://www.countyhealthrankings.org/). 
5b.	EPA has included a discussion of five broad categories of other factors that affect vulnerability in communities. Based on available information, only two of those appear to be specifically associated with the 2008 DSW rule. Is the broader discussion helpful, or would it be better to focus more specifically on the factors associated with the rule?  Please explain.
The broader discussion is helpful. The report is complex, removing the broader discussion will not simplify the report tremendously. The broader discussion, however adds thoughtful detail and clearly communicates the challenges confronting public and private decision makers. 
5c.	Are there any additional factors that should be described? If so, please describe.
If the country level examination is pursued, the agency might be able to rely upon data compiled under the country health rankings. Otherwise, the data presented in the report seems adequate. 

6. 	Conclusions
6a.	Do the results of this analysis support EPA's conclusions? 
Yes, the conclusions seem to stem directly from the analysis presented in the report. 
6b.	Has EPA clearly and accurately explained the difference between its community-level analysis and its population-level analysis? 
Yes, I think the agency has clearly differentiated between the community level analysis  -  comparing the 3 km buffer area to state, national and urban and rural areas, and the population level analysis  -  exploring the threshold risk ratio and the demographic ratio for a larger set of categories (minority, poverty, American Indian and Alaska Native, and children under 5). 
6c.	Please identify any additional information that would make a significant impact on the conclusions of this analysis. 
An environmental justice analysis of current regulatory practices for those states and facilities not adopting the 2008 DSW Final Rule would be helpful and may provide guidance if regulatory changes to the DSW rule are likely. The assumption seems to be that the status quo before the rule is neutral for communities of concern. Given the weight of the evidence in the environmental justice literature, this is not likely a reasonable assumption. 
In addition, it would be helpful to have county level comparisons for the community analysis. I am less certain that this addition would significantly change the conclusions, but it may strengthen the recommendations. 
Finally, the report would also benefit from a clear assessment of compliance under RCRA and the DSW final rule. If there are no prescriptive standards for storing waste while waiting to reclaim/recycle, are facilities more likely to have accidents?  











                             COMMENTS SUBMITTED BY

Deeohn Ferris, J.D.
President
Sustainable Community Development Group, Inc.
P.O. Box 15395
Washington, DC 20003
202-637-2467
Email: gerinc@mindspring.com
 


External Peer Review for EPA's Draft Definition of Solid Waste Final Rule
Environmental Justice Analysis
                                       
               Response to Charge Questions by Dr. Deeohn Ferris

General
Ga.		Overall Assessment
 Generally, the DSW Rule is read-able and clear.
Gb.		Information Accuracy, Clarity
Generally, information in the DSW Rule presented is clear and appears to be accurate.
Gc.	Document Adequacy, Data Summary
The DSW Rule is an extensive document with significant analysis and extensive data.

Executive Summary
ESa. 	Clarity, Readability
The Executive Summary is very clear and readable. 
The material is complex and dense which is to be expected in an agency rulemaking. 
Readers reviewing the synopsis of the Sierra Club petition in this summary (or, in the relevant section below) would be informed by an abbreviated specific list of issues identified by the Club (instead of the more generalized ""raised a number of issues about the protectiveness of the rule...' -- especially since the language of the rule emphasizes that a number of commenters "echoed"...concern..."
Summary of DSW EJ Methodology is clear and concise with a couple of exceptions. The term "damages" in Step 4 could be explicitly defined. Presumably, the language that follows the term "damages" is explanatory but that is somewhat unclear. 
Does "Information" in Step 5 include data?  Perhaps strengthening the language to "Data and Information..." is more supportive of the Agency's rationale.

ESb. 	Accurate View
The "Summary of Potential Impacts...Under Different Recycling Scenarios" is clear however the accuracy of the presumptions is challenged by the language on its face. Pursuant to Hazard Characterization, EPA demonstrates and discusses the highly hazardous nature of the 2 main sources of reclaimed hazardous secondary materials (hereafter HSM) upon which much of the Draft DSW Rule analysis is based in addition to the storage hazards including leaks, fires and explosions. Container standards do not appear to compare with the highly hazardous nature of the 2 main sources upon which much of the Draft DSW Rule analysis is based. 
The Scenarios chart shows several instances involving movement or activity related to HSM where it appears that various historic RCRA safeguards are not required see e.g., certain scenarios under Scenario 3; Scenario 5; Scenario 6; Scenario 7; Scenario 8) without, it could be said, on balance, assuring a regulatory atmosphere that influences the regulated community to handle HSW in ways that prevent hazards.  Comparatively, streamlined notice and recordkeeping would both facilitate emergency planning and response and protect human health and the environment. 
The benefits (per language in the Scenarios) don't balance the potential negative impacts. This could be more clearly set forth or demonstrated. Although the reviewer does not express agreement that any benefits would offset the potential hazards or actual hazards, more clarity or quantification of benefits would strengthen the analysis.
Does EPA plan to establish a definition or explanation of what constitutes legitimate recycling and/or recycler-facility?  This could serve as notice to generators, receivers, recyclers that the exemptions and exclusions are accompanied by affirmative duties that include compliance with statutes and regulations?
Regarding DSW Rule controversies about disproportionate negative impacts and exposures, would these communities feel more protected (versus adding to the burden of already cumulative pollution) if HSW were handled similar to those substances categorized under Universal Wastes?

1. 	Introduction
1a.	Clarity of Analysis
Summary of DSW EJ Methodology is clear and concise with a couple of exceptions. 
The term "damages" in Step 4 could be explicitly defined. Presumably, the language that follows the term "damages" is explanatory but that is somewhat unclear. 
Does "Information" in Step 5 include data?  Perhaps strengthening the language to "Data and Information..." is more supportive of the Agency's rationale.
Readers reviewing the synopsis of the Sierra Club petition would be informed by an abbreviated specific list of issues identified by the Club (instead of "raised a number of issues about the protectiveness of the rule..." 
1.3  minor typo -- see "HSMHSMs"
In view of the complexity of what is explained, Scenarios are clear.
In addition to the advantage of adhering to RCRA's decades old cradle-to-grave protective framework, consider whether requiring a streamlined notice and record-keeping process for generators, receivers and recyclers of HSM benefit EPA and states e.g., monitoring, as well as data that shows the environmental and economic benefits of HSW exclusions?
The threat of post facto CERCLA liability is not interchangeable with up-front environmental protection standards and requirements. This reviewer does not presume that the entire regulated community is villainous; however, any vigor/rigor that a generator, receiver, recycler might invest in environmental protection could wane as oversight and compliance requirements are reduced or eliminated. 

2. 	Hazard Characterization
The transition to this section should be clear. It's not clear that this section is distinct from the prior rule discussion (the "Introduction"). This is a graphics issue.
The discussion of hazards associated with mainly the 2 selected vectors is clear. Hazards are extensive in contrast to reducing the levels of regulatory oversight provided by RCRA. Should all 4 legitimacy factors should be mandatory in view of the hazards?  What about release reporting?  Should releases be grounds for losing exclusion and HSW status?  How is a significant release determined?
In view of the stated hazards, could managing HSW as a valuable commodity influence the generator, receiver, recycler and affect compliance?  This goes to the concept of legitimacy. Would the mandating a showing of economic value militate in favor of safeguards against releases, spillage?
If wastes that are going to be recycled as hazardous as wastes that are not recycled, are both equally likely to be mishandled?  Are the regulatory and enforcement tools provided by the DSW Rule commensurate with the task of ensuring compliance?
2a. 	Clarity of Potential Scenarios
The description of potential scenarios is clear. 
The chart situated in the "Overview" section, which describes the 3 relevant exclusions is clear and concise. The applicability of RCRA (or not) is readily discernible from the scenarios.
2b. 	Plausibility of Potential Scenarios
The Scenarios are plausible. 
Is there a significant possibility that small businesses could be more vulnerable to sham recycling if many won't have the capacity and staffing to thoroughly assess recyclers?
The DSW Rule could inadvertently encourage mismanagement while transitioning materials into the recycling stream. Does the Rule apply where HSW is recycled into even more toxic products? 
Should notice be the hazard quid pro quo for the exclusions which provide the benefit of different (reclamation), streamlined treatment of HSW?
Without adequate record-keeping and environmental protection safeguards (streamlined requirements would not be objectionable), the international export exclusion/scenario is troubling re potential releases, impacts, whether or not the recipient nation has genuine knowledge of the contents and consents to the receiving the export; see e.g., Circle of Poison and pesticide regulation. 
2c. 	Accuracy of EPA Hazard Analysis
The selection of electric arc furnace dust and solvent wastes as national HSM proxies appears sound based on the analysis. Excluding used oil, non-ferrous metals and scrap metals from the hazard analysis, although associated with "damages cases?" 
How and when does enforcement after a "significant" release occur?  How is the determination made as to what constitutes a significant release?  Which party has the burden of proof?  It would seem that placing the burden of proof on the party that causes or is responsible for the release should bear the burden of proof opposed to, for example, the governmental entity. 
Intervening prior to a release is clearly preferable  -  the possibility of enforcement action prior to a release would be protective and more likely in the event that there are clear regulations e.g., storage limitations, labeling requirements and requirements related to the integrity of storage containers and areas.
Does worker training (e.g., protection standards, labeling, handling) or the lack of it at relevant/specified facilities factor into the EPA hazard analysis?
Does emergency response and planning, the existence or lack of these programs, factor into the EPA hazard analysis?  Similarly, does eliminating the requirement for a Manifest (or some version of a tracking mechanism) factor into possible increases or reductions in the emergency response preparedness and planning, enforcement or compliance monitoring?  
Would streamlined, but standardized HSW reporting on a Manifest or comparable document be feasible, e.g., for state and local government, the industry?
Compliance with financial assurance can be a high hurdle given conditions imposed by the environmental insurance market. All parties under this rule should be required to comply with financial assurance. Would requiring management standards for reclamation facilities that don't have RCRA permits strengthen the DSW Rule by increasing or reducing the potential for environmental releases and/or disproportionate negative impacts?
Does the EPA hazard analysis factor imposition (or not) of container integrity and container management standards into the determination of the potential or actual hazards associated with higher storage volumes, and/or higher concentrations of HSW?  What constitutes a safe "container?"  Does the analysis factor in whether making storage subject to controls, environmental monitoring and inspections increases or reduces community protection?
Intermediate facilities and off-site HSM self policing may not be effective as a means to ensure protection of health and the environment in communities whether disproportionately affected or not.
2d. 	Accuracy of EPA Differences Analysis
The threat of post facto CERCLA liability is not interchangeable with up-front environmental protection standards, notice and record-keeping requirements. Streamlined record, keeping, tracking and monitoring would contribute to ensuring that toxic wastes are not dumped illegally dumped in disproportionately affected communities and would contribute to ensuring that HSW is safely delivered to the recycling facility.
The DSW Rule could be characterized, to some extent, as industry self-certification on HSW, which contrasts sharply with RCRA's historic cradle-to-grave framework and preventative approach to hazardous waste management and disposal.
Generators and third party reclaimers must file notices  -  they should be required or the HSW should be treated as hazardous waste.
Audits of reclaimers are not the same as or superior to periodic inspections or other agency oversight. 

3. 	Identification of Potentially Affected Communities
The transition to this section should be clear. It's not clear that this section is distinct from the prior rule discussion. This is a graphics issue.
Does the large number of damages cases support granting the transfer-based exclusion?
Diversion of materials from the waste stream is beneficial (although on it face, without safeguards, reclamation falls short of environmental justice) and should be accompanied by adequate protections for health and the environment.
3a. 	Correct Identification of Recycling Facilities
Are the facilities that gave notice located; in disproportionately affected communities or not?  The recycling exclusions should not exacerbate impacts or exposures in these communities. A troubling compliance history could signal future endangerment of the health and the environment.
3b. 	Appropriateness of EPA 25-ton Threshold
Is the 25-ton threshold empirical?
3c. 	Appropriateness of Identification of Potentially Affected Communities and Populations + 3km Circle
EPA synchronizes with Professor Paul Mohai and Professor Robin Saha who are doing cutting edge demographic analysis. Their work shows increasing disproportionality, for example, nationally, people of color now make up the majority of those living within 3-km of such facilities and that where facilities cluster they make up over two-thirds. These are the highest concentrations of people of color ever reported from a national study of the distribution of hazardous waste facilities. Furthermore, when applying their recommended statistical analysis, it was found that racial disparities persist even when controlling for socioeconomic factors such as income, housing values, education, and employment status.

4. 	Analysis of Demographics of Potentially Affected Communities
The transition to this section should be clear. It's not clear that this section is distinct from the prior rule discussion. This is a graphics issue.
4a. Appropriateness of Data Sources for Demographic Analysis
EPA synchronizes with Professor Paul Mohai and Professor Robin Saha who are doing cutting edge demographic analysis. Their work shows increasing disproportionality, for example, nationally, people of color now make up the majority of those living within 3-km of such facilities and that where facilities cluster they make up over two-thirds. These are the highest concentrations of people of color ever reported from a national study of the distribution of hazardous waste facilities. Furthermore, when applying their recommended statistical analysis, it was found that racial disparities persist even when controlling for socioeconomic factors such as income, housing values, education, and employment status.
4b. Appropriateness of Environmental Justice Analysis Characteristics
Is the preventative approach a reasonable starting point when evaluating harm to the environment and public health.
Without RCRA status or Treatment, Storage and Disposal (TSD) protections, communities will many facilities are not required to get a permit, provide notice and opportunity to comment, address public input, monitor or be subject to protective container standards or storage time limits. These are baseline historic protections that, while they do exist, as regards most environmental statutes, they haven't been updated to reflect the "real world" since original passage. 
Furthermore, taking advantage of existing public participation mandates commonly requires a massive investment of usually volunteer community resources to engage in a process that is time consuming and expensive and in which, comparatively, the private sector and government stakeholders have extensive resources and dedicated staff. 
Have either the absence of these protections or the "real world" limitations on a community's ability to publicly participate been evaluated as an impact upon low income communities and communities of color?
Notice requirements inform communities, the public and municipalities about how HSW is being managed and recycled. How would the public learn about releases, spillage without notice, monitoring, recordkeeping; how would they advocate safeguards or alternatives?  Streamlined requirements would be preferable to none.
In disproportionately affected communities, the issue is not whether there will be impacts; rather the issue is what the full extent of the hazard impacts will be. 
Should recyclers be required to demonstrate a history of compliance prior to benefiting from the exclusions?  Are the facilities that gave notice disproportionately located in low income and communities of color?
Potential impacts associated with transportation of HSW warrant a closer look.
4c. Appropriateness of Areal Apportionment Methodology
EPA synchronizes with Professor Paul Mohai and Professor Robin Saha who are doing cutting edge demographic analysis. Their work shows increasing disproportionality, for example, nationally, people of color now make up the majority of those living within 3-km of such facilities and that where facilities cluster they make up over two-thirds. These are the highest concentrations of people of color ever reported from a national study of the distribution of hazardous waste facilities. Furthermore, when applying their recommended statistical analysis, it was found that racial disparities persist even when controlling for socioeconomic factors such as income, housing values, education, and employment status.

4d. 	Scientifically Appropriateness of Statistical Analysis for Assessing Environmental Justice Concerns
EPA synchronizes with Professor Paul Mohai and Professor Robin Saha who are doing cutting edge demographic analysis. Their work shows increasing disproportionality, for example, nationally, people of color now make up the majority of those living within 3-km of hazardous waste facilities and that where facilities cluster they make up over two-thirds. These are the highest concentrations of people of color ever reported from a national study of the distribution of hazardous waste facilities. Furthermore, when applying their recommended statistical analysis, it was found that racial disparities persist even when controlling for socioeconomic factors such as income, housing values, education, and employment status.
4e. 	Propriety of Statistical Analysis
EPA synchronizes with Professor Paul Mohai and Professor Robin Saha who are doing cutting edge demographic analysis. Their work shows increasing disproportionality, for example, nationally, people of color now make up the majority of those living within 3-km of hazardous waste facilities and that where facilities cluster they make up over two-thirds. These are the highest concentrations of people of color ever reported from a national study of the distribution of hazardous waste facilities. Furthermore, when applying their recommended statistical analysis, it was found that racial disparities persist even when controlling for socioeconomic factors such as income, housing values, education, and employment status.
4f. 	Accuracy and Thoroughness of Assumptions 
Assumptions appear accurate; however at this reading, this reviewer cannot determine whether they are thorough. 
Would assessment of an additional number of damages case affect the assumptions? 
4g. 	Appropriateness of Uncertainties in Data Input in Analysis
Uncertainties in the data input in the analysis appear to be appropriate.
4h. 	Demographic Analysis Alternatives
Damages timeframe
       The reviewer is uncertain if this analysis will yield additional useful information without an assessment of additional cases.
Comparison of Potentially Affected Areas and Non-Affected Areas
       The reviewer is uncertain if this analysis will yield additional useful information other than another take on areas where disproportionate impacts occur.
Compare Host Metropolitan Areas for Facilities
       This comparison may yield information about metropolitan areas where disproportionate impacts exist.
Has EPA evaluated additional damages cases?  Is the pool of 218 adequate to reveal the widest reasonable range of damages, potential damages and types of comparisons?
Do host metropolitan areas factor in rural areas where capacity, including staffing, to oversee HSM facilities may be more limited compared to urban and suburban areas.

5. 	Other Factors That Affect Vulnerability in Communities
The transition to this section should be clear. It's not clear that this section is distinct from the prior rule discussion. This is a graphics issue.
5a. 	Accuracy of Description of Other Factors That May Affect Vulnerability
The description of other factors that may affect vulnerability is accurate. 
The DSW Rule contains sound baseline vulnerability factors. 
Where does immigrant status factor into vulnerability factors?
Should whether the generator is a small business factor into community vulnerability?  For example, some small businesses could be less likely due to capacity to conduct in-depth due diligence or examinations of HSW facilities or how their wastes are recycled.
What is the universe of industrial facilities that may be generators of HSW and how do they factor into this analysis?  How do industrial facilities factor into the potential impacts analysis?
5b. 	Broad or Short Discussion of Factor Categories
What are "structural" reasons as stated in section 5.1.4?  
Ability to participate in the decision making process is a stated environmental justice vulnerability; however, this section states explicitly (in clear contradiction) that community input is not required of facilities or regulators.
The section is clear that lower participation levels can translate into experiencing greater negative impacts since "their input has not been considered fully, particularly if competing interests are set forth more effectively."  
Does the importance and/or benefit of recycling override the value of public participation especially where disproportionality is an issue?  Additionally, the studies show that facilities which have notified already show multiple facilities, higher disease incidence and a dearth of hospitals in the 3-km area.
5c. 	Additional Factors, If Any
Immigrant status, particularly new immigrants and undocumented workers and residents constitute an important additional factor.
Unfunded mandates would affect EPA ability to ensure compliance if enforcement is left to states that don't have the budget to inspect, monitor, notify the public or implement public involvement.

6. 	Conclusions
The transition to this section should be clear. This is a graphics issue.
6a. 	Conclusions Supported By Analysis
The analysis and conclusions are extensive.
Arguably, DSW Rule conclusions supporting exclusion of HSW from RCRA protections are not thoroughly supported by the analysis as it pertains, in particular, to multiple (disproportionate?) impacts, potential hazards and public participation.
6b. 	Accuracy and Clarity of Differences Between Community-Level and Population Level Analysis
The differences between community level and population level analysis are clear. 
6c. 	Additional Information With Significant Impact on Conclusions
Specific and explicit storage timeframes and container requirements would facilitate safer handling by reducing the prospects for potential explosions, spills and other uncontrolled releases of HSM.
Off-site reclamation  -  regulatory safeguards that ensure proper management of HSW while on-site and during shipment are important.
Is there sufficient analysis of the impacts of storing HSW in land-based units such as pits, ponds and lagoons. Should standard RCRA regulatory controls apply where HSW is stored in such land-based units?
Does the DSW Rule consider the affects of tracking, monitoring and enforcement across multiple state lines or, nationally, whether HSW transport will operate smoothly across multiple state programs?  Is there favorable or negative impact on interstate commerce?
What constitutes "reasonable effort" is not clear. This may not be a sufficiently high bar to deter mismanagement of HSW and could encourage high risk conduct.
Should record-keeping include quantities of HSW generated?  This data would be useful for emergency planning and response as well as helpful to determining economic and environmental benefit.
Has EPA assessed profitability and the viability of end-use markets for various HSW and facilities?  Would lack of markets affect, encourage or discourage a facility from taking steps and investing in protecting health and the environment?
Does the DSW Rule facilitate broad adoption by the states and state implementation?
Tribes, including Inuits, will need additional resources to staff and implement the DSW Rule. Should more cases on Native lands be included in the damages cases analysis?
Encouraging recycling should be possible while retaining a reasonable assurance of regulatory protection for health and the environment. 











                             COMMENTS SUBMITTED BY

Diane S. Henshel, Ph.D.
Owner and Sole Proprietor, Henshel EnviroComm Consulting
and
Associate Professor, School of Public and Environmental Affairs
Indiana University
1315 E 10[th], #340
Bloomington, IN 47405
812-855-4556
Email: dhenshel@indiana.edu 





External Peer Review for EPA's Draft Definition of Solid Waste Final Rule
Environmental Justice Analysis
                                       
             Response to Charge Questions by Dr. Diane S. Henshel
                                       

General
Ga.	Please state your overall assessment of the content, readability and utility of this report, including any changes needed. 

   a. Content: The environmental justice assessment process undergone was a reasonably thorough approach to assessing environmental justice concerns associated with the 2008 DSW rule.  However, there was a lot of repetition that seemed to involve just copying and pasting the same statements and phrases over and over again, and rarely elaborated in depth.  In addition, the analysis never included taking the example situations (places where greater exposure by the neighbors could occur, for example) and quantifying the potential impact on the neighboring community (maybe within the 3 km diameter defined "proximity to site" area).  Using the basic approaches for predictive risk assessment, one would take the real situation, make assumptions for the average and high end of the range for the increased exposure situation, calculate the potential increased exposure to the chemicals, and then calculate the potential for increased risk, with average and high end assumptions.  At least one such scenario calculated for each situation identified as potentially increasing exposure under the new regulations would be helpful in determining just how critical revising the regulations would be.  It would have been even better to then recalculate the exposure and risk inherent in at least a few scenarios with the assumptions made under the recommendations for revision of the 2008 DSW rule, weak and non-specific as those recommendations were. 
   b. Readability.  Unfortunately, this document is pretty standard for an EPA document.  If the audience is only technical, the document is usable and readable. If the audience is intended to be the EJ communities and include a non-technical audience, this document is dense and difficult to read.  There is also a lot of "spaghetti" logic  -  explanations that are internally referencing without sufficient definition of terms or acronyms.  The document is missing some key components that would make it more readable.  These include: 
      oo A Glossary
      oo A list of abbreviations and acronyms
      oo Remembering to define every acronym the first time it is used in every chapter, and in the legend for every table.
      oo A few diagrammatic algorithms that could help explain two complexities:
            o The regulations that apply under each condition that increases exposure under the new 2008 DSW.  This algorithm could be bilaterally symmetrical  -  taking the reader through the process, and when there is a change that occurs due to the regulations, the algorithm can identify the regulations in play and not in play for each set of exposure conditions (as in longer stored waste materials, transport exposure, etc).  One side of the symmetrical algorithm would identify regulations in play before the new 2008 DSW regulations, and the other side would identify the regulations in play after the new 2008 DSW regulations.
            o A second algorithm that would be useful would be to outline the framework, the process used to analyze the environmental justice concerns about the 2008 DSW rules.  This algorithm could then tersely identify each step in the process and the implications for the outcomes of each step.
   c. Utility.  This analysis and report does demonstrate that there already is a bias in the location of the facilities that already produce hazardous waste, and this includes a bias in the facilities that might utilize this rule to increase storage time and volume of exempted otherwise hazardous materials.  The environmental justice analysis could have been more complete.  Further discussion of other approaches are included below. The report does make reasonably recommendations for some of the ways that the environmental justice concerns might by reduced, although the discussion of these abbreviated comments was both vague and needed elaboration. 

Gb.	Is this information accurately and clearly presented?  How might the document be improved?

   a. Needed improvements for clarity: As mentioned above, it would be very helpful for the general reader to provide a Table of Abbreviations/Acronyms and a Glossary, in addition to remembering to follow some other basic rules of clear communication as outlined above (Readability).   In addition, the excessive use of acronyms makes the document very hard for any lay reader to read.  It makes the document hard to read for even an expert, as one must constantly look for the meaning of the new acronyms that pop up pages after they have been defined.  It would be helpful, should the acronyms stay in such a density, to provide a list of acronyms used on each page in a footnote at the bottom of each page.
   b. The Wind Dispersal category would be better (and less confusingly) labeled as "Particulate [or Dust] Wind Dispersal."
   c. There is no need to repeat the exact same phrases multiple times without additional elaboration.
   d. In no scenario was there a consideration of generators who consistently function in violation of the regulations once they start handling waste under the 2008 DSW rule.  As there are such companies, this would seem to be a scenario to include, or at least discuss, in an EJ analysis. 
   e. No scenario considered the possibility of the impact on the community if a generator acting under the 2008 DSW rules goes out of business with accumulated, non-processed, possibly not appropriately contained excluded hazardous material waste on the property.  Yet this situation would only happen (legally) under the 2008 regulations, not under the previous HSM regulations. 
   f. The methods for analysis of environmental justice impacts (as in biases in the siting) are not  as sensitive as might be obtained with using regression analysis and grouping the data into maybe quartiles rather than doing all analyses on data that has only been divided into two categories  (ie equivalent of yes/no data).

Gc.	Does the document adequately review, present, analyze and summarize the available data? If not, what can be improved?

   a. This is a nice review of the data.  This is a nice first analysis.  Improvements can be made.  This question is a perfect example of the redundancy that could be avoided.

Executive Summary
ESa.	Is the Executive Summary clear and easy to read?

   a. The executive summary is readable.  The executive summary needs to include definitions of terms used (for example, "notification facilities," "damage case facilities," etc).  In addition, the last sentence on page ix (starts "In addition,...") is unclear and needs  rewriting.  Similarly the last sentence on page xi needs additional clarity.   The summary of Finding 4 should include some discussion of the other three categories not discussed in the executive summary (example, susceptible populations).  You might include leaking septics in the list on the top of page xiii (under multiple and cumulative effects), as leaking septics are more likely to be addressed more slowly, and thus pose a higher health risk, in low income communities.

ESb.	Does the Executive Summary provide an accurate view of the contents, findings, and conclusions of the analysis? If not, please specify how it could be improved.

   a. See above comments.  But yes, the Executive Summary does provide a reasonably accurate view of the findings and conclusions of this analysis.
   
1.	Introduction
1a.	Does the Introduction clearly lay out the contents of the analysis? If not, please describe how it could be improved.

   a. No complaints.

2.	Hazard Characterization
2a.	Do the potential scenarios under the definition of solid waste final rule clearly describe how EPA thinks facilities may react to the rule? If not, please suggest where improvements should be made. 
   
   a. What is lacking overall from the analysis, from the Hazard Characterization step onward, is any discussion or analysis of the implications (i.e secondary consequences) of each of these assumptions (about the possibility of increased exposure) for total exposure and estimated changes in risk values for the potential EJ communities (ie those communities within 3 km of each facility, according to this report).

2b. 	Are these scenarios plausible, and are there any other likely scenarios? Please explain. 
   
   a. "Unlikely" scenario 1: If the cost of recycling off-shore (Africa, China) are low enough, these lower costs will offset the additional costs of transportation to get the now excluded hazardous waste (which costs less also as an excluded waste, and one for which the same expensive containers do not have to be bought).  This is a reasonable scenario (not unlikely) unless the EPA provides calculations to prove that the costs of transportation would never be overcome by the reduced costs of recycling in Africa or China.  It is also disturbing that there is insufficient discussion and analysis of EJ concerns for the locations of the recycling plant and no discussion and analysis of EJ concerns for the locations where recycling is carried out in another country.  (This last sentence applies as well to the whole report.)
   b. Two other possible scenarios to address are listed above under "general comments".

2c.	Is EPA's analysis of the types and quantities of hazards from the definition of solid waste final rule thorough and accurate? Please explain. 
   
      a. Toxicity information: When applicable, endocrine disruption toxicity is not even mentioned.  Developmental toxicity of heavy metals would also seem to be an important, very sensitive effects indictor to mention.
      b. p 19 top of the page. There isn't mention of fugitive emissions (leaks at valves, escape at points of transfer) as part of the way in which hazardous materials/waste can be released.
   
2d.	Is EPA's analysis of the differences between hazardous waste regulations and the regulations in the definition of solid waste accurate? If not, please explain. 
   
   Detailed Comments to Improve Writing:
      a. Page 3 Inset.  Typo or residual undeleted "("  in explanation for Generator-Controlled Exclusion.
      b. Section 2.3 p 30, third paragraph down  -  This paragraph needs rewriting to make it clearer.  A particularly difficult sentence is sentence 3, and the logic between sentences 3 and 4 may be faulty.  If a material's reporting quantity is lower when it is characterized as hazardous waste then if it is characterized as non-waste, as written in sentence 3, then a smaller quantity would be needed before it exceeds the reportable quantity as a hazardous waste compared to as a non-waste, not a larger quantity (as currently written in sentence 3).  But this does mean that if the implementation of the DSW results in what would otherwise be considered hazardous waste being now considered non-waste, then indeed more of the same material (now classified as "non-waste") would be required to trigger reporting requirements then would have been true if the material were classified as hazardous waste.
      c. P 32, end of second full paragraph.  A "reasonable" effort is not defined or discussed.  Without such discussion or clarification, a "reasonable" effort remains so open to interpretation that a phone call to the foreign facility owner and asking for assurances of safety would more than satisfy that requirement, and yet not provide any real assurance that the community and workers in the foreign facility would be protected from hazardous exposures.
      d. p 32 end of the second bullet there's a typo, a misplaced period.
      e. As an example of a need to define terms and abbreviations, K061 is used in the footnote on page 27 (footnote 25) without any explanation of the meaning of the term.
      f. When requirements or criteria are referred to, it is important to specify and list those requirements or criteria, in the text, a footnote, or an inserted box.  An example of such a referral to insufficiently explained and detailed requirements are on the top of page 28.  The details about the "SQG" requirements are lacking.
      g. The discussion about reportable quantities (second full paragraph down on page 30) needs to be written more clearly.  The third sentence in that paragraph is particularly problematic.


3.	Identification of Potentially Affected Communities
3a.	Has EPA correctly identified the facilities that may be likely to begin recycling hazardous secondary materials under the definition of solid waste exclusions? If not, what facilities are missing?
3b.	Is EPA's use of a threshold of 25 tons of recyclable hazardous secondary material generated annually an appropriate way of identifying generators that might choose to recycle onsite?  If not, what threshold would be more appropriate?
3c.	Has EPA appropriately identified the communities and populations that are potentially affected by these facilities?  In particular is the 3-km circle the appropriate area of consideration?  Please explain.
   
   a. Other environmental justice analyses have used other diameters to define the proximate neighborhoods.  On studies of Indianapolis, especially, we have tested multiple diameters (all smaller than 3 km  -  0.5, 1 and 2 km) and typically found 2 km to be a good cut-off to define proximate communities.  It would seem that this question should be answered in the study, that is, that the study design would test multiple possible diameters (or donuts) around the facilities and determine which diameter provides the most sensitive distinction. It is quite possible that different types of facilities would have different cut-off diameters, based on differences in potential for dispersal/distribution of (excluded) hazardous waste (i.e. as liquid solvent waste or dust).
   b. Other problems:
In conducting similar EJ analyses, using TRI facilities, we discovered that we had to truthed the facility addresses to the GPS coordinates by on the ground determination of whether the address listed for the facility was actually the address of the releasing facility or the address of company offices.  Even when the two buildings were on the same campus, and not separated by miles, the difference between the buildings (between the addresses) could be a significant distance (on a mile long campus, they could be a mile apart), and thus could affect which neighborhoods were included in the proximate circle.  From the information provided in chapter 3 about the methods used for obtaining GPS coordinates, it appears that this truthing was never done or even attempted for this analysis.  No efforts were apparently made to ascertain whether the listed address referred to the hazardous materials processing location or to the main offices.

4.	 Analysis of Demographics of Potentially Affected Communities
4a.	Are EPA's data sources for its demographic analysis appropriate? Should EPA consider any additional data sources for this information? Please explain.

   a. Using census data is standard protocol for many EJ analyses.

4b. 	Did EPA select appropriate characteristics for analysis of environmental justice concerns in communities? 
   
   a. The same methods and break outs should have been carried out for all analyses.  There should not have been different protocols and analyses used for some types of facilities and not for others.  There should have been a break out of asian, black, other minority, and possibly immigrant populations for all facilities.
      
4c.	Was the choice of the areal apportionment method to determine the characteristics of the communities surrounding each facility appropriate and properly conducted? 

   a. It appears, yes.
      
4d. 	Is a statistical analysis of these results to determine how the demographic characteristics compare to those on a county, state, and national level, in both urban and rural communities, a scientifically appropriate way to assess the existence of environmental justice concerns in connection with this rulemaking?  Are there additional methods that should be considered for analyzing the environmental justice concerns with this rule?

   a. Yes, statistical methods are appropriate in general.  However, a more sensitive analyses seems to be to break the demographics up (quartiles, quintiles) into some binned division of the total range, create donuts around the facility, and run the analyses as a regression of the demographics against proximity.  You are able to detect (significantly) more subtle shifting of demographics over time.
   b. It is also appropriate to use the more commonly used cut-off of 2x the poverty line, since even families at 2x the poverty line are struggling just to provide food and shelter these days.  The census determined poverty line (that is, above or below the threshold) does not include childcare costs, as one example of the kind of critical expenditures that enable both parents to work.  Thus it would be appropriate to test poverty threshold versus 2x*poverty threshold as the criteria for determining bias in representation of low income populations with regard to proximity to the HSM or DSW facilities.
   c. Given the neighborhoods in which these facilities are placed, it would seem to be appropriate to at least consider other toxic stressors, such as TRI facilities or road density as part of the EJ analysis looking at cumulative exposure potential (based on permit data for the HSM/potential DSW facilities for maximal exposure estimates).
      
4e. 	Is the statistical analysis in this section properly done?

   a. There is a lot of detail provided for some of the analyses.  Other analyses (producing some of the results listed in the tables) do not have such clear or extensive or sometimes even any clear statement of how some calculations were made.  At the very least, a clear, jargonless description of how each column in each table was determined should be evident in the text in proximity to the tables.
   b. There needs to be an explanation of why some Fisher's analytical results are listed as N/A.
   c. Why are the same charts repeated twice?  Is this necessary?
   d. The analysis should be broken out by state for all the analyses, and similarly, the notification facility results could be summed across states, as was done for the other types of facilities, even though the states covered are small in number compared to the total number of states represented in the analyses for the other facilities.  That discrepancy should be pointed out in the legend.  Yet there needs to be some way to compare across analyses.  Only summing across states for most, and separating out one subset of data makes it hard to compare across results.
   e. The same set of demographics should be tested for all types of facilities for all types of analyses.  There is no justification for cutting back to 4 categories of analysis for some analyses, and none is given.
   f. The first set of analyses for each facility type would be better and more clearly represented by box and whisker plots.  When using box and whisker plots, it is important to explain (for the lay audience) what data is represented in each part of the plots at some point in the chapter.  With that explanation, the box and whisker plots provide more information for the comparisons, and provide a visual sense of variability and bias around the means.
      
4f. 	Are the assumptions in the analysis presented accurately and thoroughly?
4g. 	Are the uncertainties in the data input into the analysis appropriately accounted for and discussed?  

      oo No  -  see comments above about problems with addresses linked to facilities that have corporate offices away from the processing facilities, even when the two buildings are on the same campus.
         
4h.	Based on internal EPA review of the document, the Agency is considering additional approaches for analyzing the demographic data.  The decision to conduct additional analyses will depend in part on timing and resource concerns.  Which of the following analyses are most likely to yield new and/or improved information, and how would that information inform EPA's decision-making process?

            1.     For the category of facilities from the damage case study, analyze the demographics for the timeframe the damages occurred.
               
      oo Your meaning is not fully clear in this question, and it is not clear what analyses you would do to address this question, and how you would break up the data.  I am also concerned that the number of facilities in each group might then be so small as to really decrease power to detect significant differences.
            2.     Compare potentially affected areas to non-affected areas.  Compare the demographics in the 3-km concentric areas surrounding potentially affected facilities to demographics of areas in the state not containing facilities, rather than to averages for the entire state.
      
      oo Yes  -  this provides a better comparison group  -  a counterfactual.
         
            3.     Add comparison to host metropolitan areas for facilities situated in metropolitan areas.  
               
            - Yes, this also would be a very appropriate comparison to include.
            
5.	Other Factors that Affect Vulnerability in Communities
5a. 	Has EPA accurately described the other factors that may affect vulnerability in communities and their relevance to this analysis? 
5b.	EPA has included a discussion of five broad categories of other factors that affect vulnerability in communities.  Based on available information, only two of those appear to be specifically associated with the 2008 DSW rule.  Is the broader discussion helpful, or would it be better to focus more specifically on the factors associated with the rule?  Please explain.

         a. It is better to keep the discussion broader.  In this kind of document, it is important to include the negative results as well as the positive results.
            
5c.	Are there any additional factors that should be described? If so, please describe.

      a. Other health endpoints to consider include: low birth weight as a measure of embryonic development; some measure of endocrine disruption including, and particularly, thyroid disruption, although other endocrine function indices should also be considered, such as diabetes; and some measure of IQ in children, if this is available.
         
6. 	Conclusions
6a.	Do the results of this analysis support EPA's conclusions? 
   
   a. Yes, the results support EPA's conclusions, but likely not as well, currently, as they might with more extensive EJ analysis carried out on the data.


6b.	Has EPA clearly and accurately explained the difference between its community-level analysis and its population-level analysis?
   
   a. I think so, but I am not fully sure that my answer comes from the readability and clarity of the explanations  -  I suspect my familiarity with the information is affecting my comfort with the descriptions.
   
6c.	Please identify any additional information that would make a significant impact on the conclusions of this analysis. 
   
      a. Table 5.1 would allow for a very nice small cluster analysis (or principal component analysis) to look at which emissions and types of facilities link with which endpoints, and (using maybe MANOVA) incorporating transportation and health care availability would be interesting as confounding variables.
      b. The chapter analyses seem to be ignoring the risk ratio data. Since the "full" set of analyses (ie Fishers Exact Test, for example) did not actually carry out the full set of analyses (separating out Black and Asian, for example), it seems that the results might have been stronger with the additional analyses.
      c. Table numbers are not provided.
      d. Population level analysis table results seem to show that there is a numbers (sample size) bias in the level of statistical significance.  Once the notification facilities used in the analysis are pooled, or incorporate facilities from other states as will, to increase sample size, chances are that these results will also be significant in a pooled state analysis.  Thus the need to do both pooled state, and divided state, to assess how much of the sensitivity might be sample size.  It is also important to note that in regions with few Blacks, or few minorities, the minority population comparisons are always less of a driver than the low income analysis, as in these regions, are larger percentage of the low income community is white.  This is true in Indianapolis, where there is a significantly large percentage of poor whites.  In Northwest Indiana, however, there are more minorities, and minority population comparisons drive the EJ analysis results as strongly, or more strongly, than the low income analysis.
      e. P121  -  7[th] line down in paragraph  -  typo  -  higher total population than.... I believe was intended.
      f. P 123 Multiple and Cumulative Effects  - note the numbers with % above 70% (most) and above 60% (all but a few!!!)  -  and a cumulative frequency chart might make a nice and very visually effective summary of these observations.
      g. The proposed regulatory and implementation measures section needs solid expansion.  It's truly pitiful right now.  It should also be noted that without regulatory clarification of some of the weaknesses in the 2008 DSW rule, enforcement is only as strong as the environmental consciousness of the administration.  Even with revised regulation, enforcement and oversight is susceptible to political influence, just somewhat less so. Monitoring is not as much a part of the 2008 DSW rule, so how could monitoring data be used to help ensure facilities are not creating more of an exposure hazard for their communities under the new rule?
      h. Comments on appendices:
            oo Appendix A  -  Clarification is needed.  
                    1.                      page A-4  -  second paragraph  -  last line  -  "typically 120 days following promulgation" of what? 
                    2.                     Page A-4 still  -  last paragraph  -  5 lines down  -  please add examples here to each type of industry not subject to CAA regulation under the 2008 DSW rule.
                    3.                      Page A-6 third bullet  -  clarify if you mean leaks to only ground, to surface water, or also leaching to ground water  -  ie clarify
                    4.                     Page A-7 last bullet  -  capitalize OSH.
                    5.                     Page A-8 first bullet  -  a great example of spaghetti writing.  Spaghetti writing makes the report more difficult to read for everyone, especially the lay person.
                    6.                      Page A-9 top of the page.  A mention of the differences in monitoring would be appropriate here.
                    7.                     Page A-10  -  second full paragraph (starts... "Further...") -  This paragraph needs some clarification, and needs rewriting.
                    8.                     Page A-10 section 3.9 second two paragraphs.  These paragraphs are very unclear, and also seem to be internally inconsistent, as written.  Clarify in these paragraphs what is included in each type of exclusion.  A good example of a clearly written paragraph is the next paragraph (first paragraph of 3.10).  This paragraph includes a short but clear definition which enables the reader to comprehend the information provided more readily.  The reader does not have to go searching for the definition, which helps reduce reader confusion.
                    9.                     Page A-10 last paragraph, first sentence.  Add clarifying phrases  -  what kind of characteristics, and where are these listed.
                    10.                     Page A-11. First paragraph needs a sentence inserted clarifying what could be exempted.  Second paragraph needs clarification in the middle sentence  -  specify briefly or provide example (in parentheses) for conditions under which listed waste may not be determined to be hazardous waste.
                    11.                     Table A-11 page A-13 middle column, middle section  -  This is too dense for the average reader referring only to CFR citations and statutes (in inconsistent style no less).  Provide details or names for these sections and their overall regulation titles in clear, jargon-free writing.
                    12.                     Table A-11  -  Overall  -  this table (and the report) lacks any discussion of the implication of these changes due to the adoption of the 2008 DSW rule.  An example might be to evaluate the implication of leaking containers, which (under the current vagueness of the 2008 DSW ruling language) might not be caught for up to 2 years, or maybe longer.  What kinds of exposures might be increased to what diameter population under this scenario?
                    13.                     Page A -17  -  why the changed font size in column 1?
                    14.                     Page A-18.  Last paragraph in column 1.  Include a list of the 140 toxic and flammable regulated chemicals referred to here, and elsewhere in the report.  The easiest way to handle this might be a footnote, or adding the list to the appendices, and referring to the inserted list (footnote, endnote, or appendix) here and whenever else this list is referenced.
                    15.                     Page A-21 first column, second line  -  missing an article ("the" or "a") between "above" and "specified quantity".
                    16.                     Page A-26  -  last column, first paragraph  -  this section (and its text tie-in) is just calling out for a discussion of the implications of lack of required training on personnel at the facility if exposures indeed rise.
                    17.                     Page A-29 first column  -  need a space between paragraphs.
                    18.                     Page A-30 third column, first word "these" needs clarification  -  to which regulations is the writer referring.  Do not expect the lay audience to know for sure.  Same point for page A-32 third column, last paragraph, second line and again page A-33 second line.
                    19.                     Page A-34 third column second paragraph.  This very statement again seems to call out for a discussion of the implications of the lack of record keeping and exception reporting.  It is NOT sufficient to point out that the 2008 DSW rule changes the recording and record keeping requirements.
                    20.                     Page A-35 first column middle paragraph  -  clarify, provide more detail.  Similarly third column second paragraph  -  specify which excluded materials may not qualify as a DOT hazardous material, and provide some discussion of the potential implications of this from an EJ perspective.
                    21.                     Page A-36 third column second paragraph  -  this again needs further discussion here and in the text on the possible implication of this change due to the 2008 DSW rules.  Specifically, would any residues from reclamation ever be potentially hazardous and NOT get identified as such through the 2008 DSW process.
                    22.                     Page A-37 GCE: is something missing here?  Seems like it.
                    23.                     Page A-49 right hand column, second paragraph  -  use the full citation, not the abbreviated citation.  The full citation (ie using the same citation format throughout) is more understandable to the lay person.
                    24.                     Page A-52 right column, top line "an SPCC" fix typo
                    25.                     Page A-61 middle column, top paragraph.  Keep citation format consistent and summarize the legitimacy criteria in this paragraph, a footnote, or the next paragraph.
                    26.                     Page A-63 right column.  Provide examples of potential problematic emissions that don't meet threshold criteria and don't meet other applicability criteria.  It would be great if the implications of the resultant exposure (were this to occur) be determined quantitatively, maybe at the mean/lower assumption (9.5 TPY single type HAP) and high assumption (24.95 TPY mixed HAP) levels.
                    27.                     Page A-64  -  right hand column  -  No one in this report is discussed how much might be released before anything would trigger the need to determine if a cleanup is needed.  Surely this would help explore the potential for a significant disproportionate impact occurring? 
                    28.                     Page A-68, right column, last sentence.  Where is the "note below"?  Probably better to put the note in a footnote on the same page or in the right hand column immediately after this paragraph.
                    29.                     Page A-82 on  -  It is great to have the summary of the Federal Regulations here.  However, the formatting and depth of each discussion is not consistent and needs to be made so.  For example, the most readable formats start with a sentence or two that talks about the relevant regulation(s) more broadly, and then includes a sentence that explains the relevance of the regulation to this report/study.
                    30.                     Page A-83 Second paragraph under "Legitimate recycling"  -  include a brief summary of the six criteria referred to here (and elsewhere) and not clearly defined.  The listing could be in the paragraph, in an offset list, in a separated inset box, or in a footnote.
                    31.                     Page A-84  Top of the page  -  provide details about standards in 262.34 (d) and (e).  They are not clearly described anywhere.  Also maintain a consistent formatting for citations.
                    32.                     Page A-94 Exports first paragraph last two sentences  -  a) remember to keep citations in consistent format.  B) To make this document most readable, summarize these requirements briefly.
                    33.                     Page A-96  -  No where is there a discussion of at what point in a chain of non-compliance would the chain of permission stop and oversight identify a problem that needs to be addressed.
                    34.                     Page A-99 bullets under 5.3.3  In each bullet, first identify the key factors that determine whether processes fit as Program 1, 2 or 3 (i.e. defined for the appropriate bullet).  These three was to characterize a process are never clearly defined throughout this whole document.  This would seem to be the place to put those definitions.
                    35.                     Page A-106 5.8.2 line 3  -  TSDFs is an acronym that is never defined.
                    36.                     Appendix B is good to have, as is Appendix C and D.  Note that in Appendix B, and in a few other places, the writer uses the personal "we" instead of a noun.  This is formal writing and should not include any first person (personal) pronouns anywhere in the document.  Search to find the "we"s and rewrite to replace the personal pronoun with a clearer sentence using the impersonal 3[rd] person or noun.
                    37.                     Page B-2 first paragraph.  Need to add a sentence that explains why it is important to understand what a "wastewater" versus a "nonwastewater" is.
                    38.                     Page B-3 table bottom:  POTW and NPDES is not defined here or elsewhere in the document.  Acronyms are best spelled out the first time they are used in any given chapter, and still it is kind to the read to provide a Table of Acronyms and Abbreviations, and spelling out of all acronyms in footnotes for any table in which they are used.
                    39.                     Appendix C is written like an abbreviated technical reference manual, but without the definitions usually included in such a manual.  Most of this section reads like a jargonfest, and is not accessible to the lay reader.  So, 
                        a. Identify all acronyms and abbreviations.  There's two in the second paragraph.
                        b. In the introductory section, explain by the K, F, U, P and D codes represent, how they are used, and how understanding the coding was needed for the environmental justice analyses.
                        c. Explain all codes that have no other referent to explain them.  For example, on page C-5 top line, F037-F038 codes are referred to without placing them in any context or defining what these codes cover.
                        d. Table C-1 needs some additional explanation before it will be easily read by the lay reader.
                    40.                     Appendix D is written in a way that is essentially understandable.  However in two places, where groups are moved from the logical grouping to a not as obvious grouping, explanation is needed for why the move was made.  These points are on Page D-3 bottom of the page, and on Page D-5 top of the page.
                    41.                     Appendix E needs some clarifying verbiage written in easy to understand English.  The two ratios could, for example, be easily explained by one or two sentences.  The description of the tests are more theoretical in places.  Specific places that clarification is needed includes:
                        a. Page E-9 bottom of the last Fisher Exact Test paragraph, there is no explanation for why or when the minus sign is removed, nor where the minus sign is likely to show up and why.
                        b. Page E-10  -  it is not clear what the tables are for the various states.  Add clarifying language.
                        c. Since these tests are not always run (but should be) the verbiage should include an explanation of why they are not run in some situations in the analysis and are run in other, apparently similar situation.

