EPA-HSRB-09-10

Kevin P. Teichman, PhD

Acting Science Advisor

Office of the Science Advisor

1200 Pennsylvania Avenue, NW

Washington, DC 20460 

Subject: October 20-21, 2009 EPA Human Studies Review Board Meeting
Report

Dear Dr. Teichman,

	The United States Environmental Protection Agency (EPA or Agency)
requested the Human Studies Review Board (HSRB) to review two completed
intentional studies examining the effects of intentional exposure of
human participants to pesticides containing pyrethrins/ pyrethroids. The
Agency proposes to rely on these two  studies, conducted prior to
publication of the EPA’s expanded final rule for protection of
subjects in human research (40 CFR 26) on February 6, 2006 (71 Federal
Register 24, 6137), for regulatory actions under the pesticide laws. The
Agency asked the HSRB to advise the Agency on a range of scientific and
ethics issues regarding how the studies should be assessed against the
provisions in 40 CFR sections 26.1701 – 26.1704 of the final human
studies rule.

	The Agency also requested the HSRB to provide scientific and ethics
reviews of two proposed human studies: a proposed Carroll-Loye
Biological Research, Inc. (CLBR) insect repellent efficacy study
(LNX-003); and a proposed Antimicrobial Exposure Assessment Task Force
II (AEATF II) aerosol spray application protocol (AEA04).

	The enclosed report provides the Board’s response to EPA charge
questions presented at the October 20-21, 2009 meeting.

Assessment of Completed Research Study: Newton, J., Breslin, A. (1983)
Asthmatic reactions to a commonly used aerosol insect killer. Medical
Journal of Australia 1:378-380. 

Science

Because of the complex nature of the insecticide product that was
evaluated, the Board concluded that the Newton and Breslin study was not
relevant when considering the asthmatic or allergic respiratory response
for pyrethrins/pyrethroids as a class. The Newton and Breslin study was
relevant when considering the asthmatic or allergic respiratory response
only for the specific insecticide product that was tested.

Considering the substantial limitations of the study, the Board
concluded that the study was of limited utility. The Board recommended
that the Agency be cautious in its use of these data, limiting it to a
qualitative contribution to the overall weight of evidence. 

Ethics

Given the limited information available about this study, the Board
concluded that there neither was clear and convincing evidence that the
study was fundamentally unethical, nor clear and convincing evidence
that the study was significantly deficient relative to the ethical
standards prevailing at the time the research was conducted.

Assessment of Completed Research Study: Lisi, P. (1992) Short
Communication: Sensitization risk of pyrethroid insecticides. Contact
Dermatitis 26:349-350. 

Science

The Board concluded that the Lisi study was likely sound, but the
brevity of the report and limited details as to how the study was
conducted prevent an accurate assessment of its reliability.  

If carried out according to accepted guidelines and criteria, the Lisi
study likely provides hazard assessment data that are relevant to
assessing whether exposures to pyrethrins/pyrethroids may be associated
with allergic contact dermatitis or sensitization responses.

Although the data presented in the Lisi study suggest that the seven
compounds tested pose little risk of allergic contact dermatitis or skin
sensitization, unanswered questions about participant history and
selection, study procedures, and outcome definitions should be taken
into account when considering these data in the overall analysis.

Ethics

Given the limited information available about this study, the Board
concluded that there neither was clear and convincing evidence that the
study was fundamentally unethical, nor clear and convincing evidence
that the study was significantly deficient relative to the ethical
standards prevailing at the time the research was conducted.

Assessment of Proposed Carroll-Loye Biological Research Study LNX-003:
Efficacy Test of KBR 3023 (Picaridin; Icaridin) - Based Personal Insect
Repellents (20% Cream and 20% Spray) with Ticks Under Laboratory
Conditions.

Science

The Board concluded that the protocol submitted for review, if modified
in accordance with Agency recommendations and conducted accordingly,
will likely yield scientifically valid results on the efficacy of these
two picaridin-based insect repellent formulations against ticks.

Ethics

The Board concluded that the protocol submitted for review, if modified
in accordance with Agency recommendations and conducted accordingly, is
likely to meet the applicable requirements of 40 CFR 26, subparts K and
L.

Assessment of Proposed AEATF II Scenario and Protocol AEA04: Research on
Exposure of Janitorial Works Applying Pesticides Formulated as Aerosol
Sprays.

Science

The Board concluded that the protocol submitted for review, if modified
in accordance with Agency and HSRB recommendations, will likely generate
scientifically reliable data, useful for assessing the exposure of
handlers who apply antimicrobial pesticides formulated as aerosol
sprays.

The Board provided a number of recommendations for how the study could
be improved, including: clarifying the criteria for exclusion of
participants who deviate grossly from the protocol; acknowledging that
systematic differences in the air sampling results are likely to occur
between the two methods used; and, considering carefully other variables
that may influence exposure measurements.

Ethics

The Board concluded that the protocol submitted for review, if modified
in accordance with Agency and HSRB recommendations and conducted
accordingly, is likely to meet the applicable requirements of 40 CFR 26,
subparts K and L.

The Board recommended that the protocol be revised to address certain
concerns, including: ensuring that documents in Spanish are reviewed by
someone familiar with the dialect written and spoken in the target
community; design and posting of community notification flyers
appropriate for guests staying at the motels where study procedures are
likely to take place; and, exclusion of volunteers who may be at greater
risk of product-related harm due to immunodeficiency or other underlying
conditions.

Sincerely,

 

Sean Philpott, PhD, MSBioethics

	Chair

	EPA Human Studies Review Board

NOTICE

This report has been written as part of the activities of the EPA Human
Studies Review Board, a Federal advisory committee providing advice,
information and recommendations on issues related to scientific and
ethical aspects of human subjects research.  This report has not been
reviewed for approval by the Agency and, hence, the contents of this
report do not necessarily represent the view and policies of the
Environmental Protection Agency, nor of other agencies in the Executive
Branch of the Federal government, nor does the mention of trade names or
commercial products constitute a recommendation for use.  You may obtain
further information about the EPA Human Studies Review Board from its
website at   HYPERLINK "http://www.epa.gov/osa/hsrb" 
http://www.epa.gov/osa/hsrb .  You may also contact the HSRB Designated
Federal Officer, via e-mail at   HYPERLINK "mailto:phre@epa.gov" 
phre@epa.gov 

	In preparing this document, the Board carefully considered all
information provided and presented by the Agency presenters, as well as
information presented by public commenters.  This document addresses the
information provided and presented within the structure of the charge by
the Agency.

US ENVIRONMENTAL PROTECTION AGENCY

HUMAN STUDIES REVIEW BOARD

Chair

Sean Philpott, PhD, MSBioethics, Director for Research Ethics, Assistant
Professor of Bioethics, The Bioethics Program of Union Graduate College
and the Mount Sinai School of Medicine, Schenectady, NY

Vice Chair

Janice Chambers, PhD, DABT, William L. Giles Distinguished Professor,
Director, Center for Environmental Health Sciences, College of
Veterinary Medicine, Mississippi State University, Mississippi State, MS

Members

Suzanne C. Fitzpatrick, PhD, DABT, Senior Science Policy Analyst, Office
of the Commissioner, Office of Science and Health Coordination, US Food
and Drug Administration, Rockville, MD

Vanessa Northington Gamble, MD, PhD, University Professor of Medical
Humanities, Professor of Health Policy and American Studies, The George
Washington University, Washington, DC

Sidney Green, Jr., PhD, Fellow, ATS, Professor, Department of
Pharmacology, Howard University College of Medicine, Washington, DC

Dallas E. Johnson, PhD, Professor Emeritus, Department of Statistics,
Kansas State University, Manhattan, KS

Michael D. Lebowitz, PhD, FCCP, Retired Professor of Public Health &
Medicine, University of Arizona, Tucson, AZ 

Lois D. Lehman-Mckeeman, PhD, Distinguished Research Fellow, Discovery
Toxicology, Bristol-Myers Squibb Company, Princeton, NJ 

Jerry A. Menikoff, MD, JD, Director, Office of Human Subjects Research,
Office of the Director, National Institutes of Health, Bethesda, MD

Rebecca Parkin, PhD, MPH, Associate Dean for Research and Public Health
Practice, School of Public Health and Human Services, The George
Washington University, Washington, DC

William Popendorf, PhD, MPH, Professor, Department of Biology, Utah
State University, Logan, UT 

*Ernest D. Prentice, PhD, Associate Vice Chancellor for Academic
Affairs, University of Nebraska Medical Center, Omaha, NE

Linda J. Young, PhD, Professor, Department of Statistics, Institute of
Food and Agricultural Sciences, University of Florida, Gainesville, FL 

Human Studies Review Board Staff

Paul I. Lewis, PhD, Executive Director, Human Studies Review Board
Staff, Office of the Science Advisor, United States Environmental
Protection Agency, Washington, DC 

Not in attendance at October 20-21, 2009 Public Meeting



INTRODUCTION 

From October 20-21, 2009, the United States Environmental Protection
Agency’s (EPA or Agency) Human Studies Review Board (HSRB) met to
address scientific and ethical issues concerning: two completed human
toxicity studies involving one class of pesticide active ingredients –
pyrethrins/pyrethroids– conducted prior to publication of the EPA’s
expanded final rule for protection of subjects in human research. In
accordance with 40 CFR 26.1602, EPA sought HSRB review of these
completed studies. Each of these completed studies is discussed more
fully below.

In addition, the Agency submitted two protocols for conducting new
research involving human participants: one study measuring the efficacy
of two registered insect repellents containing picaridin against ticks
under laboratory conditions; and one study measuring levels of exposure
received by janitorial workers when applying a commercially-available
antimicrobial pesticide formulated as an aerosol spray. In accordance
with 40 CFR 26.1601, EPA sought HSRB review of these two proposed
protocols. Each of these protocols is discussed more fully below.

REVIEW PROCESS

On October 20-21, 2009, the Board conducted a public face-to-face
meeting in Arlington, Virginia. Advance notice of the meeting was
published in the Federal Register as “Human Studies Review Board;
Notice of Public Meeting” (74 Federal Register 106, 26861).

Following welcoming remarks from Agency officials, the Board heard
presentations from EPA on the following topics: two completed studies
involving intentional human exposure to pyrethrin/pyrethroid pesticides
(Newton and Breslin [1983], and Lisi [1992]); a proposed Carroll-Loye
Biological Research, Inc. (CLBR) insect repellent efficacy study
(LNX-003); and a proposed Antimicrobial Exposure Assessment Task Force
II (AEATF II) aerosol spray application protocol (AEA04).

The Board also asked clarifying questions of several study sponsors
and/or research investigators, including:

Dr. Scott Carroll, Principal, Carroll-Loye Biological Research

Dr. Jeffery Driver, Principal, infoscientific.com, Inc.

Mr. Shawn King, Director of Operations, Carroll-Loye Biological Research

Dr. Bryce Landenberger, Dow AgroSciences

Dr. Sami Selim, President, Golden Pacific Laboratories

Oral comments were provided by: 

Dr. Scott Carroll, Principal, Carroll-Loye Biological Research

Mr. Stephen McFadden, Independent Advocate

Dr. Lawrence Plumlee, Independent Advocate

No written public comments were provided.

For their deliberations, the Board considered the materials presented at
the meeting, oral comments, and Agency background documents (e.g.,
published literature, sponsor and investigator research reports, study
protocols, data evaluation records, and Agency science and ethics
reviews of proposed protocols and completed studies). A comprehensive
list of background documents is available online at
http://www.regulations.gov. 

CHARGE TO THE BOARD AND BOARD RESPONSE

Assessment of Completed Research Study: Newton, J., Breslin, A. (1983)
Asthmatic reactions to a commonly used aerosol insect killer. Medical
Journal of Australia 1:378-380.

Overview of the Study

	In the Newton and Breslin (1982) study, seven participants diagnosed
with asthma and a history of chest tightness were evaluated for airway
narrowing and chest tightening before and after a controlled exposure to
aerosol insecticide sprays. Participants were exposed to a
commercially-available aerosol insecticide (Mortein Pressure Pak insect
killer containing 3.0 g/kg pyrethrins, 0.9 g/kg tetramethrin, 15 g/kg
piperonyl butoxide, 7.5 g/kg N-octyl-bicycloheptene dicarbons,
propellants [chlorofluorocarbons and hydrocarbons] and solvents
[non-water based]). The aim of the study was three-fold: to study the
response of asthmatic participants to exposure to insecticides
containing pyrethrins and tetramethrin; to study the time-course of
exacerbation of asthma following exposure to the insecticide; and to
evaluate bronchial reactivity to histamine after exposure to the
insecticide. Data were reported as simple counts or proportions across
exposure groups.

Science

Charge to the Board

	Is the Newton and Breslin (1983) study scientifically sound, providing
reliable data?

Board Response to the Charge

  In light of the substantial limitations of the study, the Board
concluded that the study was of limited utility. The Board recommended
that the Agency be cautious in its use of these data, limiting it to a
qualitative contribution to the overall weight of evidence. The use of
these data should also be limited to assessments of the specific
insecticide product tested rather than pyrethrins/pyrethroids as a class
of pesticides.

Charge to the Board

	Is the Newton and Breslin (1983) study relevant to an assessment of the
proposition that exposures to pyrethrins/pyrethroids may be associated
with asthmatic or allergic respiratory responses?

Board Response to the Charge

 		

	The Board concluded that the Newton and Breslin study was relevant only
when considering the asthmatic or allergic respiratory response for the
specific insecticide product that was tested. It was not relevant when
considering the asthmatic or allergic respiratory response for
pyrethrins/pyrethroids as a class because of the complex nature of the
insecticide product that was evaluated.

Charge to the Board

	What limitations of the Newton and Breslin (1983) study should be taken
into account by EPA in assessing the proposition that exposures to
pyrethrins/pyrethroids may be associated with asthmatic or allergic
respiratory responses?

Board Response to the Charge

	The Board cautioned against using the Newton and Breslin study for
assessing the proposition that exposures to pyrethrins/pyrethroids may
be associated with asthmatic or allergic respiratory responses. The use
of a complex insecticide product rather than a specific chemical was of
considerable concern, as was the more robust response for respiratory
irritation rather than asthma or allergy reported in the study. 

HSRB Detailed Recommendations and Rationale

	The Newton and Breslin (1982) article provided an incomplete record of
the conduct of the study and analysis of the data, such that a number of
important data gaps were identified. 

	One primary concern was that the tested material contained a mixture of
ingredients, some of those ingredients at higher concentrations than the
pyrethrins.  One such ingredient, piperonyl butoxide, has been
identified previously as a potential respiratory allergen (EPA Office of
Pesticide Programs 2009). Accordingly, the Board concluded that the
data obtained by Newton and Breslin are directly applicable only to the
product tested, and are not broadly applicable to pyrethrins/pyrethroids
as a pesticide class.  

	Other major issues and limitations noted during the Board’s
discussion included: 

1.	Limited and likely inaccurate exposure data.

a. 	Exposures to the test material were not directly measured, but were
calculated based on the assessment of the amount of product delivered
per unit time. Study participants were likely to have been exposed to
the experimental product, but the lack of data on the actual exposure
rate or concentration precludes any quantitative evaluation.

 

b.	The Newton and Breslin article provides a description that is
inadequate to assess whether the facility was appropriate for this type
of testing. The test chamber was described as a 7-m3 room with large
windows on one side for observation purposes and an exhaust fan that
“removed fumes between sprays.” There is no indication, however,
that there was a standardized protocol that governed spraying procedures
or cleaning requirements between exposures. There also was no
indication whether the air exchange rate of the room or cleaning
procedures between exposures were sufficient to ensure that the exposure
levels intended by the investigators were delivered. Thus, there may be
a high degree of error in the estimated exposures, as compared to the
actual exposures.

c.	There was no mention of factors that may have affected actual
exposures, such as the distance between participant and spray nozzle,
spray droplet size, or any odorants or deodorants used to “mask” the
product spray from the placebo.

 

2.	Small sample size.

a.	Although study subjects were noted to have a predisposition for
asthma, and thus likely to be potentially susceptible to showing an
effect, only seven participants were evaluated.

b.	There is insufficient descriptive information for the seven
participants chosen, such as parameters of respiratory function and
disease state. It was also noted that the two oldest participants were
both male, but no consideration whether age, gender or other participant
characteristics may have affected the study results.

3.	Incomplete results and data analysis. 

a.	Two subjects did not complete the pretest histamine challenge, and
three subjects did not complete the post-exposure challenge. Maximum
mid-expiratory flow rate (MMEFR) was not measured for one subject. More
notable is the fact that the only subject with a significant decrease in
forced expiratory volume in one second (FEV1) did not complete either
histamine challenge evaluation. No explanation for participants’
failure to complete these study procedures and tests was provided.

b.	There are limited details on how FEV1 and MMEFR values were reported.
  It was noted during the Board’s discussion that standard clinical
practice is to report the maximum values, but it is unclear if this was
done in the Newton and Breslin study.  

c.	There was no indication that baseline FEV1 or MMEFR values were
within the normal ranges for these participants, nor any additional
detail on participant characteristics that could affect these values,
such as age, gender, body size, or underlying disease.

d.	There was no specific designation of the symptoms that each patient
experienced.   It is not clear which patient was the person not counted
among the “6 of 7” who experienced chest tightness with other
symptoms.

e.	There was no specific indication whether and how the observation of
respiratory irritation correlated with changes in FEV or MMEFR values.

f.	It is unclear which participants were tested at which levels of
exposure (e.g. 10 versus 20 seconds), how many times each person was
exposed at each exposure duration, or which volunteers received the
placebo on day 2.

 

4.	Potential information bias.

a.	Participants were not blinded to treatment, and there was no
description of how symptoms were self-diagnosed and reported by
volunteers. Without the use of standardized definitions as a reference,
there is no way to know whether each participant had the same
interpretation and reporting practices for self-diagnosed respiratory
symptoms.

Ethics

Charge to the Board 

Is there clear and convincing evidence that the conduct of the Newton
and Breslin (1983) study was fundamentally unethical, or significantly
deficient relative to the standards of ethical research conduct
prevailing when it was conducted?

Board Response to the Charge 

	Given the limited information available about this study, the Board
concurred with the Agency’s assessment (Sherman 2009a) that there
neither was clear and convincing evidence that the study was
fundamentally unethical, nor clear and convincing evidence that the
study was significantly deficient relative to the ethical standards
prevailing at the time the research was conducted.

HSRB Detailed Recommendations and Rationale 

The Newton and Breslin study was conducted in Australia in 1981 or 1982.
The Board concurred with the Agency’s assessment that most widely
recognized ethical standard at that time was the 1975 Declaration of
Helsinki, but also noted the importance of placing ethical standards
within a specific national context. The Australian National Health and
Medical Research Council (NHMRC) -- the largest public supporter of
research in Australia -- issued guidelines as early as 1966 that
requiring all NHMRC-funded research to be reviewed by an independent
ethics committee. A 1976 supplementary note to these guidelines,
prompted by and explicitly referencing the Declaration of Helsinki,
called upon all Australian research facilities to establish committees
to review all research involving human participants, NHMRC-funded or
otherwise. 

Limited information is available about how the Newton and Breslin study
was conducted. The Agency attempted to obtain additional information
about the conduct of this study, but was unable to do so. The only
available information is from the published article, which states that
the written informed consent of all participants was obtained. The
article also indicates that all participants were adults, and that none
of the female volunteers were pregnant. Although the article does not
describe the type of ethical oversight of this research that took place,
it is assumed that independent ethics review occurred in accordance with
NHMRC guidelines.

Considering the information available, the Board concurred with the
conclusions and factual observations of the ethical strengths and
weaknesses of the study, as detailed in the Agency’s Ethics Review
(Sherman 2009a). The Board also concluded that this study met all
applicable ethical requirements for such research involving human
participants, as required by the Final Human Studies Rule, according to
the following criteria: 

a. 	Not fundamentally unethical. When determining whether or not a study
is fundamentally unethical, the Board’s standard approach is to decide
if the research was intended to seriously harm participants, if it
failed to obtain informed consent, or if it was fundamentally unethical
for other reasons. In this case, there was no evidence that the study
was intended to seriously harm participants. The published article also
failed to provide evidence supporting a conclusion that the informed
consent of participants was not obtained. Given lack of clear and
convincing evidence that for any other reasons it might have been
fundamentally unethical, the Board concluded that it was not
fundamentally unethical.

b. 	Not significantly deficient. With regard to determining whether a
study was significantly deficient relative to the ethical standards
prevailing at the time the research was conducted, the Board’s
standard is to determine whether or not any ethical deficiencies
identified could have resulted in serious harm to participants, based on
knowledge available to researchers at the time the study was conducted,
or whether the information provided to study participants could
seriously impair informed consent. As noted above, there is limited
information available regarding this study. Only the published article
provides any information, and the information contained therein fails to
support a conclusion that the study was significantly deficient relative
to the standards at the time.

Assessment of Completed Research Study: Lisi, P. (1992) Short
Communication: Sensitization risk of pyrethroid insecticides. Contact
Dermatitis 26:349-350. 

Overview of the Study

	In the Lisi (1992) study, 230 volunteers were patch tested with each of
seven pyrethroids: allethrin, cypermethrin, deltamethrin, fenothrin
(assumed phenothrin), fenvalerate, permethrin, resmethrin. The goal was
to establish the irritation and sensitization potential of these widely
used pyrethroids. Of the volunteers who participated, 162 were men and
68 were women. Participants ranged from 19 to 78 years in age.
Approximately 35% of participants were agricultural workers, 12% had
previously worked on a farm, and the remaining 53% were engaged in
nonagricultural activities. Fifty-four subjects had irritant or allergic
contact dermatitis of the hands, 18 of which were correlated with
agricultural activities. A further 176 participants had non-allergic
skin disorders, and 16 participants were also atopic. Simple counts and
proportions were used to compare irritation and sensitization effects
across the three occupational groups. 

Science

Charge to the Board

	Is the Lisi (1992) study scientifically sound, 	providing reliable
data?

Board Response to the Charge

	The Board concluded that the Lisi study was likely sound, but the
brevity of the report and limited details as to how the study was
conducted prevent an accurate assessment of its reliability.  

Charge to the Board

	Is the Lisi (1992) study relevant to an assessment of the proposition
that exposures to pyrethrins/pyrethroids may be associated with allergic
contact dermatitis or sensitization responses?

Board Response to the Charge

	If the Lisi study was carried out according to accepted guidelines and
criteria, it likely provides hazard assessment data that are relevant to
assessing whether exposures to pyrethrins/pyrethroids may be associated
with allergic contact dermatitis or sensitization responses.

	

Charge to the Board

	What limitations of the Lisi (1992) study should be taken into account
by EPA in assessing the proposition that exposures to
pyrethrins/pyrethroids may be associated with allergic contact
dermatitis or sensitization responses?

Board Response to the Charge

	A lack of experimental detail and subject history represent two major
limitations of the Lisi study. The Board also noted that the
allergic/irritant response rate was extremely low in the study. This may
suggest that the seven compounds tested pose little risk of allergic
contact dermatitis or skin sensitization but the low response rate,
along with concerns about participant history and selection, study
procedures, and outcome definitions, should be taken into account when
considering these data in the overall weight-of-evidence analysis.

HSRB Detailed Recommendations and Rationale

The extremely brief nature of the report raised a number of questions
about how the study was conducted. Because the study was reportedly
performed according to standards and criteria established by the
International Contact Dermatitis Research Group (ICDRG), the Board
assumed that standard testing procedures were used.  

	Major issues and limitations noted during the Board’s discussion
included:  

1.	Poorly defined participant population.  

a.	It was unclear how the 230 participants were chosen for study.  

b.	Subjects were identified as “agricultural workers,”
“ex-agricultural workers,” or “other,” but there was no
explanation for creating and using these categories.

c.	Participant compliance with study procedures was not reported. There
was no indication as whether any participants withdrew or were otherwise
not observed, nor did the report provide any detail whether all test
patches were evaluated for all participants on days 2 and 3.

2.	Lack of protocol detail.

a.	ICDRG protocols may have been used, but the report provides few
details on how investigators distinguished allergic versus irritant
responses or any information on how they graded response severity. 

3.	Low response rate.

a.	The low response rate (less than 2% of all study participants)
suggests that the seven compounds tested pose little risk of allergic
contact dermatitis or skin sensitization. However, these data could also
be indicative of problems in study design and conduct that invalidate
the results. Because the appropriate positive and negative controls were
not included in the study design, the Board felt that it could not rule
out this alternative explanation for the low response rate.

Ethics

Charge to the Board

Is there clear and convincing evidence that the conduct of the Lisi
(1992) study was fundamentally unethical, or significantly deficient
relative to the standards of ethical research conduct prevailing when it
was conducted?

Board Response to the Charge 

Given the extremely limited information available about this study, the
Board concurred with the Agency’s assessment (Sherman 2009b) that
there was neither clear and convincing evidence that the study was
fundamentally unethical, nor clear and convincing evidence that the
study was significantly deficient relative to the ethical standards
prevailing at the time the research was conducted.

HSRB Detailed Recommendations and Rationale 

The Lisi study was conducted in Italy sometime in 1990 or 1991. The
Board concurred with the Agency’s assessment that the most widely
recognized ethical standard at that time was the 1989 Declaration of
Helsinki, but again noted the importance of placing ethical standards
within a specific national context. Italy had established a National
Bioethics Committee in 1990. That Committee may have developed
directives applicable to this study. Nevertheless, as explained below,
the potential existence of those national standards does not alter the
Board analysis. 

Extremely limited information is available about how the Lisi study was
conducted. The EPA attempted to obtain additional information about the
design and conduct of the study, but was unable to do so. The only
available information is from the published article, which does not
state whether written informed consent of the participants was obtained.
The published article also does not address what type of ethical
oversight took place (e.g. review by an independent ethics committee).
Study participants were reported to be adults, and there is no
indication that any of the participants were pregnant or nursing.
Although the EPA ethics review states that “there is no evidence
suggesting that any subjects came from an especially vulnerable
group,” several of participants were current and former agricultural
workers. Such participants may be socioeconomically or educationally
disadvantaged, and thus may be vulnerable. The Board recommended that
the Agency more clearly articulate or standardize its definition of
vulnerability.

Considering the information available, the Board concurred with the
conclusions and factual observations of the ethical strengths and
weaknesses of the study, as detailed in the Agency’s Ethics Review
(Sherman 2009b). The Board also concluded that this study met all
applicable ethical requirements for such research involving human
participants, as required by the Final Human Studies Rule, according to
the following criteria: 

a. 	Not fundamentally unethical. When determining whether or not a study
is fundamentally unethical, the Board’s standard approach is to decide
if the research was intended to seriously harm participants, if it
failed to obtain informed consent, or if it was fundamentally unethical
for other reasons.  In this case, there was no evidence that the study
was intended to seriously harm participants. The published article also
failed to provide evidence supporting a conclusion that the informed
consent of participants was not obtained. Given the lack of clear and
convincing evidence that for any other reasons it might have been
fundamentally unethical, the Board concluded that it was not
fundamentally unethical.

b. 	Not significantly deficient. With regard to determining whether a
study was significantly deficient relative to the ethical standards
prevailing at the time the research was conducted, the Board’s
standard is to determine whether or not any ethical deficiencies
identified could have resulted in serious harm to participants, based on
knowledge available to researchers at the time the study was conducted,
or whether the information provided to study participants could
seriously impair informed consent. Only the published article provides
any information, and the information contained therein fails to support
a conclusion that the study was significantly deficient relative to the
standards at the time.

Assessment of Proposed Carroll-Loye Biological Research Study LNX-003:
Efficacy Test of KBR 3023 (Picaridin; Icaridin) - Based Personal Insect
Repellents (20% Cream and 20% Spray) with Ticks Under Laboratory
Conditions.

Overview of the Study

The protocol describes a study to measure the effectiveness of picaridin
as a tick repellent when used in one of two compound formulations (20%
picardin KBR 3032 All-Family Insect Repellent Cream and 20% picaridin
KBR 3023 All-Family Insect Repellent Spray). Dosimetry data accumulated
in previous Carroll-Loye studies (LNX-001 and LNX-002) would be used for
dose selection. The efficacy of picaridin as a tick repellent will be
determined in a controlled laboratory setting by placing both Western
black-legged ticks (Ixodes pacificus) and American dog ticks
(Dermacentor variabilis) on picaridin-treated and untreated forearms and
measuring the speed and distance that moving ticks would penetrate into
the treated area at 15-minute intervals. A total of 40 participants will
be enrolled.

Science

Charge to the Board

	If the proposed laboratory tick repellency study protocol LNX-003 is
revised as suggested in EPA’s review and if the research is performed
as described, is the research likely to generate scientifically reliable
data, useful for assessing the efficacy of the tested materials in
repelling ticks? 

Board Response to the Charge

	

	The Board concluded that the protocol submitted for review, if modified
in accordance with Agency recommendations and conducted accordingly,
will likely yield scientifically valid results on the efficacy of these
two picaridin-based insect repellent formulations against ticks.

HSRB Detailed Recommendations and Rationale 

	

	Protocol LNX-003 from Carroll-Loye Biological Research (Carroll 2009a,
2009b) will be conducted using methods similar to those presented to and
commented on by the Board in the past. Although the study protocol was
overly long and includes redundant or unnecessary text, it was
relatively clear and addressed adequately a number of key scientific
issues, including: scientific justification, objectives, and data
collection and compilation methods.

	The proposed methods largely follow EPA’s guidelines, with the one
notable exception being the use of ten volunteers per study aim, rather
than the Agency’s existing recommendation of six volunteers per study
aim. The greater number of study participants should yield more useful
information than might otherwise be obtained. The protocol also
incorporated a number of recommendations that had been made by the Board
previously, such as the use of dosimetry-generated data rather than use
of a standardized dosage amount, which will likely generate data more
representative of real-world use by consumers.

	As has been pointed out previously in Board reviews of other repellency
protocols, the proposed statistical approach is not ideal and the
calculation of complete protection time is not the best end-use of the
study data. Calculating the proportion of individuals protected for a
given time may be a better way to report the data and should be
considered.

Ethics

Charge to the Board 

	If the proposed laboratory tick repellency study protocol LNX-003 is
revised as suggested in EPA’s review and if the research is performed
as described, is the research likely to meet the applicable requirements
of 40 CFR part 26, subparts K and L? 

Board Response to the Charge

	The Board concluded that the proposed laboratory tick repellency study
protocol LNX-003, if modified in accordance with EPA (Sherman and
Sweeney 2009) recommendations, and performed as described, will likely
meet the applicable requirements of 40 CFR 26, subparts K and L. 

HSRB Detailed Recommendations and Rationale 

	The submitted documents assert that the study will be conducted in
accordance with the ethical and regulatory standards of 40 CFR 26,
Subparts K and L, as well as the requirements the US EPA’s GLP
Standards described at 40 CFR 160, and the California State EPA
Department of Pesticide Regulation study monitoring (California Code of
Regulations Title 3, Section 6710) (Carroll 2009a, 2009b). Requirements
of FIFRA §12(a)(2)(P) also apply. The protocol was reviewed and
approved by an independent human subjects review committee, Independent
Investigational Review Board, Inc. (IIRB, Inc.), of Plantation, FL,
prior to submission. Minutes of IIRB, Inc. meetings and a copy of IIRB,
Inc. policies and procedures were provided to the EPA as a separate
document (IIRB, Inc. 2009). These documents indicate that IIRB, Inc.
reviewed this protocol pursuant to the standards of the Common Rule (45
CFR Part 46, Subpart A).

1. 	The Board concurred with the conclusions and factual observations of
the ethical strengths and weaknesses of the study, as detailed in the
EPA’s Ethics Review (Sherman and Sweeney 2009). The proposed study is
likely to meet the applicable ethical requirements for research
involving human participants, in accordance with the following criteria:

a.	Acceptable risk-benefit ratio. The risks as noted in the study
protocol are fivefold: 1) allergic reaction to test materials
themselves; 2) exposure to biting arthropods; 3) possible exposure to
arthropod-borne diseases; 4) physical stress from the test conditions;
and 5) psychological stress and/or breach of confidentiality for
pregnancy test results. These risks are minimized appropriately and are
justified by the potential societal benefits, particularly data on the
efficacy of these new formulations as personal insect repellents.

Based on toxicological data currently available for picaridin, coupled
with appropriate exclusion criteria, study participants are unlikely to
be at risk of adverse side effects with exposure.

The risk of bites is negligible and minimized by the study design; tick
questing and biting behavior is slow, and study participants are trained
to remove ticks from their forearms prior to biting. Study participants
will be trained in proper insect observation and handling techniques.

The ticks used for the study are bred and raised in a laboratory
environment and are considered to be pathogen-free, minimizing the risk
of vector-borne disease. Tick colonies and their rabbit hosts are also
screened regularly for known tick-borne diseases, including the
rickettsial illness Rocky Mountain Spotted Fever that has been observed
in the past to be transmitted within laboratory tick colonies through a
trans-ovarian mechanism.

The potential risks to participants from physical stress are minimized.
Although the 12-hour duration of the study protocol raises some concerns
about physical stress and exhaustion, the study investigators attest
that similar protocols of equivalent length have never been seen as
unduly stressful by study participants. Appropriate stopping rules and
medical management procedures are in place. Participants are also given
frequent breaks and can withdraw from the study at any time should the
investigational procedures prove too strenuous.

Minors and pregnant or lactating women are excluded from participation,
with pregnancy either confirmed by over-the-counter pregnancy testing on
the day of study or by opt-out. The potential stigma resulting from
study exclusion due to pregnancy is also appropriately minimized.

b.	Voluntary and informed consent of all participants

The study protocol includes several mechanisms designed to minimize
coercive recruitment and enrollment. For example, although many of the
research participants will be recruited from the University of
California at Davis student population, where Dr. Carroll holds an
adjunct appointment, student and employees of the Study Directory are
excluded from participation. Additional mechanisms design to minimize
coercive recruitment, developed in response to earlier HSRB concerns and
recommendations (c.f. EPA HSRB 2006a; 2006b) are also in place.

Monetary compensation is not so high as to unduly influence
participants.

c.	Equitable selection of study participants

The majority of research participants will be recruited from the
University of California at Davis student population. Study participants
are likely to reflect the ethnic and racial diversity of individuals in
the City of Davis, but the use of this convenience sample may limit the
broad applicability of the study results to the general population. The
investigators have noted this fact in the protocol.

Assessment of Proposed AEATF II Scenario and Protocol AEA04: Research on
Exposure of Janitorial Workers Applying Pesticides Formulated as Aerosol
Sprays.

Overview of the Study

	AEATF II aerosol spraying scenario is designed to measure a typical
occupational handler’s daily exposure to an antimicrobial spray
packaged in a pressurized aerosol spray can. These data will be used
generically to estimate dermal and inhalation exposures and risks for
other antimicrobial ingredients where the product is packaged in a
pressurized aerosol spray can. The Task Force expects these data to be
useful for estimating exposures for various types of aerosol spray
products, different types of surfaces, different room configurations,
different types of buildings, different handlers, and various
antimicrobial active ingredients. Eighteen professional janitors will be
enrolled in the study, and asked to apply (spray but not wipe) the
product at one of three motels in the Fresno, CA area. Study
participants will be randomized to apply different amounts of product,
from 1 to 4 cans of product in 1/2-can increments (i.e., 1 to 1.5 cans,
1.5 to 2 cans, up to 3.5 to 4 cans). Dermal and inhalation exposure will
be measured using whole-body dosimeters (inner and outer), hand and face
washes, and personal air monitors.

Science

Charge to the Board

	If the proposed AEATF II aerosol application scenario and field study
protocol AEA04 is revised as suggested in EPA’s review and if the
research is performed as described, is the research likely to generate
scientifically reliable data, useful for assessing the exposure of
handlers who apply antimicrobial pesticides formulated as aerosol
sprays? 

Board Response to the Charge  

The Board concluded that this protocol, if modified in accordance with
EPA (Leighton, Walls and Sherman 2009) and HSRB recommendations, will
likely generate scientifically reliable data, useful for assessing the
exposure of handlers who apply antimicrobial pesticides formulated as
aerosol sprays.

The Board also recommended a number of protocol modifications, as listed
below. Additional Board review of the protocol is not required prior to
study implementation.

HSRB Detailed Recommendations and Rationale

The Board concluded that the protocol (AEATF II 2009a, 2009b) and
supplemental SOPs (AEATF II 2009b) address adequately a number of key
scientific issues, including: scientific objective, quantification of
the test materials, data collection and compilation methods and summary
of test results, justification for selection of the test substances, and
QA/QC requirements. The process used to select the product to be tested
seems rational. The AEATF II has described in detail their sampling
design for the aerosol scenario, and has incorporated random elements
where feasible. The aerosol scenario is well defined, and the diversity
of daily exposures under the aerosol scenario as defined in this
proposal is likely to describe adequately a typical occupational
handler’s daily exposure to this antimicrobial application. The
environmental monitoring, exposure monitoring and analytic techniques
appear to be adequate.   The variation in exposure should be sufficient
to determine a distribution of exposures.

	The Board did raise a number of concerns about perceived inadequacies
in the study design, as summarized below:

Use of the results: The EPA’s Antimicrobials Division (AD) plans to
use the data generated from the proposed aerosol study generically to
estimate dermal and inhalation exposures and risks for other
antimicrobial ingredients where the product is packaged in a pressurized
aerosol spray can. However, other variables can affect rates of
exposure, including different nozzle sizes, spray and ejection rates,
the size of the particles generated, and the generation of nonvolatile
active ingredients. Inhalation versus dermal exposure may also vary with
these variables.

It is also unclear if occupational handler exposure can be used to
estimate exposures of nonprofessionals using similar consumer products.
However, for risk assessment purposes it was felt that exposure among
occupational users is likely to exceed that of consumers; the higher
frequency of exposures among occupational users is likely to exceed a
plausibly higher individual but less frequent exposure among
nonprofessional users. 

2. 	Sample Size & Analysis: Raw and descriptive data will be provided,
but no statistical analyses have been planned. Based on the information
provided, a sample size of 18 participants may be sufficient. Placing
these 18 participants into three clusters of six each requires
consideration of cluster effects as well, and may complicate analysis of
the results. Assumed constants related to exposure to the active
ingredient may also prove to be incorrect, and additional study
participants may need to be enrolled.

3.	Application of spray: The target number of cans sprayed in each
location by each worker on each day could be specified better within the
protocol. For example, does a worker need to continue spraying a room if
the target application is met before completion of that room? Additional
protocol clarifications needed include: addressing the issue of the
interval between sampling at different sites (currently, it appears that
no two monitoring events can occur within the same building within one
week); clarifying the issue of using empty apartments rather than motel
rooms; and clarifying the effect of exhaust vent effects on exposure,
which could be different than indicated in the protocol. 

	Quality assurance and control (QA/QC): QA/QC can be improved by: 

a. 	Determining likely differences in the air sampling results depending
on the method used (e.g. depending on the air sampling equipment used,
differences in the orifice diameters and air flow sampling rates could
affect aerosol collection efficiencies and effects).

b. 	Setting the minimal spike to two to four times that of the limit of
quantitation (LOQ; cf. AEATF II 2009b: 42, 46).

c. 	Considering other variables that may influence the measurement of
exposure, including whether the surface being treated is dry or wet at
the time of the next spray application and whether the applicant
accidentally wiped the surface after spraying. The protocol should
indicate that there will be a maximum of two workers on any given day in
the same location.

Ethics

Charge to the Board

	If the proposed AEATF II aerosol application scenario and field study
protocol AEA04 is revised as suggested in EPA’s review and if the
research is performed as described: Is the research likely to meet the
applicable requirements of 40 CFR part 26, subparts K and L?

Board response to the Charge

	The Board concluded that the protocol submitted for review, if modified
in accordance with EPA (Leighton, Walls and Sherman 2009) and HSRB
recommendations, is likely to meet the applicable requirements of 40 CFR
26, subparts K and L.

HSRB Detailed Recommendations and Rationale

	This is a protocol for third-party research involving intentional
exposure of human participants to a pesticide, with the intention of
submitting the resulting data to EPA under the pesticide laws (AEATF II
2009a, 2009b, 2009d).  Thus, it is subject to the ethical and regulatory
standards of 40 CFR 26, Subparts K and L. The requirements of FIFRA
§12(a)(2)(P) relating to informed, voluntary consent and those of the
California State EPA Department of Pesticide Regulation study monitoring
(California Code of Regulations Title 3, Section 6710) also apply. The
protocol was reviewed and approved by an independent human subjects
review committee, Independent Investigational Review Board, Inc. of
Plantation, Florida, prior to submission (AEATF 2009b, 2009c).

1. 	Except as noted below, the Board concurred with the conclusions and
factual observations of the ethical strengths and weaknesses of the
study, as detailed in the EPA’s Ethics Review (Leighton, Walls and
Sherman 2009). The proposed study is likely to meet the applicable
ethical requirements for research involving human participants, in
accordance with the following criteria:

a.	Acceptable risk-benefit ratio. The risks from participation in the
study include: 1) discomfort and heat-related illness from wearing two
layers of clothing; 2) discomfort from wearing an air sampling device;
3) reactions from exposure to the test material (including accidental
exposure) or to solvents used to obtain residues from hands, face and
neck; 4) embarrassment from disrobing in the presence of a research
technician; and 5) psychological stress and/or breach of confidentiality
for pregnancy test results. These risks are minimized appropriately and
are justified by the potential societal benefits, particularly data for
new exposure assessments for antimicrobial products applied with
pressurized aerosol cans.

The surrogate materials consist of a commonly used commercial
antimicrobial product, which is known to have low risks. The
participants will only be exposed to concentrations of the surrogate
compound at accepted exposure thresholds. Candidates who are known to be
sensitive to that compound, or who are in poor health, or who have
broken skin on their hands, face, or neck will be excluded.

Risk of heat-related illness is minimized appropriately. Testing will
take place in a temperature-controlled environment, and the heat index
monitored with associated stopping rule. There will be a limited time of
exposure with rest periods at 30-minute intervals, or more frequently if
requested. There also will be close observation of participants, with
participants alerted to the signs and symptoms of heat stress.

Participants will be reminded about safe handling practices and
procedures, wear appropriate personal protective equipment (PPE), and
will be monitored for any accidental or unintended product exposure.

 Minors and pregnant or lactating women are excluded from participation,
with pregnancy either confirmed by over-the-counter pregnancy testing on
the day of study or by opt-out. The potential stigma resulting from
study exclusion due to pregnancy is appropriately minimized.

The purpose of the study is to develop more accurate information on
worker exposures to liquid antimicrobial products applied using
pressured aerosol cans for indoor surface disinfecting. Although there
is no clear benefit to study participation, participants may benefit by
requesting their results and using these results to change their working
practices. Benefits and risks of participation are clearly articulated
in the protocol and informed consent documents.

Possible participants for this study may be undocumented immigrants.
Recruitment materials state that government-issued identification is
necessary for enrollment in this study.

A valid government-issued form of identification is necessary for
enrollment. Volunteers who lack proper identification are excluded from
study participation but no other action is taken, minimizing any social
risks to undocumented workers from disclosure of their status to
law-enforcement or immigration authorities.

b.	Voluntary and informed consent of all participants

The informed consent documents clearly state that study participants can
withdraw at any time without penalty.

The Board disagrees with the EPA comment that “no potential subjects
are from a vulnerable population” (Leighton, Walls and Sherman 2009:
11). In fact, participants in this study might represent particularly
vulnerable populations and thus be susceptible to coercion or undue
influence. Some participants may be educationally or economically
disadvantaged, or may not be native speakers of English. However, the
risk of coercion or undue influence is appropriately minimized as
follows:

Monetary compensation is roughly equivalent to the average daily wage of
janitorial workers in Fresno, California, and so is not so high as to
unduly influence participants.

Spanish translations of all informed consent documents, informational
packets, and recruitment flyers are available. If necessary, the
informed consent process will be conducted in Spanish by a member of the
research team fluent in that language.

The participants communicate directly with the researchers about their
interest in enrolling in the study. Recruitment is not conducted through
employers.

c.	Equitable selection of study participants

The study is designed to recruit an appropriately diverse population of
participants who represent professional janitorial workers in the Fresno
County area. The researchers plan to place recruitment notices in three
newspapers that cater to the general population, the African-American
population, and the Latino population.

2.	The Board recommended that the study protocol be modified to address
the few concerns noted in the EPA’s Ethics Review (Leighton, Walls and
Sherman 2009). The Board also made the following additional
recommendations:

The Board recommended that the consent form be reviewed to make sure
that it is at an appropriate reading level for the proposed participant
population. In some instances, there appear to be word choices that
could be simplified (e.g., wording such as “non-porous surfaces”,
“pressurized aerosol can” instead of “spray can”, and assuming
that the volunteer knows what a Material Safety Data Sheet is).

The Board agrees with the EPA ethics review that the informed consent
form should read, “We will pay for needed medical treatment that is
not paid for by your own insurance of a third party under which you are
covered.”

As some possible participants for this study may be undocumented
immigrants, recruitment materials should more explicitly state that a
valid government-issued form of identification is necessary for
enrollment. 

The Board recommended that the Task Force ensure that documents in
Spanish are reviewed by someone familiar with the dialects written and
spoken in the target community. The National Standards for Culturally
and Linguistically Appropriate Services in Health developed by the
Department of Health and Human Services, the Office of Minority Health
has stressed the importance of treating medical patients in a
“culturally and linguistically appropriate manner” (Office of
Minority Health 2001, 3). The same respect should be extended to
participants in EPA- or third party-conducted research, with materials
such as informed consent forms and recruitment flyers written in ways
that reflect “the dialectic and cultural nuances as well as the
acculturation, educational, and literacy levels of the local target 
population” (Office of Minority Health 2001, 80).

The protocol provides for use of a community notification flyer.
However, one neglected community includes persons who might be staying
in the hotels where the study is conducted. The flyer should be revised
so that it communicates the goals of the study and risks to that group.
These flyers should also be posted in locations so that hotel guests are
likely to see it. Alternatively, the researchers should consider
conducting the research in areas away from hotel guests.

The exclusion criteria should be revised to eliminate some groups that
might be at higher risk of physical harm but are not presently excluded.
This might include participants who might be immunosuppressed for a
variety of reasons, those with severe diabetes, and those with other
conditions that pose a health risk.

REFERENCES

Antimicrobial Exposure Assessment Task Force II (AEATF II). 2009a. AEATF
II Aerosol Study Vol. 1 Primary Documentation: Aerosol Application
Scenario Design. Dated August 4, 2009. Unpublished protocol prepared by
the AEATF II.

Antimicrobial Exposure Assessment Task Force II (AEATF II). 2009b. AEATF
II Aerosol Study Vol. 2 Primary Documentation: Study Protocol and IIRB
Approval and Documentation. Dated August 4, 2009. Unpublished protocol
and documentation prepared by the AEATF II.

Antimicrobial Exposure Assessment Task Force II (AEATF II). 2009c. AEATF
II Aerosol Study Vol. 3 Primary Documentation: IIRB Communications.
Dated August 4, 2009. Unpublished documentation prepared by the AEATF
II.

Antimicrobial Exposure Assessment Task Force II (AEATF II). 2009d. AEATF
II Aerosol Study Vol. 4 Standard Operating Procedures for a Multi-Year
Antimicrobial Chemical Exposure Monitoring Program. Dated August 4,
2009. Unpublished documentation prepared by the AEATF II.

Carroll, S. 2009a. Efficacy Test of KBR3023 (Picaridin; Icaridin)-Based
Personal

Insect Repellents (20% Cream and 20% Spray) with Ticks under Laboratory
Conditions. Dated July 30, 2009. Unpublished protocol prepared by
Carroll-Loye Biological Research. MRID 47836801.

Carroll, S. 2009b. Carroll-Loye Study LNX-003 Proposed Amended Language.
Dated September 18, 2009. Unpublished protocol amendment prepared by
Carroll-Loye Biological Research. 

EPA HSRB. 2006a. June 27-30, 2006 EPA Human Studies Review Board Meeting
Report.

EPA Human Studies Review Board. 2006b. October 18-19, 2006 Human Studies
Review Board Meeting Report.

EPA Office of Pesticide Programs. 2009. A Review of the Relationship
between Pyrethrins, Pyrethroid Exposure and Asthma and Allergies.
Revised September 2009. Unpublished document prepared by the by Office
of Pesticide Programs, United States Environmental Protection Agency.

Independent Institutional Review Board, Inc. (IIRB, Inc.). 2009.
Supplemental IIRB, Inc., Documentation Package. Unpublished materials
prepared by IIRB.

Leighton, T., C. Walls and K. Sherman. 2009. Science and Ethics Review
of AEATF II Aerosol Scenario Design and Protocol for Exposure
Monitoring. Dated September 21, 2009.  Unpublished document prepared by
Office of Pesticide Programs, United States Environmental Protection
Agency.

Lisi, P. 1992. Sensitization Risk of Pyrethroid Insecticides. Contact
Dermatitis 26:349-350. MRID 47852801.

Newton, J.G. and A.B. Breslin. 1983. Asthmatic Reactions to a Commonly
Used Aerosol Insect Killer. Medical Journal of Australia 1:378-380. MRID
47686201.

Office of Minority Health, Department of Health and Human Services.
2001. National Standards for Culturally and Linguistically Appropriate
Services in Health Care: Final Report. Washington, DC: U.S. Department
of Health and Human Services.

Sherman, K. 2009a. Ethics Review of Pre-Rule Intentional Exposure Human
Toxicity Study (Newton and Breslin 1983). Dated September 16, 2009. 
Unpublished document prepared by Office of Pesticide Programs, United
States Environmental Protection Agency.

Sherman, K. 2009b. Ethics Review of Pre-Rule Intentional Exposure Human
Toxicity Study (Lisi 1992). Dated September 16, 2009.  Unpublished
document prepared by Office of Pesticide Programs, United States
Environmental Protection Agency.

Sherman, K., and K. Sweeney. 2009. Science and Ethics Review of Protocol
for Human Study of Tick Repellent Performance. Dated September 21, 2009.
 Unpublished document prepared by Office of Pesticide Programs, United
States Environmental Protection Agency.

Proposed Final Draft Dated November 29, 2009; Do Not Cite or Quote

Proposed Final Draft Dated November 29, 2009; Do Not Cite or Quote

Page   PAGE  2  of   NUMPAGES  26 

Page   PAGE  1  of   NUMPAGES  26 

