April 16, 2007

EPA-HSRB-07-01

George Gray, Ph.D.

Science Advisor

Office of the Science Advisor

1200 Pennsylvania Avenue, NW

Washington, DC 20460 

Subject:  January 24, 2007 EPA Human Studies Review Board Meeting Report

Dear Dr. Gray:

The United States Environmental Protection Agency (EPA or Agency)
requested the Human Studies Review Board (HSRB) to review scientific and
ethical issues addressing: (1) 

two completed human studies (EMD 003 and EMD 004) evaluating repellent
efficacy of formulations containing the active ingredient IR3535; and
(2) a research proposal to evaluate the efficacy in the field of
multiple formulations of the repellent DEET against mosquitoes.  

The enclosed HSRB report addresses the Board’s response to EPA charge
questions at its January 24, 2007 meeting.  A summary of the Board’s
conclusions is provided below.

Completed IR3535 Insect Repellent Efficacy Studies 

Studies EMD-003.1 and EMD 003.2 

Scientific Considerations

The reported studies on the efficacy of lotion and pump spray
formulations of IR3535 (studies EMD-003.1 and EMD-003.2) on repelling
ticks are sufficiently sound, from a scientific perspective, to be used
to assess the repellent efficacy of the two formulations against ticks.

The Board also recognized that advances in statistical analyses mean
that there are probably ways of measuring efficacy of individual
products that would be an improvement over traditional techniques. The
Board encouraged the Agency to proceed in its efforts to examine how a
transition to more appropriate methods of calculating efficacy for
specific data sets can be introduced so that consumers can not only
compare relative efficacy of products based on traditional methods but
also have better information on the degree of protection individual
products provide.

The Board commended Dr. Carroll for conducting the preliminary dosimetry
phase of the study. Since the Agency noted that this was the first
repellency study to have a specific dosimetry phase, the HSRB suggested
EPA might wish to provide guidance concerning whether the method
employed in this study was the most valid way to determine dose.  The
Board recommended inclusion of a description of the sampling frame and
definition of eligible subjects to help justify subject
generalizability.

Ethical Considerations

The Board concurred with the initial assessment of the Agency that the
studies EMD–003.1 and EMD-003.2 submitted for review by the Board meet
the applicable requirements of 40 CFR part 26, subparts K and L. 

The Board also noted that there were a series of deviations from Subpart
K that, while not adversely affecting the rights and welfare of human
subjects of the study, reflected a lack of familiarity with IRB
procedures and protocol requirements described in Subpart K. The HSRB
advised the Agency to recommend that investigators undergo human
research protection training, and include evidence of completion of such
training as part of their submission of protocols or completed studies
to the Agency.  Examples of such training could include the on-line
training program offering by NIH/NCI or a similar program that might be
developed by the Agency.   

Studies EMD-004.1 and EMD-004.2 

 

Scientific Considerations

The reported studies on the efficacy of lotion and pump spray
formulations of IR3535 (studies EMD-004.1 and EMD-004.2) on repelling
mosquitoes are sufficiently sound, from a scientific perspective, to be
used to assess the repellent efficacy of the two formulations against
mosquitoes.

The Board also recognized that advances in statistical analyses mean
that there are probably ways of measuring efficacy of individual
products that would be an improvement over traditional techniques. The
Board encouraged EPA to proceed in its efforts to examine how a
transition to more appropriate methods of calculating efficacy for
specific data sets can be introduced so that consumers can not only
compare relative efficacy of products based on traditional methods but
also have better information on the degree of protection individual
products provide.

The Board commended Dr. Carroll for conducting the preliminary dosimetry
phase of the study. Since the Agency noted that this was the first
repellency study to have a specific dosimetry phase, the HSRB suggested
EPA might wish to provide guidance concerning whether the method
employed in this study was the most valid way to determine dose.  The
Board recommended inclusion of a description of the sampling frame and
definition of eligible subjects to help justify subject
generalizability.

Ethical Considerations

The Board concurred with the initial assessment of the Agency that
studies EMD 004.1 and EMD 004.2 submitted for review by the Board met
the applicable requirements of 40 CFR part 26, subparts K and L.  

The Board also noted that there were a series of deviations from Subpart
K that while not adversely affecting the rights and welfare of human
subjects of the study, reflected a lack of familiarity with IRB
procedures and protocol requirements described in Subpart K. The HSRB
advised the Agency to recommend that investigators undergo human
research protection training, and include evidence of completion of such
training as part of their submission of protocols or completed studies
to the Agency.  Examples of such training could include the on-line
training program offering by NIH/NCI or a similar program that might be
developed by the Agency.   

Insect Repellent Efficacy Protocol SCI-001

Scientific Considerations

The Board raised several concerns about sample size, population
generalizability and statistical analysis in SCI-001 that should be
addressed.   If the recommendations provided by EPA and those suggested
by the Board are followed, protocol SCI-001 appears likely to generate
scientifically valid data to assess the efficacy of the test products
against mosquitoes.   

The protocol would satisfy the scientific criteria recommended by the
HSRB, namely, producing important information that cannot be obtained
except by research with human subjects, and having a clear scientific
objective and study design that should produce adequate data to test the
hypothesis.   

 

Ethical Considerations

The Board concluded that the protocol should meet the applicable
requirements of 40 CFR part 26, subparts K and L if the points raised in
the EPA review and in this report are adequately addressed.  

In conclusion, the EPA HSRB appreciated the opportunity to advise the
Agency on the scientific and ethical aspects of research with human
subjects and looks forward to future opportunities to continue advising
the Agency in this endeavor.

Sincerely,

Celia B. Fisher, Ph.D. Chair

EPA Human Studies Review BoardNOTICE

This report has been written as part of the activities of the EPA Human
Studies Review Board, a Federal advisory committee providing advice,
information and recommendations on issues related to scientific and
ethical aspects of human subjects research.  This report has not been
reviewed for approval by the Agency and, hence, the contents of this
report do not necessarily represent the view and policies of the
Environmental Protection Agency, nor of other agencies in the Executive
Branch of the Federal government, nor does the mention of trade names or
commercial products constitute a recommendation for use.  Further
information about the EPA Human Studies Review Board can be obtained
from its website at http://www.epa.gov/osa/hsrb/.  Interested persons
are invited to contact Paul Lewis, Designated Federal Officer, via
e-mail at lewis.paul@epa.gov.

	In preparing this document, the Board carefully considered all
information provided and presented by the Agency presenters, as well as
information presented by public commenters.  This document addresses the
information provided and presented within the structure of the charge by
the Agency.

United States Environmental Protection Agency Human Studies Review
Board

Chair

Celia B. Fisher, Ph.D., Marie Ward Doty Professor of Psychology,
Director, Center for Ethics Education, Fordham University, Department of
Psychology, Bronx, NY 

Vice Chair

William S. Brimijoin, Ph.D., Chair and Professor, Molecular Pharmacology
and Experimental Therapeutics, Mayo Foundation, Rochester, MN  *  

Members 

David C. Bellinger, Ph.D., Professor of Neurology, Harvard Medical
School

Professor in the Department of Environmental Health, Harvard School of
Public Health

Children's Hospital, Boston, MA  

Alicia Carriquiry, Ph.D., Professor, Department of Statistics, Iowa
State University, Ames, IA 

Gary L. Chadwick, PharmD, MPH, CIP, Associate Provost, Director, Office
for Human Subjects Protection, University of Rochester, Rochester, NY 

Janice Chambers, Ph.D., D.A.B.T., William L. Giles Distinguished
Professor, Director, Center for Environmental Health Sciences, College
of Veterinary Medicine, Mississippi State University, Mississippi State,
MS 

Richard Fenske, Ph.D., MPH, Professor, Department of Environmental and
Occupational Health Sciences, University of Washington, Seattle WA  *

Susan S. Fish, PharmD, MPH, Professor, Biostatistics & Epidemiology,
Boston University School of Public Health, Co-Director, MA in Clinical
Investigation, Boston University School of Medicine, Boston, MA 

Suzanne C. Fitzpatrick, Ph.D., DABT, Senior Science Policy Analyst,
Office of the Commissioner, Office of Science and Health Coordination,
U.S. Food and Drug Administration, Rockville, MD 

Kannan Krishnan, Ph.D., Professor, Département de santé
environnementale et santé au travail, Faculté de medicine, Université
de Montréal, Montréal, Canada

KyungMann Kim, Ph.D., CCRP, Professor & Associate Chair, Department of
Biostatistics & Medical Informatics, School of Medicine and Public
Health, University of Wisconsin-Madison, Madison, WI  

Michael D. Lebowitz, Ph.D., FCCP, Professor of Public Health & Medicine.
University of Arizona, Tucson, AZ 

Lois D. Lehman-Mckeeman, Ph.D., Distinguished Research Fellow, Discovery
Toxicology, Bristol-Myers Squibb Company, Princeton, NJ  *

Jerry A. Menikoff, M.D., Associate Professor of Law, Ethics & Medicine,
Director of the Institute for Bioethics, Law and Public Policy,
University of Kansas Medical Center, 

Kansas City, KS *

Sean Philpott, PhD., MS Bioethics, Policy and Ethics Officer, Global
Campaign for

Microbicides, Program for Appropriate Technology in Health, Washington
D.C.

Richard Sharp, PhD., Assistant Professor of Medicine, Center for Medical
Ethics and Health Policy, Baylor College of Medicine, Houston, TX

Human Studies Review Board Staff

Paul I. Lewis, Ph.D., Designated Federal Officer, United States
Environmental Protection Agency, Washington, DC 

* Not in attendance at the January 24, 2007 MeetingTABLE OF CONTENTS

  TOC \o "1-3" \h \z \u    HYPERLINK \l "_Toc160603981"  INTRODUCTION	 
PAGEREF _Toc160603981 \h  8  

  HYPERLINK \l "_Toc160603982"  REVIEW PROCESS	  PAGEREF _Toc160603982
\h  9  

  HYPERLINK \l "_Toc160603983"  CHARGE TO THE BOARD AND BOARD RESPONSE	 
PAGEREF _Toc160603983 \h  10  

  HYPERLINK \l "_Toc160603984"  Completed IR3535 Insect Repellent
Efficacy Studies	  PAGEREF _Toc160603984 \h  10  

  HYPERLINK \l "_Toc160603986"  Insect Repellent Efficacy Protocol
SCI-001	  PAGEREF _Toc160603986 \h  22  

  HYPERLINK \l "_Toc160603988"  REFERENCES	  PAGEREF _Toc160603988 \h 
28  

 INTRODUCTION

On January 24, 2007, the United States Environmental Protection
Agency’s (EPA or Agency) Human Studies Review Board (HSRB) met to
address scientific and ethical issues concerning two categories of
research:  

(1) two completed human studies (EMD 003 and EMD 004) evaluating
repellent efficacy of formulations containing the active ingredient
IR3535.   

In its last two meetings (June 27-30, 2006 and October 18-19, 2006) the
HSRB reviewed and commented on materials relating to two insect
repellent efficacy protocols from Carroll-Loye Biological Research. 
These two protocols described proposed research to evaluate the efficacy
of new formulations of repellent products containing the active
ingredient IR-3535.  The protocol identified as EMD-003 described a
laboratory study of efficacy of the test formulations against ticks. 
The protocol identified as EMD-004 described a field study of efficacy
of the test formulations against mosquitoes.  

Following the June 2006 meeting, Dr. Carroll revised the protocols to
address comments from the HSRB.  EPA reviewed Dr. Carroll’s revised
protocols and concluded that they appeared likely to generate
scientifically sound, useful information and to meet the applicable
provisions of the EPA regulations in 40 CFR part 26, subparts K and L. 
When the HSRB reconsidered the revised protocols at its October 2006
meeting, it concurred with EPA’s assessment and suggested some minor
additional refinements.  Dr. Carroll proceeded to conduct the research
and had submitted the results to EPA for review.  

The Agency’s regulation, 40 CFR §26.1602, requires EPA to seek HSRB
review of an EPA decision to rely on the results of these studies.  The
sponsor has not yet submitted an application to register these products,
but with Agency concurrence submitted the completed studies ahead of the
applications so that HSRB review would not compromise EPA’s ability to
review the application within the time allowed by statute.  The Agency
expects to receive such an application in the near future.  In order to
facilitate timely review of the application, EPA has reviewed the
studies, applying the standard in 40 CFR §26.1705.  That provision
states:

§ 26.1705  Prohibition on reliance on unethical research with
non-pregnant, non-nursing adults conducted after April 7, 2006

Except as provided in §26.1706, in actions within the scope of §
26.1701, EPA shall not rely on data from any research initiated after
April 7, 2006, unless EPA has adequate information to determine that the
research was conducted in substantial compliance with subparts A through
L of this part . . . This prohibition is in addition to the prohibition
in § 26.1703.

The Agency’s reviews concluded that the data were scientifically sound
and that the research was conducted in a manner that deviates at least
technically from some of the requirements of subparts K and L of EPA’s
final rule establishing Protections for Subjects in Human Research—the
only subparts of the rule which apply to third-party research.  The
Agency sought the Board’s advice on whether the available information
supports a determination of “substantial compliance” with the
applicable rules.  Assuming a potential determination of substantial
compliance, and because EPA would like to rely on these data to support
an application for registration of these formulations, EPA presented
these studies for review at the Board’s January 2007 meeting.

(2) research proposal to evaluate the efficacy in the field of multiple
formulations of the repellent DEET against mosquitoes.  

EPA requires data from efficacy studies using appropriate insect species
to support claims of greater efficacy than have previously been
approved.  

An applicant for new or amended registration typically conducts such
research prior to submitting an application.  If such a study is to be
initiated after April 7, 2006, EPA’s regulation, 40 CFR §26.1125,
requires the sponsor or investigator to submit to EPA, before conducting
the study, materials describing the proposed human research in order to
allow EPA to conduct scientific and ethics reviews.  In addition,
EPA’s regulation, 40 CFR §26.1601, requires EPA to seek HSRB review
of the research proposal.  

Dr. Scott Carroll submitted a description of proposed research to be
performed by Carroll-Loye Biological Research.  The proposal, identified
as SCI-001, described a study to evaluate the efficacy of four
formulations of repellent products containing the active ingredient DEET
(one formulation includes two other active ingredients as well).  The
study would measure the efficacy of three test formulations and one
”comparison article”—the US military standard repellent—against
mosquitoes under field conditions.  The proposal bears many similarities
to the protocol EMD-004 that the HSRB had previously reviewed.   EPA had
reviewed Dr. Carroll’s protocol and concluded that, with some required
refinements, it appeared likely to generate scientifically sound, useful
information and to meet the applicable provisions of the EPA regulations
in 40 CFR part 26, subparts K and L.  

In its initial review, EPA identified some relatively easily corrected
deficiencies in the protocol, which must be corrected before execution. 
In the interest of providing a thorough and timely response to the
proposal, and since EPA found the protocol generally meeting applicable
scientific and ethical standards, EPA presented this protocol for review
at the Board’s January 2007 meeting.

  This report transmits the HSRB’s comments and recommendations from
its January 24, 2007 meeting.        

REVIEW PROCESS

On January 24, 2007 the Board had a public face-to-face meeting in
Arlington, Virginia.  Advance notice of the meeting was published in the
Federal Register “Human Studies Review Board: Notice of Public Meeting
(71 Federal Register 249).  At the public meeting, following welcoming
remarks from Agency officials, Celia B. Fisher, HRSB Chair, summarized
the Board’s process for its review. The Board then heard presentations
from the Agency on the following topics: (1) Insect Repellent Completed
Efficacy Studies EMD-003 and EMD-004; and (2) Insect Repellent Efficacy
Protocol SCI-001. 

The Board received written public comments from Dr. Scott Carroll
representing Carroll- Loye Biological Research.  In addition, Dr.
Carroll, representing Carroll-Loye Biological Research, provided oral
comments at the meeting. 

For their deliberations, the Board considered the materials presented at
the meeting, written public comments and Agency background documents
(e.g. pesticide human study, Agency data evaluation record (DER) of the
pesticide human study, weight of evidence review, ethics review,
pesticide human study protocol/study and Agency evaluation of the
protocol/study). 

CHARGE TO THE BOARD AND BOARD RESPONSE

Completed IR3535 Insect Repellent Efficacy Studies 

Background

Protocol EMD-003 proposed testing 3 formulations of repellent (lotion,
pump spray, and aerosol) containing the active ingredient IR-3535 for
efficacy in repelling ticks under laboratory conditions.  Protocol
EMD-004 proposed testing of the same 3 formulations for efficacy in
repelling mosquitoes under field conditions in two habitats (dense
forest and moist pasture or marshland).  EPA guidelines recommend
testing in two habitats to assess efficacy in the presence of different
mosquito species with different behaviors.  Both protocols had a
dosimetry phase to establish a “typical consumer dose” that would be
used in the efficacy phases of the trials.  Twelve subjects participated
in dosimetry testing of the 3 formulations whose results were used for
both protocols.  An error in formulation of the aerosol test material
caused a delay in testing this formulation, and the reports considered
during this meeting addressed testing of only the lotion and pump spray
formulations.  Separate reports for each formulation (lotion and pump
spray) were submitted for each protocol, and then a subsequent report
including both formulations was re-submitted.

Studies EMD-003.1 and EMD 003.2

Charge to the Board

Scientific Considerations

Are these studies sufficiently sound, from a scientific perspective, to
be used to assess the repellent efficacy of the formulations tested
against ticks?  

Board Response to the Charge

The active ingredient IR 3535 was tested for its ability to repel ticks
on the forearms of volunteers by the protocol presented and modified by
Carroll-Loye.  The protocol had been modified based on the suggestions
and input of EPA and HSRB.  The results were reported in EMD-003.1 and
EMD-003.2

The active ingredient was formulated into two products, a pump spray and
a lotion, but data on the originally proposed aerosol was not provided
because of an error in the formulation. The products were produced using
Good Manufacturing Practices. All experiments were conducted using Good
Laboratory Practices.  A passive dosimetry experiment was done, as
suggested previously by the HSRB both at its June and October 2006
meeting reviewing this protocol, to determine the amount of product that
would be utilized by people using the product as directed.  This passive
dosimetry experiment was used to determine a grand mean of the 12
individuals tested (3 subsamples each) per product that was then used
for all 10 individuals per product participating in the subsequent tick
repellency tests for each product  (It should be noted that the
dosimetry experiment was common for both this study and the mosquito
repellency study, EMD-004, since the same formulated products were used
for both).

The experiment was a laboratory study and was conducted according to the
approved protocol with only very minor deviations, and none of these
deviations would have affected the quality of the data or the safety of
the subjects.  The number of 10 subjects was justified in the text as
leading to sufficient statistical power while exposing only a small
number of people to the potential risks.  Each subject had one limb
treated. Each of the subjects served as a negative control in that each
tick was tested first on the untreated limb to guarantee that the ticks
demonstrated typical questing behavior (all did) prior to being tested
on the treated limb.  All ticks were laboratory reared with no history
of tick-borne pathogens.  Each tick was used only once. Repellency was
tested during a 3-min interval each 15 minutes, starting 15 minutes
after product application, using the criterion of First Confirmed
Crossing (FCC) for each individual (replicate) to calculate Complete
Protection Time (CPT) for the study.  Stopping rules were employed.  The
study identified a range of 5-12 hr with a mean CPT of 9.1 hr for the
lotion and a range of 6.5 to 15 hr with a mean CPT of 12.1 hr for the
pump spray.  The CPT is probably conservative as a number of the
subjects reported no crossings at all, and the experiment was terminated
before a FCC. 

With respect to the science criteria established earlier by the HSRB for
completed studies: 

General HSRB Scientific Criteria

The scientific question was stated (i.e., to test the efficacy of IR3535
in repelling ticks).

Existing data were not adequate to answer the question of efficacy of
these new formulations.

Because existing data were not adequate to answer the question of
efficacy, new studies involving human subjects are necessary.

The potential benefits of the study were clear, i.e., that an effective
repellent would be available that would have either greater efficacy
and/or fewer drawbacks than what was currently approved.

It is likely that the benefits would be realized because repellent
efficacy was determined in controlled experiments.

The risks are minimal because the formulation products are of very low
toxicity and ticks are laboratory-reared with no evidence of pathogens.

The most likely relevant risk would be irritation from tick bites, but
participants were instructed to remove ticks before they were bitten.

Study Design Criteria

The purpose of the study was clearly defined (i.e., efficacy testing).

There were specific objectives/hypotheses (i.e., that IR3535 in the
proposed formulations is an effective repellent).

The study as described tested this hypothesis.

The sample size was 10 individuals per product with each individual
serving as his/her own negative control to test for tick questing
behavior. A dosimetry experiment prior to the field experiment
determined the amount of repellent to be tested. 

There was a plan allocating individuals to treatments.

It is anticipated that the findings from this study can be generalized
beyond the study sample.

Participation Criteria

There was justification for the selection of the target population.

The participants were representative of some of the population of
concern; however, there are others in the population unlike these
participants who are likely to use these products, but it would either
be unethical to test them or would be less appropriate to test them. 
The participating population is considered appropriate and reasonable.

The inclusion/exclusion criteria were appropriate.

The sample was not a vulnerable group.

Measurement Criteria

The measurements were accurate and reliable.

The measurements were appropriate to the question being asked.

Quality assurance was addressed; however, some of the quality assurance
was not as precise as it should have been.

Statistical Analysis Criteria

The data can be analyzed to calculate CPT with a range of variability.

The statistical method will be commented upon in more detail in the
Board’s response to protocol SCI-001 below. 

Measures of uncertainty were addressed.

Laboratory and Field Conditions

Laboratory experiments were appropriate.

Field experiments were not conducted.

The study included a stop rule plan, medical management plan, and a
safety monitor.

HSRB Consensus and Rationale

The reported studies on the efficacy of lotion and pump spray
formulations of IR3535 (studies EMD-003.1 and EMD-003.2) on repelling
ticks are sufficiently sound, from a scientific perspective, to be used
to assess the repellent efficacy of the two formulations against ticks.

The Board also recognized that advances in statistical analyses mean
that there are probably ways of measuring efficacy of individual
products that would be an improvement over traditional techniques. The
Board encouraged the Agency to proceed in its efforts to examine how a
transition to more appropriate methods of calculating efficacy for
specific data sets can be introduced so that consumers can not only
compare relative efficacy of products based on traditional methods but
also have better information on the degree of protection individual
products provide.

The Board commended Dr. Carroll for conducting the preliminary dosimetry
phase of the study. Since the Agency noted that this was the first
repellency study to have a specific dosimetry phase, the HSRB suggested
EPA might wish to provide guidance concerning whether the method
employed in this study was the most valid way to determine dose.  The
Board recommended inclusion of a description of the sampling frame and
definition of eligible subjects to help justify subject
generalizability.

Charge to the Board

Ethical Considerations

Does available information support a determination that these studies
were conducted in substantial compliance with subparts K and L of EPA
regulations at 40 CFR part 26?

Brief Overview of the Study

The protocol for these two studies was initially reviewed at the June
2006 meeting of the HSRB, at which time the Board concluded that the
study failed to meet the requirements established in the Environmental
Protection Agency’s final human studies rule (40 CFR Part 26, subpart
K.)  The Board also raised questions about: 1) equitable study subject
selection and recruitment; and 2) whether or not the documentation and
process of study subject enrollment was sufficient to meet prevailing
standards of voluntary informed consent. A revised Institutional Review
Board (IRB)-approved protocol was submitted and reviewed at the October
2006 meeting of the HSRB, at which the Board concluded that the revised
research protocol, as submitted to the EPA, was compliant with the
applicable ethical requirements of 40 CFR Part 26, subpart K and L. 

Subsequent to the aforementioned October meeting of the HSRB, two
dosimetry and efficacy studies for tick repellents containing IR-3535
were conducted from October 23 through November 8, 2006 (Carroll 2006a;
Carroll 2006b). The studies were performed in Davis, California by
researchers at Carroll-Loye Biological Research. The studies were
sponsored by EMD Chemicals, Inc., Gibbstown, New Jersey; EMD Chemicals
is the North American subsidiary of Merck KGaA, Darmstadt, Germany. The
documents provided by Carroll-Loye specifically state that each study
was conducted in compliance with the requirements of the U.S. EPA Good
Laboratory Practice Regulations for Pesticide Programs, as promulgated
at 40 CFR Part 160 (Carroll 2006a, 3; Carroll 2006b, 3). Each study was
also reviewed and approved by a commercial human subjects review
committee, Independent Investigational Review Board (IIRB), Inc.,
Plantation, FL. Documentation provided to the EPA by IIRB indicates that
it reviewed these studies pursuant to the standards of the Common Rule
(45 C.F.R. Part 46, Subpart A) and determined them to be in compliance
with that Rule.

As submitted to the EPA, each completed study consisted of two
interdependent analyses: 1) a dosimetry study designed to determine the
amount of an insect-repelling compound, known as IR-3535, that typical
users would typically apply when provided with one of two compound
formulations (lotion or pump spray); and 2) an efficacy study designed
to measure the efficacy of IR-3535 as a tick repellent for each compound
formulation. Dosimetry was determined either by passive dosimetry using
self-adhesive roll-gauze (pump spray formulation) or by direct
measurement of compound application (lotion formulation). The efficacy
of IR-3535 as a tick repellent was determined by placing Western
black-legged ticks (Ixodes pacificus) on IR-3535-treated and untreated
forearms and measuring the speed and distance that moving insects would
penetrate into the treated area; thus, each subject served as his/her
own control. The scientific strengths and weaknesses of each study
design were described above.

The dosimetry study enrolled a total of 12 individuals, seven women and
five men, each of whom tested both the lotion and pump spray
formulations. The efficacy study for each formulation enrolled 10
subjects: seven women and three men tested the lotion formulation, and
four women and six men tested the pump spray formulation. Two subjects
enrolled in the dosimetry study participated in both the lotion and pump
spray efficacy studies. All remaining subjects participated in only one
of the three analytic phases of EMD-003.1 and EMD-003.2, giving a total
of 28 subjects enrolled. In addition, three alternate subjects were
enrolled to: 1) replace any subject who withdrew; and 2) protect the
confidentiality of any subject excluded from the study as a result of
pregnancy or other potentially stigmatizing condition, as described
below. Study documents, however, also include limb measurement
information for an additional nine subjects who were not enrolled in
either the dosimetry or the efficacy studies. These subjects appear to
be enrolled in two additional studies also submitted the EPA by
Carroll-Loye Biological Research, EMD-004.1 (Completed Efficacy Studies
for Mosquito Repellents Containing IR-3535 – Lotion) and EMD-004.2
(Completed Efficacy Studies for Mosquito Repellents Containing IR-3535
– Pump Spray) (Carroll 2006c; Carroll 2006d).

Critique of Study

The Board concurred with the factual observations of the ethical
strengths and weaknesses of the study, as detailed in the EPA’s Ethics
Review (Carley 2006a). In general, the research described in EMD-003.1
and EMD-003.2 comports with the applicable requirements of 40 CFR Part
26, subparts K and L. The risks to study participants were minimal and
were justified by the likely societal benefits, including data on the
efficacy of IR-3535 as a tick repellent. As IR-3535 is commercially
available and has been used as a repellent in Europe for years with no
evidence of toxic effects, the subjects enrolled in this study were
unlikely to be at increased risk of experiencing adverse side effects
upon exposure. The ticks used for the study were reared in a laboratory
environment and are considered to be pathogen-free, minimizing the risk
of vector-borne disease. Clear stopping rules also were developed, as
were plans for the medical management of any side effects or adverse
events; no side effects or adverse events were reported. The study
protocol also included several mechanisms designed to minimize coercive
subject recruitment and enrollment, compensation was not considered to
be so high as to unduly influence participation, and minors and pregnant
or lactating women were explicitly excluded from volunteering (pregnancy
being confirmed by requiring all female volunteers to undergo a
self-administered over-the-counter pregnancy test on the day of the
study). The potential stigmatization resulting from study exclusion was
minimized by the use of so-called “alternate” subjects, allowing for
volunteers to withdraw or be excluded from participating without unduly
compromising their confidentiality.

The revised protocol and informed consent documents were reviewed and
approved by IIRB, Inc., on November 1, 2006, nine days after study
subject enrollment began. In email correspondence between Dr. Scott
Carroll of Carroll-Loye Biological Research and Mr. John Carley of the
EPA’s Office of Pesticide Programs, dated December 18-19, 2006, Dr.
Carroll reported that subjects were enrolled using a previously approved
protocol and consent form, dated September 12, 2006; these were modified
to reflect protocol and consent form changes under review but not yet
approved by the IIRB. For example, Dr. Carroll reported that, “to each
of the 12 September consent forms used for subject enrollment …
corrections were made by hand, and acknowledged by initialing by the
subject and Study Director” (Carley and Carroll 2006). 

Although it is unlikely that these changes knowingly and/or seriously
impaired the informed consent process, enrollment of subjects using
unapproved protocols and consent forms represents a significant and
serious departure from accepted review and approval practices. EPA
regulations regarding review and approval of human subjects research,
for example, prohibit investigators from implementing any protocol
changes without prior IRB approval unless such changes are necessary to
prevent immediate, serious harm to study participants. The regulations
also require investigators to only obtain consent using IRB-approved
forms. Furthermore, it is the policy of the IIRB, available online at  
HYPERLINK "http://iirb.com"  http://iirb.com , that all “significant
protocol deviations [be] reported to the Independent Investigational
Review Board, Inc. in a timely manner.” Protocol violations or
deviations occur when there is a variance in a research study between
what is described in the protocol approved by the IRB and the actual
activities performed by the research team. The failures of Carroll-Loye
Biological Research to 1) obtain IRB approval of the revised protocol
and consent forms prior to enrollment of study subjects, and 2) report
these deviations to the Independent Investigational Review Board, are
serious regulatory breaches. The Board thus recommended that
Carroll-Loye Biological Research report these deviations to the IIRB as
soon as possible and work with that organization to develop and
implement a corrective course of action.  

HSRB Consensus and Rationale

The Board concurred with the initial assessment of the Agency that the
studies EMD–003.1 and EMD-003.2 submitted for review by the Board meet
the applicable requirements of 40 CFR part 26, subparts K and L. 

The Board also noted that there were a series of deviations from Subpart
K that, while not adversely affecting the rights and welfare of human
subjects of the study, reflected a lack of familiarity with IRB
procedures and protocol requirements described in Subpart K. The HSRB
advised the Agency to recommend that investigators undergo human
research protection training, and include evidence of completion of such
training as part of their submission of protocols or completed studies
to the Agency.  Examples of such training could include the on-line
training program offering by NIH/NCI or a similar program that might be
developed by the Agency.   

Completed IR3535 Insect Repellent Efficacy Studies 

Studies EMD-004.1 and EMD 004.2

Charge to the Board

Scientific Considerations

Are these studies sufficiently sound, from a scientific perspective, to
be used to assess the repellent efficacy of the formulations tested
against mosquitoes?  

Board Response to the Charge

The active ingredient IR 3535 was tested for its ability to repel
mosquitoes from the forearms or legs of volunteers by the protocol
presented and modified by Carroll-Loye.  The protocol had been modified
based on the suggestions and input of EPA and HSRB.  The results were
reported in EMD-004.1 and EMD-004.2

The active ingredient was formulated into two products, a pump spray and
a lotion, but data on the originally proposed aerosol was not provided
because of an error in the formulation. The products were produced using
Good Manufacturing Practices. All experiments were conducted using Good
Laboratory Practices.  A passive dosimetry experiment was done, as
suggested by the HSRB, to determine the amount of product that would be
utilized by people using the product as directed.  This passive
dosimetry experiment was used to determine a grand mean of the 12
individuals tested (3 subsamples each) per product that was then used
for all 10 individuals per product participating in the subsequent
mosquito repellency tests for each product. (It should be noted that the
dosimetry experiment was in common for both this study and the tick
repellency study, EMD-003, since the same formulated products were used
for both.)

The experiment was a field study and was conducted according to the
approved protocol with only very minor deviations, and none of these
deviations would have affected the quality of the data or the safety of
the subjects.  Two locations in California were used, one a dense forest
and the other a moist pasture marshland; the two locations had
differences in the composition and relative abundance of mosquito
species. Neither location showed evidence of the presence of West Nile
Virus (WNV). The number of 10 subjects per product was justified in the
text as leading to sufficient statistical power while exposing only a
small number of people to the potential risks.  

	Each subject had one limb treated, and the remainder of the body was
covered with material impervious to mosquitoes. There were two
experienced persons serving as negative controls (i.e., without any
repellant product) to confirm mosquito biting pressure (and biting
pressure was maintained throughout the period of the study, defined as
at least one Landing with Intent to Bite, LIBe, per min). Experimental
subjects, in pairs, monitored LIBe’s during a one min interval each 15
min, until the First Confirmed LIBe (FCLIBe) was determined. Stopping
rules were employed.  The Complete Protection Time (CPT) was calculated
as the mean for all participants for each product.  For the lotion the
study identified a range of 6-8.5 hr with a mean CPT of 7.3 hr for the
forest, and a range of 7.75 to 10 hr with a mean CPT of 8.5 hr for the
marsh.  For the pump spray the study identified a range of 5 to 8 hr
with a mean CPT of 7.1 hr for the forest and a range of 7.7 to 10 hr
with a CPT of 8.4 hr for the marsh.  The CPT is probably conservative as
a number of the subjects reported no LIBe’s at all, and the experiment
was terminated before a FCLIBe was observed.

With respect to the science criteria established earlier by the HSRB for
completed studies: 

General HSRB Scientific Criteria

The scientific question was stated (i.e., to test the efficacy of IR3535
in repelling mosquitoes).

Existing data were not adequate to answer the question of efficacy of
these new formulations.

Because existing data were not adequate to answer the question of
efficacy, new studies involving human subjects are necessary.

The potential benefits of the study were clear, i.e., that an effective
repellent would be available that would have either greater efficacy
and/or fewer drawbacks than what was currently approved.

It is likely that the benefits would be realized because repellent
efficacy was determined in carefully designed field experiments.

The risks are minimal because the formulation products are of very low
toxicity, the mosquitoes were aspirated before they had an opportunity
to bite, and the regions selected did not have evidence of WNV.

The most likely relevant risk would be irritation from mosquito bites,
but participants were instructed to remove mosquitoes before they were
bitten, or the possibility of infection with WNV, but the regions
selected had no evidence of the virus.

Study Design Criteria

The purpose of the study was clearly defined (i.e., efficacy testing).

There were specific objectives/hypotheses (i.e., that IR3535 in the
proposed formulations is an effective repellent).

The study as described tested this hypothesis.

The sample size was 10 individuals per product along with 2 experienced
individuals to confirm mosquito biting pressure. A dosimetry experiment
prior to the field experiment determined the amount of repellent to be
tested. 

There was a plan allocating individuals to treatments.

It is anticipated that the findings from this study can be generalized
beyond the study sample.

Participation Criteria

There was justification for the selection of the target population.

The participants were representative of some of the population of
concern; however, there are others in the population unlike these
participants who are likely to use these products, but it would either
be unethical to test them or would be less appropriate to test them. 
The participating population is considered appropriate and reasonable.

The inclusion/exclusion criteria were appropriate.

The sample was not a vulnerable group.

Measurement Criteria

The measurements were accurate and reliable.

The measurements were appropriate to the question being asked.

Quality assurance was addressed; however, some of the quality assurance
was not as precise as it should have been.

Statistical Analysis Criteria

The data can be analyzed to calculate CPT with a range of variability.

The statistical method will be commented upon in more detail by the
Board in its response to protocol SCI-001 below. 

Measures of uncertainty were addressed.

Laboratory and Field Conditions

Laboratory experiments were not conducted.

Field experiments were appropriate.

The study included a stop rule plan, medical management plan, and a
safety monitor.

HSRB Consensus and Rationale

The reported studies on the efficacy of lotion and pump spray
formulations of IR3535 (studies EMD-004.1 and EMD-004.2) on repelling
mosquitoes are sufficiently sound, from a scientific perspective, to be
used to assess the repellent efficacy of the two formulations against
mosquitoes.

The Board also recognized that advances in statistical analyses mean
that there are probably ways of measuring efficacy of individual
products that would be an improvement over traditional techniques. The
Board encouraged EPA to proceed in its efforts to examine how a
transition to more appropriate methods of calculating efficacy for
specific data sets can be introduced so that consumers can not only
compare relative efficacy of products based on traditional methods but
also have better information on the degree of protection individual
products provide.

 The Board commended Dr. Carroll for conducting the preliminary
dosimetry phase of the study. Since the Agency noted that this was the
first repellency study to have a specific dosimetry phase, the HSRB
suggested EPA might wish to provide guidance concerning whether the
method employed in this study was the most valid way to determine dose. 
The Board recommended inclusion of a description of the sampling frame
and definition of eligible subjects to help justify subject
generalizability.

Charge to the Board

Ethical Considerations

Does available information support a determination that these studies
were conducted in substantial compliance with subparts K and L of EPA
regulations at 40 CFR part 26?

Brief Overview of the Study

The protocol for these two studies was initially reviewed at the June
2006 meeting of the Human Studies Review Board, at which time the Board
concluded that the study failed to meet the requirements established in
the Environmental Protection Agency’s final human studies rule (40 CFR
Part 26, subpart K.)  The Board also raised questions about: 1)
equitable study subject selection and recruitment; 2) description and
minimization of risks to study participants; and 3) whether or not the
documentation and process of study subject enrollment was sufficient to
meet prevailing standards of voluntary informed consent. A revised,
Institutional Review Board (IRB)-approved protocol was submitted and
reviewed at the October 2006 meeting of the Human Studies Review Board,
at which the Board concluded that the revised research protocol, as
submitted to the EPA, was compliant with the applicable ethical
requirements of 40 CFR Part 26, subpart K and L.

Subsequent to the aforementioned October meeting of the HSRB, two
dosimetry and efficacy studies for mosquito repellents containing
IR-3535 were conducted from October 23 through November 8, 2006 (Carroll
2006c; Carroll 2006d). The studies were performed at a laboratory site
in Davis, California, and at two field sites in Butte and Glenn
Counties, California, by researchers at Carroll-Loye Biological
Research. The studies were sponsored by EMD Chemicals, Inc., Gibbstown,
New Jersey; EMD Chemicals is the North American subsidiary of Merck
KGaA, Darmstadt, Germany. The documents provided by Carroll-Loye
specifically state that each study was conducted in compliance with the
requirements of the U.S. EPA Good Laboratory Practice Regulations for
Pesticide Programs, as promulgated at 40 CFR Part 160 (Carroll 2006c, 3;
Carroll 2006d, 3). Each study was also reviewed and approved by a
commercial human subjects review committee, Independent Investigational
Review Board (IIRB), Inc., Plantation, FL. Documentation provided to the
EPA by IIRB indicates that it reviewed these studies pursuant to the
standards of the Common Rule (45 C.F.R. Part 46, Subpart A) and
determined them to be in compliance with that Rule.

As submitted to the EPA, each completed study consists of two
interdependent analyses: 1) a dosimetry study designed to determine the
amount of a formulation (lotion or pump spray) containing an
insect-repelling compound, known as IR-3535, that users would typically
apply when provided with one of two compound formulations (lotion or
pump spray); and 2) efficacy studies designed to measure the efficacy of
IR-3535 as a mosquito repellent for each formulation. Dosimetry was
determined either by passive dosimetry using self-adhesive roll-gauze
(pump spray formulation) or by direct measurement of compound
application (lotion formulation). The efficacy of IR-3535 as a mosquito
repellent was determined by measuring the ability of the three
formulations to prevent mosquito landings (defined as “Lite with
Intent to Bite”; LIBe) under field conditions. Mosquitoes were
aspirated mechanically prior to biting; prior to initiation of the
efficacy study, all volunteers were trained both to recognize a mosquito
landing with the intent to bite and to remove such mosquitoes with an
aspirator using laboratory-raised, pathogen-free mosquitoes in a
controlled laboratory setting. During the field studies, subjects worked
in pairs to facilitate identification and aspiration of LIBing
mosquitoes during brief exposure periods. The scientific strengths and
weaknesses of each study design are described above.

The dosimetry study enrolled a total of 12 individuals, seven women and
five men, each of whom tested both the lotion and pump spray
formulations. The field-based efficacy study for each formulation
enrolled 10 subjects: seven women and three men tested both the lotion
and pump spray formulation over two days at a “forest” site in Butte
County, and four women and six men tested the pump spray formulation
both the lotion and pump spray formulation over two days at a
“marsh/pasture” site in Glenn County. One subject enrolled in the
dosimetry study participated in the “forest” efficacy study, three
additional subjects participated in both the “forest” and
“marsh/pasture” studies, and a fifth subject participated in the
dosimetry, “forest”, and “marsh/pasture” studies. All remaining
subjects participated in only one of the analytic phases of EMD-004.1
and EMD-004.2. Two control subjects, described as “experienced
personnel” (Carroll 2006c, 9; Carroll 2006d, 9) and who were untreated
with either repellent formulation, also participated to determine
ambient LIBe pressure, giving a total of 26 subjects enrolled. In
addition, three alternate subjects were enrolled to: 1) replace any
subject who withdrew; and 2) protect the confidentiality of any subject
excluded from the study as a result of pregnancy or other potentially
stigmatizing condition, as described below. Study documents, however,
also include limb measurement information for additional subjects who
were not enrolled in either the dosimetry or the efficacy studies. These
subjects appear to be enrolled in two additional studies submitted to
the EPA by Carroll-Loye Biological Research, EMD-003.1 (Completed
Efficacy Studies for Tick Repellents Containing IR-3535 – Lotion) and
EMD-003.2 (Completed Efficacy Studies for Tick Repellents Containing
IR-3535 – Pump Spray) (Carroll 2006a; Carroll 2006b).

Critique of Study

The Board concurred with the factual observations of the ethical
strengths and weaknesses of the study, as detailed in the EPA’s Ethics
Review (Carley 2006b). In general, the research described in EMD-004.1
and EMD-004.2 comports with the applicable requirements of 40 CFR Part
26, subparts K and L. The risks to study participants were minimal and
were justified by the likely societal benefits, including data on the
efficacy of IR3535 as a mosquito repellent. IR3535 is commercially
available and has been used as a repellent in Europe for years with no
evidence of toxic effects, so the subjects enrolled in this study were
unlikely to be at increased risk of experiencing adverse side effects
upon exposure. Reactions to mosquito bites are usually mild and easily
treated with over-the-counter steroidal creams. The study also excluded
subjects who have a history of such severe skin reactions to further
minimize the risk of a subject experiencing a severe physical reaction
to a mosquito bite. In addition, the study protocol was designed
specifically to minimize the likelihood that a mosquito will bite,
through the use of clear stopping rules, limited exposure periods, and
paired observation; no side effects or adverse events were reported. To
minimize the risk that study subjects would be exposed to disease causal
agents like WNV, the study protocol called for field tests of repellent
efficacy to be conducted only in areas where known vector-borne diseases
have not been detected by county and state health or vector/mosquito
control agencies for at least one month. Although it would have been
ideal if the mosquitoes collected during the field studies were
subjected to serologic or molecular analyses to confirm that they were
free of known pathogens, it is unlikely that failure to do so
compromised participant safety in any significant way.  Finally, the
study protocol also included several mechanisms designed to minimize
coercive subject recruitment and enrollment, compensation was not
considered to be so high as to unduly influence participation, and
minors and pregnant or lactating women were explicitly excluded from
volunteering (pregnancy being confirmed by requiring all female
volunteers to undergo a self-administered over-the-counter pregnancy
test on the day of the study). The potential stigmatization resulting
from study exclusion was minimized by the use of so-called
“alternate” subjects, allowing for volunteers to withdraw or be
excluded from participating without unduly compromising their
confidentiality.

As with the two tick repellent studies (EMD-003.1 and EMD-003.2), the
revised protocol and informed consent documents used for these mosquito
repellent studies were reviewed and approved by IIRB several days after
study subject enrollment began; some subjects participating in these
studies were re-consented using IIRB-approved documents, but not all
were. Although it is unlikely that these changes knowingly and/or
seriously impaired the informed consent process, enrollment of subjects
using unapproved protocols and consent forms represents a significant
and serious departure from accepted review and approval practices. The
failures of Carroll-Loye Biological Research to 1) obtain IRB approval
of the revised protocol and consent forms prior to enrollment of study
subjects, and 2) report these deviations to IIRB, are serious regulatory
breaches. The Board thus recommended that Carroll-Loye Biological
Research report these deviations to the IIRB as soon as possible and
work with that organization to develop and implement a corrective course
of action.

Second, the IIRB-approved protocol and consent documents specifically
stated that they are to be conducted only in areas where known
vector-borne diseases have not been detected by county and state health
or vector/mosquito control agencies for at least one month (see, for
example, Carroll 2006c, 75). One sentinel poultry flock in the area,
however, did test positive for WNV during the month prior to conduct of
the field studies (Carroll 2006c, 7). Sentinel flocks closer to the two
study sites did not test positive for arboviruses during this period,
and a leading vector control ecologist consulted by Carroll-Loye
reported that “WNV activity in Northern Calfornia [was] effectively
concluded for 2006” (Carroll 2006c, 7), so it is unlikely that
participant safety was compromised in any significant way. Nevertheless,
initiation of field studies following the detection of WNV in a sentinel
chicken flock represents a deviation from the approved protocol and
should be reported to the IIRB as soon as possible.

Finally, even though two IR-3535-untreated control subjects were
enrolled in the study, the IIRB-approved consent documents provided for
review do not list the unique risks that these two volunteers faced.
These control subjects were “experienced” personnel who were likely
aware of these risks, but nonetheless should have been consented using
documents that listed these dangers. 

HSRB Consensus and Rationale

The Board concurred with the initial assessment of the Agency that
studies EMD 004.1 and EMD 004.2 submitted for review by the Board met
the applicable requirements of 40 CFR part 26, subparts K and L.  

The Board also noted that there were a series of deviations from Subpart
K that while not adversely affecting the rights and welfare of human
subjects of the study, reflected a lack of familiarity with IRB
procedures and protocol requirements described in Subpart K. The HSRB
advised the Agency to recommend that investigators undergo human
research protection training, and include evidence of completion of such
training as part of their submission of protocols or completed studies
to the Agency.  Examples of such training could include the on-line
training program offering by NIH/NCI or a similar program that might be
developed by EPA.   

Insect Repellent Efficacy Protocol SCI-001

Background

The objectives of this study will be to test the mosquito repellent
efficacy characteristics of three test materials, to compare them to one
another, reinforce measurements of time for which they are effective,
and to contrast them with the U.S. military issue topical insect
repellent.  Test Material #1 is LipoDEET, which contained 30% DEET that
had lipid spheres and inhibits evaporation, improved field, and reduced
plasticizing and odor.  Test Material # 2, Coulston’s Duranon, is 20%
DEET in a controlled-release, low-odor formulation.  Test Material #3 is
Insect Guard II, which contains as active ingredients 17.5% DEET, 5%
N-octyl bicycloheptane dicarboximide (synergist), and 2.5% Di-n-propyl
isocinchomerate (fly repellent).  Test Material # 4, 3M Ultrathon
(military issue repellent), contained 34.34% DEET in a polymer-based
lotion to extend efficacy and reduce plasticizing.

This study will be similar to EMD-004 in terms of the dosimetry phase,
efficacy measurements (time to “first confirmed landing with intent to
bite”), and training of subjects in aspirating mosquitoes before they
bite.  The field conditions and timing of exposure also will be  similar
(treated subjects work in pairs, untreated controls work with 2
assistants to aspirate landing mosquitoes, and both treated and
untreated subjects are exposed to the mosquitoes for 1 minute every 15
minutes).  The field testing sites will be in the California Central
Valley or Florida Keys, with expected wild mosquito populations of Aedes
vexans, Ochlerotatus melanimon, O. taeniorhynchus, and Culex pipens. 
The test results would be analyzed using unspecified statistics. 
Measurements would be reported with 95% confidence intervals of the mean
and associated standard deviations.  The efficacy of each treatment
would be compared to that of Ultrathon.  The sample size reflects a
compromise between financial and ethical concerns, although it was
difficult to pre-determine sample size without knowing the distribution
of outcome values.  EPA guidelines recommend 6 replicates, which is
considered sufficient to show statistical significance at P<0.05.  EPA
recommended changes to the protocol to include developing a full
description of the statistical analysis plan to compare means and to
assess within-treatment variability, and to define a testable
hypothesis. 

Charge to the Board

Scientific Considerations

If the proposed research described in Protocol SCI-001 from Carroll-Loye
Biological Research is revised as suggested by EPA, does the research
appear likely to generate scientifically reliable data, useful for
assessing the efficacy of the test substances for repelling mosquitoes? 

Board Response to the Charge

The proposal intends to test the efficacy of three novel formulations of
N,N-diethyl m-toluamide (DEET).  Three topical formulations containing
DEET will be tested against a positive control, Ultrathon (35% DEET). 
The objectives, design and methods are adequately described.  The plans
for statistical analyses of the data, however, require significant
revision as detailed below. 

Comments and suggestions for revision or clarification:

Experimental design: While this is not identified as a limitation of the
study, nowhere is it justified the randomization to left and right
limbs.  Thus is there any reason to believe that products will be more
or less effective on the two limbs? In addition the Board questioned why
use right/left as a blocking variable (that is subsequently ignored in
the analysis) in the design? 

Statistical analysis:  The investigator proposed computing the means and
confidence intervals around those means for CPT in each treatment group
(three test products plus the positive control). But this simple
analysis makes comparisons across products more difficult. If
comparisons are to be made  the following approach should be considered:

Let CPTij denote the CPT measured on the ith subject in the jth
treatment, where i=1,…,10 and j=1,…,4. 

Fit a linear model to the 40 measurements, with a fixed effect for
treatment. Other fixed effects can also be included if they happen to be
of interest. 

This approach permits direct comparisons among products. In particular,
it is possible to obtain a point estimate and a confidence interval for
the difference in CPT in a test product and in the comparison product,
one of the objectives of the study.

If the entire study is replicated in two locations using different
subjects, then an even better approach is to fit a model to the entire
set of 80 measurements, but adding an effect for location (and perhaps
an interaction between location and treatment) to the model.

The investigator can easily fit the model using JMP. Just use the Fit X
by Y option in the Analyze menu and then choose ANOVA.

Interpretation of results:  Results from this study need to be
interpreted judiciously. Given the large variability in individual
attractiveness to mosquitoes, the small sample size seriously limits
conclusions that the sample is representative of the population of
individuals who might eventually be users of these products. While the
long list of exclusions is justifiable, one consequence is that the
population represented in the sample is different from the population of
potential users. 

Sample size:  Including 10 subjects per treatment is probably
sufficient, but the justification provided by investigators is not
convincing. First, in order to estimate power an estimate of the
within-treatment variance in the response variable is needed.  The
investigator does not provide such an estimate in the discussion. Thus
it is unclear how they can argue that “from the standpoint of
statistical power, six treated and one untreated subject are sufficient
to demonstrate a significant effect at P<0.05”. Second, the argument
used to justify no more than 10 subjects per treatment states that
“adding subjects beyond six increases the precision of the means
estimate only slowly”. This argument relies on the assumption that the
between-person variance in CPT does not change as sample size increases,
which in general is not true.  The information on interindividual
variability drawn from studies completed by these investigators may be
used to guide and justify sample size. Submission of the completed
protocol to EPA should include evidence that steps like those described
above were taken to justify sample size.

Sample Size Considerations for Subject Drop-Outs. In previous studies,
subjects dropped out at different points potentially confounding the
quantification of the CPT. Criteria need to be established for how long
subjects must remain in the study in order for their data to be used.
Criteria for when a new subject must be run as a substitute to meet the
sample size requirements must also be determined.

Assumption of normality of CPT measurements:  In choosing statistical
analyses the investigator must select the appropriate model for the
distribution of the data that will be used. The methods for the
statistical analysis of the data rely heavily on the assumption that
measurements are normal. Because of the small sample size, departures
from normality can have important consequences on the validity of the
methodology proposed here. In this case, the assumption of normality is
probably justifiable and in any case can be easily tested and corrected
for. There seems to be some confusion) regarding the exponential family
of distributions and the exponential distribution. The latter is a
standard probability model for variables such as time which are strictly
positive and tend to exhibit a rounded L shape when plotted. The former
has nothing to do with the study at hand. If CPTs can all be expected to
be noticeably larger than 0, then approximating the exponential model
with a normal model may be justifiable.

Measurement variables: Although it is clear from the rest of the
protocol discussion, the investigator might consider adding CPT to the
list of variables given in Section 10.1.

Dose:  Even though it is suggested that the typical consumer exposure
should be far below the dermal toxicity benchmarks, there is no
indication of such toxicological data in the MSDS included with this
submission. Typical consumer dose and known toxicity benchmarks should
be clearly identified.

HSRB Consensus and Rationale

The Board raised several concerns about sample size, population
generalizability and statistical analysis in SCI-001 that should be
addressed.   If the recommendations provided by EPA and those suggested
by the Board are followed, protocol SCI-001 appears likely to generate
scientifically valid data to assess the efficacy of the test products
against mosquitoes.   In addition, the protocol would satisfy the
scientific criteria recommended by the HSRB, namely, producing important
information that cannot be obtained except by research with human
subjects, and having a clear scientific objective and study design that
should produce adequate data to test the hypothesis.   

Charge to the Board

Ethical Considerations

If the proposed research described in Protocol SCI-001 from Carroll-Loye
Biological Research is revised as suggested by EPA, does the research
appear to meet the applicable requirements of 40 CFR part 26, subparts K
and L?

Board Response to the Charge

The Board concurred with the factual observations of the ethical
strengths and weaknesses of the study, as detailed in the EPA’s
Science and Ethics Review (Carley 2006c). In general, the research
described in Protocol SCI-001comports with the applicable requirements
of 40 CFR Part 26, subpart K and L. The risks to study participants are
limited and appropriate steps have been taken to minimize these risks.
The risks to participants are justified by the likely societal benefits,
including data on the efficacy of new topical formulations containing
DEET as a mosquito repellent. DEET is commercially available and has
been used as a repellent for years with no evidence of substantial toxic
effects, so the subjects enrolled in this study are unlikely to be at
increased risk of experiencing adverse side effects upon exposure.
Reactions to mosquito bites are usually mild and easily treated with
over-the-counter steroidal creams. In addition, the study protocol is
designed to minimize the likelihood that a mosquito will bite, through
the use of clear stopping rules, limited exposure periods, and paired
observation. To minimize the risk that study subjects will be exposed to
illnesses resulting from WNV, the study protocol calls for field tests
of repellent formulations to be conducted only in areas where known
vector-borne diseases have not been detected by county and state health
or vector/mosquito control agencies. 

The Board recommended that the investigator collect mosquitoes during
the field studies and that they be subject to serologic or molecular
analyses to confirm absence of known pathogens.  Finally, the study
protocol included several mechanisms designed to minimize coercive
subject recruitment and enrollment, compensation was not considered to
be so high as to unduly influence participation, and minors and pregnant
or lactating women were explicitly excluded from volunteering (pregnancy
being confirmed by requiring all female volunteers to undergo a
self-administered over-the-counter pregnancy test on the day of the
study). The potential stigmatization resulting from study exclusion was
minimized by the use of so-called “alternate” subjects, allowing for
volunteers to withdraw or be excluded from participating without unduly
compromising their confidentiality.

The Board concluded that research described in Protocol SCI-001
minimizes risks to subjects and has appropriate stopping rules in place.
The safety monitoring proposed seems reasonable and appropriate in light
of the level of risk to subjects. Despite this generally favorable
assessment, the Board considered several additional matters relevant to
subject recruitment and the overall conduct of the study.

First, as noted in the Agency’s review of Protocol SCI-001, the
protocol does not describe how untreated controls would be recruited.
The protocol implies that controls will be recruited in the same manner
as subjects in the “exposure” arm—via “word-of-mouth” and a
Volunteer Data Base maintained by the Principal Investigator. The
protocol should clarify how untreated controls will be recruited. . The
Board also found it a bit unusual that the IRB did not ask to review a
script of the proposed recruitment phone call as most IRBs regard
recruitment as the first step in the IC process and require that all
recruitment activities be reviewed. This would include any fliers,
emails, letters, or local ads as well, which should be submitted to IIRB
for review.

Second, the Board discussed several issues related to subject
recruitment and consent. First, the risks associated with DEET exposure
during the course of the study are mischaracterized in the submitted
informed-consent document, which refers to sprayed applications
containing alcohol.  Since the study involved the application of lotions
to the skin, these risks should be redescribed.  In addition, the
informed-consent document is structured in a manner that does not apply
to unexposed control subjects.  Also, the submitted informed-consent
document indicates that up to 40 subjects may participate in the study
when the correct number should be 48 (10 exposed and 2 controls per arm
of the study).

Third, the Board discussed the fact that the proposed sample size is
slightly larger than what EPA has historically required (10 exposed
subjects vs. the historical norm of 6 exposed subjects). The protocol
provides a rationale for this approach (pp. 13-15), which is meant to
reduce the probability that the sample over-represents individuals who
are “inherently unattractive” to mosquitoes.  In light of the
limited risks to subjects, this departure from the historical norm was
viewed as acceptable by the Board.

Fourth, the Board found it difficult to assess the qualifications of the
IIRB based on the materials that were supplied. Although the Board did
not have significant concerns about the overall quality of the IRB’s
review of the protocol, it would be reassuring to the Board if some type
of documentation of the IRB’s qualifications were provided to the
Agency for review (e.g., evidence of member training, accreditation by
an external professional body, etc.).

HSRB Consensus and Rationale

The Board concluded that the protocol should meet the applicable
requirements of 40 CFR part 26, subparts K and L if the points raised in
the EPA review and in this report are adequately addressed.  

REFERENCES 

Carley, J.M. 2006a. Ethics Review of EMD-003 Reports of Completed
Efficacy Studies for Tick Repellents Containing IR-3535. Dated December
19, 2006.  Unpublished document prepared by Office of Pesticide
Programs, United States Environmental Protection Agency.

Carley, J.M. 2006b. Ethics Review of EMD-004 Reports of Completed
Efficacy Studies for Mosquito Repellents Containing IR-3535. Dated
December 19, 2006.  Unpublished document prepared by Office of Pesticide
Programs, United States Environmental Protection Agency.

Carley, J.M. 2006c.   Ethics Review of SCI-001.  Unpublished document
prepared by Office of Pesticide Programs, United States Environmental
Protection Agency.

Carley, J.M., and S. Carroll. 2006. E-mail Exchange Between Scott P.
Carroll of Carroll-Loye Biological Research and John M. Carley of EPA
Concerning Repellent Studies EMD-003 and EMD-004 on December 18-19,
2006. Unpublished document prepared by Office of Pesticide Programs,
United States Environmental Protection Agency.

Carroll, S. 2006a. Test of Personal Insect Repellents: Study EMD-003.1
(Lotion) – Revised. Dated December 15, 2006.  Unpublished document
prepared by 

Carroll-Loye Biological Research.

Carroll, S. 2006b. Test of Personal Insect Repellents: Study EMD-003.2
(Pump Spray) – Revised. Dated December 15, 2006.  Unpublished document
prepared by Carroll-Loye Biological Research.

Carroll, S. 2006c. Test of Personal Insect Repellents: Study EMD-004.1
(Lotion) – Revised. Dated December 15, 2006.  Unpublished document
prepared by 

Carroll-Loye Biological Research.

Carroll, S. 2006d. Test of Personal Insect Repellents: Study EMD-004.2
(Pump Spray) – Revised. Dated December 15, 2006.  Unpublished document
prepared by Carroll-Loye Biological Research.

  PAGE  1  of   NUMPAGES  28 

