FIFRA Scientific Advisory Panel Meeting, August 25-27, 2009

CHARGE QUESTIONS

 

Evaluation of the Expert System in the Context of the Organization for
Economic Cooperation and Development (OECD) Validation Principles

As discussed in the Preface and Introduction of the white paper, EPA’s
development of the Quantitative Structure Activity Relationship
(QSAR)-based expert system was guided by the OECD principles for (Q)SAR
validation.  The five principles include demonstration of:

a well defined endpoint,

mechanistic interpretation of the model,

defined domain of model applicability,

an unambiguous algorithm, and

appropriate measures of goodness of fit, robustness, and predictivity. 

A prototype of the expert system was the subject of an OECD convened
peer-consultation in February 2009, at which time the system was
evaluated using the (Q)SAR validation principles.  Based on input from
this peer consultation, the Agency further refined the expert system,
particularly as it related to the OECD validation principles.  The
components to Charge Question 1 address specific issues concerning the
(Q)SAR validation principles and the subject expert system in the
context of its use to determine the order in which chemicals (i.e.,
food use inert ingredients and antimicrobial pesticides) will be subject
to Tier 1 screening under EPA’s Endocrine Disrupter Screening Program
(EDSP)  (i.e., for prioritization).   

 

A.		Biological Endpoint 

The biological endpoint that is predicted by the expert system is
relative binding affinity to the cytosolic rainbow trout estrogen
receptor (ER).  Based on preliminary studies it was anticipated that
food use inert ingredients and antimicrobial pesticides would have low
relative binding affinities.  In addition, an evaluation of the two
respective inventories indicated a wide range of structures and
associated physical-chemical properties (e.g., solubility, Kow, etc).
Consequently, assay methods used to measure binding affinity to
establish the training set were designed to detect low levels of binding
affinity (e.g., testing to solubility in binding assays and cytotoxicity
or solubility, as appropriate, in slice assays). Confirmatory binding
studies (Ki experiments) and transactivation assays were employed to
systematically verify that apparent low affinity binding levels
represented competitive displacement and translated to ER-mediated
transactivation. 

Question A1. Please comment on the methods employed and their adequacy
for detecting and measuring ER binding affinity for the compounds in the
two chemical inventories of immediate interest. 

Question A2. In the context chemical prioritization for Tier 1
screening, please comment on the decision to measure binding affinity up
to the maximum concentration based on the solution properties of the
chemical, rather than using ligand concentration ‘cut-off’ values of
-4 Log Molar to -3 Log Molar, which have typically been used to conclude
a compound does not bind to the ER.  

Question A3. Please also comment on how any in vivo studies that are
available for compounds with low receptor binding affinity could be used
to provide a relative binding affinity ‘cut-off’ value either alone
or in combination with cut-off values based on the maximum solubility of
a ligand in the buffer solution. 

B.	Mechanistic Interpretation     

Numerous studies have established the alignment of estrogen and other
high affinity ligands within the ER binding domain and indicate that a
distance of 10.2 to 11A between the two H-bonding sites and stable
(non-flexible) ring structure is optimal for binding. These and other
studies lead to the assumption that acyclic compounds would not bind to
the ER, although a systematic analysis across a diversity of structures
was not available in the literature.  In the current study 25 acyclic
compounds across 10 classes present in the inventories were evaluated in
the training set and none were found to bind at a Relative Binding
Affinity (RBA) detection limit of 0.00001%.  

Question B1. Please comment on the adequacy of this training set for
supporting the expert system’s rule that acyclic compounds do not bind
to the ER.  

Based on studies by Katzenellenbogen et al. (2003 and references
therein) a working hypothesis in developing the training set was that
compounds in the inventories of interest could bind at one site; i.e.,
the A site or the B site, based on the presence of a hydrogen bond donor
or acceptor substituent.  The development of the training sets and the
resultant ER expert system use a chemical hierarchy based on different
binding mechanisms (i.e., A-B binding sites, A binding site only, B
binding site only).  

Question B2. Please comment on strengths and limitations of this
mechanistic interpretation for selecting chemicals in the training sets
and for interpreting the observed binding data. 

While ER binding can be an initial step in a toxicity pathway leading to
adverse reproductive outcomes (see Figure 1 in the white paper), the ER
binding data in the training set, and the associated expert system, were
not designed to predict in vivo responses.  Rather the expert system was
designed to predict relative ER binding affinity from a chemical’s
structure to support the prioritization of food use inert ingredients
and antimicrobial pesticides for in vitro and in vivo Tier 1 screening,
which is designed to ascertain if a compound has the potential to
interact with the estrogen system.

	Question B3. Please comment on the clarity of the white paper in
describing the differences in (Q)SAR development when the goal is to
predict in vitro ER receptor binding from chemical structure vs. when
the goal is to predict in vivo reproductive/developmental responses from
chemical structure.  Please indicate if additional discussion in the
white paper is needed to establish the relevance of ER binding affinity
(either measured or predicted) to interpret the potential for in vivo
outcomes.

C.	Model Domain     

The domain of the current ER expert system includes rules to support
predictions of ER binding affinity for chemicals in the food use inert
ingredient and antimicrobial pesticide inventories.  

Question C1. Please comment on the adequacy of the approach that was
used to select chemicals for the training sets in terms of these two
inventories.  

D.	Algorithm

The ER expert system provides predictions for each chemical, with each
individual prediction traceable to chemical subgroups, binding mechanism
and endpoint databases.  In developing the expert system several
chemical subgroups were identified as chemicals that contain multiple
functional groups. 

Question D1. Please make suggestions for improvements in presenting the
expert rules and their underlying rationale, especially with regard to
groups with multiple functional groups.

Question D2. Please also comment on the ability of the expert rules to
identify chemicals outside the model domain.

E.	Goodness of Fit, Robustness, and Predictivity

Consistent with suggestions by the EDSTAC (1998), and typical processes
for (Q)SAR development, the expert system rules were established through
an iterative process of defining subgroups, gathering empirical data to
refine subgroups rules, followed by collection of additional empirical
data to cover the structural domain and/or until a consistent pattern of
structural rules and activity emerged.  The expert rules permit each
chemical to be assigned to subgroups and an associated estimated binding
affinity value, accompanied by an explanation of the basis for the
estimate as well as of how the estimate compares to measured data for
other members of the same subgroup. The 2009 OECD expert consultation
report on the expert system recognized that standard statistical methods
such as those used to assess regression model QSARs are not necessarily
applicable to expert systems whereas transparency and usefulness as
described in the white paper are more appropriate parameters for
assessing the validity of an expert system. The peer consultation report
found the current approach, with individual predictions traceable to
chemical subgroups, binding mechanism and endpoint databases, to be
appropriate although the report noted that if additional information
could be made available it would facilitate future peer-review on this
issue.  

 

Question E1. Please comment on the adequacy of information presented in
the white paper to evaluate the scientific rationale of how a chemical
is processed through the decision logic; i.e., how a chemical is
assigned to a subgroup with an associated binding affinity value; the
mechanistic rationale for estimates of binding affinity data, including
data for related chemicals; and how it is determined that a chemical is
outside of the domain of the expert system. 

Question E2. While to date the Agency is not aware of statistical
approaches that would provide the means to assess goodness-of-fit or
predictivity of expert systems such as the one described here, is the
SAP aware of any statistical approaches or data presentations that could
be amenable for such evaluations?

F.	Transparency and Clarity of the Expert System

In its validation principles, OECD recognized the importance of a
transparent validation process for the development of (Q)SAR models in
order to further enhance their regulatory acceptance of (Q)SAR models.

 

Question F1. Please provide any additional comment on how well the white
paper’s summary of the expert system conforms to the OECD validation
principles and provide suggestions, as appropriate, to enhance the
clarity or transparency of the expert system’s development and
intended use with regard to the validation principles.

 

The white paper and associated presentations at the SAP meeting form the
basis of the documentation of the expert system.

Question F2. Please provide any suggestions for preparing the system
documentation that will enhance clarity and understanding for users.  

Acyclic Compounds

Acyclic compounds comprise ~58% of the food use inert and antimicrobial
inventories.  As discussed in Question 1c, acyclic compounds were found
to not bind to the ER.  Generally, the absence of hydrogen bonding
groups, or inappropriate geometry can explain the failure of these
chemicals to bind to ER (e.g., see Katzenellenbogen et al., 2003). 
Prior to the EPA research described in the SAP review, a diverse set of
acyclic structures had not been evaluated for ER binding affinity.

Question 2-1. Please comment on the extent to which the finding with
acyclic compounds in the FI and AM inventories may be broadly applicable
to other acyclic compounds. Suggestions on an approach to empirically
and efficiently assess a hypothesis that acyclic compounds will not bind
to the ER in other chemical inventories would be welcomed.

Question 2-2.  Please comment on the extent to which the finding with
acyclic compounds in the FI and AM inventories can be applied to other
nuclear steroid receptors in general. Suggestions on an approach to
empirically and efficiently assess a hypothesis that acyclic compounds
will not bind to the androgen receptor would be welcomed.

Prioritization for EDSP Tier 1 Screening 

OECD member countries have long recognized the potential of (Q)SAR for
initial assessments for thousands of untested chemicals and to
establish priorities for follow up actions.  The OECD "Integrated
Approaches to Testing and Assessment" framework has encourage the use of
existing knowledge including (Q)SAR to effectively assess and manage
large chemical inventories
(http://www.oecd.org/dataoecd/45/52/40705314.pdf).  In its final
report, the EDSTAC (US EPA 1998a) recommended a tiered approach for
detecting chemicals with endocrine disrupting potential using a
resource-efficient manner that is similar to  OECD's Endocrine
Disruptor Testing and Assessment Framework
(http://www.oecd.org/document/58/0,3343,en_2649_34377_2348794_1_1_1_1,00
.html).  Like the OECD approach, the framework proposed by the EDSTAC
includes use of (Q)SARs and high through put screening assays.

Question 3-1. Based on the characteristics of the (Q)SAR-based expert
system presented in the white paper, please comment on the Agency’s
view that the expert system could be employed to support  "sorting and
prioritizing"  food use inert ingredients and antimicrobial pesticides
for EDSP Tier 1 screening

Cross Species Applicability

 

ERα, binding assays using full-length recombinant human   SEQ CHAPTER
\h \r 1 ERα and transactivation assays in human T47D cells are in
progress with food use inert ingredients and antimicrobial active
ingredients. Chemicals selected for human ER testing are based on
predictions from the current expert system and designed to cover each
chemical group and bracket the Log Kow ranges within the group.  To date
results show good species concordance with  chemical groups that have
members that bind trout ER also having chemicals that bind human   SEQ
CHAPTER \h \r 1 ER, although the trend is toward fewer members of a
chemical group binding to human ER than to rainbow trout ER (e.g., a
more restrictive Log Kow range for binding within a chemical group for
human ER). Therefore rainbow trout ER appears to bind more low affinity
chemicals within a group but bind the same type of chemicals as does
human ER.   

h±}

☊଀၆㜀$␸䠀$摧⑜

䠀$摧綱

☊଀ཆ㜀$␸䠀$摧綱

<

K

O

P

Q

R

Ž

©

ª

ᄀ킄㜂$␸䠀$葠ː摧澮Q

ༀ炄㜈$␸䠀$葞ࡰ摧寣Ä

ༀ炄㜈$␸䠀$葞ࡰ摧澮Q

ༀ㢄ᄄ预㟾$␸䠀$葞и葠ﺘ摧澮Q

ༀ킄㜂$␸䠀$葞ː摧፪ý

p

t

‰

Ž

‘

’

”

š

œ

§

ª

h±}

h±}

ༀ炄㜈$␸䠀$葞ࡰ摧綱

ༀꂄ㜅$␸䠀$葞֠摧綱

ༀࢄ㜇$␸䠀$葞܈摧፪ý

ഀ׆Āࡰༀࢄ㜇$␸䠀$葞܈摧澮Q

h±}

h±}

ᄀ킄㜂$␸䠀$葠ː摧綱

䠀$葞֠摧綱

h±}

h±}

ༀ炄㜈$␸䠀$葞ࡰ摧寣Ä

ༀ炄㜈$␸䠀$葞ࡰ摧澮Q	㜀$␸䠀$摧፻¼

ᄀ킄㜂$␸䠀$葠ː摧綱

ༀ炄㜈$␸䠀$葞ࡰ摧Ừ3

ༀࢄ㜇$␸䠀$葞܈摧፪ý

ༀꂄ㜅$␸䠀$葞֠摧綱

	㜀$␸䠀$摧綱

h±}

h±}

ༀ炄㜈$␸䠀$葞ࡰ摧㱀y

ༀꂄ㜅$␸䠀$葞֠摧寣Ä

ༀꂄ㜅$␸䠀$葞֠摧㱀y

ༀ炄ᄈ预㟾$␸䠀$葞ࡰ葠ﺘ摧㿘"

ᄀ킄㜂$␸䠀$葠ː摧綱

h±}

h±}

ༀꂄጅ㜀$␸䠀$葞֠摧寣Ä

ༀ킄㜂$␸䠀$葞ː摧綱

䠀$葞ː摧綱

☊଀ཆഀ׆ĀȜ㜀$␸䠀$摧綱

䠀$摧綱

䠀$摧綱

ༀꂄ㜅$␸䠀$葞֠摧寣Ä

ༀ킄㜂$␸䠀$葞ː摧䌏Ë

ᔧ屨褤ᘀh±}

ༀꂄጅ㜀$␸䠀$葞֠摧瀳O

ༀꂄ㜅$␸䠀$葞֠摧寣Ä

䠀$葞ː摧綱

ഀfor prioritizing food use inert ingredients and antimicrobial
pesticides for EDSP Tier I screening. 

 PAGE   

 PAGE   6 

