Here
are
three
items
regarding
our
conference
call
on
April
20.

Edit
to
the
preamble
on
p.
68
of
the
redline
version
that
we
emailed
to
you
on
April
9
Edit
to
the
preamble
on
p.
47
Excerpts
from
the
Residual
Risk
Report
to
Congress
regarding
probabilistic
analysis
(
this
was
a
comment
from
preamble
p.
50).

(
See
attached
file:
Preamble_
Mar15
omb
page
42.
pdf)(
See
attached
file:
Preamble_
Mar15
omb
page
68.
pdf)

Below
is
text
in
the
Residual
Risk
Report
to
Congress
that
supports
that
"
in
general"
probabilistic
analysis
will
be
considered.
Text
supporting
a
tiered
approach
where
decisions
are
made
using
analysis
that
propagates
variability
or
uncertainty
through
the
risk
assessment
is
in
bold
(
emphasis
added).
(
I
read
sensitivity
analysis
as
requiring
this
kind
of
analysis.)

*
*
*
March
1999

Page
97
*
*
*

Approaches
to
Addressing
Uncertainty
and
Variability
in
the
Estimation
of
Residual
Risks
Systematic
uncertainty
and
variability
analyses
have
been
used
in
support
of
risk
assessment
in
a
number
of
fields,
most
notably
nuclear
engineering,
for
over
three
decades.
The
use
of
uncertainty
analysis
in
health
risk
assessment
for
exposure
to
chemical
agents
did
not
become
widespread
until
the
1980s
(
Bogen
and
Spear
1987).
Since
then,
a
wide
range
of
techniques
for
quantitative
uncertainty
analysis
have
been
developed
and
applied
to
risk­
related
policy
analysis
(
e.
g.,
Morgan
and
Henrion
1990;
Frey
1992;
Hoffman
and
Hammonds
1994;
McKone
1994;
Hattis
and
Barlow
1996).
In
its
1994
report,
Science
and
Judgment
in
Risk
Assessment,
NRC
recommended
that,
when
possible,
uncertainty
and
variability
should
be
quantified
and
the
distinction
between
them
maintained
throughout
risk
assessment
(
NRC
1994).
As
discussed
below,
a
number
of
techniques
are
available
that
allow
the
separate
analysis
of
the
impacts
of
uncertainty
and
variability
on
the
overall
dispersion
in
risk
estimates.

The
EPA
has
long
recognized
the
need
to
consider
uncertainty
and
variability
in
risk
assessment.
Agency
guidance
on
these
issues
has
gradually
evolved
over
more
than
a
decade,
with
major
documents
including:


Initial
set
of
risk
assessment
guidance
documents
(
e.
g.,
EPA
1986f,
b);


Risk
Assessment
Council
(
RAC)
guidance
("
the
Habicht
Memorandum,"
EPA
1992e);


Guidelines
for
Exposure
Assessment
(
EPA
1992a);


Policy
and
guidance
for
risk
characterization
("
the
Browner
Memorandum,"
EPA
1995a,
f);


Summary
Report
of
the
Workshop
on
Monte
Carlo
Analysis
(
EPA
1996g);
and

Policy
for
Use
of
Probabilistic
Analysis
in
Risk
Assessment
(
EPA
1997k)
and
Guiding
Principles
for
Monte
Carlo
Analysis
(
EPA
1997c).

Among
these
documents,
the
1992
exposure
assessment
guidance,
the
1997
Policy
for
Use
of
Probabilistic
Analysis
in
Risk
Assessment,
and
1997
Guiding
Principles
for
Monte
Carlo
Analysis
provide
the
most
detailed
recommendations
for
uncertainty
and
variability
analysis.
The
former
document
primarily
provides
technical
guidance
on
uncertainty
evaluation
in
the
context
of
exposure
assessment,
while
the
latter
two
provide
refined
technical
guidance,
as
well
as
recommendations
on
presentation
of
uncertainty
information
to
decision­
makers.
The
1997
Policy
also
documents
EPA's
judgment
that
probabilistic
methods
should
be
used
wherever
the
circumstances
justify
these
approaches.
Thus,
the
Agency
is
committed
to
carefully
considering
use
of
quantitative
methods
for
evaluating
uncertainty
and
variability
in
its
residual
risk
assessments.
The
Agency
has
also
recently
released
a
revised
version
of
the
Exposure
Factors
*
*
*
March
1999

Page
98
*
*
*
Handbook
(
EFH)
that
supports
probabilistic
approaches
to
the
treatment
of
a
number
of
commonly
employed
risk
assessment
input
variables
(
EPA
1997g).
In
April
1998,
the
EPA
Risk
Assessment
Forum
convened
a
workshop
on
uncertainty
analysis
in
which
the
problems
associated
with
defining
probability
distributions
for
uncertainty
and
variability
analyses
were
discussed.

As
techniques
for
uncertainty
analysis
have
matured,
the
Agency
has
come
to
endorse
a
tiered
approach
to
such
analyses.
In
residual
risk
and
other
air
toxics
analyses,
EPA
plans
on
addressing
uncertainty
in
a
tiered
approach.
In
this
way,
EPA
can
efficiently
utilize
resources,
mirroring
the
level
of
uncertainty
analysis
to
the
overall
level
of
analysis.
In
the
Policy
for
the
Use
of
Probabilistic
Risk
Analysis
in
Risk
Assessment
(
EPA
1997k),
four
general
steps
(
tiers)
in
the
recommended
approach
to
quantitative
uncertainty
analysis
are
identified:


Single­
value
estimates
of
high­
end
and
mid­
range
risk;


Qualitative
evaluation
of
model
and
scenario
sensitivity;


Quantitative
sensitivity
analysis
of
high­
end
or
mid­
point
estimates;
and

Fully
quantitative
characterization
of
uncertainty
and
uncertainty
importance.
This
approach
starts
with
simple
assessments
of
potential
risks
using
both
representative
and
more
conservative
scenarios,
models,
and
input
values,
using
point
estimates
of
the
major
parameters.
This
approach
may
provide
sufficient
information
for
the
policy
question
being
addressed
in
some
cases.
For
example,
if
risks
for
a
suitably
defined
high­
end
receptor
are
far
below
levels
of
concern,
then
no
additional
uncertainty
analysis
(
or
risk
analysis)
may
be
needed
to
support
a
risk
management
decision.
Such
screening
analyses
will
probably
be
appropriate
as
the
first
step
in
the
analysis
of
residual
risk
uncertainty
for
all
of
the
source
categories.
Where
the
single­
value
high­
end
and
mid­
range
estimates
do
not
provide
sufficient
information
about
residual
risk,
additional
analyses
can
be
conducted
to
determine
the
likely
range
of
uncertainty
in
these
estimates,
and
the
major
factors
that
contribute
to
the
uncertainty
of
the
estimates.
The
sensitivity
of
the
high­
end
and
mid­
point
estimates
to
the
specification
of
scenarios
and
models
can
usually
be
evaluated
by
conducting
a
manageable
number
of
case
studies
using
different
model
specifications
and
observing
the
resulting
changes
in
risks.
If
scenario
or
model
specification
turns
out
to
strongly
affect
risk
estimates,
a
more
refined
analysis
(
see
below)
may
be
necessary.

In
addition
to
the
evaluation
of
scenario
and
model
uncertainty,
it
may
be
desirable
to
evaluate
the
sensitivity
of
the
point
estimates
of
risks
to
variability
and
uncertainty
in
model
input
parameters.
This
may
be
done
through
sensitivity
analysis
or
through
the
use
of
more
detailed
probabilistic
methods.
If
sensitivity
analyses
are
used,
care
must
be
taken
to
insure
that
the
combinations
of
parameter
values
that
have
the
greatest
impact
on
risks
are
identified.
For
example,
the
greatest
contributions
to
uncertainty
may
arise
where
two
or
more
variables
take
values
that
are
only
moderately
different
from
their
mean
values,
rather
than
where
either
one
of
them
takes
an
extreme
value.

For
some
source
categories,
systematic
sensitivity
analyses
would
provide
sufficient
information
regarding
residual
risks,
and
the
uncertainties
associated
with
these
risks.
If
they
do
not,
the
next
step
is
explicit
probability
modeling,
most
likely
Monte
Carlo
or
related
simulation
methods.
Using
such
approaches,
uncertainty
and
variability
distributions
can
be
defined
for
the
major
parameter
values
used
in
the
derivation
of
the
mid­
range
and
high­
end
risk
estimates.
These
distributions
would
then
be
used
to
develop
Monte
Carlo
estimates
of
risk
and
risk
uncertainty.
There
are
many
precedents
for
the
application
of
such
methods
(
Frey
and
Rhodes
1996)
in
the
evaluation
of
potential
risks
from
HAP
sources.

Whether
sensitivity
analysis
or
simulation
modeling
is
used,
it
is
important
to
consider
both
uncertainty
and
variability
at
this
stage
of
the
analysis.
Very
often,
key
parameters
in
the
residual
risk
assessment
will
be
highly
uncertain.
Experience
to
date
indicates
that
the
emissionrelated
parameters
with
a
particularly
high
degree
of
uncertainty
include
measurements
of
emission
rates,
emissions
inventories,
ambient
levels,
and
facility
operating
patterns
that
affect
HAP
releases.
On
the
risk
side,
uncertainties
in
dose­
response
models,
dose­
response
parameters,
populations
exposed,
and
behavior
patterns
associated
with
exposures
seem
to
contribute
significantly
to
the
overall
uncertainties
in
population
risk
estimates.
Where
data
are
lacking
or
limited,
it
may
be
necessary
to
extrapolate
beyond
the
range
of
available
information,
or
use
surrogate
data
where
direct
observations
are
not
available,
in
order
to
develop
estimates
of
parameter
variability
and
uncertainty.
The
Agency
is
currently
exploring
a
number
of
promising
techniques
in
this
area.
Where
relatively
few
data
are
available,
statistical
techniques
such
as
bootstrap
analysis
may
be
used
to
develop
variability
and
uncertainty
distributions.
Where
important
data
are
lacking,
techniques
for
eliciting
expert
opinion
(
Morgan
and
Henrion
1990)
may
be
useful
in
developing
estimates
of
the
uncertainty
and
variability
of
key
parameters.

While
these
techniques
can
be
very
helpful
in
characterizing
uncertainty,
it
is
important
that
all
assumptions
and
methods
be
fully
documented,
and
that
the
available
data
sources
be
fully
exploited
before
extrapolation
or
surrogate
data
are
used.
Decisions
regarding
the
appropriate
methods
to
be
used
in
developing
uncertainty
distributions
must
be
made
on
a
casebycase
basis,
carefully
considering
the
specific
needs
of
the
analysis.
The
final
step
in
the
analysis
is
a
fully
quantitative
analysis
of
uncertainty
and
uncertainty
importance.
This
approach
is
basically
a
more
comprehensive
extension
of
the
previously
described
methods.
In
this
case,
however,
rather
than
starting
from
pre­
defined
central­
tendency
and
high­
end
risk
estimates,
all
scenarios
and
models
(
to
the
extent
possible)
and
all
parameters
are
included
in
the
modeling
process
as
uncertainty
and
variability
representations.
Using
standard
two­
dimensional
Monte
Carlo
simulation
methods,
the
effects
of
variability
and
uncertainty
on
the
overall
dispersion
in
risk
estimates
can
be
separated
and
quantified.
In
addition,
the
relative
importance
of
individual
sources
of
uncertainty
can
be
evaluated
through
partial
correlation
coefficients,
regression
methods,
contributions
to
variance,
or
related
methods.
However,
the
data
requirements
of
such
an
analysis
often
limit
its
ability
to
be
truly
comprehensive.

Within
the
residual
risk
program,
this
option
will
be
appropriate
for
sources
or
source
categories
where
potential
risks
may
indicate
the
need
for
a
risk
management
action.
The
importance
analysis
could
be
used
to
guide
data
gathering
to
parameters
where
uncertainty
is
the
greatest,
or
to
define
conditions
(
e.
g.,
average
emissions
or
operating
conditions)
for
which
risk
estimates
would
not
exceed
levels
of
concern
with
a
high
degree
of
confidence.

*
*
*
March
1999

Page
122
*
*
*
5.3.5
Refined
Analyses
For
source
categories
that
proceed
from
screening
to
refined
risk
assessments,
additional
data
collection
will
be
required,
with
a
greater
emphasis
on
site­
specific
data
for
affected
facilities.
As
mentioned
previously,
some
assessments
may
begin
at
the
refined
level.
In
some
cases,
this
data
collection
effort
may
be
relatively
extensive,
although
it
should
be
able
to
be
focused
based
on
the
results
of
the
screening
assessment,
when
done,
on
the
HAPs,
types
of
effects
(
i.
e.,
endpoints),
sources,
locations,
exposure
pathways,
and
receptors
of
most
concern.
Data
collection
to
support
the
refined
assessment
may
involve
more
detail
about
data
elements
used
in
the
screening
assessment
(
e.
g.,
HAP
emission
rates,
source
characteristics)
as
well
as
information
about
additional
data
elements
(
e.
g.,
exposed
populations
and
subpopulations,
epidemiology
and
disease
registry
information,
actual
ecosystems
and
endangered
and
threatened
species
that
might
be
exposed).
This
data
collection
step
is
also
more
likely
to
include
collection
of
data
from
industry
sources
and
possibly
other
stakeholders,
in
addition
to
more
extensive
data
collection
from
State
and
local
agencies.

The
sources
of
this
additional
information
for
the
refined
assessment
will
vary.
It
is
assumed
that
State,
local,
and
EPA
Regional
offices
should
have
information
that
is
more
sitespecific,
especially
about
which
facilities
are
subject
to
a
particular
MACT
rule,
which
have
applied
for
operating
permits,
and
which
are
in
compliance
at
a
particular
time.
Other
facilityspecific
information
that
is
needed
to
conduct
the
more
detailed
exposure
and
risk
analysis
may
have
to
be
obtained
from
the
information
request
mechanisms
that
were
used
to
gather
data
for
the
MACT
process.
Other
information
needed
may
come
from
existing
data
bases,
such
as
U.
S.
Census
data,
geographic
information
systems
(
GIS),
or
other
types
of
data
bases
that
may
provide
needed
inputs
for
modeling.
EPA
may
also
work
together
with
industry
to
obtain
needed
data.
Considerable
professional
judgment
is
required
to
carry
out
and
interpret
a
more
refined
residual
risk
assessment,
and
the
steps
taken
and
approaches
used
may
vary
from
one
source
to
the
next,
even
within
the
same
source
category.
As
noted
earlier,
refinement
might
be
necessary
for
some
or
all
components
of
the
analysis.
Evaluating
the
sensitivity
of
the
risk
results
to
different
components
of
the
risk
analysis
can
help
identify
which
components
are
most
important
and
allow
us
to
preferentially
refine
the
more
sensitive
components
or
assumptions.

Human
Health
The
refined
analyses
will
be
based
on
the
methods
and
approaches
described
in
Chapter
3
and
will
incorporate
more
site­
specific
data,
fewer
simple
default
assumptions,
and
more
comprehensive
and
complex
models
(
e.
g.,
ISCST3
for
atmospheric
dispersion
and
deposition).
In
general,
these
analyses
will
be
probabilistic
and
will
produce
estimates
of
risk
distributions
(
in
addition
to
point
estimates).
The
theoretical
MEI
risk
estimate
will
not
be
used
in
refined
assessments;
instead,
the
MIR
estimate
for
areas
that
people
are
believed
to
occupy
will
be
used
to
provide
input
for
risk
management
decisions
that
may
call
for
additional
controls
or
regulatory
actions
.
Criteria
for
Evaluating
Refined
Analysis
Results.
The
refined
analysis,
like
the
screening
analysis,
may
be
iterative
with
increasing
complexity
at
each
iteration.
General
assumptions
and
criteria
are
summarized
in
Exhibit
21.
In
refined
risk
assessment,
the
level
of
confidence
is
increased
through
the
use
of
EPA
or
comparable
consensus
toxicity
values
that
reflect
currently
available
information.
This
ensures
that
toxicity
criteria
of
consistently
high
quality
and
derived
by
a
consistent
methodology
are
used
in
the
assessment.
At
this
level
of
analysis,
additional
available
credible
and
relevant
data
for
all
toxicity
values
used
will
be
considered
by
the
Agency.
In
the
exposure
assessment,
more
site­
specific
data
and
more
refined
models
are
used
to
estimate
exposure
concentrations
and
intakes.
In
addition,
the
refined
analysis
considers
the
number
of
people
exposed
at
different
levels.
