1
Measuring
Program
Progress
Work
Group
Update
January
7­
8,
2003
2
Purpose
°
Summarize
work
group
activities
conducted
since
September
2002
°
Propose
future
activities,
tasks,
deliverables,

and
schedules
°
Get
feedback
from
the
Subcommittee,
and
revise
future
activities
based
on
input
3
MPP
Work
Group:
Major
Tasks
°
Comments
on
EPA's
progress
measure
White
Papers
("
consultative
comments")

 
Population
risk
reduction
indicator
(
done)

 
Ecological
risk
reduction
indicator
 
Land
use
performance
indicator
°
Develop/
propose
additional
progress
measures
for
Superfund
4
MPP
Work
Group
Presentation
and
Discussion
°
Comments
on
the
Population
Risk
Reduction
White
Paper
(
David
Cooper/
EPA)

°
Work
group
activities
since
September
meeting
°
Performance
measures
developed
by
the
work
group
°
Subcommittee
discussion
5
Population
Risk
Reduction
Measure:

Proposed
Methodology
°
Primary
components
of
proposed
methodology
°
Summary
of
comments
from
MPP
work
group
°
Next
steps
for
this
measure
°
Other
new
measures
under
development
6
Population
Risk
Reduction
Measure:

Primary
Components
°
Measure
the
population
benefiting
from
Superfund
actions.

°
Two
components:

 
Identification
of
populations
that
may
be
exposed
at
Superfund
Sites
via
relevant
pathways
based
on
proximity
 
Identification
of
risk
reduction
category
achieved
by
response
actions
°
Risk
Reduction
Categories:

 
No
risk
reduction
(
no
action
yet
taken)

 
Partial
risk
reduction
(
some
action
taken)

 
Current
Risks
controlled
(
human
exposures
controlled)

 
Current
and
7
Population
Risk
Reduction
Measure:

Work
Group
Comments
°
Indicator
must
be
comprehensible
at
site
level
and
national
level
°
Population
aspect
is
misleading
°
Accurate
measure
of
populations
beyond
the
scope
of
what
can
be
accomplished
°
Progress
measure
is
on
target,
and
could
be
enhanced
with
additional
detail
8
Other
New
Measures
Under
Development
°
Ecological
Risk
Indicator:
to
work
group
by
end
of
the
month
°
Land
Reuse:
to
work
group
this
spring
°
Implementation:
first
time
reporting
in
FY2004
9
MPP
Work
Group
Activities­

September
02­
January
03
°
Reviewed/
commented
on
Population
Risk
Reduction
indicator
°
Developed
work
group
work
plan
°
Reduced
and
consolidated
initial
list
of
measures
°
Work
group
met
on
November
21
10
MPP
Work
Group
Work
Plan
°"
Screening
criteria"
for
selecting
progress
measures
for
development:

 
Can
the
concept
be
quantified
(
measured)?

 
Does
it
align
with
a
desired
program
outcome?

 
Are
measurement
data
available?

 
Is
the
effort
to
score
worth
the
potential
benefit?

 
Is
it
duplicative?

°
Develop
a
program
"
report
card"
that
combines
several
measures
11
MPP
Work
Group
Activities­

September
02­
January
03
°
Drafted
introduction
to
performance
measure
paper
°
Drafted
progress
measures:

 
Alternate
framework
for
a
human
health
protection
 
Remedy
failure
effectiveness
measure
 
Institutional
coordination
measure
12
Introduction
to
Performance
Measures
­
Key
Points
°
Describes
various
kinds
of
measures
°
National
vssite­
specific
performance
measures
°
Program
progress
vsprogram
merit
°
MPP
work
group
focus
is
on
program
progress
°
Other
important
performance
areas
to
consider:

 
Budget
transparency
 
General
program
tracking
 
Tracking
performance
of
institutional
controls
13
Introduction
to
Performance
Measures
°
Topics
of
discussion
and
debate
 
Cost­
benefit
measures
 
Measures
related
to
protection
"
mandate"

 
Performance
of
institutional
controls
14
Framework
for
Human
Health
Protection
Measure
°
Framework
principles:

 
A
population
exposed
to
contaminants
is
at
greater
risk
than
a
population
that
is
not
exposed
 
More
contamination,
or
more
toxic
contaminants
at
a
site
represent
an
increased
residual
risk
to
the
population
 
Stronger
exposure
controls
afford
greater
protection
than
weak
controls
15
Framework
for
Human
Health
Protection
Measure
°
Framework
principles:

 
Measurement
framework
should
reflect
progress
in
public
health
protection
through
actions
that
precede
construction
complete
 
Measurement
has
to
work
at
the
national
and
site
specific
levels
16
Framework
for
Human
Health
Protection
Measure
°
Components:

 
Control
of
exposure
 
Residual
contamination
 
Effectiveness
and
reliability
of
controls
17
Framework
for
Human
Health
Protection
Measure
ComponentScoring
Phase
1
Control
of
exposureY/
N
Phase
2
Residual
contamination
5
category
scale
from
2­
10
Effectiveness
and
reliability
of
controls
10
category
scale
from
5
­
50
18
Framework
for
Human
Health
Protection
Measure
°
Two
phases
of
scoring:

 
Phase
1:
Based
solely
on
"
Control
of
Exposure"
(
Y/
N)

 
Phase
2:

°
Applied
only
to
sites
that
score
"
Yes"
in
Phase
1
°
Calculate
a
"
Human
Health
Protection
Indicator"

HHPI
=
(
Residual
Contamination)
*
(
Effectiveness
and
Reliability
of
Controls)
19
Framework
for
Human
Health
Protection
Measure
°
Program
scoring
and
evaluation
 
Phase
1:

°
Number
of
sites
with
exposure
controlled
°
Program
ratio­
Exposure
controlled
:
Not
controlled
 
Phase
2:

°
Initial
site
evaluation
score
normalized
to
"
baseline"
score
of
1.0
°
Site­
specific
progress
in
human
health
protection
reflected
in
changes
in
baseline
°
Program
evaluation/
progress
reflected
in
change
in
the
total
program
score:
(
 
Site­
specific
scores)
20
Framework
for
Human
Health
Protection
Measure
°
Topics
of
discussion
and
debate:

 
Need
for
an
ecological
risk
indicator
 
Potential
for
misinterpretation
that
a
site
is
"
clean"

 
Weighting
toward
use
of
interim
measures
 
Link
measure
to
the
pace
of
cleanup
 
Simplicity
of
Phase
1
score
(
Y/
N)
is
deceptive
 
Difficulty
in
quantifying
mass
of
hazardous
substances
 
What's
the
appropriate/
most
important
indicator
of
"
extent"
21
Framework
for
Human
Health
Protection
Measure
°
Topics
of
discussion
and
debate:

 
Insufficient
weighting
of
highly
contaminated
materials
 
Application
of
the
"
toxicity"
subcomponent
 
Subjectivity
of
the
"
Effectiveness
and
reliability
of
controls
(
E&
RC)"
component
 
Proposed
number
of
E&
RC
categories
 
Potential
to
score
only
selected
sites
to
make
the
program
look
good
 
Application
at
the
site,
OU,
or
pathway
level
22
Remedy
Effectiveness
Measure
°
Diverging
opinions
on
what
constitutes
"
remedy
failure"
and
how
to
measure
it:

 
Remedies
aren't
allowed
to
fail:

°
Problems
with
remedy
effectiveness
trigger
additional
obligations
°
Once
triggered,
"
contingency
obligations"
require
corrective
responses
 
The
program
record
doesn't
conform
to
the
"
no
failures"
model,
and
failures
can
be
measured
23
Remedy
Effectiveness
Measure
°
Options
for
tracking
(
measuring)
remedy
effectiveness/
failure:

 
Significant
changes
in
response
actions
°
Would
scoring
"
changes"
be
a
disincentive
for
improving
remedies?

 
Results
of
Five­
Year
Reviews
(
track/
score
results
of
"
protectiveness
determination")
24
Remedy
Effectiveness
Measure
°
Options
for
tracking
(
measuring)
remedy
effectiveness/
failure:

 
Track/
score
sites
where
ROD
is
reopened
°
Need
to
distinguish
reasons
for
reopening
 
Track/
score
sites
that
are
re­
listed
 
Track/
score
effectiveness/
failure
of
institutional
controls
25
Institutional
Coordination
Measure
°
Three
measures:

 
State
coordination
 
Tribal
consultation
 
Community
measure
°
Each
measure
has
narrative
and
specific
measurable
components
26
Institutional
Coordination
Measure­

States
°
Governor's
concurrence
°
ROD
concurrence
°
Agency­
lead
agreements
°
State
NRD
trustee
communication
process
°
Agreed
approach
to
cleaning
up
sites
outside
of
NPL
°
MOU
for
State's
VCP
°
Routine
State­
EPA
Region
meetings
°
Performance
partnership
agreements
27
Institutional
Coordination
Measure­

Tribal
Consultation
°
Tribal
concurrence
on
listing
(
how
many
sites
have
it/
don't
have
it)

°
Concurrence
on
lead
agency
designation
°
MOU
with
EPA
°
Proximity
of
tribal
lands
to
NPL
sites
°
Consultation
meetings
between
Tribe
and
EPA
°
Existence
of
Tribal
Environmental
Agreements
(
TEA)

°
Inclusion
of
Superfund
in
TEA
°
Technical/
financial
environmental
assistance
to
the
Tribe
(
several
measures)
28
Institutional
Coordination
Measure­

Tribal
Consultation
°
Tribal
NRD
trustee
coordination
°
Formal
agreements
between
Tribe
and
State
°
Tribal
concurrence
with
ROD
29
Institutional
Coordination
Measure­

Community
°
Underlying
premise:

 
Right
to
participate
in
policy
process
is
equal
across
all
groups,
including
individual
community
members
 
Develop
firm
rules
for
communication
and
decision
making
30
Community
Coordination:
Objective
Measures
°
Presence
of
CAG
°
Application
for/
granting
of
TAG
°
Public
notices
for
meetings
°
Public
comments
on
key
documents
°
Written
responses
to
public
comments
°
Formal
outreach
to
seek
community
input
°
Evaluation
of
applicability
of
environmental
justice
guidelines
31
Community
Coordination:

Consultation
and
Outreach
Measures
°
Local
medical
professionals
°
Natural
scientists
with
local
knowledge
°
Social
scientists
°
Indigenous
and
environmental
justice
groups
°
Social
service
professionals
°
Local
media
°
CAG
°
Community
acceptance
of
proposed
remedy
°
Reports/
deliverables
to
the
community
32
Measuring
Program
Progress
Work
Group
Update
Subcommittee
Discussion
