1
Response
to
Comments
from
the
American
Chemistry
Council
et
al.
and
Accompanying
Triangle
Economic
Research
(
TER)
Report
Docket
ID:
OW­
2004­
0020­
0005
I.
Responses
to
Comments
from
the
American
Chemistry
Council
et
al.
on
EPA's
Proposed
Information
Collection
Request
Many
of
the
comments
from
the
American
Chemistry
Council
et
al.
are
drawn
from
the
accompanying
TER
report,
which
EPA
comments
on
below.
However,
EPA
would
like
to
emphasize
a
number
of
misleading
statements
in
the
introductory
statement
preceding
the
TER
report.
The
material
up
to
page
4
is
introductory
in
nature,
and
draws
no
conclusions
regarding
the
ICR.
Thus,
no
response
is
required.
However,
EPA
finds
a
variety
of
incorrect
statements
on
pages
4­
7.
Some
of
the
more
notable
of
these
statements
include
the
following:

 
Page
4:
" 
EPA
has
already
provided
a
meaningful
way
to
assess
the
potential
magnitude
of
nonuse
benefits .
This
approach
allows
for
the
value
of
forage
fish
to
be
reflected
in
the
value
of
recreational
and
commercial
fish."

This
statement
reveals
a
misunderstanding
of
the
multiple
primary
ways
in
which
forage
fish
may
influence
the
utility
of
individuals.
Specifically,
it
fails
to
recognize
the
distinction
between
two
legitimate
sources
of
economic
value
related
to
forage
fish:
indirect
use
benefits
and
non­
use
benefits.
A
more
detailed
response
to
this
comment
is
provided
in
EPA's
response
to
Section
1.1
of
the
TER
report.

 
Page
4:
"
EPA
has
ignored
relevant
empirical
literature."

The
TER
report
identifies
no
"
relevant
economic"
literature
that
EPA
has
not
considered
in
the
development
of
the
ICR.
The
noted
literature
regarding
hypothetical
bias
is
well
known
to
EPA,
as
noted
below.
This
comment
is
dealt
with
in
greater
detail
below,
in
the
response
to
Section
X
of
the
TER
report.

 
Page
4:
"
TER
does
not
believe
that
it
is
possible
for
EPA
to
develop
reliable
estimate
of
nonuse
benefits
for
the
Phase
III
rule."

This
statement
is
an
opinion
that
is
largely
unsubstantiated
by
empirical
findings.
TER
does
not
provide
empirical
evidence
for
this
statement,
but
rather
based
it
on
the
"
difficulty"
of
the
proposed
benefit
measurement.
EPA
disagrees
based
on
its
research
of
empirical
literature.
Additional
comments
regarding
this
issue
are
provided
below
in
EPA's
response
to
Section
2
of
the
TER
report.
2
 
Page
5:
"
As
the
TER
Analysis
shows,
the
proposed
ICR
has
a
number
of
difficulties
both
conceptual
and
practical."

The
TER
report
identifies
no
conceptual
problems
with
the
ICR.
Rather,
the
TER
report
focuses
almost
exclusively
on
arguments
regarding
the
difficulty
of
non­
use
benefits
measurement
in
the
proposed
Phase
III
context.
In
fact,
the
TER
report
focuses
primarily
on
issues
not
directly
relevant
to
the
ICR,
which
describes
the
proposed
use
of
focus
groups
for
survey
design.
In
contrast,
the
TER
report
focuses
on
survey
design
and
stated
preference
valuation
in
general.
While
the
TER
report
identifies
many
complexities
and
challenges
associated
with
the
measurement
of
non­
use
benefits
for
the
Phase
III
policy
context,
none
of
these
represent
insurmountable
hurdles,
and
none
represent
conceptual
mistakes
on
the
part
of
EPA.
In
sum,
the
report
identifies
no
theoretical
or
conceptual
faults
with
EPA's
proposed
focus
groups
or
non­
use
benefits
measurement
approach
 
rather
only
opinions
regarding
the
perceived
difficulty
of
the
task.

 
Page
6:
" 
it
would
appear
to
be
a
better
use
of
agency
resources
to
focus
more
on
the
actual
benefits
to
be
realized
from
regulating
Phase
III
facilities."

The
implied
claim
that
non­
use
benefits
are
not
"
actual
benefits"
represents
a
misrepresentation
of
well­
documented
economic
theory.
As
clearly
stated
by
Freeman
(
2003)
and
many
others
(
including
the
attached
TER
report),
non­
use
benefits
are
legitimate
components
of
total
value.
"[
T]
he
hypothesis
of
nonuse
values
has
gained
wide
acceptance
among
economists
working
in
the
field
of
environmental
and
resource
economics ."
Freeman
(
2003,
p.
137).
Ignoring
such
values
"
in
natural
resource
policymaking
could
lead
to
serious
errors
and
resource
misallocations"
(
p.
138).
Furthermore,
the
TER
report
contradicts
itself
elsewhere
in
the
document
by
incorporating
nonuse
values
in
the
total
value
(
e.
g.,
page
one
of
the
TER
report
states
"
we
certainly
agree
 
that
nonuse
values
here
are
likely
to
be
positive.").

 
Page
6:
"
nonuse
benefits
cannot
be
greater
than
use
benefits"

There
is
no
theoretical
or
empirical
basis
for
this
comment.
The
commenter
does
not
provide
any
or
empirical
or
theoretical
evidence
suggesting
that
the
non­
use
benefits
from
reductions
in
I&
E
losses
cannot
exceed
use
benefits.
3
II.
Responses
to
the
TER
Report
General
Response
The
ICR
concerns
focus
groups
that
are
to
be
used
in
the
development
of
a
stated
preference
survey.
Most
of
the
material
in
this
section
does
not
address
focus
groups
explicitly,
but
rather
the
more
general
topic
of
resource
valuation.
Many
of
the
claims
made
in
the
Triangle
Economic
Research
(
TER)
report
represent
empirical
questions
that
are
appropriately
addressed
within
the
survey
design
process.
Page
one
of
the
TER
report
states
"
we
certainly
agree
 
that
nonuse
values
here
are
likely
to
be
positive."
Hence,
the
primary
question
is
not
whether
these
values
exist,
but
rather
whether
they
may
be
measured
appropriately
using
stated
preference
methodology.

EPA
believes
 
following
guidance
in
the
literature
 
that
stated
preference
methods
are
capable
of
measuring
such
values,
if
surveys
and
approaches
are
appropriately
designed.
As
clearly
stated
by
Johnston
et
al.
(
1995),
Mitchell
and
Carson
(
1989),
Bateman
et
al.
(
2002),
Desvousges
et
al.
(
1984),
Desvousges
and
Smith
(
1988)
and
many
others,
the
use
of
focus
groups
to
assist
in
the
design
of
stated
preference
surveys
is
well
established.
Moreover,
focus
groups
represent
one
of
the
primary
means
of
assessing
whether
many
of
the
as­
of­
yet
unsubstantiated
claims
of
the
TER
report
(
e.
g.,
regarding
the
inability
of
survey
instruments
to
measure
non­
use
values
for
fish
affected
by
entrainment
and
impingement)
are
indeed
accurate.
EPA
would
like
to
emphasize
that
the
TER
report
reflects
many
of
the
issues
that
will
be
addressed
in
focus
groups
 
indeed,
such
issues
are
one
of
the
primary
justifications
for
such
methods.
In
sum,
EPA
views
the
TER
report
as
additional
justification
as
to
why
focus
groups
are
clearly
needed
within
survey
design.
EPA
recognizes
the
difficulties
faced
in
the
design
of
appropriate
stated
preference
survey
instruments
 
indeed
this
is
why
the
focus
group
ICR
was
issued
 
but
does
not
consider
these
difficulties
to
be
insurmountable.
EPA
believes
that
this
position
is
in
correspondence
with
the
economic
literature,
as
noted
in
more
detail
below.

Specific
Responses
Section
1:
EPA's
View
of
Non­
Use
Values
as
Presented
in
the
Proposed
Rule
and
ICR
is
Conceptually
Flawed
The
TER
report
claims
that
EPA's
view
of
non­
use
values
is
conceptually
flawed,
but
offers
no
evidence
for
the
claimed
flaws.
Rather,
the
report
speaks
to
the
"
difficulty
of
measuring
nonuse
values
in
the
Phase
III
316b
context."
EPA
agrees
that
the
development
of
appropriate
stated
preference
instruments
requires
substantial
work
to
address
potential
biases,
including
those
that
may
lead
to
an
overstatement
of
willingness
to
pay.
For
this
reason,
EPA
has
proposed
the
use
of
focus
groups
for
survey
design.
4
Section
1.1:
Production
Foregone
Methodology
Provides
a
Reliable
Alternative
to
EPA's
Proposed
CV
Survey
The
commenter
argues
that
the
values
of
protecting
fish
from
impingement
and
entrainment
are
indirect
use
values
and
not
non­
use
values
and
these
values
are
already
captured
in
EPA's
analysis.
This
statement
reveals
a
misunderstanding
of
the
multiple
primary
ways
in
which
forage
fish
affect
the
utility
of
individuals.
Specifically,
it
fails
to
recognize
the
distinction
between
two
legitimate
sources
of
economic
value
related
to
forage
fish:
indirect
use
benefits
and
non­
use
benefits.
First,
forage
fish
may
influence
utility
indirectly,
through
their
influence
on
the
productivity
of
recreational
and
commercial
fish.
Such
benefits
may
be
largely
captured
by
a
trophic
transfer
approach.
However,
forage
fish
may
also
enter
individuals'
utility
functions
directly,
providing
non­
use
benefits
that
have
little
or
no
relationship
to
recreational
and
commercial
fish.
These
benefits
can
not
be
measured
using
a
trophic
transfer
approach.
EPA
refers
to
Freeman
(
2003),
who
provides
a
clear
distinction
between
indirect
use
benefits
and
true
nonuse
benefits.
EPA
also
emphasizes
the
well­
known
tenet
of
economic
theory
which
states
that
benefits
or
costs
from
multiple
direct
(
primary)
effects
on
markets
should
be
counted
within
a
benefit­
cost
analysis;
this
is
not
double
counting,
and
is
indeed
appropriate
(
Just
et
al.
1982).

The
TER
report
also
incorrectly
states
that
EPA
has
rejected
the
production
foregone
methodology
used
in
the
Phase
II
analysis.
Contrary
to
this
statement,
EPA
continues
to
use
the
production
foregone
methodology
to
estimate
the
contribution
of
forage
species
on
productivity
of
commercial
and
recreational
stocks.
For
detail
see
Chapter
A­
X
of
"
The
Regional
Benefits
Assessment
for
the
Proposed
Section
316(
b)
Rule
for
Phase
III
Facilities"
(
U.
S.
EPA
2004).

EPA
also
disagrees
with
the
TER
statement
that
the
two
approaches
(
i.
e.,
a
stated
preference
survey
to
estimate
total
value
of
fish
species
affected
by
impingement
and
entrainment
and
the
production
foregone
model
that
estimates
contribution
of
forage
fish
to
commercial
and
recreational
harvest)
are
potentially
inconsistent.
As
noted
above
there
are
two
legitimate
sources
of
economic
value
related
to
forage
fish:
indirect
use
benefits
and
non­
use
benefits.
EPA
emphasizes
that
the
production
foregone
model
captures
indirect
use
value
only
and
does
not
capture
non­
use
values.

The
TER
report
also
makes
another
misleading
statement
regarding
non­
use
values
for
forage
species.
"
If
forage
species
are
intrinsically
valuable
in
the
a
non­
use
context,
it
would
seem
that
their
value
is
independent
of
the
manner
in
which
they
die"
and
thus
"
welfare
gains
associated
with
commercial
and
recreational
fishing
should
also
net
out
the
non­
use
values
associated
with
the
forage
fish
biomass
these
predators
consume
as
they
grow
to
recreational
and
commercial
sizes."

First,
the
TER
report
fails
to
recognize
that
forage
fish
provide
a
number
of
ecological
services
that
are
not
directly
related
to
production
of
commercial
and
recreational
species.
For
example,
some
small
forage
fish
can
consume
macroalgae
on
a
massive
scale
and
contribute
to
reducing
diseases
of
submerged
aquatic
vegetation
and
corals
and
therefore
contributing
to
the
overall
health
of
an
ecosystem.
The
loss
of
benthic
fauna
(
including
forage
fish)
leads
to
"
shifts
from
ecosystems
once
dominated
by
benthic
production
to
those
dominated
by
planktonic
primary
production"
(
Bjernstad
and
Grenfell,
2001).
This
may
lead
to
ecosystem
collapse.
Therefore,
assigning
only
one
type
of
ecological
value
(
i.
e.,
food
for
predators)
to
forage
fish
is
misleading
and
inconsistent
with
ecological
literature.
5
Second,
the
TER
statement
that
"
welfare
gains
associated
with
commercial
recreational
fishing
should
also
net
out
the
non­
use
values
associated
with
the
forage
fish
biomass
these
predators
consume
as
they
grow
to
recreational
and
commercial
sizes"
is
unsupported
by
either
economic
theory
or
empirical
literature,
and
confuses
the
baseline
from
which
welfare
measures
are
to
be
assessed.
There
are
two
distinct
and
independent
issues
here,
a
fact
that
the
TER
report
obscures.
The
first
issue
concerns
the
appropriate
measurement
of
nonuse
values
associated
with
forage
fish.
The
second
concerns
the
appropriate
measurement
of
social
welfare
associated
with
landed
recreational
and
commercial
fish.

The
focus
groups
(
and
associated
stated
preference
survey)
address
the
first
issue
only­
the
appropriate
measurement
of
nonuse
values
associated
with
forage
fish.
EPA
is
well
aware­
as
reflected
in
prior
316b
materials
and
the
economic
literature­
that
nonuse
values
should
be
assessed
only
for
those
portion
of
forage
fish
affected
the
proposed
policies­
whether
losses
or
ultimate
population
effects.
The
numbers
used
by
EPA
as
a
basis
for
benefit
calculations
reflect
losses
of
forage
fish
due
to
water
intake
(
e.
g.,
in
Phase
III
facilities),
and
the
best
available
estimates
of
resultant
impacts
on
fish
populations.
That
is,
EPA
seeks
to
address
nonuse
values
associated
with
the
marginal
increment
to
fish
losses
and
populations
directly
associated
with
the
proposed
Phase
III
rule.
The
proposed
focus
groups
will
be
used,
in
part,
to
assess
whether
nonuse
values
are
associated
with
reduction
in
annual
fish
losses,
changes
in
long­
term
populations,
or
both.
In
all
cases,
the
baseline
from
which
changes
are
estimated
already
reflects
gains
and
losses
due
to
other
exogenous
factors­
both
natural
and
anthropogenic­
including
the
effect
of
natural
predation.
The
TER
report's
implication
that
this
represents
some
type
of
"
inconsistency"
is
incorrect.

As
noted
above,
the
second
issue
concerns
the
appropriate
measurement
of
social
welfare
associated
with
landed
recreational
and
commercial
fish.
EPA
emphasizes
that
this
is
NOT
a
concern
of
the
ICR
in
question,
or
of
the
proposed
focus
groups.
However,
as
it
is
mentioned
in
the
TER
report,
EPA
will
offer
a
brief
response.
EPA
once
again
emphasizes
the
importance
of
recognizing
the
appropriate
baseline.
Here,
reducing
I&
E
losses
in
Phase
III
facilities
will
result
in
a
net
reduction
in
fish
mortality
and
a
potential
increase
in
populations
of
both
forage
and
recreational
fish,
compared
to
the
baseline
in
which
I&
E
losses
remain
unabated.
Any
estimated
increase
in
population
and
harvest
is
measured
as
an
equilibrium
condition,
after
accounting
for
natural
predation
and
other
ecological
events.
Hence,
whether
predators
have
consumed
a
certain
quantity
of
forage
fish
during
their
life
span
is
immaterial,
as
the
relevant
comparisons
for
welfare
estimation
here
are
the
net
equilibrium
(
marginal)
changes
in
annual
fish
losses,
fish
populations,
and
harvest
resulting
from
the
Phase
III
rule.
(
Whether
the
"
manner
of
a
fish's
demise"
is
relevant
to
individuals
is
an
issue
that
will
be
address
in
the
proposed
focus
groups.)
The
discussion
of
predator­
prey
relationships
presented
by
TER
has
no
bearing
on
the
fundamental
comparison
of
pre­
and
post­
rule
ecological
services
to
be
valued
by
EPA.
(
As
an
aside,
EPA
emphasizes
that
if
one
follows
the
logic
of
the
TER
report
regarding
the
valuation
of
fish
harvest,
then
all
prior
valuations
of
predator
species
would
be
required
to
"
net
out"
nonuse
values
associated
with
prey
species
consumed
during
their
lifetimes.
EPA
is
aware
of
no
precedent
in
the
literature
for
such
analysis.)

EPA
recognizes
that
the
valuation
of
ecological
services
presents
a
number
of
challenges,
and
proposes
to
conduct
such
valuation
using
the
standards
set
by
the
most
recent
and
appropriate
economic
literature.
The
TER
comment­
while
raising
complex
issues­
neither
reflects
current
practice
in
the
valuation
literature,
nor
identifies
any
inconsistencies
in
EPA's
proposed
focus
groups
or
methods
of
nonuse
value
estimation.
6
Section
1.2:
EPA
fails
to
Identify
Distinct
Non­
use
Benefit
Categories,
Which
Would
Lead
to
Double
Counting
of
Benefits.
The
commenter
argues
that
because
EPA
failed
to
identify
distinct
non­
use
benefits
categories
applicable
to
the
Phase
III
regulation,
the
stated
preference
survey
would
substantially
overstate
non­
use
impacts
of
the
rule
by
double
counting
intermediate
and
final
impacts.
According
to
the
TER
report
there
is
only
one
benefit
category
affected
by
the
316(
b)
regulation
 
fish
populations.
EPA
points
out
that
the
destruction
of
individual
fish
and
larvae
may
lead
to
changes
in
fish
population
and
it
may
also
affect
higher­
level
ecosystem
functions
or
structural
attributes,
such
as
the
rates
of
material
and
energy
cycling
through
the
aquatic
ecosystem
or
the
harvestable
biomass
maintained
at
the
top
of
the
aquatic
food
web.
For
additional
discussion
of
ecosystem
services
provided
by
fish
see
Section
1.1
above.
EPA
also
points
out
that
the
stated
preference
study
proposed
by
EPA
is
intended
to
measure
total
values
of
reducing
fish
losses
from
impingement
and
entrainment
and
not
the
distinct
value
categories.
Therefore,
the
TER
statement
regarding
double
counting
of
non­
use
benefits
categories
in
EPA's
analysis
is
unsubstantiated.

Section
1.3:
EPA
Fails
to
Consider
the
Potential
of
Use
Values
in
Understanding
Non­
Use
Values
First,
the
commenter
refers
to
the
Fisher
and
Raucher
approach
to
estimate
non­
use
values
of
impingement
and
entrainment
reductions
used
in
the
proposed
Phase
II
rule
analysis
and
the
public
comments
on
this
approach.
EPA
notes
that
the
Fisher
and
Raucher
approach
was
not
used
in
the
benefits
analysis
of
the
final
Phase
II
regulation.
For
more
detail
on
EPA's
responses
to
public
comments
on
the
Phase
II
proposed
rule
and
Notice
of
Data
Availability
see
"
The
316(
b)
Phase
II
Rule
Comment
Response
Document"(
U.
S.
EPA
2004;
http://
www.
epa.
gov/
ost/
316b/
commentph2.
htm).

Second,
the
commenter
asserts
that
because
fish
are
renewable
resources
that
are
harvested
for
personal
consumption,
the
realization
of
use
benefits
impacts
non­
use
values
negatively.
The
commenter
further
argues
that
"
if
it
is
worth
more
to
recreational
anglers
to
leave
the
fish
in
the
water
and
never
use
them
(
i.
e.,
have
true
non­
use
value
based
on
existence
motive),
then
those
anglers
would
never
decide
to
consume
the
fish."
EPA
points
out
that
recreational
anglers
who
catch
fish
for
personal
consumption
represent
a
relatively
small
percentage
of
the
U.
S.
population.
The
majority
of
the
U.
S.
population
does
not
have
direct
use
values
for
fish
species
affected
by
impingement
and
entrainment.
Furthermore,
recent
studies
of
recreational
fishing
behavior
suggest
that
non­
use
values
of
fish
species
caught
by
recreational
anglers
may
in
fact
exceed
consumptive
use
values.
For
example,
Hicks
et
al.
(
2002)
found
that
only
24
percent
of
recreational
saltwater
anglers
in
the
Northeastern
United
States
keep
most
of
the
fish
they
catch,
19
percent
release
most
of
all
species
they
catch,
and
57
percent
release
some
fish
(
Hicks
et
al.
2002).
One
of
the
leading
causes
for
releasing
fish
was
that
"
the
angler
caught
the
amount
planned
to
eat."
This
findings
clearly
illustrate
that
non­
use
values
for
fishery
resources
are
likely
to
be
non­
trivial
and
that
while
the
direct
use
value
of
catching
the
first
fishes
exceeds
the
non­
use
value,
this
relationship
does
not
hold
as
the
number
of
fish
caught
increases.

EPA
also
disagrees
that
the
fishery
management
practice
to
limit
harvest
and
consumption
of
fish
would
imply
that
"
the
foregone
use
value
of
increased
fish
populations
is
a
reasonable
upper
bound
any
potential
non­
use
values."
The
value
of
increased
fish
populations
includes
the
use
value
of
an
increase
in
the
future
harvest
of
commercial
and
recreational
fish
and
the
value
of
ecological
services
provided
by
these
species.
7
Therefore,
the
total
value
of
changes
in
fish
population
associated
with
fishery
management
practices
is
likely
to
be
higher
than
the
use
value
of
foregone
fishery
yield
associated
with
the
catch
limit.

The
TER
report
also
claims
that
"
nonuse
benefits
cannot
be
greater
than
use
benefits":
There
is
no
theoretical
or
empirical
basis
for
this
comment.
There
is
no
literature
or
empirical
evidence
or
justification
in
accepted
theory
suggesting
that
the
non­
use
benefits
from
reductions
in
entrainment
and
impingement
losses
cannot
exceed
use
benefits.

Section
1.4:
EPA
Fails
to
Identify
Rationale
for
Significant
Benefits
The
TER
report
claims
that
EPA
fails
to
identify
a
rationale
for
significant
non­
use
benefits.
However,
the
evidence
provided
is
largely
based
on
speculation.
EPA
emphasizes
that
the
magnitude
of
non­
use
benefits
associated
with
the
Phase
III
rule
is
an
empirical
question
 
and
that
the
potential
significance
of
such
benefits
can
only
be
assessed
using
appropriate
empirical
methods.
For
example,
the
arguments
regarding
"
limited
existence
motive"
are
based
on
assumptions
regarding
behavior
and
underlying
utility
that
have
not
been
substantiated
by
any
literature
cited
in
the
TER
report.
EPA
is
aware
of
no
research
 
empirical
or
otherwise
 
that
demonstrates
that
individuals
have
limited
existence
motives
for
the
types
of
fish
that
would
be
protected
by
the
Phase
III
rule.
Indeed,
the
TER
report
explicitly
states
that
the
non­
use
benefits
of
fish
have
not
been
studied
 
hence
there
is
no
evidence
to
suggest
that
these
benefits
are
trivial,
as
implied
by
the
TER
report.
The
second
argument
presented
by
TER
is
that
the
resources
to
be
protected
by
the
Phase
III
rule
have
many
substitutes,
and
hence
likely
have
relatively
low
non­
use
values.
EPA
recognizes
the
position
that
larger
non­
use
values
would
be
expected
for
unique
resources
with
few
substitutes.
However,
EPA
again
emphasizes
that
the
magnitude
of
non­
use
benefits
in
the
Phase
III
context
is
an
empirical
issue
that
can
only
be
addressed
with
appropriate
research
methods.
Even
if
nonuse
values
for
fish
are
small
on
a
per­
capita
basis,
they
may
be
large
in
the
aggregate.
The
TER
claims
for
trivial
non­
use
benefits
in
the
Phase
III
policy
context
are
based
on
ad
hoc
assessments
unsubstantiated
by
the
empirical
literature.

Section
2:
Unreliable
Empirical
Estimates
Most
of
the
material
in
this
section
again
does
not
address
focus
groups
explicitly,
but
rather
the
more
general
topic
of
resource
valuation.
Nonetheless,
it
is
the
opinion
of
EPA
that
these
comments
reflect
the
perspective
of
a
relatively
narrow
portion
of
the
literature,
and
one
that
is
out
of
step
with
accepted
practices
in
benefit­
cost
analysis.
While
EPA
recognizes
that
some
researchers
consider
stated
preference
methods
to
be
controversial,
it
is
also
emphasized
that
standard
benefit­
cost
practice
considers
these
methods
to
provide
useful
information
concerning
resource
values
(
e.
g.,
Freeman
2003;
Carson
et
al.
1999;
Mitchell
and
Carson
1989;
Louviere
et
al.
2000;
Bateman
et
al.
2002).
For
example,
that
which
the
TER
report
describes
as
"
well­
documented
empirical
flaws"
are
more
appropriately
characterized
by
the
preponderance
of
the
literature
 
including
the
NOAA
Blue
Ribbon
Panel
on
Contingent
Valuation
(
Arrow
et
al.
1993)
 
as
potential
biases
that
may
be
avoided
or
largely
ameliorated
with
appropriate
best­
practices
in
stated
preference
survey
design
and
implementation.

EPA
is
aware
of
the
potential
biases
associated
with
poorly
conceptualized
and
implemented
stated
preference
studies.
However,
EPA's
proposed
use
of
these
methods
for
non­
use
WTP
estimation
reflects
the
standards
of
the
broader
literature,
which
considers
the
use
of
such
methods
appropriate.
As
stated
by
the
NOAA
Blue
Ribbon
Panel
on
Contingent
Valuation
(
Arrow
et
al.
1993,
p.
4610),
" 
some
antagonists
of
the
CV
approach
go
so
far
as
to
suggest
that
there
can
be
no
useful
information
content
to
CV
results.
8
The
Panel
is
unpersuaded
by
these
extreme
arguments."
The
panel
later
follows
with
the
conclusion
"[
t]
he
Panel
concludes
that
under
[
conditions
described
in
the
report],
CV
studies
convey
useful
information.
We
think
it
is
fair
to
describe
such
information
as
reliable
by
the
standards
that
seem
to
be
implicit
in
similar
contexts,
like
market
analysis
for
new
and
innovative
products "
In
the
opinion
of
EPA,
the
responses
in
this
report
largely
reflect
the
type
of
opinions
that
the
NOAA
panel
found
unconvincing
in
its
original
assessment.
Indeed,
many
of
the
citations
provided
in
the
TER
report
(
e.
g.,
Diamond
et
al.
1993;
Diamond
and
Hausman
1993)
reflect
research
that
was
reviewed
(
prior
to
publication
in
book
form)
by
the
NOAA
Blue
Ribbon
panel
during
its
deliberations.

In
the
opinion
of
EPA,
the
TER
comments
reflect
a
relatively
extreme
criticism
of
stated
preference
methods.
This
critical
assessment
is
at
odds
with
current
practice
as
reflected
by
the
NOAA
panel
(
p.
4610),
which
states
that
"
CV
studies
can
produce
results
reliable
enough
to
be
the
starting
point
of
a
judicial
process
of
damage
assessment,
including
lost
passive­
use
values."
Moreover,
as
stated
by
Freemen
(
2003,
p.
154),
"
stated
preference
valuation
methods 
are
likely
to
offer
the
only
feasible
approaches
to
estimating
nonuse
values."

More
broadly,
it
is
the
position
of
EPA
that
an
ICR
for
focus
groups
alone
need
not
incorporate
a
full
discussion
of
all
potential
biases
associated
with
stated
preference
methods
in
general.
The
potential
for
these
biases
in
poorly
designed
surveys
are
well
known
(
Mitchell
and
Carson
1989;
Bateman
et
al.
2002)
 
and
indeed
one
of
the
primary
purposes
of
focus
groups
is
the
amelioration
of
potential
biases
in
resulting
survey
instruments
(
Johnston
et
al.
1995).
For
this
reason,
EPA
considers
focus
groups
to
be
an
important
tool
in
designing
a
stated
preference
survey.

Section
2.1:
The
contingent
valuation
(
CV)
approach
that
EPA
intends
to
implement
to
measure
nonuse
values
for
the
Phase
III
rule
contains
hypothetical
bias,
which
would
lead
to
substantial
overestimates
of
potential
benefits.
While
recognizing
the
potential
for
hypothetical
bias
in
stated
preference
responses,
EPA
believes
that
this
comment
presents
a
one­
sided
view
of
the
literature
regarding
the
potential
for
such
biases.
EPA
also
emphasizes
that
many
of
the
citations
used
within
this
section
were
published
more
than
ten
years
ago,
based
on
research
conducted
prior
to
the
NOAA
Blue
Ribbon
Report
(
Arrow
et
al.
1993).
For
example,
the
NOAA
Panel
carefully
reviewed
the
research
underlying
Diamond
et
al.
(
1993)
prior
to
drawing
its
conclusion
that
"[
w]
e
think
it
is
fair
to
describe
[
information
from
well­
designed
CV
surveys]
as
reliable
by
the
standards
that
seem
to
be
implicit
in
similar
contexts,
like
market
analysis
for
new
and
innovative
products "
Hence,
the
NOAA
Panel
clearly
did
not
consider
such
evidence
overwhelming.

EPA
notes
that
this
comment
fails
to
consider
recent
work
when
drawing
conclusions
regarding
the
inevitability
of
hypothetical
bias.
A
more
thorough
review
of
the
recent
literature
reveales
many
advances
have
been
made
since
the
older
studies
cited
by
the
TER
report
were
published.
These
recent
works
provide
various
means
for
the
amelioration
of
hypothetical
bias.
For
example,
many
researchers
have
had
success
using
cheap
talk
methodology
and
other
methods
shown
to
have
the
promise
to
largely
eliminate
or
at
a
minimum
reduce
hypothetical
biases
(
e.
g.,
Champ
et
al.
1997;
2004;
Cummings
and
Taylor
1999).
While
the
evidence
regarding
the
ability
of
researchers
to
ameliorate
hypothetical
biases
is
best
characterized
as
mixed,
there
is
no
evidence
that
such
biases
are
unavoidable.
9
While
critics
of
stated
preference
methods
often
claim
insurmountable
divergences
between
stated
and
real
market
behavior,
evidence
from
comparisons
of
binding
referenda
to
stated
preference
responses
contradicts
this
opinion.
Existing
studies
find
close
correspondence
between
choices
in
stated
preference
surveys
and
voting
behavior
in
binding,
consequential
referenda
(
Bateman
et
al.
2002,
p.
318;
Schläpfer
et
al.
2004).
Hence,
in
these
true
tests
of
criterion
validity,
responses
in
hypothetical
and
real
contexts
comport
quite
closely,
suggesting
that
stated
preference
surveys
can
provide
useful
indications
of
that
which
individuals
"
are
willing
to
forgo
to
enjoy
a
particular
benefit"
(
Circular
A­
4,
as
quoted
in
the
received
comment).

EPA
notes
that
many
of
research
efforts
that
provide
evidence
for
the
presence
of
hypothetical
biases
are
carried
out
in
simulated
markets,
and
hence
do
not
reflect
true
assessments
of
criterion
validity
(
Bateman
et
al.
2002).
While
simulated
markets
 
experimental
or
otherwise
 
might
generate
unbiased
WTP
estimates,
they
are
often
conducted
in
institutional
settings
unfamiliar
to
respondents.
Such
settings
may
generate
novel
or
unusual
behavior,
confusion,
distrust
or
other
behaviors
that
may
lead
to
misstatements
of
true,
underlying
WTP.
For
such
reasons
Bateman
et
al.
(
2002)
argue
that
comparisons
involving
WTP
estimates
obtained
through
simulated
or
experimental
markets
do
not
qualify
as
true
tests
of
criterion
validity
(
i.
e.,
they
do
not
provide
unambiguous
measures
of
true
willingness
to
pay
in
actual
markets).
The
most
common
alternative
 
the
use
of
individual
or
group
donation
mechanisms
(
e.
g.,
Foster
et
al.
1997;
Seip
and
Strand
1992)
 
suffers
from
confounding
effects
of
free­
riding
and
the
associated
lack
of
incentive
compatibility
(
Bateman
et
al.
2002;
Shläpfer
et
al.
2004).
As
a
result,
as
stated
by
Murphy
and
Stevens
(
2004,
p.
185),
one
cannot
unambiguously
claim
that
values
estimated
using
laboratory
experiments
are
any
more
representative
of
true
willingness
to
pay
than
values
estimated
using
hypothetical
markets.

The
TER
report
also
states
that
"[
b]
ecause
the
respondent
does
not
actually
pay
the
stated
amount
in
a
CV
survey,
there
is
no
penalty
for
giving
an
answer
that
does
not
reflect
true
preferences."
EPA
emphasizes
that
this
statement
fails
to
recognize
the
role
of
incentive
compatibility
in
minimizing
strategic
and
hypothetical
biases
(
Sudgden
1999).

EPA
certainly
recognizes
the
potential
for
hypothetical
biases.
However,
EPA
intends
to
incorporate,
to
the
extent
possible,
the
important
steps
that
have
been
taken
in
recent
years
to
develop
means
to
ameliorate
such
biases.
EPA
is
mindful
of
the
important
role
of
appropriate
survey
design,
including
focus
groups,
in
the
development
of
surveys
that
minimize
biases
such
as
those
discussed
in
this
section
(
cf.
Mitchell
and
Carson
1989;
Desvousges
and
Smith
1988;
Desvousges
et
al.
1984;
Johnston
et
al.
1995).
To
reduce
the
potential
for
hypothetical
bias,
EPA
intends
to
use
focus
groups
to
frame
and
define
[
CVM]
survey
questions
and
to
pretest
questionnaires.

Section
2.2:
Marginal
Changes
in
Fish
Population
Present
Unique
Empirical
Valuation
Changes.
EPA
agrees
that
the
representation
of
marginal
changes
in
fish
stocks
represents
an
empirical
challenge.
However,
the
TER
report
provides
no
evidence
that
this
challenge
 
as
implied
by
the
section
title
 
is
"
unique"
or
otherwise
more
difficult
than
challenges
facing
numerous
other
stated
preference
studies
addressing
marginal
changes.
The
TER
report
again
cites
an
older
study
arguing
that
stated
preference
respondents
are
unable
to
comprehend
the
scope
of
resource
changes.
However,
since
that
time
there
have
been
numerous
studies
showing
that
stated
preference
WTP
estimates
are
indeed
sensitive
to
scope
(
e.
g.,
Smith
and
Osborne
1996).
Such
findings
extend
to
stated
preference
estimates
addressing
aquatic
resources,
habitat
and
wildlife
(
Johnston
et
al.
2002;
2003).
10
Section
2.3:
EPA
has
failed
to
identify
the
appropriate
regulatory
baseline
for
non­
use
values.
EPA
emphasizes
that
this
comment
has
little
to
do
with
the
submitted
focus
group
ICR.
EPA
strongly
agrees
that
any
valuation
associated
with
the
Phase
III
rule
must
appropriately
account
for
baselines.
We
also
agree
that
this
is
an
empirical
issue,
which
can
only
be
addressed
by
appropriate
stated
preference
survey
design,
including
focus
groups.
EPA
emphasizes
that
there
is
nothing
in
this
section
that
suggests
in
any
way
that
appropriately
representing
baselines
for
the
Phase
III
is
an
insurmountable
challenge.
EPA
also
recognizes
the
challenges
associated
with
the
communication
of
uncertainty
within
CVM
surveys
 
this
is
again
one
of
the
topics
that
will
be
addressed
in
the
proposed
focus
groups.

Section
3.
Proposed
ICR
Contains
Numerous
Flaws
that
Will
Result
in
Unreliable
Benefit
Estimates
The
TER
report
claims
that
EPA
does
not
appreciate
the
magnitude
of
the
difficulties
that
"
the
Agencies
is
going
to
encounter
in
developing
its
CV
questionnaire."
EPA
agrees
that
the
development
of
appropriate
stated
preference
instruments
requires
substantial
work
to
address
potential
biases
and
problems,
including
those
discussed
in
the
TER
report.
For
this
reason,
EPA
has
proposed
the
use
of
focus
groups
for
survey
design
because
focus
groups
are
considered
by
many
stated
preference
practitioners
to
be
an
critical
component
of
contingent
valuation
survey
design.

3.1
Proposed
ICR
fails
to
address
crucial
linkages
between
Phase
III
CMIS
purported
ecological
effects
and
hypothesized
benefits.

This
section
of
the
TER
comments
raises
a
number
of
concerns
regarding
the
type
of
fish
valued
in
the
proposed
stated
preference
study,
the
ability
to
distinguish
between
Phase
II
and
Phase
III
facility
impacts
on
fish,
people's
limited
knowledge
of
fish
populations,
and
the
effect
of
distance
on
non­
use
values.
EPA
recognizes
difficulties
faced
in
the
design
of
appropriate
stated
preference
survey
instruments
and
thus
many
of
the
issues
raised
in
this
section
will
be
addressed
in
focus
groups.
In
addition
EPA,
points
out
that
the
ability
to
distinguish
between
Phase
II
and
Phase
III
facility
impacts
is
irrelevant
to
the
proposed
ICR.
Phase
II
and
Phase
III
facilities
affect
similar
fish
species.
The
main
difference
between
Phase
II
and
phase
III
facility
impacts
is
the
magnitude
of
fish
losses
from
impingement
and
entrainment.
Hence,
the
primary
question
here
is
not
whether
EPA's
analysis
can
distinguish
between
Phase
II
and
Phase
III
facility
impacts,
but
rather
whether
the
benefit
valuation
function
developed
based
on
the
stated
preference
survey
would
allow
to
estimate
willingness­
to­
pay
values
based
on
different
policy
scenarios
(
i.
e.,
different
changes
in
fish
losses
from
impingement
and
entrainment).
EPA
emphasizes
that
this
issue
can
be
appropriately
address
in
the
survey
design
process.

3.2
EPA
provides
no
factual,
conceptual
or
empirical
arguments
to
support
the
contention
that
significant
non­
use
values
are
not
being
included
in
the
proposed
Phase
III
Rule.
A
detailed
response
to
this
comment
is
provided
in
Section
1.4
above.
11
3.3
The
focus
group
script
provided
in
the
ICR
demonstrates
numerous
problems
that
EPA
is
going
to
face
in
trying
to
elicit
non­
use
values
for
CWIS
regulations.
Specific
comments
from
this
section
are
addressed
below.

3.3.1:
The
materials
do
not
indicate
where
the
changes
in
fish
population
are
going
to
occur
and
why
they
would
be
relevant
to
a
particular
respondent.
A
detailed
response
to
this
comment
is
provided
in
Section
3.1
above.

Section
3.3.2:
The
ICR
fails
to
recognize
the
role
of
embedding
in
people's
preferences
for
environmental
commodities.
EPA
once
again
emphasizes
that
these
comments
extend
far
beyond
the
ICR
at
hand.
The
TER
comment
also
reflects
a
clear
conceptual
misunderstanding
of
the
role
of
focus
groups
in
stated
preference
survey
design.
For
example,
there
is
no
evidence
from
any
of
the
focus
group
literature
that
one
should
first
attempt
to
have
respondents
"
rank"
the
importance
of
fish
prior
to
engaging
in
general
discussions
of
their
perceptions
of
fish
and
fish
habitat.
To
the
contrary,
Johnston
et
al.
(
1995)
explicitly
emphasize
the
importance
of
commencing
focus
groups
with
general
"
grand
tour"
discussions
of
broad
issues,
in
the
participants'
language,
prior
to
introducing
sometimes
foreign
concepts
such
as
ranking
the
importance
of
policy
issues.
Hence,
before
one
can
begin
to
assess
the
importance
of
such
issues
to
participants,
one
must
obtain
an
understanding
of
the
language
that
they
use
to
describe
such
resources,
and
their
past
experiences
(
Johnston
et
al.
1995).

Many
of
the
TER
comments
in
this
section
do
not
refer
to
the
ICR
or
focus
groups
in
general,
but
rather
to
participant
comments
(
taken
out
of
context)
regarding
very
early
drafts
of
survey
questions.
It
is
the
express
purpose
of
focus
groups
to
obtain
reactions
such
as
these,
in
order
to
ameliorate
such
problems
in
the
final
instrument.
Indeed,
TER's
comments
again
further
support
EPA's
rationale
for
conducting
focus
groups.
EPA
strongly
agrees
that
the
design
of
the
CV
questionnaire
is
critical
in
this
case
 
this
is
why
focus
groups
are
required.

Section
3.3.3:
The
ICR
indicates
that
double
counting
could
be
a
significant
problem
with
the
implementation
of
the
proposed
CV
survey.
The
TER
comment
reflects
a
misunderstanding
of
the
distinction
between
the
use
of
focus
groups
and
the
implementation
of
actual
stated
preference
surveys.
It
also
reflects
a
mistaken
perception
that
people
may
not
have
non­
use
values
for
fish
of
the
types
that
are
harvested
by
recreational
and
commercial
fishers.
It
is
the
position
of
EPA
that
individuals
may
have
non­
use
values
for
any
and
all
fish
 
including
forage
fish
as
well
as
fish
harvested
by
humans.
The
TER
comment
blurs
the
distinctions
between
stated
preference
questions
 
which
must
be
worded
carefully
to
avoid
double
counting
 
and
focus
group
discussions,
whose
purpose
is
to
elicit
the
perceptions
and
language
of
respondents,
and
to
assess
their
reactions
to
draft
survey
questions.
Questions
such
as
that
quote
by
the
report
(
pg.
21)
are
important
to
understand
the
motivations
underlying
responses
to
potential
survey
questions
 
and
to
better
understand
participants'
underlying
utility.
12
Moreover,
the
TER
comment
reflects
a
lack
of
understanding
that
focus
group
questions
are
not
the
same
as
CVM
survey
questions.
The
TER
report
quotes
a
suggested
focus
group
question
(
p.
21)
 
a
question
to
be
asked
verbally
of
a
small
group
of
focus
group
respondents,
for
the
purposes
of
eliciting
discussion.
The
report
then
goes
on
to
discuss
this
question
as
if
it
were
a
question
to
be
included
in
an
actual
stated
preference
survey.
The
subsequent
discussion
of
anchoring
specifically
concerns
questions
included
in
CV
surveys,
not
questions
to
be
used
in
focus
groups.
Hence,
the
TER
comment
fails
to
distinguish
between
questions
to
be
asked
for
exploratory
purposes
in
a
focus
group,
and
final
questions
that
would
appear
in
a
contingent
valuation
survey.

The
last
paragraph
of
section
3.3.3
expresses
the
opinion
that
individuals
may
have
difficulty
separating
the
"
value
of
protecting
fish
from
the
value
of
fishing."
EPA
emphasizes
again
that
this
is
an
empirical
issue,
and
one
that
will
be
addressed
in
the
proposed
focus
groups.

Section
3.3.4:
Potential
questions
about
habitat
would
result
in
biased
values
for
marginal
changes
in
fish
populations.
EPA
again
emphasizes
the
distinction
between
questions
asked
in
focus
groups
and
questions
asked
in
a
final
stated
preference
survey.
EPA
agrees
that
the
distinction
between
values
for
habitat
and
values
for
fish
is
an
important
distinction,
and
that
it
represents
"
essential
information"
to
be
obtained
in
focus
groups.
The
fact
that
habitat
issues
are
raised
in
the
focus
group
(
to
obtain
this
essential
information),
does
not
suggest
that
they
will
appear
in
any
survey
that
results.

Section
3.3.5:
The
ICR
indicates
that
the
Agency
does
not
clearly
understand
the
geographical
scope
of
the
survey
and
the
potential
effects
of
that
scope
of
benefits
estimation.
Once
again,
EPA
would
like
to
emphasize
the
distinction
between
questions
asked
in
focus
groups
and
questions
asked
in
a
final
stated
preference
survey.
The
purpose
of
focus
groups
is
NOT
to
estimate
values.
On
page
23,
the
TER
response
states
"
each
answer
will
present
the
Agency
with
a
considerable
challenge
from
a
benefits
measurement
criterion."
It
is
also
stated:
"
how
the
Agency
would
attempt
to
aggregate
those
values
into
some
type
of
national
benefits
estimate
is
also
unclear."
Both
these
comments
seem
to
imply
that
focus
group
responses
are
somehow
to
be
used
to
estimate
values
 
a
misrepresentation
of
the
purposes
of
focus
groups
clearly
laid
out
by
the
literature
(
Desvousges
et
al.
1984;
Desvousges
and
Smith
1988;
Johnston
et
al.
1995;
Mitchell
and
Carson
1989).

EPA
agrees
that
a
clearly
defining
geographical
scope
is
an
important
part
of
CVM
survey
design.
However,
the
Agency
emphasizes
that
general
questions
asked
in
focus
groups
 
for
the
purposes
of
eliciting
general
perceptions
and
understandings
of
participants
relative
to
the
resource
in
question
 
neither
reflect
a
"
lack
of
understanding"
on
the
part
of
the
Agency,
nor
imply
that
similar
questions
will
appear
in
any
subsequent
survey
instrument.
13
Section
3.3.6:
The
ICR
appropriately
raises
questions
about
natural
variability
in
fish
populations,
which
raises
further
doubts
as
to
whether
the
survey
would
be
effective.
EPA
recognizes
the
challenges
associated
with
the
communication
of
effects
of
natural
variability
in
fish
population
and,
as
a
result,
uncertainty
of
the
impact
of
reduced
impingement
and
entrainment
on
fish
populations.
However,
communicating
uncertainty
in
the
316(
b)
regulation
context
is
again
one
of
the
topics
that
will
be
addressed
in
the
proposed
focus
groups.
EPA
emphasizes
that
here
is
nothing
in
this
section
that
suggests
in
any
way
that
appropriately
representing
baseline
factors
that
may
influence
fish
populations
in
the
survey
instrument.

References
Arrow,
K.
,
R.
Solow,
E.
Leamer,
P.
Portney,
R.
Rander,
and
H.
Schuman.
1993.
Report
of
the
NOAA
Panel
on
Contingent
Valuation,
Federal
Register
58
(
10):
4602­
4614.

Bateman,
I.
J.,
R.
T.
Carson,
B.
Day,
M.
Hanemann,
N.
Hanley,
T.
Hett,
M.
Jones­
Leee,
G.
Loomes,
S.
Mourato,
E.
Ozdemiroglu,
D.
W.
Pierce,
R.
Sugden,
and
J.
Swanson.
2002.
Economic
Valuation
with
Stated
Preference
Surveys:
A
Manual.
Northampton,
MA:
Edward
Elgar.

Bjernstad,
O.
N.
and
B.
T.
Grenfell.
2001.
"
Noisy
Clockwork:
Time
Series
Analysis
of
Population
Fluctuations
in
Animals."
Science,
Vol.
293,
July
27,
2001.

Boardman,
A.
E.,
D.
H.
Greenberg,
A.
R.
Vining,
and
D.
L.
Weimer.
2001.
Cost­
Benefit
Analysis:
Concepts
and
Practice,
2nd
edition.
Upper
Saddle
River,
NJ:
Prentice
Hall.

Carson,
R.
T.,
N.
E.
Flores,
and
R.
C.
Mitchell.
1999.
The
Theory
and
Measurement
of
Passive­
Use
Value,
in
Valuing
Environmental
Preferences:
Theory
and
Practice
of
the
Contingent
Valuation
Method
in
the
US,
EU,
and
Developing
Countries,
edited
by
I.
J.
Bateman
and
K.
G.
Willis,
Oxford
University
Press,
Oxford,
UK.

Champ,
P.
A.,
R.
C.
Bishop,
T.
C.
Brown,
and
D.
W.
McCollum.
1997.
"
Using
Donation
Mechanisms
to
Value
Nonuse
Benefits
from
Public
Goods."
Journal
of
Environmental
Economics
and
Management
33:
151­
162.

Champ,
P.
A.,
R.
C.
Bishop,
T.
C.
Brown,
and
D.
W.
McCollum.
1997.
"
Using
Donation
Mechanisms
to
Value
Nonuse
Benefits
from
Public
Goods."
Journal
of
Environmental
Economics
and
Management
33:
151­
162.

Champ,
P.
A.,
R.
Moore,
and
R.
C.
Bishop.
2004.
"
Hypothetical
Bias:
The
Mitigating
Effects
of
Certainty
Questions
and
Cheap
Talk."
Selected
Paper
prepared
for
presentation
at
the
American
Agricultural
Economics
Association
Annual
Meeting,
Denver,
Colorado,
August
2004.
14
Cummings,
R.
and
L.
Taylor.
1999.
Unbiased
Value
Estimates
for
Environmental
Goods:
A
Cheap
Talk
Design
for
the
Contingent
Valuation
Method.
American
Economic
Review
89:
649­
665.

Desvousges,
W.
H.
and
V.
K
Smith.
1988.
Focus
Groups
and
Risk
Communication:
the
Science
of
Listening
to
Data.
Risk
Analysis
8:
479­
484.

Desvousges,
W.
H.,
V.
K.
Smith,
D.
H.
Brown,
and
D.
K.
Pate.
1984.
The
Role
of
Focus
Groups
in
Designing
a
contingent
Valuation
Survey
to
Measure
the
Benefits
of
Hazardous
Waste
Management
Regulations.
Research
Triangle
Institute:
Research
Triangle
Park,
NC.

Diamond,
P.
A.
J.
A.
Hausman,
G.
K.
Leonard,
M.
Denning.
1993.
"
Does
Contingent
Valuation
Measure
Preferences?
Experimental
Evidence."
In
Contingent
Valuation:
A
Critical
Assessment,
J.
A.
Haousman.
Ed.
New
York:
Elsevier
Science
Publishers
B.
V.

Diamond,
P.
A.
and
J.
A.
Hausman.
1993.
"
CV:
Is
Some
Number
Better
Than
No
Number?"
Journal
of
Economic
Perspectives
8
(
4):
45
64.

Foster,
V.,
I.
J.
Bateman,
and
D.
Harley.
1997.
"
Real
and
Hypothetical
Willingness
to
Pay
for
Environmental
Preservation:
A
Non­
experimental
Comparison."
Journal
of
Agricultural
Economics
48(
2):
123­
138.

Freeman,
A.
M.
III.
2003.
The
Measurement
of
Environmental
and
Resource
Values:
Theory
and
Methods.
Washington,
DC:
Resources
for
the
Future.

Hicks,
R.
2002.
Stated
Preference
Methods
for
Environmental
Management:
Recreational
Summer
Flounder
Angling
in
the
Northeastern
United
States.
Final
report
prepared
for
Fisheries
Statistics
and
Economics
Division,
Office
of
Science
and
Technology,
National
Marine
Fisheries
Service.
Requisition
Request
#
NFFKS­
18.
March.

Johnston,
R.
J.,
E.
Y.
Besedin
and
R.
F.
Wardwell.
2003.
Modeling
Relationships
Between
Use
and
Nonuse
Values
for
Surface
Water
Quality:
A
Meta­
Analysis.
Water
Resources
Research
39(
12),
p.
1363.

Johnston,
R.
J.,
G.
Magnusson,
M.
Mazzotta
and
J.
J.
Opaluch.
2002.
Combining
Economic
and
Ecological
Indicators
to
Prioritize
Salt
Marsh
Restoration
Actions.
American
Journal
of
Agricultural
Economics
84(
5):
1362­
1370.

Johnston,
R.
J.,
T.
F.
Weaver,
L.
A.
Smith
and
S.
K.
Swallow.
1995.
Contingent
Valuation
Focus
Groups:
Insights
From
Ethnographic
Interview
Techniques.
Agricultural
and
Resource
Economics
Review
24(
1):
56­
69.

Just,
R.
E.,
D.
L.
Hueth,
and
A.
Schmitz.
1982.
Applied
Welfare
Economics
and
Public
Policy.
Englewood
Cliffs,
NJ:
Prentice
Hall.

Louviere,
J.
J.,
D.
A.
Hensher,
and
J.
D.
Swait.
2000.
Stated
Preference
Methods:
Analysis
and
Application.
Cambridge,
UK:
Cambridge
University
Press.
15
Mitchell,
R.
C.
and
R.
T.
Carson.
1989.
Using
Surveys
to
Value
Public
Goods:
The
Contingent
Valuation
Method.
Resources
for
the
Future,
Washington,
D.
C.

Murphy,
J.
J.
and
T.
H.
Stevens.
2004.
Contingent
Valuation,
Hypothetical
Bias,
and
Experimental
Economics.
Agricultural
and
Resource
Economics
Review
33(
2):
182­
192.

Schkade,
D.
A.
and
J.
W.
Payne.
1994.
How
People
Respond
to
Contingent
Valuation
Questions:
A
Verbal
Protocol
Analysis
of
Willingness
to
Pay
for
an
Environmental
Regulation.
Journal
of
Environmental
Economics
and
Management
26:
88­
109.

Schläpfer,
F.,
A.
Roschewitz,
and
N.
Hanley.
2004.
Validation
of
Stated
Preferences
for
Public
Goods:
A
Comparison
of
Contingent
Valuation
Survey
Response
and
Voting
Behavior.
Ecological
Economics
51(
1):
1­
16.

Seip,
K.
and
Strand,
J.
1992.
Willingness
to
Pay
for
Environmental
Goods
in
Norway:
A
Contingent
Valuation
Study
with
Real
Payments.
Environmental
and
Resource
Economics
2:
91­
106.

Smith,
V.
K.
and
L.
Osborne.
1996.
Do
Contingent
Valuation
Estimates
Pass
the
Scope
Test?
A
Meta
Analysis.
Journal
of
Environmental
Economics
and
Management
31(
3):
287­
301.

Spencer,
M.
A.
S.
K.
Swallow,
and
C.
J.
Miller.
1998.
Valuing
Water
Quality
Monitoring:
A
Contingent
Valuation
Experiment
Involving
Hypothetical
and
Real
Payments.
Agricultural
and
Resource
Economics
Review
27(
1),
28­
42.

Sugden,
R.
1999.
Alternatives
to
the
Neo­
Classical
Theory
of
Choice
in
Valuing
Environmental
Preferences:
Theory
and
Practice
of
the
Contingent
Valuation
Method
in
the
US,
EU,
and
Developing
Countries,
edited
by
I.
J.
Bateman
and
K.
G.
Willis,
Oxford
University
Press,
Oxford,
UK.

U.
S.
EPA.
2000.
Guidelines
for
Preparing
Economic
Analyses.
(
EPA
240­
R­
00­
003).
U.
S.
EPA,
Office
of
the
Administrator,
Washington,
DC,
September
2000.

U.
S.
EPA.
2004.
The
Regional
Benefits
Assessment
for
the
Proposed
Section
316(
b)
Rule
for
Phase
III
Facilities.
(
EPA­
821­
R­
04­
017).
U.
S.
EPA,
Office
of
Water,
Washington,
DC,
November
2004.
