Technical
Support
Document
for
the
Assessment
of
Detection
and
Quantitation
Approaches
February
2003
U.
S.
Environmental
Protection
Agency
Office
of
Water
(
4303T)
1200
Pennsylvania
Avenue,
NW
Washington,
DC
20460
EPA­
821­
R­
03­
005
Technical
Support
Document
for
the
Assessment
of
Detection
and
Quantitation
Approaches
Engineering
and
Analysis
Division
Office
of
Science
and
Technology
U.
S.
Environmental
Protection
Agency
Washington,
DC
20460
February
2003
Acknowledgments
and
Disclaimer
This
document
was
prepared
by
Maria
Gomez­
Taylor,
Henry
D.
Kahn,
William
A.
Telliard,
Khouane
Ditthavong,
and
Leonid
Kopylev,
of
the
Engineering
and
Analysis
Division
in
EPA s
Office
of
Water.
Harry
McCarty,
Lynn
Riddick,
Ken
Miller,
and
Joan
Cuddeback,
with
DynCorp
I&
ET,
and
Dale
Rushneck
with
Interface,
Inc.
provided
assistance
under
EPA
Contract
No.
68­
C­
01­
091.
Sidina
Dedah
and
Kathleen
Stralka
with
Science
Applications
International
Corporation
provided
assistance
under
EPA
Contract
No.
68­
C­
99­
233.

Questions
or
comments
about
general
aspects
of
this
assessment
should
be
addressed
to:

William
A.
Telliard
USEPA
(
4303T)
1200
Pennsylvania
Avenue,
NW
Washington,
DC
20460
telliard.
william@
epa.
gov
Questions
or
comments
about
statistical
issues
related
to
this
assessment
should
be
addressed
to:

Henry
Kahn
USEPA
(
4303T)
1200
Pennsylvania
Avenue,
NW
Washington,
DC
20460
kahn.
henry@
epa.
gov
This
document
has
been
reviewed
and
approved
for
publication
by
the
Engineering
and
Analysis
Division,
Office
of
Science
and
Technology.
Neither
the
United
States
Government
nor
any
of
its
employees,
contractors,
or
their
employees
make
any
warranty,
expressed
or
implied,
or
assumes
any
legal
liability
or
responsibility
for
any
third
party s
use
of
or
the
results
of
such
use
of
any
information,
apparatus,
product,
or
process
discussed
in
this
report,
or
represents
that
its
use
by
such
party
would
not
infringe
on
privately
owned
rights.

February
2003
i
ii
February
2003
Table
of
Contents
Acknowledgments
and
Disclaimer
.......................................................
i
Table
of
Contents
....................................................................
iii
Chapter
1
Introduction
..........................................................
1­
1
1.1
Background
..........................................................
1­
1
1.2
Clause
6
Settlement
Agreement
Requirements
...............................
1­
1
1.2.1
Clause
6a......................................................
1­
1
1.2.2
Clause
6b......................................................
1­
2
1.2.3
Clause
6d......................................................
1­
2
1.2.4
Clause
6e......................................................
1­
2
1.2.5
Clause
6f
......................................................
1­
2
1.3
EPA s
Approach
to
Conducting
this
Assessment
.............................
1­
3
1.3.1
Study
Plan.....................................................
1­
4
1.3.2
Material
and
Data
used
in
the
Assessment
............................
1­
4
1.4
Peer
Review
of
the
Agency s
Assessment...................................
1­
9
1.5
Terminology
used
in
this
Document
......................................
1­
10
Chapter
2
Overview
and
History
of
Detection
and
Quantitation
Limit
Approaches
...........
2­
1
2.1
Currie s
Call
for
Standardization..........................................
2­
1
2.2
Development
of
the
MDL
and
ML
as
Practical
Embodiments
of
Currie s
Proposal
.
.
.
2­
3
2.2.1
Method
Detection
Limit
..........................................
2­
3
2.2.2
Minimum
Level
of
Quantitation
....................................
2­
4
2.3
Approaches
Advanced
by
Other
Organizations
...............................
2­
5
2.3.1
EPA
Approaches................................................
2­
5
2.3.2
Industry­
supported
Approaches
....................................
2­
7
2.3.3
Approaches
Advocated
by
the
Laboratory
Community
and
Voluntary
Consensus
Standards
Bodies
................................................
2­
8
2.3.4
Approaches
Advocated
by
Other
U.
S.
Government
Agencies
and
Other
Governments
...................................................
2­
9
Chapter
3
Issues
Pertaining
to
Detection
and
Quantitation
..............................
3­
1
3.1
Analytical
Chemistry
Approaches
to
Detection
and
Quantitation
.................
3­
2
3.1.1
Blank
versus
Zero
Concentration
...................................
3­
2
3.1.2
Lack
of
Instrument
Response
......................................
3­
2
3.1.3
Matrix
Effects
..................................................
3­
4
3.1.4
Recovery
Correction
.............................................
3­
6
3.1.5
Measurement
Quality
over
the
Life
of
a
Method.......................
3­
8
3.2
CWA
Regulatory
Issues
Affecting
Detection
and
Quantitation
..................
3­
9
3.2.1
Detection
and
Quantitation
Limit
Applications
Under
CWA
..............
3­
9
3.2.2
Descriptive
versus
Prescriptive
Uses
of
Lower
Limits
to
Measurement
....
3­
13
3.2.3
Compliance
Evaluation
Thresholds
................................
3­
14
3.2.4
Accepting
the
Procedures
of
Voluntary
Consensus
Standards
Bodies
......
3­
14
February
2003
iii
Assessment
of
Detection
and
Quantitation
Approaches
3.2.5
National
versus
Local
Standards
for
Measurement
....................
3­
16
3.2.6
Cost
and
Implementation
Issues
...................................
3­
16
3.2.7
Use
of
a
pair
of
related
detection
and
quantitation
procedures
in
all
Clean
Water
Act
applications.
...............................................
3­
17
3.2.8
Alternative
Procedures
..........................................
3­
18
3.3
Statistical
Issues
......................................................
3­
18
3.3.1
Sources
of
Variability
...........................................
3­
19
3.3.2
Censoring
Measurement
Results...................................
3­
20
3.3.3
Outliers
......................................................
3­
22
3.3.4
Criteria
for
the
Selection
and
Appropriate
Use
of
Statistical
Models
.......
3­
24
3.3.5
Methodology
for
Parameter
Estimation
.............................
3­
28
3.3.6
False
Positives
and
False
Negatives
................................
3­
28
3.3.7
Statistical
Prediction
and
Tolerance
................................
3­
30
3.3.8
Design
of
Detection
and
Quantitation
Studies
........................
3­
33
Chapter
4
Evaluation
Criteria.....................................................
4­
1
4.1
Criterion
1
...........................................................
4­
1
4.2
Criterion
2
...........................................................
4­
1
4.3
Criterion
3
...........................................................
4­
2
4.4
Criterion
4
...........................................................
4­
4
4.5
Criterion
5
...........................................................
4­
4
4.6
Criterion
6
...........................................................
4­
6
Chapter
5
Assessment...........................................................
5­
1
5.1
Detection
Limit
Approaches
.............................................
5­
1
5.1.1
Evaluation
of
the
MDL
...........................................
5­
1
5.1.2
Evaluation
of
the
ASTM
International
Interlaboratory
Detection
Estimate
(
IDE)
.............................................................
5­
7
5.1.3
Evaluation
of
the
ACS
Limit
of
Detection
...........................
5­
11
5.1.4
Evaluation
of
the
IUPAC/
ISO
Critical
Value
(
CRV)
...................
5­
13
5.1.5
Evaluation
of
the
IUPAC/
ISO
Detection
Limit
.......................
5­
15
5.2
Quantitation
Limit
Approaches
..........................................
5­
17
5.2.1
Assessment
of
the
EPA
Minimum
level
of
Quantitation
(
ML)
...........
5­
17
5.2.2
Assessment
of
the
IQE
..........................................
5­
20
5.2.3
Assessment
of
the
ACS
Limit
of
Quantitation
........................
5­
24
5.2.4
Assessment
of
the
IUPAC/
ISO
Limit
of
Quantitation
..................
5­
26
Chapter
6
Conclusions
..........................................................
6­
1
References
........................................................................
6­
1
Appendix
A
Literature
Search
Regarding
Detection
and
Quantitation
Limit
Approaches
Appendix
B
Characterizing
Measurement
Variability
as
a
Function
of
Analyte
Concentration
for
a
Variety
of
Analytical
Techniques
Appendix
C
Computation
of
Detection
and
Quantitation
Limits
iv
February
2003
Chapter
1
Introduction
1.1
Background
On
June
8,
1999
(
64
FR
30417),
EPA
promulgated
(
i.
e.,
published
in
a
final
rule)
Method
1631B:
Mercury
in
Water
by
Oxidation,
Purge
and
Trap,
and
Cold
Vapor
Atomic
Fluorescence
Spectro­
metry
(
the
"
method")
for
use
in
EPA's
Clean
Water
Act
programs.
The
method
was
developed
specifically
to
measure
mercury
at
ambient
water
quality
criteria
levels
and
includes
a
method
detection
limit
(
MDL;
see
40
CFR
part
136,
Appendix
B)
of
0.2
nanograms
per
liter
(
ng/
L).

Following
promulgation,
a
lawsuit
was
filed
challenging
EPA
on
the
validity
of
the
method.
The
basis
of
the
challenge
included
several
specific
aspects
of
Method
1631
as
well
as
the
general
procedures
used
to
establish
the
MDL
and
minimum
level
of
quantitation
(
ML)
published
in
the
method.
In
order
to
settle
the
lawsuit,
EPA
entered
into
a
settlement
agreement
(
the
"
Settlement
Agreement")
with
the
Alliance
of
Automobile
Manufacturers,
Inc.,
the
Chemical
Manufacturers
Association,
and
the
Utility
Water
Act
Group
(
collectively
the
 
Petitioners )
and
the
American
Forest
and
Paper
Association
( 
Intervenor )
on
October
19,
2000.
Under
Clause
6
of
the
Settlement
Agreement,
EPA
agreed
to
perform
an
assessment
of
detection
and
quantitation
limit
concepts.
The
complete
text
of
Clause
6
is
provided
in
Exhibit
1­
1
of
this
chapter.
A
summary
of
Clause
6
is
provided
in
Section
1.2.
The
summary
is
followed
by
a
description
of
EPA s
approach
to
the
assessment,
including
the
material
and
data
evaluated
(
Section
1.3),
the
use
of
an
independent
peer
review
to
evaluate
the
Agency's
assessment
(
Section
1.4),
and
a
brief
discussion
of
the
terminology
used
in
this
document.

1.2
Clause
6
Settlement
Agreement
Requirements
Clause
6
of
the
Settlement
Agreement
is
titled
Reassessment
of
Method
Detection
Limit
and
Minimum
Level
Procedures.
Clause
6
consists
of
five
subclauses,
a
­
b
and
d
­
f.
(
There
is
no
subclause
c).

1.2.1
Clause
6a
Clause
6a
broadly
defines
the
scope
of
the
assessment
and
provides
a
schedule
for
completing
the
initial
phase.
Specifically,
Clause
6a
requires
EPA
to:

 
Sign
and
forward
to
the
Office
of
Federal
Register
(
OFR)
a
notice
inviting
public
comment
on
a
reassessment
of
existing
EPA
procedures
for
determining
the
detection
and
quantitation
limits
of
contaminants
in
aqueous
samples.
 
Forward
the
notice
to
the
OFR
on
or
before
February
28,
2003.
 
Provide
a
period
of
at
least
120
days
for
public
comment
on
the
notice.
 
At
a
minimum,
include
the
MDL
procedure
published
at
40
CFR
part
136,
Appendix
B,
and
the
ML
procedure
described
in
Section
17.8
of
Method
1631B,
in
the
reassessment
of
detection
and
quantitation
limits.
 
Invite
comment
on
one
or
more
alternative
procedures
for
determining
and
describing
test
sensitivity.

Clause
6a
also
provides
EPA
with
the
option
of
proposing
modifications
to
the
existing
procedures.

February
2003
1­
1
Assessment
of
Detection
and
Quantitation
Approaches
1.2.2
Clause
6b
Clause
6b
requires
that
EPA
obtain
a
peer
review
of
its
reassessment,
and
describes
six
specific
topics
that
must
be
included
in
the
charge
to
the
peer
reviewers.
Specifically,
Clause
6b
requires
EPA
to:

 
Submit
the
reassessment
of
existing
procedures
(
including
any
proposed
modifications
thereof)
and
any
evaluation
of
alternatives
for
peer
review
by
experts
in
the
field
of
analytical
chemistry
and
the
statistical
aspects
of
analytical
data
interpretation.
 
Conduct
the
peer
review
in
accordance
with
EPA s
peer
review
policies.
 
Prepare
a
charge
to
the
peer
review
panel
that
requests
the
peer
reviewers
to
consider:
<
Criteria
for
selection
and
appropriate
use
of
statistical
models
<
Methodology
for
parameter
estimation
<
Statistical
tolerance
and
prediction
<
Criteria
for
design
of
detection
and
quantitation
studies,
including
selection
of
concentration
levels
( 
spiking
levels )
<
Interlaboratory
variability,
and
<
Incorporation
of
elements
of
probability
design.

1.2.3
Clause
6d
Clause
6d
requires
EPA
to
provide
the
Petitioners
and
Intervenor
(
the
 
litigants )
with
an
opportunity
for
review
of
the
Agency s
assessment
concurrent
with
the
Clause
6b
peer
review.

1.2.4
Clause
6e
Clause
6e
requires
EPA
to
provide
the
litigants
with:

 
An
opportunity
to
meet
periodically
(
i.
e.,
every
six
months)
to
discuss
the
Agency s
progress
during
development
of
the
assessment,
 
A
plan
for
performing
the
assessment
on
or
before
the
second
of
these
meetings,
and
 
Copies
of
relevant
documents,
where
appropriate,
in
advance
of
these
meetings.

1.2.5
Clause
6f
Clause
6f
establishes
a
schedule
and
requirements
concerning
final
action
on
the
notice
described
in
Clause
6a.
Specifically:

 
On
or
before
September
30,
2004,
EPA
is
to
sign
and
forward
to
the
OFR
a
notice
taking
final
action
on
the
notice
described
in
Clause
6a,
and
 
Coincident
with
publication
of
this
notice
of
final
action,
EPA
is
to
provide
the
litigants
with
an
opportunity
to
meet
and
discuss
the
implications
of
the
final
notice
and/
or
the
need
for
any
subsequent
EPA
action
in
light
of
the
final
notice.

1­
2
February
2003
Chapter
1
Exhibit
1­
1.
Full
Text
of
Clause
6
of
the
Settlement
Agreement
6.
Reassessment
of
Method
Detection
Limit
and
Minimum
Level
Procedures
a.
On
or
before
February
28,
2003,
EPA
shall
sign
and
forward
to
the
Office
of
the
Federal
Register
for
prompt
publication
a
notice
inviting
public
comment
on
a
reassessment
of
the
existing
Agency
procedures
for
determination
of
sensitivity
of
analytic
test
methods
for
aqueous
samples,
specifically,
EPA
procedures
for
determining
the
detection
limits
and
levels
of
quantitation
of
contaminants
in
aqueous
samples,
including,
at
a
minimum,
the
 
Definition
and
Procedure
for
Determination
of
the
Method
Detection
Limit 
published
at
40
C.
F.
R.
Part
136,
Appendix
B,
as
well
as
the
 
minimum
level 
procedures,
which
is
described
in
section
17.8
of
Method
1631B.
The
notice
shall
invite
comment
on
EPA s
evaluation
of
one
or
more
alternative
procedures
for
determining
and
describing
test
sensitivity.
The
notice
also
may
propose
modifications
to
the
existing
procedures.
The
notice
shall
invite
public
comment
for
a
period
of
no
less
than
one
hundred
twenty
(
120)
days.

b.
Prior
to
publishing
the
notice
inviting
public
comment
on
EPA
procedures
for
determining
test
sensitivity,
EPA
shall
submit
its
reassessment
of
existing
procedures
(
including
any
proposed
modifications
thereof)
and
its
evaluation
of
alternatives
for
peer
review
by
experts
in
the
field
of
analytical
chemistry
and
the
statistical
aspects
of
analytical
data
interpretation.
In
its
charge
to
the
peer
review
panel,
EPA
shall
request
that
the
peer
review
consider:
criteria
for
selection
and
appropriate
use
of
statistical
models;
methodology
for
parameter
estimation;
statistical
tolerance
and
prediction;
criteria
for
design
of
detection
and
quantitation
studies,
including
selection
of
concentration
levels
( 
spiking
levels );
interlaboratory
variability;
and
incorporation
of
elements
of
probability
design.
EPA
(
or
its
authorized
representative)
shall
conduct
the
peer
review
in
accordance
with
EPA s
current
peer
review
policies
in
the
January
1998
Science
Policy
Council
Handbook
(
EPA
100­
B­
98­
00)
[
sic],
including
any
subsequently­
developed
EPA
peer
review
documents
that
may
revise
or
amend
that
Handbook.

[
Note
­
the
correct
document
number
for
the
Science
Policy
Council
Handbook
is
EPA
100­
B­
98­
001[

[
c.
Note
­
there
is
no
clause
"
6.
c"
in
the
Settlement
Agreement]

d.
During
the
peer
review
period,
EPA
shall
also
provide
an
opportunity
for
concurrent
review
and
comment
by
the
Petitioners
and
Intervenor.

e.
In
the
development
of
the
reassessment/
assessment
of
alternatives,
EPA
shall
provide
the
Petitioners
and
Intervenor
with
a
periodic
opportunity
to
meet
(
i.
e.,
every
six
(
6)
months)
on
the
Agency s
progress.
EPA
shall
prepare
and
present
the
Petitioners
and
Intervenor
with
the
Agency s
 
plan 
for
conducting
the
reassessment/
assessment
of
alternatives
on
or
before
the
second
such
periodic
meeting.
Where
appropriate,
EPA
shall
provide
the
Petitioners
and
Intervenor
with
copies
of
relevant
documents
in
advance
of
such
meetings.

f.
On
or
before
September
30,
2004,
EPA
shall
sign
and
forward
to
the
Office
of
the
Federal
Register
for
prompt
publication
a
notice
taking
final
action
on
the
notice
described
in
subparagraph
6.
a.
Coincident
with
publication
of
the
notice
of
final
action,
EPA
shall
provide
Petitioners
and
Intervenor
an
opportunity
to
meet
to
discuss
the
implications
of
the
final
notice
and/
or
the
need
for
any
subsequent
EPA
action
in
light
of
the
final
notice.

1.3
EPA s
Approach
to
Conducting
this
Assessment
This
document
details
the
Agency s
assessment
of
methodology
for
the
determination
of
method
sensitivity,
specifically:
detection
and
quantitation
limits.
This
assessment
is
being
conducted
in
accordance
with
a
plan
summarized
in
Section
1.3.1
and
is
based,
in
part,
on
an
assessment
of
the
data
described
in
Section
1.3.2.

February
2003
1­
3
Assessment
of
Detection
and
Quantitation
Approaches
1.3.1
Study
Plan
EPA
developed
a
technical
approach
for
1)
conducting
the
assessment,
and
2)
complying
with
all
applicable
requirements
of
the
Settlement
Agreement.
The
approach
was
documented
in
a
draft
study
plan
that
has
since
formed
the
general
framework
for
the
assessment
described
in
this
Assessment
Document.
EPA
also
conducted
a
literature
search
to
identify
and
review
issues
and
concepts
that
should
be
considered
when
developing
the
plan.
A
summary
of
this
literature
review
is
provided
in
Appendix
A
to
this
Assessment
Document.

The
study
plan
described
roles
and
responsibilities
for
implementing
the
plan,
provided
a
background
discussion
of
detection
and
quantitation
limit
concepts,
including
the
MDL
and
ML,
and
outlined
a
series
of
11
events
associated
with
the
Agency s
assessment
of
detection
and
quantitation
limit
approaches.
The
relationship
between
those
planned
events
and
this
Assessment
Document
is
summarized
in
Exhibit
1­
2
at
the
end
of
this
chapter.

Although
the
Settlement
Agreement
did
not
require
that
EPA
seek
formal
peer
review
on
its
draft
plan,
the
Agency
chose
to
conduct
a
peer
review
of
the
draft
plan.
The
peer
review
was
initiated
in
December
2001,
conducted
in
accordance
with
EPA s
peer­
review
policies,
and
performed
by
two
statisticians
and
two
chemists.
EPA
reviewed
the
comments
and
recommendations
offered
by
these
reviewers,
and
where
appropriate,
revised
the
plan
to
reflect
the
peer­
review
comments.
EPA
also
reviewed,
and
where
appropriate,
revised
the
plan
to
reflect
comments
provided
by
the
litigants
following
their
concurrent
review.

1.3.2
Material
and
Data
used
in
the
Assessment
In
order
to
perform
the
assessment
described
in
this
document,
EPA
sought
to
collect
documentation
describing
existing
detection
and
quantitation
limit
concepts
and
procedures
and
data
that
could
be
used
to
evaluate
these
concepts
and
procedures.

Documentation
concerning
the
existing
concepts
and
procedures
was
obtained
by
performing
a
literature
search
as
described
in
Appendix
A
to
this
Assessment
Document,
and
where
appropriate,
by
purchasing
copies
of
documents
describing
concepts
or
procedures
from
the
organizations
that
published
them.

In
performing
this
assessment,
EPA
hoped
to
identify
a
substantial
amount
of
data
containing
results
of
direct
relevance
to
the
determination
of
detection
and
low­
level
measurement
capability.
That
is,
measurement
results
in
the
low
concentration
region.
To
date,
EPA
has
been
able
to
identify
only
six
data
sets
that
were
of
use
in
fully
evaluating
variability
in
the
range
of
analytical
detection
and
quantitation.
Three
of
the
six
were
developed
by
EPA
for
the
express
purpose
of
studying
the
relationship
between
measurement
variation
and
concentration
across
a
wide
variety
of
measurement
techniques
and
analytes.
EPA
refers
to
these
data
sets
as
 
EPA s
ICP/
MS
Study
of
Variability
as
a
Function
of
Concentration, 
 
EPA s
Multi­
technique
Variability
Study 
(
also
referred
to
as
the
 
Episode
6000
study ),
and
 
EPA s
GC/
MS
Threshold
Study 
(
also
referred
to
as
 
the
Episode
6184
study ).
In
all
three
cases,
replicate
measurement
results
from
each
combination
of
analyte
and
measurement
technique
were
produced
by
a
single
laboratory
over
a
wide
range
and
large
number
of
concentrations.
The
fourth
data
set
was
developed
by
the
American
Automobile
Manufacturer s
Association
(
AAMA)
for
the
purpose
of
estimating
one
particular
kind
of
quantitation
value.
That
quantitation
value
is
called
an
alternative
minimum
level
(
AML;
see
Gibbons
et
al.,
1997).
In
the
AAMA
study,
replicate
results
were
measured
at
a
limited
number
of
concentrations
by
multiple
laboratories
using
EPA
Method
245.2
(
cold
vapor
atomic
absorption;
CVAA)
for
mercury
and
EPA
Method
200.7
(
inductively
coupled
plasma/
atomic
emission
spectroscopy;
ICP/
AES)
for
twelve
other
metals.
The
final
two
data
sets
were
1­
4
February
2003
Chapter
1
jointly
gathered
by
EPA
and
the
Electric
Power
Research
Institute
(
EPRI)
to
support
interlaboratory
validation
of
EPA
Methods
1631
and
1638.

The
studies
from
which
these
six
data
sets
were
obtained
are
summarized
in
sections
1.3.2.1
­
1.3.2.6
below.
Additional
information
about
these
studies
can
be
found
in
Appendices
B
and
C
to
this
Assessment
Document.

Although
the
litigants
offered
specific
suggestions
for
other
data
sets
that
they
believed
should
be
considered
in
this
assessment,
EPA
found
that
these
data
sets
did
not
include
a
sufficient
number
of
results
in
the
region
of
detection
and
quantitation
to
yield
information
for
the
assessment,
overlapped
with
data
already
used
in
the
assessment,
or
exhibited
signs
of
significant
contamination
that
made
the
data
inappropriate
for
inclusion
in
the
assessment.
These
data,
and
EPA s
decisions
regarding
the
data,
are
discussed
in
Section
1.3.2.7
below.

1.3.2.1
EPA s
ICP/
MS
Study
of
Variability
as
a
Function
of
Concentration
The
objective
of
the
ICP/
MS
study
was
to
characterize
variability
as
a
function
of
concentration
using
EPA's
draft
Method
1638
for
determination
of
nine
metals
by
inductively
coupled
plasma
with
mass
spectroscopy
(
ICP/
MS).
The
nine
metals
were
silver,
cadmium,
copper,
nickel,
lead,
antimony,
selenium,
thallium,
and
zinc.
The
ICP/
MS
instrument
used
in
this
study
averages
triplicate
scans
to
produce
a
single
measurement
of
each
element
at
each
concentration.
Such
averaging
is
typical
of
ICP/
MS
design
and
use.

In
preparation
for
the
study,
the
ICP/
MS
was
calibrated
using
triplicate
scans
averaged
to
produce
a
single
measurement
of
100,
1,000,
5,000,
10,000,
and
25,000
nanograms
per
liter
(
ng/
L)
for
each
element.
Originally,
the
instrument
was
calibrated
using
unweighted
least
squares
estimates
under
the
assumption
of
linearity.
Subsequently,
the
analytical
results
were
adjusted
with
weighted
least
squares
estimates.
Weighted
least
squares
estimates
are
based
on
the
knowledge
that
variability
(
expressed
as
the
standard
deviation)
increases
with
increasing
analyte
concentration.

Although
the
instrumentation
has
the
capability
to
provide
intensity
results
for
each
of
the
three
scans
at
each
concentration,
averaging
the
three
scans
to
produce
a
single
measurement
is
the
normal
operating
mode,
and
the
average
was
used
to
produce
the
measurements
in
this
study.
Draft
Method
1638
specifies
the
use
of
average
response
factors
rather
than
least
squares
estimation
of
a
linear
calibration,
although
it
does
allow
for
the
use
of
such
procedures.

All
nine
metals
were
spiked
into
reagent
water
to
produce
solutions
at
concentrations
of:
0,
10,
20,
50,
100,
200,
500,
1,000,
2,000,
5,000,
10,000,
and
25,000
ng/
L.
Each
solution
was
divided
into
seven
replicate
aliquots
for
subsequent
analysis.
The
aliquots
were
analyzed
beginning
with
the
blank
(
zero
concentration)
followed
by
analyses
from
the
highest
to
the
lowest
concentration.
This
sequence
was
chosen
to
minimize
carry­
over
effects
and
to
allow
the
analyst
to
stop
at
the
concentration
that
returned
zero
results.
Carry­
over
is
caused
by
residual
sample
remaining
in
the
inlet
system
of
the
instrument,
in
this
case,
the
ICP/
MS.
Carry­
over
can
occur
when
analysis
of
a
high­
concentration
sample
is
followed
by
analysis
of
a
relatively
low­
concentration
sample,
as
could
occur
if
the
replicates
were
analyzed
in
random
order.
Use
of
the
highest
to
lowest
analytical
sequence
ensured
that
each
successive
concentration
analyzed
was
close
enough
to
the
previous
concentration
that
any
effects
of
carryover
would
be
negligible
and,
therefore,
would
not
compromise
study
results.
(
A
more
in­
depth
discussion
of
the
randomized
design
and
the
effects
of
carry­
over
issues
is
provided
in
Chapter
3,
Section
3.3.8.2).

Results
at
multiple
mass­
to­
charge
ratios,
or
m/
z's,
were
reported
for
each
metal,
although
draft
Method
1638
specifies
only
one
m/
z
for
eight
of
the
nine
metals.
For
lead,
m/
z's
206,
207,
and
208
are
February
2003
1­
5
Assessment
of
Detection
and
Quantitation
Approaches
specified.
Only
data
associated
with
m/
z's
specified
in
draft
Method
1638
were
used
in
the
ICP/
MS
study.

1.3.2.2
EPA s
Multi­
technique
Variability
Study
(
the
 
Episode
6000
Study )

In
1997
and
1998,
EPA
conducted
a
study
of
variability
vs.
concentration
for
a
number
of
analytical
methods.
Five
laboratories
were
employed
for
the
analyses;
each
analyte
and
method
combination
was
tested
by
one
of
these
laboratories.
Details
of
the
study
design
are
described
in
EPA s
Study
Plan
for
Characterizing
Variability
as
a
Function
of
Concentration
for
a
Variety
of
Analytical
Techniques
(
July
1998).
Based
on
the
sampling
episode
number
assigned
to
the
study
by
the
EPA
Sample
Control
Center,
the
study
and
results
have
become
known
as
the
Episode
6000
study
and
data.
The
analytes
and
analytical
techniques
studied
were:

 
Total
suspended
solids
(
TSS)
by
gravimetry
 
Metals
by
graphite
furnace
atomic
absorption
spectroscopy
(
GFAA)
 
Metals
by
inductively­
coupled
plasma
atomic
emission
spectrometry
(
ICP/
AES)
 
Hardness
by
ethylene
diamine
tetraacetic
acid
(
EDTA)
titration
 
Phosphorus
by
colorimetry
 
Ammonia
by
ion­
selective
electrode
 
Volatile
organic
compounds
by
purge­
and­
trap
capillary
column
gas
chromatography
with
a
photoionization
detector
(
GC/
PID)
and
electrolytic
conductivity
detector
(
GC/
ELCD)
in
series
 
Volatile
organic
compounds
by
gas
chromatography
with
a
mass
spectrometer
(
GC/
MS)
 
Available
cyanide
by
flow­
injection/
ligand
exchange/
amperometric
detection
 
Metals
by
inductively­
coupled
plasma
spectrometry
with
a
mass
spectrometer
(
ICP/
MS)

In
this
study,
an
initial
(
range
finding)
MDL
was
determined
for
each
combination
of
analyte
and
analytical
technique
using
minor
modifications
to
the
MDL
procedure
at
40
CFR
part
136.
Specifically.
the
modifications
made
the
optional
iterative
step
7
of
the
MDL
procedure
mandatory
and
required
the
spike
concentration
to
be
no
more
than
a
factor
of
three
times
the
determined
MDL
(
instead
of
a
factor
of
five
times).
During
the
study,
however,
two
of
the
laboratories
found
that
the
reduction
in
the
allowable
spike
range
necessitated
an
unreasonably
large
number
of
iterations.
In
continuing
the
study,
EPA
returned
to
the
spike­
to­
MDL
ratio
of
five
published
in
the
40
CFR
part
136,
Appendix
B
procedure.

After
determining
the
initial
MDL,
each
laboratory
analyzed
7
replicate
samples
spiked
at
concentrations
that
were
100,
50,
20,
10,
7.5,
5.0,
3.5,
2.0,
1.5,
1.0,
0.75,
0.50,
0.35,
0.20,
0.15,
and
0.10
times
the
initial
MDL.
In
a
few
instances,
laboratories
analyzed
more
than
7
replicates.
As
often
as
possible,
the
replicate
analyses
at
each
concentration
level
were
produced
using
the
same
calibration
that
was
used
in
determining
the
initial
MDL.
Where
laboratory
reports
indicated
that
multiple
calibrations
were
conducted,
each
result
was
associated
with
its
calibration
in
the
data
analysis.

Spiked
aqueous
solutions
were
analyzed
in
order
from
the
highest
concentration
(
100
times
the
MDL)
to
the
concentration
at
which
3
or
more
non­
detects
(
zeros)
were
encountered
among
the
7
replicates,
or
the
lowest
concentration
specified
(
0.1
times
the
MDL),
whichever
occurred
first.
This
analysis
order
(
1)
minimized
carryover
that
could
occur
in
some
methods
if
a
low­
concentration
sample
had
followed
a
high­
concentration
sample
(
as
may
happen
when
samples
are
analyzed
in
random
order),
and
(
2)
prevented
collection
of
a
large
number
of
zeros
if
the
signal
disappeared.

For
methods
that
do
not
produce
a
signal
for
a
blank,
the
signal
will
disappear
somewhere
below
the
MDL,
i.
e.,
a
zero
will
be
reported.
Laboratories
were
instructed
that
when
three
nondetects
(
out
of
seven
measurements)
were
reported,
it
was
not
necessary
to
move
to
the
next
lower
concentration,

1­
6
February
2003
Chapter
1
because
it
would
be
of
no
practical
value
to
have
laboratories
measure
seven
zeros,
move
to
a
lower
level,
measure
seven
zeros,
etc.

A
variant
of
the
iterative
procedure
for
determining
the
MDL
was
used
for
organic
compounds
determined
by
chromatographic
methods.
Methods
for
organics
normally
list
many
(
15
to
100)
analytes,
and
the
response
for
each
analyte
is
different.
Therefore,
to
determine
an
MDL
for
each
analyte,
the
concentration
of
the
spike
would
need
to
be
inversely
proportional
to
the
response.
Making
a
spiking
solution
with
15
to
100
different
concentrations
is
cumbersome
and
error
prone.
The
approach
used
in
the
study
was
to
run
seven
replicates
at
decreasing
concentrations
until
signal
extinction,
then
select
the
concentration(
s)
appropriate
for
the
determining
the
MDL
for
each
analyte
according
to
the
MDL
procedure.
In
some
cases,
the
laboratories
selected
the
concentrations,
in
others
cases,
EPA
did.
This
approach
was
generally
applied
for
organics
analysis.
However,
laboratories
also
had
the
option
of
using
some
combination
of
the
monotonically
decreasing
concentrations
described
above
and
a
few
selected
concentrations
to
achieve
the
desired
spiking
levels.

1.3.2.3
EPA s
GC/
MS
Threshold
Study
(
the
 
Episode
6184
Study )

Data
from
the
Episode
6184
study
of
variability
vs.
concentration
were
used
to
evaluate
the
effect
of
GC/
MS
thresholds
on
the
ability
to
identify
semivolatile
organic
compounds
at
low
concentrations.
Details
of
the
design
of
this
study
are
described
in
EPA s
Study
Plan
for
Characterizing
Error
as
a
Function
of
Concentration
for
Determination
of
Semivolatiles
by
Gas
Chromatography/
Mass
Spectrometry
(
December
1998).
Data
were
generated
for
82
semivolatile
organic
compounds
using
EPA
Method
1625C
(
semivolatile
organic
compounds
by
GC/
MS).
MDLs
were
not
determined
for
these
compounds.
Instead,
solutions
of
the
analytes
were
prepared
and
analyzed
at
concentrations
of
50.0,
20.0,
10.0,
7.50,
5.00,
3.50,
2.00,
1.50,
1.00,
0.75,
0.50,
0.35,
0.20,
0.15,
0.10,
0.075
and
0.050
ng/:
L
(
or
:
g/
mL).
Each
solution
was
injected
into
the
GC/
MS
in
triplicate
with
the
mass
spectrometer
threshold
set
to
0,
and
again
in
triplicate
with
the
mass
spectrometer
threshold
set
to
a
level
typical
of
that
used
in
routine
environmental
analyses.
As
with
the
ICP/
MS
study
and
the
Episode
6000
study,
and
for
the
same
reasons
described
in
Section
1.3.2.1,
samples
were
analyzed
in
order
from
the
highest
to
the
lowest
concentration.

1.3.2.4
AAMA
Metals
Study
of
Methods
200.7
and
245.2
The
American
Automobile
Manufacturer s
Association
conducted
an
interlaboratory
study
of
EPA
Method
200.7
(
metals
by
ICP/
AES)
and
Method
245.2
(
mercury
by
CVAA).
The
study
was
designed
to
estimate
a
quantitation
value
based
on
a
concept
termed
the
alternative
minimum
level
(
AML)
that
had
been
described
in
the
literature
(
Gibbons
et
al.,
1997).
Nine
laboratories
participated
in
the
study,
and
each
reported
data
for
the
following
13
metals:
aluminum,
arsenic,
cadmium,
chromium,
copper,
lead,
manganese,
mercury,
molybdenum,
nickel,
selenium,
silver
and
zinc.
Study
samples
were
analyzed
by
EPA
Method
200.7
for
12
of
the
metals.
Mercury
was
determined
by
EPA
Method
245.2.

As
part
of
the
study
design,
the
nine
laboratories
were
randomized
prior
to
the
start
of
the
study.
Five
sample
matrices
(
including
reagent
water)
were
studied,
including
four
wastewater
matrices
that
are
representative
of
the
automotive
industry.
Starting
from
a
blank,
or
unspiked
sample,
all
target
analytes
were
spiked
at
four
concentrations
to
yield
a
total
of
five
concentrations
per
matrix.
Concentrations
ranged
from
0.01
to
10
:
g/
L
for
mercury
and
selenium
on
the
low
end,
and
from
2.0
and
1000
:
g/
L
for
mercury
and
selenium
on
the
high
end.
In
addition,
the
concentrations
were
matrix­
dependent.
The
same
concentration
ranges
for
each
metal
by
matrix
combination
were
used
for
all
five
weeks
of
the
study.

Matrix
A
(
reagent
water)
was
analyzed
in
all
nine
laboratories,
and
three
laboratories
analyzed
each
of
the
other
four
matrices.
All
analyses
were
repeated
weekly
over
a
five­
week
period.
As
a
result,

February
2003
1­
7
Assessment
of
Detection
and
Quantitation
Approaches
a
total
of
6,825
observations
were
obtained,
which
includes
2,925
observations
for
matrix
A
(
9
labs
×
13
metals
×
5
spike
concentrations
×
5
weeks),
and
975
observations
(
3
labs
×
13
metals
×
5
spike
concentrations
×
5
weeks)
for
each
of
the
other
four
matrices
(
6,825
=
2,925
+
(
975
×
4)).
There
were
two
missing
values
for
chromium
in
matrix
A
from
laboratories
1
and
9.

1.3.2.5
Method
1631
Interlaboratory
Validation
Study
The
Method
1631
interlaboratory
validation
study
was
conducted
by
EPA
to
evaluate
performance
of
the
method
and
to
gather
data
to
evaluate
existing
performance
specifications,
including
detection
and
quantitation
limits.
To
accommodate
stakeholder
interests
and
expand
the
scope
of
the
study,
the
Electric
Power
Research
Institute
(
EPRI)
funded
the
distribution
of
additional
samples
to
study
participants.

This
jointly
funded
study
involved
an
international
community
of
twelve
participating
laboratories
and
one
referee
laboratory.
Each
participating
laboratory
analyzed
four
different
matrices,
each
containing
mercury
at
a
concentration
selected
to
allow
for
characterization
of
method
performance
across
the
measurement
range
of
the
method.
Each
of
the
12
participating
laboratories
was
provided
with
13
sample
pairs
(
a
total
of
26
blind
samples).
These
included
1
filtered
effluent
pair,
1
unfiltered
effluent
pair,
4
filtered
freshwater
pairs,
1
filtered
marine
water
pair,
1
unfiltered
marine
water
pair,
and
5
spiked
reagent
water
pairs.
All
12
laboratories
received
and
analyzed
the
same
sample
pairs
(
a
total
of
312
analyses).
To
measure
the
recovery
and
precision
of
the
analytical
system,
and
to
monitor
matrix
interferences,
the
laboratories
were
instructed
to
analyze
matrix
spike
and
matrix
spike
duplicate
samples
on
specified
field
samples
for
each
filtered
and
unfiltered
matrix,
spiked
at
1­
5
times
the
background
concentration
of
mercury
determined
by
analysis
of
an
unspiked
aliquot
of
the
sample.
The
laboratories
were
instructed
to
perform
all
other
QC
tests
described
in
Method
1631,
including
the
analysis
of
blanks,
and
to
conduct
MDL
studies
in
reagent
water
following
the
procedure
at
40
CFR
part
136.

1.3.2.6
Method
1638
Interlaboratory
Validation
Study
The
Method
1638
interlaboratory
validation
study
was
conducted
by
EPA
to
evaluate
performance
of
the
method
and
to
gather
data
that
would
allow
revision
of
existing
performance
specifications,
including
detection
and
quantitation
limits.
To
accommodate
stakeholder
interests
and
expand
the
scope
of
the
study,
the
Electric
Power
Research
Institute
funded
the
distribution
of
additional
samples
to
study
participants.

A
total
of
eight
laboratories
(
and
a
referee
laboratory)
participated
in
the
study.
The
study
was
designed
so
that
each
participating
laboratory
would
analyze
sample
pairs
of
each
matrix
of
interest
at
concentrations
that
would
span
the
analytical
range
of
the
method.
Each
laboratory
was
provided
with
11
sample
pairs
(
a
total
of
22
blind
samples).
These
included
1
filtered
effluent
pair,
1
unfiltered
effluent
pair,
4
filtered
freshwater
pairs,
and
5
spiked
reagent
water
pairs.
All
eight
laboratories
received
and
analyzed
the
same
sample
pairs
(
a
total
of
176
analyses).
To
measure
the
recovery
and
precision
of
the
analysis,
and
to
monitor
matrix
interferences,
the
laboratories
were
instructed
to
analyze
a
matrix
spike
and
matrix
spike
duplicate
of
specified
field
samples
in
each
filtered
and
unfiltered
matrix,
spiked
at
1­
5
times
the
background
concentration
of
the
analytes
determined
by
analysis
of
an
unspiked
aliquot
of
the
sample.
The
laboratories
were
instructed
to
perform
all
other
QC
tests
described
in
Method
1638,
including
the
analysis
of
blanks,
and
to
conduct
MDL
studies
in
reagent
water
following
the
procedure
at
40
CFR
part
136.

1­
8
February
2003
Chapter
1
1.3.2.7
Data
Considered
but
not
Used
in
this
Assessment
The
Petitioners
and
Intervenor
to
the
Settlement
Agreement
suggested
ten
specific
data
sets
that
EPA
should
consider
in
its
assessment
of
detection
and
quantitation
limits.
EPA
evaluated
each
of
these
data
sets
to
determine
if
the
design
of
the
study,
including
the
concentrations
targeted
in
the
study,
would
provide
sufficient
data
for
evaluating
measurement
variability
in
the
region
of
interest
(
i.
e.,
at
concentrations
below,
at,
and
above
the
region
of
detection
and
quantitation).
If
such
data
were
determined
to
be
present,
EPA
further
evaluated
the
data
set
to
ensure
that
it
was
of
sufficient
quality
to
support
the
Agency s
assessment.
Four
of
the
ten
data
sets
met
these
requirements
and
were
used
in
EPA s
assessment.
Table
1
identifies
each
of
the
data
sets
suggested
by
the
petitioners
along
with
EPA s
rationale
for
using
or
excluding
the
data
from
this
assessment.

Table
1.
Dataset
Source
and
Year
Analytes
and
technology
EPA
Decision
Regarding
Use
AAMA
1996­
1997
Metals
by
ICP/
AES
(
200.7)
Used
in
this
assessment
and
described
in
Section
1.3.2.4
AAMA
1996­
1997
Mercury
by
CVAA
(
245.2)
Used
in
this
assessment
and
described
in
Section
1.3.2.4
EPA/
EPRI
1997­
1998
Mercury
by
CVAF
(
1631)
Used
in
this
assessment
and
described
in
Section
1.3.2.5
EPA/
EPRI
1997­
1998
Metals
by
ICP/
MS
(
1638)
Used
in
this
assessment
and
described
in
Section
1.3.2.6
EPRI
1987
Metals
by
GFAA
(
EPA
200)
Not
used
in
this
assessment
because
of
insufficient
low­
level
data
EPRI
1990
Metals
by
ICP/
AES
(
EPA
200.7)
Not
used
in
this
assessment
because
of
insufficient
low­
level
data
EPRI
1994
Al,
Be,
Tl
by
GFAA
(
EPA
200)
Not
used
in
this
assessment
because
of
overlap
with
EPA's
Episode
6000
Study,
which
provides
data
on
the
same
analytes
but
covers
a
larger
number
of
concentrations
in
the
region
of
interest
AAMA
1996­
1997
PCBs
by
GC/
ECD
(
608.2)
Not
used
in
this
assessment
because
of
overlap
with
EPA's
Episode
6000
Study,
which
provides
data
on
the
same
analytes
but
covers
a
larger
number
of
concentrations
in
the
region
of
interest
EPRI
1996
Cd,
As,
Cr
by
GFAA
(
EPA
200)
Not
used
in
this
assessment
because
of
overlap
with
EPA's
Episode
6000
Study,
which
provides
data
on
the
same
analytes
but
covers
a
larger
number
of
concentrations
in
the
region
of
interest
MMA
2000­
2001
Aroclors
1016
and
1260
by
GC/
ECD
Not
used
in
this
assessment.
hough
the
study
examined
the
region
of
detection
and
quantitation,
samples
spiked
with
low
levels
of
Aroclors
exhibited
average
recoveries
>
500%,
with
RSDs
>
200%
across
10
laboratories,
indicating
contamination
of
the
samples
from
an
unknown
source.
Data
Sets
Suggested
by
Petitioners
Alt
1.4
Peer
Review
of
the
Agency s
Assessment
In
August
2002,
EPA
conducted
a
formal
peer
review
of
the
Agency s
assessment.
This
peer
review,
which
satisfied
requirements
in
Clause
6b
of
the
Settlement
Agreement,
was
conducted
in
accordance
with
EPA s
peer
review
policies
described
in
the
Science
Policy
Council
Handbook
(
EPA
100­
B­
00­
001).
The
review
was
performed
by
two
experts
in
the
field
of
analytical
chemistry
and
two
experts
in
the
statistical
aspects
of
analytical
data
interpretation.
Each
reviewer
was
provided
with
a
draft
February
2003
1­
9
Assessment
of
Detection
and
Quantitation
Approaches
version
of
this
Assessment
Document,
which
documented
the
Agency s
approach
to
the
assessment
and
the
Agency s
preliminary
findings
and
conclusions.
Reviewers
also
were
provided
with
copies
of
all
data
evaluated
in
the
assessment,
statistical
programs
used
to
analyze
the
data,
and
copies
of
the
detection
and
quantitation
concepts
and
procedures
evaluated
by
EPA.
In
accordance
with
the
Agency s
peer
review
policies,
the
reviewers
also
were
provided
with
a
written
 
charge 
intended
to
ensure
the
evaluation
would
meet
EPA
needs.

In
its
charge
to
the
peer
reviewers,
EPA
requested
a
written
evaluation
of
whether
the
assessment
approach
described
by
EPA
is
valid
and
conceptually
sound.
Reviewers
also
were
asked
to
consider
and
address
eight
specific
questions
pertaining
to
the
adequacy
of
the
concepts
and
issues
considered,
the
evaluation
criteria
developed
by
EPA,
EPA s
assessment
and
conclusions,
the
data
used
to
perform
the
assessment,
suggested
improvements
to
the
procedures
discussed,
and
EPA s
consideration
of
interlaboratory
vs.
intralaboratory
issues.
Comments
from
peer
reviewers
were
generally
supportive
of
EPA s
assessment
and
its
presentation
of
the
assessment
in
the
draft
Assessment
Document.
Where
appropriate,
EPA
revised
the
Assessment
Document
to
reflect
specific
suggestions
and
comments
offered
by
the
peer
reviewers.
This
version
of
the
Assessment
Document
reflects
those
revisions.
Copies
of
all
materials
associated
with
the
peer
review,
including
the
peer
review
charge,
the
materials
provided
to
the
peer
reviewers
for
review,
complete
copies
of
the
peer
reviewers 
comments,
and
detailed
EPA
responses
to
each
of
the
comments
are
provided
in
the
public
docket
supporting
the
Agency s
assessment.

1.5
Terminology
used
in
this
Document
We
use
the
term
"
quantitation"
in
this
document
because
of
its
common
usage
among
analytical
chemists,
even
though
we
recognize
that
the
term
"
quantification"
(
i.
e.,
the
act
of
quantifying)
is
the
term
listed
in
most
dictionaries.
Also,
when
referring
to
detection
and
quantitation,
we
use
the
words
"
approach"
or
"
concept"
to
refer,
generically,
to
the
procedures
used
to
establish
detection
and
quantitation
limits
or
the
theories
on
which
those
procedures
are
based.
We
use
the
word
"
limit"
rather
than
"
level"
to
indicate
that
the
detection
and
quantitation
concepts
are
directed
at
the
lowest
concentration
or
amount
at
which
an
analyte
is
determined
to
be
present
(
detection)
or
may
be
measured
(
quantitation).
In
choosing
the
word
 
limit 
we
do
not
mean
to
imply
any
sense
of
permanence.
We
recognize
that
measurement
capabilities
generally
improve
over
time,
and
that
detection
or
quantitation
 
limits 
established
today
may
be
superseded
by
future
developments
in
analytical
chemistry.

1­
10
February
2003
Chapter
1
Exhibit
1­
2.
Relationship
of
Assessment
Document
to
Assessment
of
Detection
and
Quantitation
Limit
Approaches
Event
1,
Develop
a
detailed
plan
for
responding
to
Clause
6
the
Settlement
Agreement:
This
event
was
completed
in
April
2002
when
the
draft
plan
was
revised
to
reflect
peer
review
and
Litigant
comments.

Event
2,
Identify
and
explore
issues
to
be
considered:
The
Settlement
Agreement
identified
six
specific
issues
that
should
be
considered
during
the
assessment
of
detection
and
quantitation
limit
concepts,
and
subjected
to
formal
peer
review.
During
development
of
the
technical
approach,
EPA
identified
a
number
of
other
issues
that
should
be
considered
during
the
assessment.
EPA
listed
and
described
each
of
these
issues
in
the
study
plan
and
noted
that
identification
of
issues
is
likely
to
be
a
dynamic
process,
in
that
as
a
suite
of
issues
is
identified
and
discussed,
other
issues
may
surface.
Finally,
EPA
stated
its
intent
to
prepare
an
 
issue
paper 
that
fully
explained
and
discussed
each
of
the
identified
issues.
Chapter
3
of
this
Assessment
Document
serves
the
function
of
the
issue
paper
described
in
the
plan.

Event
3,
Develop
criteria
against
which
concepts
can
be
evaluated:
After
fully
considering
all
relevant
issues,
EPA
developed
a
suite
of
criteria
that
could
be
used
to
evaluate
the
suitability
of
various
detection
and
quantitation
procedures
for
use
in
CWA
programs.
Chapter
4
of
this
Assessment
Document
provides
and
describes
the
criteria
selected
by
EPA
after
its
consideration
of
all
pertinent
issues.

Event
4,
Evaluate
existing
procedures
for
establishing
detection
and
quantitation
levels:
EPA
evaluated
existing
detection
and
quantitation
limit
concepts
used
or
advanced
1)
by
voluntary
consensus
standards
bodies
(
VCSBs),
2)
in
the
published
literature,
3)
by
EPA.
As
per
the
terms
of
the
Settlement
Agreement,
the
MDL
and
ML
were
explicitly
targeted
for
inclusion.
EPA
committed
to
evaluating
concepts
published
by
ASTM
International
and
ISO
and
to
consider
approaches
and
procedures
offered
by
other
organizations
such
as
the
American
Chemical
Society
(
ACS)
and
the
International
Union
of
Pure
and
Applied
Chemistry
(
IUPAC),
as
well
as
other
approaches
that
have
been
adopted
by
EPA
for
use
in
other
programs
or
that
were
identified
during
EPA s
review
of
the
published
literature.
Chapter
2
describes
the
concepts
that
EPA
evaluated
in
the
assessment.
Where
appropriate,
these
approaches
also
are
discussed
in
context
to
the
issues
that
are
identified
and
discussed
in
Chapter
3.
Chapter
5
presents
the
results
of
EPA s
assessment
of
each
approach
against
the
evaluation
criteria
established
in
Chapter
4.
Appendices
B
and
C
of
this
document
present
additional
details
of
EPA's
assessment
of
each
approach,
using
the
data
described
in
Chapter
1,
Section
1.3.

Event
5,
Develop
and
evaluate
alternative
procedures:
EPA
planned
to
develop
and
evaluate
alternative
procedures
and
modifications
to
existing
procedures
only
if
the
Agency s
assessment
of
existing
procedures
suggested
that
modifications
or
alternatives
to
the
existing
procedures
were
needed.
EPA
noted
that
its
primary
objective
in
developing
such
alternatives
(
or
modifications)
would
be
to
address
deficiencies
noted
in
Event
4
and
improve
the
performance
of
the
procedures
that
best
meet
the
criteria
established
in
Event
3.
In
accordance
with
the
plan
and
with
EPA s
findings
during
the
assessment,
this
Assessment
Document
includes
suggested
modifications
to
the
existing
MDL
and
ML
procedures.

Event
6,
Conduct
peer
review
of
the
Agency s
assessment:
EPA
documented
results
of
the
Agency s
assessment
in
a
draft
Assessment
Document
that
was
completed
in
August,
2002.
EPA
conducted
a
formal
peer
review
of
the
assessment
in
accordance
with
the
Agency s
peer­
review
policies
and
guidance.
The
peer
review
was
performed
by
two
experts
in
the
field
of
analytical
chemistry
and
two
experts
in
the
statistical
aspects
of
analytical
data
interpretation.

Events
7
­
11,
Actions
taken
following
peer
review.
After
considering
peer
review
comments,
EPA
revised
its
assessment
and
the
draft
Assessment
Document
to
reflect
peer
review
comments.
EPA
also
finalized
its
strategy
regarding
the
FR
notices
to
be
published
per
the
terms
of
Settlement
Agreement
Clause
6a
and
took
the
actions
necessary
to
ensure
publication
of
those
notices.

February
2003
1­
11
Chapter
2
Overview
and
History
of
Detection
and
Quantitation
Limit
Approaches
It
is
not
possible
to
measure
the
concentration
of
a
substance
in
water
all
the
way
down
to
zero.
As
an
analogy,
consider
the
following
example:
imagine
measuring
an
object
less
than
16th
of
an
inch
in
length
with
a
ruler
marked
in
1/
16th­
inch
increments.
How
well
can
the
length
of
the
object
be
measured
using
only
the
ruler?
Similar
issues
arise
as
chemists
try
to
measure
ever
smaller
concentrations
of
substances
in
water.
In
response
to
the
challenges
associated
with
measuring
low
concentrations,
chemists
have
defined
numerical
values
that
provide
points
of
reference
for
reporting
and
using
measurement
results.
These
values
are
usually
referred
to
as
detection
and
quantitation
limits.
This
chapter
provides
an
overview
of
detection
and
quantitation
approaches
and
procedures
in
analytical
chemistry
and
their
use
in
Clean
Water
Act
applications.

2.1
Currie s
Call
for
Standardization
Since
1968,
most
of
the
literature
regarding
detection
and
quantitation
has
referenced
the
work
of
Dr.
Lloyd
Currie,
recently
retired
from
the
National
Institutes
of
Science
and
Technology
(
NIST,
formerly
the
National
Bureau
of
Standards).
In
1968,
Currie
published
a
paper
in
which
he
reviewed
the
then
current
state
of
the
art
regarding
detection
and
quantitation,
presented
a
three­
tiered
concept,
and
demonstrated
his
concept
with
operational
equations
for
a
single
laboratory.
In
his
paper,
Currie
reviewed
eight
existing
definitions
for
the
concept
of
detection,
and
reported
that
when
these
eight
operational
definitions
were
applied
to
the
same
data,
they
resulted
in
numerical
values
that
differed
by
nearly
three
orders
of
magnitude.
These
results
made
it
impossible
to
compare
the
detection
capabilities
of
measurement
methods
using
available
publications.
Currie
proposed
standardizing
the
terminology
using
theoretical
definitions
that
he
called
the
critical
value,
the
detection
limit,
and
the
determination
limit.
(
In
1995,
writing
on
behalf
of
International
Union
of
Pure
and
Applied
Chemistry
(
IUPAC),
Currie
used
the
term
 
quantification
limit 
instead
of
his
original
term
 
determination
limit. 
Substantial
agreement
with
the
International
Organization
for
Standardization
(
also
known
as
"
ISO")
on
the
meaning
and
language
of
detection
and
quantitation
was
achieved
later,
although
some
 
subtle
differences
in
perspective 
remain
[
Currie,
2000]).
His
purpose
for
these
definitions
was
to
create
a
system
in
which
the
standard
documentation
of
any
measurement
method
would
include
a
statement
of
capabilities
that
were
directly
comparable
to
any
other
method
for
measuring
the
same
substance.

Currie
used
terms
from
statistical
decision
theory
as
the
basis
for
his
three­
tiered
system.
In
1968
and
1995,
Currie
defined
the
critical
value
as
the
measured
value
at
which
there
is
a
small
chance
that
the
concentration
in
the
sample
is
zero.
Consequently,
any
measured
result
greater
than
or
equal
to
the
critical
value
is
considered
evidence
that
the
sample
contains
the
substance
of
interest.
Currie
was
careful
to
emphasize
that
the
decision
as
to
whether
the
substance
has
been
detected
is
made
bycomparing
the
measurement
result
to
the
critical
value.
Figure
2­
1
shows
a
critical
value
selected
such
that
measurements
greater
than
the
critical
value
have
less
than
a
1%
chance
of
being
associated
with
a
sample
that
does
not
contain
the
substance
of
interest.
The
area
under
the
curve
to
the
right
of
the
critical
value
represents
the
probability
that
a
Figure
2­
1
measured
value
will
exceed
the
critical
value.
The
area
February
2003
2­
1
Assessment
of
Detection
and
Quantitation
Approaches
under
the
curve
to
the
left
of
the
critical
value
represents
the
(
much
greater)
probability
of
observing
a
value
that
is
less
than
the
critical
value
when
the
true
concentration
is
zero.

Currie
(
1968
and
1995)
used
the
term
detection
limit
to
refer
to
a
true
concentration
that
has
a
high
probability
of
generating
measured
values
greater
than
the
critical
value.
That
is,
measurements
on
samples
that
contain
concentrations
equal
to
the
detection
limit
have
a
high
probability
of
exceeding
the
critical
value
and
are,
therefore,
unlikely
to
result
in
a
decision
that
the
substance
is
not
detected
in
the
sample.
In
Currie s
concept,
the
critical
value
and
the
detection
limit
are
related
and
functionally
dependent,
but
it
is
clear
that
the
detection
decision
is
made
on
the
basis
of
comparing
sample
by
sample
measurements
to
the
critical
value.
While
Currie s
terminology
is
consistent
with
standard
statistical
decision
theory,
it
is
in
all
likelihood
responsible
for
a
great
deal
of
confusion
among
chemists
and
others
who
may
associate
the
term
 
limit 
with
some
sort
of
decision
point.
Currie
(
1995)
states:
 
The
single,
most
important
application
of
the
detection
limit
is
for
planning.
It
allows
one
to
judge
whether
the
CMP
Figure
2­
2
(
Chemical
Measurement
Process)
under
consideration
is
adequate
for
the
detection
requirements. 
Figure
2­
2
shows
a
detection
limit
selected
such
that
99%
of
the
measurements
on
a
sample
containing
this
concentration
are
expected
to
be
above
the
critical
value.
The
bell­
shaped
curve
centered
at
the
detection
limit
illustrates
how
likely
various
measurement
responses
are
when
the
concentration
of
the
substance
in
a
sample
is
equal
to
the
detection
limit.
That
is,
the
figure
shows
the
probability
density
of
values
measured
in
a
sample
with
a
true
concentration
equal
to
the
detection
limit.
The
area
under
the
curve
to
the
left
of
the
critical
value
is
equal
to
1%
of
the
total
area,
while
the
area
to
the
right
is
equal
to
99%.

Currie
(
1968,
1995)
defined
the
determination
limit,
later
renamed
the
quantification
limit,
as
(
quoting
Currie,
1995)
 
performance
characteristics
that
mark
the
ability
of
a
CMP
to
adequately
 
quantify 
an
analyte. 
Quantification
limits
 
serve
as
benchmarks
that
indicate
whether
the
CMP
can
adequately
meet
the
measurement
needs.
The
ability
to
quantify
is
generally
expressed
in
terms
of
the
signal
or
analyte
(
true)
value
that
will
produce
estimates
having
a
specified
relative
standard
deviation
(
RSD)
commonly
10%. 
This
translates
into
a
quantification
limit
equal
to
a
multiplier
of
10
times
the
standard
deviation
(
a
measure
of
measurement
variability)
at
the
limit.
The
multiplier
of
10
(
equal
to
the
inverse
of
the
10%
RSD)
is
arbitrary,
but
has
been
used
widely.
IUPAC
selected
10
as
a
 
default
value 
(
Currie,
1995),
implying
other
values
are
possible.
In
papers
published
in
1980
and
1983,
the
American
Chemical
Society s
Committee
on
Environmental
Improvement
also
recommended
the
use
of
a
multiplier
of
10
for
determining
quantitation
limits
(
see
MacDougall,
et
al.,
1980
and
Keith,
et
al.,
1983).
Measured
concentrations
greater
than
the
quantitation
limit
are
considered
to
be
reliable
by
chemists,
although
from
a
statistical
perspective,
any
measured
value,
along
with
knowledge
of
the
precision
of
the
measurement,
is
useful.

Currie s
goal
of
having
method
developers
publish
directly
comparable
descriptions
of
detection
and
quantitation
capability
remains
elusive
more
than
thirty
years
after
publication
of
his
first
paper
on
this
topic.
Even
if
Currie s
three­
tiered
concept
were
used,
the
treatment
of
related
issues
causes
difficulty
in
comparing
methods.
Some
of
these
issues
include
interlaboratory
variability,
selection
of
appropriate
statistical
models,
design
of
detection
and
quantitation
capability
studies,
and
statistical
prediction
and
tolerance.
These
and
other
issues
are
discussed
in
Chapter
3
of
this
document.

2­
2
February
2003
Chapter
2
2.2
Development
of
the
MDL
and
ML
as
Practical
Embodiments
of
Currie s
Proposal
In
1981,
staff
at
EPA s
Environmental
Monitoring
and
Support
Laboratory
in
Cincinnati,
Ohio,
published
a
procedure
for
determining
what
they
referred
to
as
a
method
detection
limit
(
MDL)
(
Glaser
et
al.,
1981).
The
MDL
functions
as
a
practical,
general
purpose
version
of
Currie s
critical
value.
The
MDL
was
subsequently
promulgated
for
use
in
CWA
programs
on
October
26,
1984
(
49
FR
43234)
at
40
CFR
part
136,
Appendix
B.
Prior
to
formal
development
of
the
MDL
in
1981,
the
EPA
Office
of
Water
had
included
the
term
 
minimum
level 
(
ML)
or
 
minimum
level
of
quantitation 
in
some
methods
for
analysis
of
organic
pollutants.
These
methods
were
proposed
on
December
3,
1979
and
subsequently
promulgated
on
October
26,
1984,
along
with
the
MDL.
Additional
information
about
the
MDL
and
ML
is
provided
below
in
Sections
2.2.1
and
2.2.2.

2.2.1
Method
Detection
Limit
Conscious
of
the
definitions
provided
by
Currie
and
others,
Glaser
et
al.
(
1981)
stated
 [
t]
he
fundamental
difference
between
our
approach
to
detection
limit
and
former
efforts
is
the
emphasis
on
the
operational
characteristics
of
the
definition.
[
The]
MDL
is
considered
operationally
meaningful
only
when
the
method
is
truly
in
the
detection
mode,
i.
e.,
[
the]
analyte
(
the
substance
of
interest)
must
be
present. 
Expanding
on
this
reasoning,
Glaser
et
al.
(
1981)
developed
MDL
estimates
for
methods
that
produce
a
result
of
zero
for
blanks,
such
as
EPA
Methods
624
and
625
for
determination
of
organic
pollutants
by
gas
chromatography/
mass
spectrometry
(
GC/
MS).
Blank
variability
exists,
whether
or
not
it
can
be
detected
by
measurement
processes.
Failure
to
detect
this
variability
may
be
attributed
to
insufficient
sensitivity
of
the
measurement
process
or,
as
is
the
case
with
some
measurement
processes,
thresholds
that
are
built
into
equipment
which
censor
measurements
below
certain
levels.
Currie s
critical
value
is
dependent
on
the
ability
to
estimate
measurement
variability
of
blank
samples.
In
cases
where
the
substance
is
not
detected
in
direct
measurements
on
blanks,
an
alternative
approach
to
estimating
blank
variability
must
be
used.
One
option
is
to
estimate
measurement
variability
at
concentrations
that
represent
the
lowest
possible
levels
where
a
signal
can
be
detected.
This
is
the
basic
approach
of
the
MDL,
which
provides
a
general
purpose,
straightforward,
operational
procedure
for
estimating
a
quantity
analogous
to
the
Currie
critical
value
when
measurement
processes
applied
to
blank
samples
do
not
produce
detectable
signals.
More
complex
statistical
procedures
for
estimating
blank
variability
are
possible
and
may
be
preferable
from
a
rigorous
statistical
perspective,
but
the
MDL
has
been
found
to
be
satisfactory
by
chemists
in
a
wide
range
of
applications.

In
1984,
the
MDL
became
a
regulatory
option
for
wastewater
discharge
permits
authorized
under
the
Clean
Water
Act.
To
determine
the
MDL,
at
least
seven
replicate
samples
with
a
concentration
of
the
pollutant
of
interest
near
the
estimated
detection
capabilities
of
the
method
are
analyzed.
The
standard
deviation
among
the
replicate
measurements
is
determined
and
multiplied
by
the
t­
distribution
for
n­
1
degrees
of
freedom
(
in
the
case
of
7
replicates,
the
multiplier
is
3.143,
which
is
the
value
for
6
degrees
of
freedom).
The
decision
to
base
the
MDL
on
a
minimum
of
seven
replicates
reflected
a
consensus
among
EPA
chemists
and
statisticians
that
a
requirement
of
seven
replicates
is
not
overly
burdensome
for
laboratories
and
that
laboratories
could
reasonably
be
expected
to
perform
the
analyses
in
a
single
batch.

Both
the
MDL
concept
and
the
specific
definition
at
part
136
have
been
used
within
EPA
by
the
Office
of
Ground
Water
and
Drinking
Water
(
OGWDW),
the
Office
of
Solid
Waste
(
OSW),
the
Office
of
Emergency
and
Remedial
Response
(
OERR),
and
others.
The
MDL
also
has
been
used
outside
of
EPA
in
Standard
Methods
for
the
Examination
of
Water
and
Wastewater,
published
jointly
by
the
American
Public
Health
Association
(
APHA),
the
American
Water
Works
Association
(
AWWA),
and
the
Water
Environment
Federation
(
WEF),
and
in
methods
published
by
the
ASTM
International,
and
elsewhere.

February
2003
2­
3
Assessment
of
Detection
and
Quantitation
Approaches
Despite
such
widespread
use,
some
members
of
regulated
industry
and
others
have
claimed
that
the
MDL
is
a
less
than
ideal
concept
for
detection.
Specifically,
critics
have
faulted
the
MDL
because:

 
There
are
some
inconsistencies
between
the
definition
and
the
procedure
 
It
does
not
account
explicitly
for
false
negatives
 
It
does
not
account
for
bias
 
A
prediction
or
tolerance
limit
adjustment
is
not
provided,
and
 
It
does
not
account
for
interlaboratory
variability
These
issues
are
discussed
later
in
this
document.

2.2.2
Minimum
Level
of
Quantitation
The
minimum
level
of
quantitation
(
ML)
was
originally
proposed
on
December
5,
1979
(
44
FR
69463)
in
footnotes
to
Table
2
of
EPA
Method
624
and
to
Tables
4
and
5
of
EPA
Method
625.
The
ML
was
defined
as
the
"
level
at
which
the
entire
analytical
system
must
give
recognizable
mass
spectra
and
acceptable
calibration
points"
(
in
the
footnote
to
Table
2
in
Method
624)
and
as
the
"
level
at
which
the
entire
analytical
system
must
give
mass
spectral
confirmation"
(
in
the
footnotes
to
Tables
4
and
5
in
EPA
Method
625).

Between
1980
and
1984,
EPA
also
developed
Methods
1624
and
1625
and
promulgated
these
methods
along
with
the
final
versions
of
EPA
Methods
624
and
625
on
October
26,
1984
(
49
FR
43234).
The
definitions
of
the
ML
in
the
promulgated
versions
of
EPA
Methods
1624
and
1625
were
the
"
level
at
which
the
analytical
system
shall
give
recognizable
mass
spectra
(
background
corrected)
and
acceptable
calibration
points"
(
in
footnote
2
to
Table
2
in
Method
1624)
and
as
the
"
level
at
which
the
entire
GC/
MS
system
must
give
recognizable
mass
spectra
(
background
corrected)
and
acceptable
calibration
points"
(
in
footnotes
2
to
Tables
3
and
4
in
Method
1625).

As
EPA
developed
additional
methods
over
the
next
decade,
the
definition
of
the
ML
was
generalized
to
"
the
lowest
level
at
which
the
entire
analytical
system
must
give
a
recognizable
signal
and
acceptable
calibration
point
for
the
analyte"
(
see,
e.
g.,
Section
24.2
of
EPA
Method
1613
at
40
CFR
part
136,
Appendix
A).
In
generating
actual
numerical
values
for
MLs,
the
lowest
calibration
point
was
estimated
from
method
development
studies
and
included
in
the
methods,
although
a
specific
calculation
algorithm
was
not
used.
EPA
methods
that
include
the
ML
generally
specify
the
number
of
calibration
standards
to
be
used
and
the
concentrations
of
those
standards.
As
a
result,
laboratories
using
those
methods
calibrate
their
analytical
systems
with
a
multi­
point
calibration
(
i.
e.,
calibrate
using
a
series
of
standards
at
different
concentrations
over
the
range
of
the
instrument)
that
includes
a
standard
at
the
lowest
calibration
point
listed
in
the
method
(
i.
e.,
the
ML).

In
response
to
a
need
top
establish
a
compliance
evaluation
threshold
when
the
water
quality­
based
permit
limit
is
below
the
detection
limit
of
the
most
sensitive
analytical
method
published
at
40
CFR
part
136,
EPA
refined
the
definition
of
the
ML
in
1994
as
10
times
the
same
standard
deviation
used
to
calculate
the
MDL1
.
Because
the
MDL
is
commonly
determined
as
3.14
times
the
standard
deviation
of
seven
replicate
measurements,
the
ML
was
commonly
calculated
as
3.18
times
the
MDL.
(
The
figure
of
3.18
was
derived
by
dividing
10
by
3.14;
if
more
than
7
replicates
were
used
to
determine
the
MDL,

1The
refined
definition
of
the
ML
first
appeared
in
EPA's
1994
draft
National
Guidance
for
the
Permitting,
Monitoring,
and
Enforcement
of
Water
Quality­
based
Effluent
Limitations
Set
Below
Analytical
Detection/
Quantitation
Levels
The
draft
guidance
was
very
controversial
and
never
finalized.
However,
the
refined
definition
of
the
ML
has
remained
in
use
or
newer
analytical
methods.

2­
4
February
2003
Chapter
2
both
the
MDL
and
the
ML
multipliers
are
adjusted
accordingly,
based
on
values
from
the
t­
distribution.)
This
calculation
makes
the
ML
analogous
to
Currie s
quantification
limit
and
the
American
Chemical
Society s
limit
of
quantitation
(
LOQ),
which
is
defined
as
ten
times
the
standard
deviation
of
replicate
or
low
concentration
measurements
(
MacDougall,
et
al.,
1980
and
Keith,
et
al.,
1983).

To
simplify
implementation
of
the
ML,
the
definition
also
was
expanded
to
state
that
the
calculated
ML
is
rounded
to
the
whole
number
nearest
to
(
1,
2,
or
5),
times
10n
,
where
n
is
an
integer.
The
reason
for
this
simplification
is
that
calibration
of
an
analytical
system
at
some
exact
number
(
e.
g.,
6.27)
is
difficult
and
prone
to
error,
whereas
rounding
to
the
whole
number
nearest
to
(
1,
2,
or
5)
x
10n
provides
a
practicable
value.
The
most
recent
definition
of
the
ML
is
"
the
lowest
level
at
which
the
entire
analytical
system
must
give
a
recognizable
signal
and
acceptable
calibration
point
for
the
analyte.
It
is
equivalent
to
the
concentration
of
the
lowest
calibration
standard,
assuming
that
all
method­
specified
sample
weights,
volumes,
and
cleanup
procedures
have
been
employed.
The
ML
is
calculated
by
multiplying
the
MDL
by
3.18
and
rounding
the
result
to
the
number
nearest
to
(
1,
2,
or
5)
x
10n
,
where
n
is
an
integer,"
and
this
definition
was
contained
in
the
version
of
EPA
Method
1631
that
was
promulgated
on
June
8,
1999
(
64
FR
30417)
(
see
Section
17.8
of
EPA
Method
1631
Revision
B).

The
ML
will
generally
be
somewhat
lower
than
Currie s
quantitation
limit,
even
when
similar
sample
sizes
and
estimation
procedures
are
used.
This
is
because
the
standard
deviation
used
to
calculate
the
ML
will
generally
be
smaller
than
the
standard
deviation
at
the
lowest
concentration
at
which
the
relative
standard
deviation
is
10%.
This
is
due
to
the
fact
that,
in
almost
all
cases,
standard
deviation
is
non­
decreasing
with
increasing
concentration,
e.
g.,
it
generally
tends
to
increase
as
concentration
increases.

Although
the
ML
has
been
used
successfully
in
EPA
methods
for
more
than
20
years,
some
members
of
the
regulated
industry
and
others
have
claimed
that
the
ML
is
less
than
an
ideal
concept
for
quantitation
because
it:

C
Does
not
account
for
interlaboratory
variability,
and
C
Is
based
on
a
multiple
of
the
standard
deviation
rather
than
a
fitted
model
These
concerns
are
discussed
later
in
this
document.

2.3
Approaches
Advanced
by
Other
Organizations
To
expand
somewhat
on
Currie
(
1968),
standardizing
the
operational
definitions
of
detection
and
quantitation
would
benefit
society
by
making
it
easier
to
compare
and
select
measurement
methods
based
on
low­
level
measurement
capability
and
requirements
in
particular
applications.
Unfortunately,
in
spite
of
agreement
on
general
principles
and
definitions
advanced
by
Currie
and
his
supporters,
consensus
on
procedures
that
would
result
in
comparable
detection
and
quantitation
estimates
has
been
elusive.
Sections
2.3.1
­
2.3.3,
which
are
by
no
means
an
exhaustive
list
of
the
various
approaches
advanced
to
date,
highlight
approaches
that
have
been
most
widely
advanced
for
environmental
applications.

2.3.1
EPA
Approaches
Over
the
years,
a
number
of
detection
and
quantitation
limit
approaches
have
been
developed,
suggested,
or
used
by
EPA
among
the
various
organizations
charged
with
responding
to
differing
program
mandates.
In
part,
this
situation
reflects
actual
differences
in
the
mandates,
and
in
part,
it
reflects
the
fact
that
no
concept
advanced
to
date
has
emerged
as
a
clear
 
winner 
that
meets
all
needs
for
all
people.
Approaches
that
have
been
used
or
suggested
by
EPA
include
the:

February
2003
2­
5
Assessment
of
Detection
and
Quantitation
Approaches
 
MDL
and
ML
(
described
in
Sections
2.2.1
and
2.2.2)
 
Instrument
detection
limit
(
IDL)
 
Practical
quantitation
limit
(
PQL)
 
Estimate
quantitation
limit
(
EQL)
 
Contract­
required
detection
limit
(
CRDL)
and
contract­
required
quantitation
limit
(
CRQL)

Instrument
Detection
Limit:
EPA
methods
for
analysis
of
metals
have
historically
included
an
instrument
detection
limit,
or
IDL.
Functionally,
the
IDL
is
similar
to
the
MDL
except
that
the
IDL
includes
temporal
variability
(
it
is
determined
on
3
non­
consecutive
days)
and
does
not
include
all
sample
processing
steps
(
the
IDL
characterizes
the
detection
capabilities
of
the
instrument
as
opposed
to
the
method).
Because
IDLs
do
not
reflect
the
entire
measurement
process
and,
for
the
most
part,
have
been
used
only
for
measurement
of
metals,
EPA
did
not
consider
the
IDL
as
a
potential
alternate
to
the
MDL
when
conducting
the
assessment
described
in
this
Assessment
Document.

Practical
Quantitation
Limit:
The
practical
quantitation
limit,
or
PQL,
was
established
in
the
1980s
by
EPA s
drinking
water
program
as
the
lowest
concentration
at
which
reliable
measurements
can
be
made.
The
PQL
is
defined
as
"
the
lowest
concentration
of
an
analyte
that
can
be
reliably
measured
within
specified
limits
of
precision
and
accuracy
during
routine
laboratory
operation
conditions"
(
52
FR
25690,
July
8,
1987).
The
PQL
is
a
means
of
integrating
information
on
the
performance
of
approved
analytical
methods
into
the
development
of
a
drinking
water
regulation.
The
PQL
incorporates
the
following:

 
Quantitation,
 
Precision
and
bias,
 
Normal
operations
of
a
laboratory,
and
 
The
programmatic
need
to
have
a
sufficient
number
of
laboratories
available
to
conduct
compliance
monitoring
analyses
of
drinking
water
samples.

EPA
uses
two
main
approaches
to
determine
a
PQL
for
an
analyte
under
the
Safe
Drinking
Water
Act
(
SDWA).
One
approach
is
to
use
the
data
from
Water
Supply
(
WS)
studies
(
e.
g.,
laboratory
performance
evaluation
studies
conducted
by
the
Agency
as
part
of
the
certification
process
for
drinking
water
laboratories).
The
PQL
is
established
at
the
concentration
at
which
at
least
75%
of
the
laboratories
in
the
study,
or
the
subset
representing
EPA
Regional
laboratories
and
state
laboratories,
obtain
results
within
some
predetermined
percentage
of
the
true
value
of
the
test
samples
(
e.
g.,
±
30%).
This
approach
is
used
in
most
cases
when
WS
data
are
available
to
calculate
a
PQL.
The
WS
data
approach
was
used
to
determine
the
PQLs
for
Phase
V
inorganic
chemicals
such
as
antimony,
beryllium,
cyanide,
nickel
and
thallium
(
July
17,
1992;
57
FR
31776),
as
well
as
many
other
contaminants
regulated
under
the
SDWA.

In
the
absence
of
WS
data,
the
second
approach
that
EPA
uses
is
the
multiplier
method.
In
this
approach,
the
PQL
is
calculated
by
multiplying
the
EPA­
derived
MDL
by
a
factor
between
5
and
10.
The
exact
multiplier
varies
and
sometimes
depends
on
the
degree
of
concern
about
the
specific
contaminant
(
i.
e.,
based
on
a
human
health
risk
assessment
for
consumption
of
drinking
water).

Application
of
the
PQL
has
been
traditionally
limited
to
drinking
water.
Furthermore,
the
PQL
may
not
be
related
to
the
lowest
quantitation
limit
because
1)
the
PQL
is
associated
with
the
analyte
and
may
have
been
determined
irrespective
of
a
specific
analytical
method
(
e.
g.,
using
data
from
a
variety
of
methods
approved
for
that
analyte
at
40
CFR
part
141),
2)
the
performance
evaluation
(
PE)
samples
from
which
it
is
derived
contain
pollutant
concentrations
that
may
be
well
above
the
true
limit
of
quantitation,
3)
the
multiplier
used
to
calculate
a
PQL
when
PE
data
are
not
available
is
somewhat
dependent
on
concerns
about
risks
from
human
exposure
to
contaminants
in
drinking
water,
and
4)
the
resulting
PQLs
may
be
too
high
for
purposes
other
than
the
Safe
Drinking
Water
Act
(
e.
g.,
other
EPA
programs).
In
addition,
because
EPA
has
privatized
the
performance
evaluation
program
for
drinking
water
laboratory
2­
6
February
2003
Chapter
2
certification,
it
is
not
yet
clear
that
appropriate
data
will
be
available
in
the
future.
Based
on
these
facts,
EPA
did
not
conduct
an
assessment
of
the
PQL
for
CWA
applications.

In
the
late
1980s,
EPA s
Office
of
Solid
Waste
(
OSW)
adopted
a
different
version
of
the
PQL
as
a
quantitation
limit.
No
procedure
for
establishing
the
limits
was
given;
instead
values
were
extrapolated
from
the
Contract
Laboratory
Program
CRQLs
(
see
below).
Since
1994,
OSW
has
actively
removed
the
term
"
PQL"
from
its
revised
methods,
replacing
it
with
the
term
"
estimated
quantitation
limit"
(
EQL).
The
term
PQL
and
the
original
numerical
values
remain
in
a
few
older
OSW
guidance
documents.

Estimated
Quantitation
Limit:
EPA's
Office
of
Solid
Waste
has
defined
the
EQL
as:

"
The
lowest
concentration
that
can
be
reliably
achieved
within
specified
limits
of
precision
and
accuracy
during
routine
laboratory
operating
conditions.
The
EQL
is
generally
5
to
10
times
the
MDL.
However,
it
may
be
nominally
chosen
within
these
guidelines
to
simplify
data
reporting.
For
many
analytes
the
EQL
analyte
concentration
is
selected
as
the
lowest
non­
zero
standard
in
the
calibration
curve.
Sample
EQLs
are
highly
matrix
dependent.
The
EQLs
in
SW­
846
are
provided
for
guidance
and
may
not
always
be
achievable."
(
see
SW­
846,
Chapter
1).

As
noted
in
most
newer
SW­
846
methods,
the
EQLs
are
provided
for
guidance
and
may
not
always
be
achievable.
Because
the
EQL
is
not
rigorously
defined
and
is
guidance,
because
the
EQL
may
be
based
on
the
MDL,
and
because
the
EQL
can
be
the
lowest
calibration
point
and
would,
therefore,
overlap
the
ML,
EPA
did
not
consider
the
EQL
further
in
its
assessment
of
detection
and
quantitation
approaches.

Contract­
Required
Detection
and
Quantitation
Limits:
EPA s
Superfund
program
has
adopted
the
use
of
contractually­
required
limits
that
are
based
on
consensus
among
analytical
chemists
about
levels
that
can
realistically
be
achieved
in
commercial
laboratories
using
a
contractually­
specified
method.
Laboratories
that
participate
in
the
Superfund
Contract
Laboratory
Program
(
CLP)
are
required
to
demonstrate
that
they
can
achieve
the
specified
CRDLs
and
CRQLs.
The
CRDLs
are
consensus
values
that
apply
to
the
analyses
of
metals
using
CLP
methods.
The
CRQLs
apply
to
organic
analytes
and
are
based
on
the
concentration
of
the
lowest
non­
zero
calibration
standard
specified
in
the
CLP
methods,
in
a
fashion
analogous
to
the
original
derivation
of
the
ML.
Because
few
CWA
applications
involve
the
use
of
the
CLP
methods,
EPA
did
not
consider
the
CRDL
or
the
CRQL
as
viable
alternatives
to
the
MDL
and
ML
when
conducting
the
assessment
described
in
this
document.

2.3.2
Industry­
supported
Approaches
The
regulated
community
has
demonstrated
an
interest
in
detection
limit
approaches
since
EPA
first
promulgated
the
MDL
and
ML
for
use
in
CWA
programs
in
1984
(
49
FR
43234).
As
part
of
that
rule,
EPA
promulgated
Methods
601
through
613,
624,
625,
1624,
and
1625
for
organic
compounds
at
40
CFR
part
136,
Appendix
A
and
EPA
Method
200.7
for
metals
by
inductively
coupled
plasma
spectrometry
(
ICP)
at
40
CFR
part
136,
Appendix
C.
EPA
also
promulgated
the
MDL
procedure
at
40
CFR
part
136,
Appendix
B.
The
Virginia
Electric
Power
Company
(
VEPCO)
brought
suit
against
EPA,
challenging
the
Agency's
use
of
the
MDL
in
the
promulgated
methods.
In
a
settlement,
EPA
agreed
that
the
MDL
would
be
applicable
only
to
the
600­
series
organic
methods,
as
these
methods
already
contained
MDL
values;
i.
e.,
it
would
not
be
applicable
to
EPA
Method
200.7.
The
settlement
agreement
did
not
preclude
future
use
of
the
MDL
by
EPA
or
the
right
of
VEPCO
to
bring
suit
in
such
future
use.

After
the
VEPCO
settlement,
the
regulated
community,
mainly
through
efforts
of
the
Electric
Power
Research
Institute
(
EPRI),
remained
involved
in
detection
and
quantitation
approaches
to
be
used
under
EPA's
CWA
programs.
The
first
approaches
that
industry
advanced
were
the
compliance
February
2003
2­
7
Assessment
of
Detection
and
Quantitation
Approaches
monitoring
detection
level
(
CMDL)
and
compliance
monitoring
quantitation
level
(
CMQL)
(
Maddalone,
et
al.,
1993).
The
CMDL/
CMQL
were
variants
of
EPA's
MDL/
ML
that
attempted
to
adjust
for
interlaboratory
variability.

The
regulated
community
continued
its
efforts
to
develop
alternative
detection
and
quantitation
approaches
with
development
of
the
"
alternate
minimum
level"
(
AML)
in
the
mid­
1990s
(
Gibbons
et
al.,
1997).
The
AML
is
based
on
statistical
modeling
of
standard
deviation
versus
concentration,
which
requires
large
amounts
of
data.

Most
recently,
the
regulated
community
has
funded
development
of
the
interlaboratory
detection
estimate
(
IDE)
and
interlaboratory
quantitation
estimate
(
IQE).
The
IDE
and
IQE
have
been
balloted
and
approved
by
ASTM's
Committee
D­
19
for
water
as
Standard
Practices
D­
6091
and
D­
6512,
respectively.
These
approaches
take
into
account
nearly
all
sources
of
variability
to
arrive
at
detection
and
quantitation
limits
that
are
higher,
on
average,
than
the
limits
produced
by
other
approaches
(
see
Appendix
C
of
this
Assessment
Document).
Because
the
regulated
community
has
shifted
support
from
the
CMDL/
CMQL
and
the
AML
to
the
IDE
and
IQE,
and
because
EPA
is
not
aware
of
other
organizations
that
currently
advocate
the
earlier
approaches,
EPA
did
not
consider
industry
approaches
other
than
the
IDE/
IQE
in
its
assessment
of
possible
alternatives
to
the
MDL
and
ML.

As
with
all
other
approaches
advocated
to
date,
the
IDE
and
IQE
have
fallen
short
of
being
ideal
approaches
for
detection
and
quantitation
for
all
organizations
and
applications.
To
date,
EPA
is
not
aware
of
a
demonstrated
implementation
of
the
IDE
or
IQE
in
the
development
of
an
analytical
method.
Specific
concerns
that
have
been
raised
about
the
IDE
and
IQE
are
that:

 
They
contain
an
allowance
for
false
negatives
that
may
be
inappropriate,
 
The
IDE
and
IQE
are
based
on
the
use
of
prediction
and/
or
tolerance
intervals,
which
may
be
inappropriate,
 
The
IDE
and
IQE
require
a
large
amount
of
data
in
order
to
be
able
to
model
variability
versus
concentration,
including
data
generated
in
multiple
laboratories,
and
 
The
complex
statistical
procedures
involved
in
calculating
an
IDE
and
IQE
would
place
a
heavy
burden
on
the
analytical
chemists
that
typically
develop,
modify,
and
use
methods.

These
concerns
are
discussed
in
detail
later
in
this
document.

2.3.3
Approaches
Advocated
by
the
Laboratory
Community
and
Voluntary
Consensus
Standards
Bodies
In
1980
(
MacDougall
et
al.,
1980)
and
1983
(
Keith
et
al.,
1983),
the
American
Chemical
Society's
Committee
on
Environmental
Improvement
(
CEI)
advanced
approaches
for
the
Limit
of
Detection
(
LOD)
and
Limit
of
Quantitation
(
LOQ).
The
ACS
LOD
is
defined
as
the
lowest
concentration
level
that
can
be
determined
to
be
statistically
different
from
a
blank.
The
recommended
value
for
the
LOD
is
three
times
the
standard
deviation
of
replicate
measurements
of
a
blank
or
low­
level
sample.
The
LOD
is
roughly
equivalent
to
the
MDL
in
numerical
terms
and
conceptually
equivalent
to
Currie s
critical
value.

The
ACS
LOQ
is
defined
as
the
level
above
which
quantitative
results
may
be
obtained
with
a
specified
degree
of
confidence.
The
recommended
value
for
the
LOQ
is
10
times
the
standard
deviation
of
replicate
measurements
of
blanks
or
low­
level
samples.
Because
the
LOD
and
LOQ
are
still
used
by
the
analytical
community,
they
have
been
included
in
EPA's
reassessment
of
detection
and
quantitation
approaches.

2­
8
February
2003
Chapter
2
In
the
mid­
1980s,
the
ACS
CEI
introduced
the
concept
of
the
Reliable
Detection
Limit
(
RDL)
and
the
Reliable
Quantitation
Limit
(
RQL).
The
RDL
and
RQL
were
attempts
at
simplification
of
the
LOD
and
LOQ.
Both
the
RDL
and
the
RQL
involved
applying
a
multiplier
to
the
standard
deviation
derived
from
replicate
measurements
of
a
low­
level
sample.
Neither
concept
received
acceptance
by
the
analytical
community.
Because
the
RDL
and
RQL
are
no
longer
being
advanced
by
ACS,
they
were
not
considered
for
evaluation
in
EPA's
assessment
of
detection
and
quantitation
approaches.

In
1999
(
Currie,
1999a
and
1999b),
IUPAC
and
ISO
reached
substantial
agreement
on
the
terminology
and
approaches
documented
by
Currie
(
1995),
although
 
subtle
differences
in
perspective 
of
the
organizations
remain
(
Currie,
2000).
IUPAC
and
ISO
have
not,
to
date,
published
methods
that
include
limits
reflecting
these
standards.
Similarly,
although
ASTM
International
adopted
the
IDE
in
1997
and
the
IQE
in
2000,
ASTM
International
has
not
included
any
IDE
or
IQE
values
in
methods
approved
through
the
ASTM
ballot
process.
On
the
other
hand,
ISO
and
ASTM
International
have
published
methods
that
employ
the
MDL.
Because
IUPAC
and
ISO
have
approved
the
critical
value,
detection
limit,
and
quantification
limit,
and
because
ASTM
International
has
approved
through
ballot
the
IDE
and
IQE,
EPA
has
included
these
approaches
in
its
assessment
of
detection
and
quantitation
approaches.

At
the
ACS
Annual
Meeting
held
in
August,
2002,
CEI
members
discussed
the
issue
of
detection
and
quantitation,
with
the
objective
of
determining
if
the
LOD
and
LOQ
approaches
should
be
re­
visited.
At
that
meeting,
several
members
suggested
that
the
committee
consider
adopting
a
sample­
specific
detection
limit
approach
in
which
the
ratio
of
instrument
signal
to
background
noise
is
used
to
estimate
a
detection
limit
for
each
analyte
in
each
sample
analyzed.
EPA
did
not
include
the
signal­
to­
noise
ratio
concept
in
this
assessment
because
its
application
is
limited
to
specific
types
of
measurement
techniques,
such
as
gas
chromatography/
mass
spectrometry.
Limitations
of
this
concept
for
use
in
general
environmental
chemistry
are
best
illustrated
by
the
fact
that
it
would
not
apply
to
any
of
the
techniques
traditionally
used
to
determine
the
"
conventional
pollutants"
cited
in
the
Clean
Water
Act
(
the
only
pollutants
cited
by
name
in
the
Act),
i.
e.,
biochemical
oxygen
demand
(
BOD),
total
suspended
solids
(
TSS),
fecal
coliforms,
and
pH.

2.3.4
Approaches
Advocated
by
Other
U.
S.
Government
Agencies
and
Other
Governments
Within
the
U.
S.,
EPA
found
that
other
Federal
agencies
tend
to
rely
on
the
detection
and
quantitation
limit
approaches
described
above
or
on
variants
of
those
procedures.
For
example,
the
USGS
National
Water
Quality
Laboratory
(
NWQL)
began
using
the
EPA
MDL
procedure
in
1992.
USGS
has
since
developed
a
variant
of
the
MDL
called
the
long­
term
MDL
(
LT­
MDL)
that
employs
at
least
24
spiked
samples
prepared
and
analyzed
by
multiple
analysts
on
multiple
instruments
over
a
6­
to
12­
month
period.
(
Unlike
EPA
programs
that
rely
on
hundreds
of
commercial,
Federal,
state,
and
local
laboratories
for
sample
analysis,
nearly
all
samples
analyzed
for
USGS
programs
are
analyzed
by
the
USGS
National
Water
Quality
Laboratory.)
As
described
by
USGS,
the
long­
term
MDL
is
based
on
many
of
the
same
fundamental
assumptions
as
the
MDL,
namely:

1.
Normal
data
distribution,
2.
Constant
standard
deviation,
and
3.
Best­
case
detection
condition
(
because
LT­
MDLs
typically
are
determined
by
spiking
the
analyte
in
a
clean
matrix,
e.
g.,
reagent
water).

The
primary
differences
between
the
EPA
MDL
and
the
USGS
LT­
MDL
are
the
longer
time
period
and
mixing
of
instruments
and
analysts
(
Oblinger
Childress,
et
al.,
1999).
Because
the
MDL
and
LT­
MDL
approaches
otherwise
are
so
similar,
EPA
did
not
evaluate
the
long­
term
MDL
approach
in
this
February
2003
2­
9
Assessment
of
Detection
and
Quantitation
Approaches
assessment.
Instead,
EPA
considered
the
underlying
differences
between
the
two
approaches
(
namely
the
effects
of
temporal,
instrument,
and
analyst
variability)
in
its
assessment
of
issues
(
see
Chapter
3).

Outside
the
U.
S.,
EPA
found
that
the
European
Union
(
EU)
relies
on
the
terminology
and
conventions
developed
by
Currie,
IUPAC,
and
others
(
Eurachem,
2000).
The
EU
advocates
reporting
all
results
along
with
an
estimate
of
the
uncertainty
associated
with
each
value.
In
its
discussion
of
the
issue,
the
EU
indicates
that
use
of
the
term
 
limit
of
detection 
only
implies
a
level
at
which
detection
becomes
problematic
and
is
not
associated
with
any
specific
definition.
Instead,
the
EU
focuses
its
attention
on
ways
to
estimate
uncertainty,
basing
its
approach
on
the
ISO
Guide
to
the
Expression
of
Uncertainty
in
Measurement
(
1993).
However,
the
EU
also
notes
that
the
use
of
uncertainty
estimates
in
compliance
statements
and
the
expression
and
use
of
uncertainty
at
low
levels
may
require
additional
guidance.
The
United
Kingdom s
Valid
Analytical
Measurement
Programme
(
VAM)
has
adopted
a
similar
approach
that
is
based
on
both
the
ISO
and
the
Eurachem
guidance
(
Barwick
and
Ellison,
2000).
Because
these
approaches
are
focused
on
estimating
uncertainty
rather
than
at
establishing
or
defining
limits
for
detection
and
quantitation,
EPA
did
not
consider
the
European
approaches
in
this
assessment.

2­
10
February
2003
Chapter
3
Issues
Pertaining
to
Detection
and
Quantitation
As
part
of
the
Settlement
Agreement
concerning
EPA's
reassessment
of
detection
and
quantitation
limit
approaches,
the
Agency
agreed
to
consider
several
specific
issues
pertaining
to
these
approaches.
These
issues
included:

 
Criteria
for
selection
and
appropriate
use
of
statistical
models,
 
Methodology
for
parameter
estimation,
 
Statistical
tolerance
and
prediction,
 
Criteria
for
design
of
detection
and
quantitation
studies,
including
selection
of
concentration
levels
( 
spiking
levels ),
 
Interlaboratory
variability,
and
 
Incorporation
of
elements
of
probability
design.

In
developing
its
plan
for
conducting
this
assessment,
EPA
identified
a
number
of
other
issues
that
should
be
considered.
These
issues
include:

 
Concepts
of
the
lower
limit
of
measurement,
 
The
need
for
approaches
that
can
support
CWA
programs,
including:
­
method
performance
verification
at
a
laboratory,
­
method
development
and
promulgation,
­
National
Pollutant
Discharge
Elimination
System
(
NPDES)
applications,
­
non­
regulatory
studies
and
monitoring,
­
descriptive
versus
prescriptive
uses
of
lower
limits
to
measurement,
and
­
use
of
a
pair
of
related
detection
and
quantitation
procedures
in
all
OW
applications
 
Censoring
of
measurement
results,
 
Sources
of
variability
(
including,
but
not
limited
to
interlaboratory
variability),
 
False
positives
and
false
negatives,
 
Measurement
quality
over
the
life
of
a
method,
 
Matrix
effects,
 
Background
contamination,
 
Outliers,
 
Instrument
non­
response,
 
Accepting
the
procedures
of
voluntary
consensus
standards
bodies
(
VCSBs),
 
National
versus
local
standards
for
measurement,
 
Ease
of
use
(
i.
e.,
ability
of
study
managers,
bench
chemists,
and
statisticians
to
do
what
is
required
by
a
detection
or
quantitation
limit
procedure),
 
Cost
to
implement
the
procedures,
and
 
Laboratory­
specific
applications.

Approaches
to
establishing
the
lower
limits
of
measurement
were
discussed
in
Chapter
2.
For
clarity
and
brevity,
EPA
has
organized
the
remaining
issues
into
three
subsections
that
follow.
Section
3.1
discusses
the
issues
that
are
primarily
driven
by
analytical
chemistry
concerns,
Section
3.2
discusses
the
issues
that
are
primarily
driven
by
CWA
regulatory
considerations,
and
Section
3.3
discusses
issues
that
are
primarily
driven
by
statistical
concerns.
Table
3­
1,
at
the
end
of
this
chapter,
provides
a
summary
of
each
issue
discussed
in
Sections
3.1
­
3.3.

February
2003
3­
1
Assessment
of
Detection
and
Quantitation
Approaches
3.1
Analytical
Chemistry
Approaches
to
Detection
and
Quantitation
3.1.1
Blank
versus
Zero
Concentration
Analytical
chemists
rarely,
if
ever,
say
that
a
sample
contains
zero
concentration
of
a
substance
of
interest.
Even
when
the
sample
is
created
in
a
laboratory
for
the
purpose
of
containing
as
little
substance
of
interest
as
possible
(
a
blank),
analytical
chemists
recognize
the
possible
contribution
of
the
blank
to
the
final
measurement
result.
The
ability
of
a
laboratory
to
reduce
the
concentration
of
a
substance
in
the
blank
is
often
the
limiting
factor
in
attempts
to
make
measurements
at
ever
lower
levels.

A
classic
example
of
the
potential
problem
is
illustrated
by
the
seminal
works
of
Patterson
in
the
late
1960s
and
1970s
(
e.
g.,
Patterson
and
Settle,
1976).
Patterson
demonstrated
that
the
majority
of
concentrations
of
lead
reported
in
the
literature
for
such
diverse
matrices
as
urban
dust,
open
ocean
waters,
and
biological
tissues
were
in
error
by
several
orders
of
magnitude.
The
source
of
the
"
gross
positive
errors"
was
contamination
introduced
during
sample
collection,
handling,
and
analysis.
Interlaboratory
studies
of
the
day
designed
to
determine
consensus
values
for
reference
materials
were,
in
fact,
determining
the
consensus
values
for
background
contamination
across
laboratories.
Patterson
recognized
the
value
in
running
blank
samples
(
samples
thought
not
to
contain
the
substance
of
interest)
to
demonstrate
that
the
sample
collection,
handling,
and
analysis
processes
were
not
introducing
contamination.
Patterson
subsequently
developed
the
techniques
for
"
evaluating
and
controlling
the
extent
and
sources
of
industrial
lead
contamination
introduced
during
sample
collecting,
handling,
and
analysis"
that
form
the
basis
of
the
"
clean
techniques"
used
for
metals
analysis
today,
and
that
are
incorporated
in
EPA
Method
1631
among
others.

The
most
common
analytes
for
which
contamination
problems
are
encountered
in
environmental
measurements
are
metals,
primarily
zinc
because
of
its
ubiquity
in
the
environment.
On
the
other
hand,
it
is
rare
to
find
contamination
in
the
measurement
of
organic
compounds,
except
for
methylene
chloride,
acetone,
and
a
few
other
volatile
organic
compounds
used
as
solvents
in
analytical
laboratories.
Therefore,
for
determination
of
metals,
a
blank
is
usually
included
or
compensated
in
the
calibration
whereas,
for
organics,
except
for
the
solvents,
the
concentration
in
the
blank
is
generally
assumed
to
be
zero
and
there
is
no
compensation
of
the
calibration.

Measurement
methods
designed
to
determine
substances
at
very
low
concentrations
may
include
requirements
for
the
preparation
and
analysis
of
a
variety
of
blanks
that
are
designed
to
identify
the
extent
and
the
sources
of
contamination.
Analysts
understand
that
"
blank"
does
not
mean
zero
concentration,
but
that
through
careful
control
and
evaluation,
it
is
possible
make
measurements
for
which
the
blank
contribution
is
sufficiently
small
to
be
considered
negligible.

Useful
detection
and
quantitation
limit
approaches
should
address
the
potential
contribution
of
the
blank
through
both
the
design
of
the
study
that
generates
the
detection
and
quantitation
limit
estimates
and
the
evaluation
of
the
study
results.

3.1.2
Lack
of
Instrument
Response
Instruments
do
not
always
produce
a
result
from
an
appropriately
prepared
sample.
Sometimes
this
is
attributable
to
uncontrollable
instrument
limitations,
sometimes
it
is
attributable
to
controllable
instrument
settings
(
thresholds)
established
by
the
manufacturer
or
the
laboratory,
and
sometimes
it
occurs
randomly.
As
an
example,
gas
chromatograph/
mass
spectrometer
(
GC/
MS)
instruments
often
contain
thresholds
below
which
no
instrument
signal
is
reported.
With
no
instrument
signal
reported,
no
measurement
result
can
be
reported,
and
the
instrument
will
report
zero
to
indicate
the
lack
of
a
signal.

3­
2
February
2003
Chapter
3
To
understand
how
instrument
thresholds
are
used,
it
may
be
helpful
to
think
of
background
static
heard
on
a
citizen­
band
(
CB)
radio
or
a
walkie­
talkie.
The
static
is
present,
but
it
has
no
meaning.
Turning
the
"
squelch"
knob
to
the
point
at
which
the
static
is
filtered
out
also
may
make
it
impossible
to
hear
the
caller.
In
the
context
of
detection,
increasing
the
instrument
threshold
may
cause
the
instrument
to
miss
a
substance
of
interest
at
a
low
level.

In
1997,
EPA
conducted
study
of
82
semivolatile
(
acid
and
base/
neutral)
organic
compounds
measured
by
EPA
Method
1625
in
order
to
observe
the
performance
of
a
GC/
MS
instrument
both
with
and
without
an
instrument
threshold
(
see
Chapter
1,
Section
1.3.2.3).
In
the
study,
solutions
at
up
to
17
concentration
levels
were
analyzed
with
the
threshold
on
(
i.
e.,
low
level
signals
are
automatically
suppressed
)
and
with
the
threshold
off
(
i.
e.,
there
is
no
suppression
of
signals).
Samples
were
analyzed
at
decreasing
concentrations,
including
a
blank,
with
triplicate
determinations
at
each
concentration.
For
measurement
results
obtained
with
the
threshold
turned
on,
all
of
the
measurements
made
on
the
blank
were
reported
as
zero.
This
was
not
a
surprising
result,
given
the
purpose
of
the
instrument
threshold.
For
measurements
obtained
without
the
threshold,
27
of
230
measurements
on
the
blank
(
11%)
were
reported
as
0.000
ng/
mL
and
no
measurement
results
were
reported
lower.
Instrument
non­
response
at
a
low
concentration
has
both
direct
and
indirect
impacts
on
estimating
detection
and
quantitation
limits.

The
main
direct
impact
of
non­
response
at
low
concentrations
is
that
it
is
not
possible
to
estimate
the
standard
deviation
of
measurements
at
zero
concentration.
By
definition,
however,
this
standard
deviation
is
required
to
calculate
the
Currie
critical
value.
The
EPA
MDL
procedure
was
constructed
to
deal
with
this
problem
by
providing
for
a
means
estimation
of
a
standard
deviation
at
a
low
concentration.
The
MDL
procedure
includes
step­
by­
step
instructions
for
determination
of
a
concentration
as
close
to
zero
as
is
possible
that
will
generate
a
measurement.

In
order
to
meet
the
requirements
of
the
MDL
definition,
it
is
necessary
to
find
the
concentration
at
which
the
measurement
method
ceases
to
generate
measurement
results,
and
many
laboratories
have
run
repeat
measurements
in
order
to
find
this
concentration.
This
problem
manifested
itself
in
EPA's
variability
versus
concentration
(
Episode
6000)
studies.
The
40
CFR
part
136,
Appendix
B
procedure
suggests
iteration
until
the
calculated
MDL
is
within
a
factor
of
5
of
the
spike
level.
For
the
Episode
6000
studies,
EPA
instructed
laboratories
to
use
a
factor
of
3
instead
of
5
in
an
attempt
to
more
narrowly
define
the
lowest
spike
level
at
which
measurements
could
be
made.

This
change
to
a
factor
of
3
also
was
suggested
by
one
of
the
peer
reviewers
charged
with
evaluating
EPA's
assessment
of
detection
and
quantitation
limits,
who
noted:

"
However,
the
use
of
as
much
as
five
times
the
critical
level
for
the
spike
concentrations
could
be
problematic.
The
inflation
of
the
MDL
by
using
a
spike
at
the
critical
level
is
only
25%
for
a
method
with
a
high­
level
CV
of
20%
(
this
and
other
calculations
here
are
done
with
the
Rocke
and
Lorenzato
1995
variance
function
assuming
a
sample
size
of
7).
A
spike
concentration
of
3
times
the
critical
level
inflates
the
MDL
to
a
value
140%
higher,
which
even
there
may
be
tolerable.
Use
of
a
value
5
times
the
critical
level
gives
an
inflation
of
over
280%.
..."

Following
some
theoretical
example
calculations
that
are
not
reproduced
here,
the
reviewer's
comment
continues
with:

"
Thus,
I
would
recommend
that
the
procedure
be
altered
to
use
concentrations
that
are
no
more
than
3
times
the
detection
limit,
and
perhaps
to
permit
concentrations
lower
then
the
critical
level,
including
possibly
blanks"
(
Rocke,
2002).

February
2003
3­
3
Assessment
of
Detection
and
Quantitation
Approaches
The
reviewer's
calculations
suggest
that
MDL
may
be
strongly
inflated
for
a
spike
level
of
5
times
the
MDL,
but
only
moderately
inflated
at
a
spike
level
of
3
times
the
MDL.
However,
during
the
Episode
6000
studies,
several
laboratories
asked
for
relief
from
the
requirement,
and
EPA
relented
after
learning
of
the
difficulties
in
attempting
to
achieve
the
factor
of
3.
If
the
reviewer's
example
calculations
are
correct
and
a
practical
procedure
for
determining
the
MDL
using
the
factor
of
3
could
be
implemented,
it
could
exacerbate
the
concern
from
the
regulated
community
that
MDL
values
are
too
low.

Given
the
competing
theoretical
and
practical
considerations,
one
conclusion
that
can
be
drawn
is
that
detection
limits
are
somewhat
variable
and
not
easy
to
define.
Further
details
are
in
the
results
of
the
studies
given
in
Appendices
B
and
C
to
this
Assessment
document.

In
summary,
both
Currie's
approach
and
EPA's
approach
have
theoretical
problems
with
addressing
instrument
non­
response.
Any
operational
approach
to
detection
or
quantitation
should
take
this
issue
into
account.

3.1.3
Matrix
Effects
"
Sample
matrix"
is
a
term
used
to
describe
all
of
the
substances,
other
than
the
substance(
s)
of
interest,
present
in
an
environmental
sample.
In
the
case
of
a
wastewater
sample,
this
would
include
the
water
itself,
as
well
as
any
other
dissolved
or
suspended
materials.
For
any
given
measurement,
some
of
the
substances
may
interfere
with
the
measurement,
while
others
may
be
substances
that
have
no
effect
on
the
measurement.
Interferences
in
the
sample
may
act
either
positively
(
i.
e.,
increasing
the
measured
result),
negatively
(
i.
e.,
decreasing
the
measured
result),
or
even
preventing
the
measurement
from
being
made.

"
Matrix
effect"
is
a
term
used
to
describe
a
situation
in
which
a
substance
or
combination
of
substances
in
the
sample
(
other
than
the
substance[
s]
of
interest)
influence
the
results
of
the
measurement.
Positive
interferences
may
inflate
the
results
for
the
substance
or
make
it
difficult
to
distinguish
one
substance
from
another.
However,
unless
the
positive
bias
is
consistent
and
predictable,
the
measurement
result
may
be
unreliable.
Negative
interferences
may
suppress
the
results
for
the
substance
to
the
point
that
the
results
cannot
be
distinguished
from
background
instrument
noise.

In
some
cases,
finding
a
matrix
effect
indicates
that
the
analyst
should
select
a
more
appropriate
method.
For
example,
a
colorimetric
method
for
the
measurement
of
sulfide
may
be
a
poor
choice
for
the
analysis
of
a
sample
that
is
very
cloudy
or
darkly
colored.
In
other
cases,
characteristics
of
the
sample
such
as
its
pH
may
destroy
the
substance
of
interest,
effectively
preventing
analysis
for
that
substance.

Nearly
all
of
the
newer
analytical
methods
approved
at
40
CFR
part
136
describe
the
preparation
and
analysis
of
quality
control
samples
that
are
designed
to
indicate
the
presence
of
matrix
effects
(
e.
g.,
matrix
spike
and/
or
matrix
spike
duplicate
samples).
Many
of
these
methods
also
contain
techniques
for
addressing
matrix
effects.
Further,
EPA
has
developed
guidance
documents
that
amplify
the
discussions
in
those
methods
(
e.
g.,
Guidance
on
Evaluation,
Resolution,
and
Documentation
of
Analytical
Problems
Associated
with
Compliance
Monitoring,
June
1993,
EPA
821­
B­
93­
001).
For
determination
of
mercury
by
EPA
Method
1631
that
is
the
subject
of
the
Settlement
Agreement,
additional
guidance
on
resolving
matrix
interferences
to
achieve
specified
detection
and
quantitation
limits
is
provided
in
EPA s
Guidance
for
Implementation
and
Use
of
EPA
Method
1631
for
the
Determination
of
Low­
Level
Mercury
(
March
2001,
EPA
821­
R­
01­
023).
Following
the
techniques
in
the
methods
and
guidance
will
usually
reduce
adverse
effects
of
the
sample
matrix
on
detection/
quantitation
limits
and
measurement
results.

3­
4
February
2003
Chapter
3
3.1.3.1
Allowance
for
Matrix
Effects
in
Detection
and
Quantitation
Limits
There
are
those
who
believe
that
detection
and
quantitation
limits
should
be
determined
in
 
real­
world 
matrices,
rather
than
in
reference
matrices
intended
to
simulate
method
performance
in
a
particular
matrix
type.
Problems
with
such
an
approach,
however,
are
that:

 
Many
 
real­
world 
matrices
contain
the
target
pollutant
at
levels
well
above
the
detection
or
quantitation
limit,
making
it
impossible
to
characterize
what
can
and
cannot
be
detected
at
low
levels.
Diluting
the
sample
to
dilute
the
target
pollutant
concentration
is
an
option.
However,
this
also
has
the
potential
to
dilute
any
interferences
that
might
be
present,
thereby
defeating
the
purpose
of
using
the
real­
world
matrix.
 
It
is
not
possible
to
anticipate
and
obtain
samples
of
every
possible
matrix
on
which
a
method
might
be
used
when
the
method
is
being
developed
and
detection/
quantitation
limits
are
being
established.
 
Although
use
of
a
reference
matrix
to
establish
detection
and
quantitation
limits
allows
the
results
to
be
reproduced
(
i.
e.,
confirmed)
by
an
independent
party,
such
a
confirmation
may
not
be
possible
with
many
real
world
matrices
that
may
be
subject
to
seasonal,
diurnal,
or
other
types
of
variability.
 
The
cost
of
determining
detection
and
quantitation
limits
in
every
possible
matrix
would
be
prohibitive.

Given
these
difficulties,
EPA
believes
that
a
reference
matrix
or
reference
matrices
should
be
used
to
establish
method
detection
and
quantitation
limits,
but
that
the
procedures
for
defining
these
limits
should
allow
for
evaluation
of
data
collected
in
particular
matrices
of
concern.
EPA
also
believes
that
such
matrix­
specific
determinations
should
only
be
used
when
all
efforts
to
resolve
matrix
interferences
have
been
exhausted.

3.1.3.2
Repository
of
Reference
Matrices
Two
of
the
four
peer
reviewers
charged
with
evaluating
EPA s
assessment
of
detection
and
quantitation
limit
approaches
suggested
that
EPA
create
a
repository
of
reference
matrices,
similar
to
those
developed
by
NIST,
and
that
these
reference
matrices
be
used
to
challenge
a
test
method
and
to
establish
detection
and
quantitation
limits
(
Cooke,
2002
and
Wait,
2002).
EPA
has
considered
such
a
repository
from
time
to
time
and
again
in
response
to
this
suggestion,
but
has
been
unable
to
resolve
all
of
the
issues
surrounding
such
a
repository.
Some
of
these
issues
are:

C
The
stability
of
aqueous
samples,
C
The
holding
times
necessary
to
assure
stability,
C
The
argument
that
no
matrix
from
a
given
industrial
discharge
in
industrial
category
or
subcategory
reflects
the
characteristics
of
another
discharge
in
that
or
other
industrial
categories
or
subcategories,
C
The
cost
of
maintaining
such
a
repository,
and
C
The
potential
conflict
with
NIST
and
with
non­
governmental
organizations
that
provide
reference
matrices.

Given
these
issues,
EPA
believes
that
the
development
and
maintenance
of
standard
reference
materials
(
SRMs)
and
certified
reference
materials
(
CRMs)
are
best
left
to
NIST
and
the
commercial
marketplace.
EPA
agrees
that
such
reference
materials
are
a
useful
means
of
challenging
a
test
method
and
has
suggested
in
recent
methods
that
reference
matrices
be
analyzed,
when
available,
as
an
additional
QC
measure.
For
example,
when
EPA
developed
an
appendix
to
Method
1631
for
application
to
matrices
other
than
water,
EPA
specified
use
of
a
quality
control
sample
(
QCS)
with
the
statement
that
"
many
certified
reference
materials
(
CRMs)
are
available
for
total
mercury
in
plants,
animals,
fish,
sediments,
soils,
and
sludge"
and
the
requirement
that
"
recovery
and
precision
for
at
least
one
QCS
per
batch
of
samples
must
meet
the
performance
specifications
provided
by
the
supplier."

February
2003
3­
5
Assessment
of
Detection
and
Quantitation
Approaches
Although
EPA
agrees
that
SRMs
and
CRMs
could
be
useful
in
establishing
detection
and
quantitation
limits,
EPA
believes
that
practical
considerations
are
likely
to
preclude
their
use
for
this
purpose
in
most
situations.
This
is
because
the
materials
would
need
to
contain
the
analytes
of
interest
at
levels
that
are
near
the
detection
limit
(
e.
g.,
within
1
to
5
times
the
concentration
of
a
determined
MDL).
Such
concentrations
are
unlikely
to
occur
in
an
SRM
produced
by
NIST
or
a
CRM
produced
by
a
vendor,
and
diluting
the
CRM/
SRM
would
diminish
matrix
effects,
as
indicated
in
Section
3.1.3.1.

As
an
alternative
to
using
standard
reference
materials,
EPA
commonly
tests
its
analytical
methods
on
a
variety
of
real­
world
matrices,
and
allows
for
this
variability
in
the
QC
acceptance
criteria
for
the
matrix
spike
(
MS)
and
matrix
spike
duplicate
(
MSD)
samples.
For
example,
EPA
published
performance
data
in
Table
3
of
EPA
Method
1631B
for
reagent
water,
fresh
water,
unfiltered
and
filtered
marine
water,
and
unfiltered
and
filtered
secondary
effluent,
and
allowed
for
the
variability
among
these
matrices
in
the
QC
acceptance
criteria
for
the
MS/
MSD
in
the
method.
ASTM
Committee
D
19
allows
this
approach
in
development
of
QC
acceptance
criteria
for
methods
(
see
Section
6.5.1.1
of
ASTM
D
5847:
Standard
Practice
for
Writing
Quality
Control
Specifications
for
Standard
Test
Methods
for
Water
Analysis.)

3.1.4
Recovery
Correction
This
section
addresses
correction
for
recovery
in
detection
and
quantitation
limits.
The
purpose
of
a
recovery
correction
is
to
adjust
a
measured
concentration
in
a
sample
for
the
amount
by
which
the
measured
concentration
differs
from
the
true
concentration
(
if
known).

To
illustrate
the
potential
need
for
recovery
correction,
consider
the
case
of
certain
compounds
such
as
organic
bases
(
e.
g.,
benzidine)
and
acids
(
e.
g.,
phenols)
that
are
either
not
totally
(
100%)
recovered
in
the
extraction
process,
or
are
adsorbed
on
the
surface
of
a
GC
column
at
low
(
nanogram)
levels.
As
a
result,
the
measured
concentration
of
such
compounds
is
always
less
than
the
true
concentration
in
the
water
sample.
These
incomplete
recoveries
have
led
some
developers
of
detection
and
quantitation
limit
approaches
to
believe
that
these
limits
should
be
recovery
corrected
(
i.
e.,
that
the
detection
or
quantitation
limit
should
be
adjusted
inversely
proportional
to
the
recovery).
For
example,
if
an
analyte
is
recovered
at
50%,
the
detection
and/
or
quantitation
limit
should
be
doubled.
EPA
believes
that
recovery
correction
may
be
appropriate
if
(
1)
the
recovery
is
consistent
across
laboratories,
matrices,
and
conditions,
and
(
2)
the
relative
variability
(
as
relative
standard
deviation)
remains
constant
as
the
recovery
decreases.
These
two
requirements
are
rarely
met;
therefore,
recovery
correction
would
be
appropriate
only
in
rare
circumstances.

The
first
requirement
(
consistent
recovery)
would
need
to
be
tested
under
a
variety
of
conditions
because,
if
the
recovery
varies
among
laboratories,
matrices,
and
analytical
conditions,
then
a
detection
and/
or
quantitation
limit
would
need
to
be
developed
for
each
of
these
conditions.
EPA's
experience
is
that
poor
recovery
is
rarely
consistent;
i.
e.,
if
one
laboratory
measures
a
recovery
of
40%,
another
laboratory
may
measure
20%,
or
60%,
but
not
exactly
40%.

The
normal
condition
in
environmental
analytical
measurements
is
that
the
variability
(
as
standard
deviation)
remains
approximately
constant
as
the
recovery
decreases
(
i.
e.,
the
relative
precision
[
as
RSD]
is
poorer
at
low
recovery).
For
example,
if
the
RSD
is
10%
at
100%
recovery,
the
RSD
may
be
50%
at
50%
recovery,
and
may
be
100%
at
10%
recovery.
This
increase
in
relative
variability
is
not
the
result
of
measurements
being
made
at
lower
levels,
as
is
the
normal
case,
but
as
a
result
of
variability
in
the
extraction
(
partitioning)
process.
For
examples
of
the
effect
of
poor
recovery
on
precision,
see
the
quality
control
(
QC)
acceptance
criteria
for
the
semivolatile
organic
compounds
in
Table
8
of
EPA
Method
1625
(
see
40
CFR
part
136,
Appendix
A).
Because
nearly
all
detection
and
quantitation
limits
3­
6
February
2003
Chapter
3
are
based
on
precision
(
as
standard
deviation),
including
a
recovery
correction
for
an
analyte
with
poor
precision
at
low
recovery
is,
in
effect,
a
double
counting
for
poor
precision.

A
third
concern
with
the
issue
of
recovery
correction
is
 
where
does
it
stop? 
It
makes
little
sense
to
recovery
correct
those
measurements
made
in
the
region
of
detection
and
quantitation
if
similar
recovery­
correction
steps
are
not
employed
for
measurements
at
higher
concentrations.
EPA
has
traditionally
viewed
recovery
correction
with
great
caution,
and
has
preferred
to
require
that
laboratories
analyze
quality
control
samples
to
demonstrate
that
analytes
are
recovered
within
an
acceptable
level.
For
example,
EPA's
Office
of
Water
methods
require
that
laboratories
prepare
and
analyze
both
a
reference
matrix
and
a
sample
matrix
that
have
been
spiked
with
the
analytes
of
interest,
and
that
these
analytes
be
recovered
within
method­
specified
acceptance
criteria.
If
the
recovery
criteria
are
met,
then
samples
analyzed
in
the
batch
are
considered
to
be
reliable
within
the
overall
level
of
error
associated
with
the
method,
and
results
are
reported
without
correcting
for
the
recovery.
EPA
believes
that
it
would
be
inconsistent
to
correct
for
recovery
in
measurements
made
at
the
detection
or
quantitation
limit,
if
such
corrections
are
not
made
to
results
obtained
at
higher
concentrations
(
e.
g.,
during
the
routine
analysis
of
samples).

EPA
acknowledges
that
recovery­
correction
techniques
are
employed
in
some
Agency
methods.
Most
notably
are
those
methods
that
employ
isotope
dilution
techniques,
in
which
a
stable,
isotopically
labeled
analog
of
each
target
analyte
is
spiked
into
each
sample.
Because
of
their
structural
similarity
to
the
analytes
of
interest,
the
labeled
analogs
are
assumed
to
behave
exactly
like
their
unlabeled
analogs
(
the
target
analytes).
Because
the
recovery
of
the
labeled
analog
will
be
similar
to
that
of
the
target
analyte,
the
technique
allows
for
recovery
correction
of
each
target
analyte
and
is
particularly
useful
in
highly
complex
matrices.
In
these
methods,
recovery
correction
techniques
are
specified
as
part
of
the
procedures
for
calculating
and
reporting
results
and
are
dependent
on
the
one­
to­
one
relationship
of
the
target
analyte
and
the
labeled
analog.
Inclusion
of
a
further
procedure
for
recovery­
correction
in
a
detection
and
quantitation
limit
approach
could
result
in
double­
counting
of
bias.

Few
of
the
 
traditional 
approaches
to
establishing
detection
and
quantitation
limits
include
procedures
for
recovery
correction.
For
example,
the
issue
was
not
addressed
by
Currie
in
his
original
proposal
of
a
critical
value
or
quantitation
limit.
Similarly,
neither
EPA's
MDL
and
ML
nor
the
American
Chemical
Society s
LOD
and
LOQ,
all
of
which
are
based
on
the
approaches
advanced
by
Currie,
include
a
mechanism
for
recovery
correction.
When
Currie
introduced
his
critical
value,
he
defined
it
as
"
the
minimum
significant
value
of
an
estimated
net
signal
or
concentration,
applied
as
a
discriminator
against
background
noise"
(
Currie,
1995).
Because
the
critical
value
is
defined
as
a
measured
concentration
rather
than
a
true
concentration,
a
recovery
correction
is
not
included.

The
use
of
recovery
correction
has
been
included
in
several
of
the
most
recently
developed
approaches
for
detection
and
quantitation.
For
example,
the
minimum
detectable
value
(
MDV)
recently
adopted
by
ISO
and
IUPAC,
and
the
interlaboratory
detection
estimate
(
IDE)
and
interlaboratory
quantitation
estimate
(
IQE)
adopted
by
ASTM
include
procedures
for
recovery
correction.
The
IQE
also
contains
a
further
correction
that
we
have
termed
a
"
bias"
correction.

In
the
MDV
approach,
recovery
is
treated
as
a
linear
function
versus
concentration,
and
an
extrapolation
is
used
to
estimate
the
recovery
at
zero
concentration.
EPA
has
found
that
this
projection
of
the
regression
line
to
zero
concentration
can
lead
to
errors
because,
depending
on
the
intercept
(
in
concentration
units),
the
recovery
at
zero
concentration
can
be
positive,
zero,
or
negative,
resulting
in
an
inflated
MDV,
an
MDV
very
close
to
zero,
or
a
negative
MDV.
For
further
details,
see
the
section
titled
"
Negative
detection
limits
for
the
ISO/
IUPAC
MDV"
in
Appendix
C
to
this
Assessment
Document,
and
the
data
in
Table
2
of
that
appendix.

February
2003
3­
7
Assessment
of
Detection
and
Quantitation
Approaches
The
IDE
and
IQE
fit
recovery
versus
concentration
in
a
way
analogous
to
the
fitting
in
the
MDV.
The
difference
between
the
treatment
of
recovery
in
the
MDV
and
the
IDE/
IQE
is
that
an
unweighted
model
is
used
in
the
MDV,
whereas
the
linear
model
in
the
IDE
and
IQE
is
weighted
as
determined
by
the
model
of
standard
deviation
versus
concentration
that
is
used
in
calculating
the
IDE
and
IQE.
(
If
this
model
is
the
constant
model,
the
weighting
is
the
same
as
for
the
MDV.)

The
IQE,
but
not
the
IDE,
includes
an
additional
correction
for
the
bias
associated
with
an
estimate
of
the
true
standard
deviation
at
each
concentration
as
compared
to
the
measured
standard
deviation
at
each
concentration.
In
this
context
(
a
"
bias"
correction
to
the
IQE),
the
word
"
bias"
means
the
amount
by
which
the
estimated
sample
standard
deviation
differs
from
the
true
population
standard
deviation,
and
should
not
be
confused
with
common
use
of
the
word
"
bias"
in
analytical
chemistry
measurements
(
the
deviation
of
a
result
from
the
true
value,
usually
expressed
as
percent).

The
effect
of
these
corrections
on
detection
and
quantitation
limits
was
calculated
using
data
generated
in
EPA s
Multi­
technique
Variability
Study
(
the
 
Episode
6000
Study ).
Details
of
these
effects
are
discussed
in
Appendix
C.

3.1.5
Measurement
Quality
over
the
Life
of
a
Method
We
have
all
heard
the
expression
"
Practice
makes
perfect."
Although
there
is
no
such
thing
as
a
 
perfect 
measurement,
the
idea
that
results
get
better
with
practice
applies
to
the
quality
of
measurements
made
with
a
given
method
over
time.
We
can
demonstrate
it
using
simple
techniques
like
laboratory
control
charts.
The
improvements
are
a
result
of
experience,
as
well
as
improvements
in
equipment
over
time.
EPA
expects
changes
in
performance
when
new
staff
are
trained.
For
this
reason,
many
EPA
methods
specify
that
"
start
up
tests"
be
repeated
each
time
new
staff
arrive.
It
is
not
unusual
to
see
slight
increases
in
measurement
variability
as
new
staff
are
trained.
However,
when
new
staff
become
as
good
as
the
existing
staff,
control
charts
should
show
it.

The
use
of
quality
control
(
QC)
charts
as
a
means
of
tracking
method
and
laboratory
improvement
as
a
function
of
time
is
described
in
EPA's
Handbook
for
Analytical
Quality
Control
in
Water
and
Wastewater
Laboratories
(
referenced
in
the
40
CFR
part
136,
Appendix
A
methods).
Although
these
charts
are
instructive
in
tracking
improvement,
they
have
two
significant
drawbacks:
(
1)
they
do
not
establish
an
absolute
limit
within
which
an
analysis
must
be
operated
and
(
2)
continued
improvement
can
lead
to
unusually
stringent
limits
that,
eventually,
will
not
be
met.
As
long
as
absolute
QC
acceptance
criteria
(
limits),
such
as
those
found
in
EPA
methods,
are
established
for
the
determination,
and
as
long
as
there
is
a
recognition
that
stringent
limits
may
be
an
artifact
of
improvement
beyond
what
is
routinely
achievable,
QC
charts
can
be
instructive
in
identifying
statistically
significant
losses
of,
or
improvements
in,
analyte
responses
in
the
region
of
interest.
ASTM
Committee
D
19
adopted
the
philosophy
of
establishing
absolute
limits
for
analytical
methods
in
approving
Standard
Practice
D
5847.

As
with
most
other
areas
of
technology,
measurement
instruments
continue
to
improve.
Instrument
manufacturers
and
laboratories
are
increasing
data
processing
power,
speed
of
analysis,
and
the
reduction
of
chemical
or
electronic
"
noise."
Any
of
these
instrument
improvements
can
be
expected
to
improve
the
measurement
method
in
determining
the
concentrations
of
environmental
pollutants.
This
process
can
be
illustrated
for
a
variety
of
EPA
methods.
A
case
in
point
is
EPA
Method
1613
for
determination
of
polychlorinated
dibenzo­
p­
dioxins
and
polychlorinated
dibenzofurans.
Development
of
this
method
began
in
1988.
At
the
time,
high
resolution
mass
spectrometer
systems
that
were
commercially
available
were
able
to
achieve
a
detection
limit
of
approximately
4
pg/
L
and
an
ML
of
10
pg/
L.
By
the
time
that
EPA
proposed
the
method
in
1991,
the
Canadian
government
published
its
own
version
that
included
a
quantitation
limit
5
pg/
L.
By
the
time
EPA
officially
promulgated
Method
1613
3­
8
February
2003
Chapter
3
in
1997,
many
laboratories
performing
the
analysis
had
replaced
or
supplemented
their
old
instruments
with
newer
models.
As
a
result,
many
laboratories
performing
analyses
using
Method
1613
routinely
measure
sample
results
at
levels
10
times
lower
than
those
analyzed
routinely
only
10
years
earlier.

Given
that
measurement
capabilities
tend
to
improve
over
time,
EPA
believes
that
a
detection
and
quantitation
limit
approach
should
be
supported
by
procedures
that
will
allow
individual
laboratories
and
other
organizations
to
affordably
characterize
such
improvements.

3.2
CWA
Regulatory
Issues
Affecting
Detection
and
Quantitation
Section
3.2.1
below
provides
a
brief
overview
and
a
discussion
of
Clean
Water
Act
activities
that
involve
chemical
measurements
and
are,
therefore,
directly
impacted
by
detection
and
quantitation
limit
approaches.
Specific
issues
that
must
be
considered
in
the
context
of
these
CWA
applications
and
EPA s
regulatory
obligations
are
discussed
in
Sections
3.2.2
­
3.2.6.

3.2.1
Detection
and
Quantitation
Limit
Applications
Under
CWA
The
Clean
Water
Act
directs
EPA,
States,
and
local
governments
to
conduct
a
variety
of
data
gathering,
permitting,
and
compliance
monitoring,
and
enforcement
activities.
Many
of
these
activities
depend
directly
on
environmental
measurements
and,
therefore,
are
affected
by
detection
and
quantitation
limit
approaches
as
discussed
in
the
subsections
that
follow.

3.2.1.1
Method
Development
and
Promulgation
Section
304(
h)
of
the
Clean
Water
Act
(
CWA;
the
"
Act")
requires
EPA
to
promulgate
test
procedures
(
analytical
methods)
to
be
used
for
data
gathering
to
support
certification,
permitting,
and
monitoring
under
the
Act.
These
methods
are
promulgated
at
40
CFR
part
136,
and
include
methods
developed
by
EPA
as
well
as
those
developed
by
other
organizations,
such
as
the
publishers
of
Standard
Methods
for
the
Examination
of
Water
and
Wastewater,
as
well
as
AOAC­
International,
ASTM
International,
the
U.
S.
Geological
Survey,
instrument
manufacturers,
and
others.
Upon
request
by
a
laboratory,
permittee,
instrument
manufacturer,
or
other
interested
party,
EPA
considers
alternate
testing
procedures
(
ATPs).
If
EPA
deems
these
ATPs
to
be
acceptable
for
nationwide
use,
they
too,
may
be
published
at
40
CFR
part
136.
A
primary
objective
in
promulgating
methods
developed
by
EPA
and
by
other
organizations
is
to
provide
the
regulatory
community,
permittees,
and
laboratories
with
multiple
options
so
that
they
may
choose
the
method
that
yields
the
best
performance
at
the
lowest
cost
for
the
application.

In
recent
years,
EPA
has
focused
on
developing
methods
for
promulgation
at
40
CFR
part
136
where
no
other
methods
are
available
that
meet
an
immediate
or
anticipated
regulatory
need.
The
National
Technology
Transfer
and
Advancement
Act
of
1995
(
NTTAA)
urges
government
agencies
to
consider
methods
published
by
voluntary
consensus
standards
bodies
(
VCSBs),
such
as
Standard
Methods
and
ASTM
International,
when
VCSB
methods
are
available.
EPA
accepts
that
many
of
these
methods
have
been
through
a
sufficient
level
of
testing,
peer
review,
and
scientific
acceptance
to
warrant
proposal
if
they
meet
EPA's
regulatory
needs.
When
an
individual
laboratory,
permittee,
or
other
organization
submits
a
request
for
approval
of
an
alternate
test
procedure,
however,
EPA
generally
requires
that
the
procedure
be
subjected
to
a
level
of
testing
that
demonstrates
that
the
method
provides
sensitivity,
accuracy,
and
other
measures
of
performance
comparable
to
an
approved
method.

The
lack
of
widespread
consensus
on
detection
limits
has
obvious
impacts
on
EPA s
responsibility
to
promulgate
methods
under
CWA.
Most
organizations
that
develop
methods
use
February
2003
3­
9
Assessment
of
Detection
and
Quantitation
Approaches
different
approaches,
and
many
organizations
have
changed
approaches
over
the
years.
The
result
is
that
a
number
of
different
approaches
for
detection
and
quantitation
are
embodied
in
the
methods
approved
at
40
CFR
part
136.
The
vast
majority
of
the
approved
methods
include
the
MDL
which,
as
noted
in
Section
2.2.1,
has
been
used
by
several
EPA
Offices,
Standard
Methods,
AOAC,
ASTM,
and
others.
Other
approaches
embodied
in
the
methods
at
40
CFR
part
136
include,
but
are
not
limited
to:
1)
a
method
 
range 
that
is
usually
not
defined,
but
is
often
interpreted
as
the
lower
end
of
the
range
in
which
pollutants
either
can
be
identified
or
quantified,
2)
an
 
instrument
detection
limit 
that
has
been
defined
by
a
variety
of
procedures,
but
is
intended
to
capture
instrument
sensitivity
only,
3)
an
"
estimated
detection
limit"
that
may
be
based
on
best
professional
judgement,
single
laboratory
data,
or
some
other
source
of
information,
4)
a
"
practical
quantitation
limit,"
that
has
typically
been
determined
according
to
one
of
the
scenarios
described
in
Section
2.3.1,
and
5)
"
sensitivity"
that
is
an
undefined
concept
similar
in
result
to
the
MDL.

The
most
obvious
solution
to
this
problem
would
be
for
the
Office
of
Water
to
force
all
methods
promulgated
at
40
CFR
part
136
to
contain
uniform
approaches
for
detection
and
quantitation.
Unfortunately,
taking
such
action
would
confound
methods
promulgation.
Problems
with
this
solution
are
that:

 
To
date,
no
single
detection
and
quantitation
limit
approach
has
emerged
to
meet
the
needs
of
all
organizations
for
all
applications.
 
If
the
Office
of
Water
were
to
select
an
approach
that
differs
from
those
of
other
organizations,
those
organizations
would
be
required
to
conform
their
method
to
accommodate
OW s
approach.
Doing
so
would
mean
that
these
organizations
would
have
to
invest
additional
laboratory
resources
to
develop
detection
and
quantitation
limits
that
conformed
to
OW
definitions.
 
If
outside
organizations
decided
against
conforming
their
approaches
to
that
of
OW,
fewer
methods
would
be
promulgated
at
40
CFR
part
136.
This
would
result
in
fewer
options
for
the
regulatory,
permittee,
and
laboratory
communities.
 
If
EPA
selected
an
approach
that
has
burdensome
procedures
for
developing
detection
and
quantitation
limits,
it
could
discourage
development
of
innovative
technology
or
method
modifications.

Given
these
issues,
and
EPA s
desire
to
1)
encourage
the
development
of
improved
measurement
techniques,
and
2)
provide
the
stakeholder
community
with
a
variety
of
measurement
options
whenever
possible,
EPA
believes
it
would
be
impractical
to
force
standardization
on
a
single
detection
or
quantitation
limit
approach
on
method
developers
and
promulgate
only
those
methods
that
contain
this
approach.
The
Agency
also
believes,
however,
that
there
are
real
benefits
to
standardization,
and
that
1)
all
new
methods
developed
by
EPA
for
promulgation
at
40
CFR
part
136
should
reflect
such
standardization,
and
2)
EPA
should
strongly
encourage
outside
organizations
to
include
these
standardized
approaches
in
their
methods.

3.2.1.2
Method
Performance
Verification
at
a
Laboratory
Just
as
sensitivity
is
important
for
evaluating
measurement
method
performance,
it
is
important
to
verify
that
a
laboratory
using
a
method
can
achieve
acceptable
levels
of
sensitivity
for
making
measurements.
Such
demonstrations
can
take
many
forms
and
should
be
viewed
in
the
context
of
the
decision
to
be
made.
The
analytical
methods
published
at
40
CFR
part
136
are
designed
for
monitoring
compliance
with
CWA
permits.
Most
pollutants
in
permits
have
a
numeric
limit,
and
compliance
with
this
limit
is
determined
by
laboratory
analysis
of
samples
from
the
waste
stream
or
water
body
regulated
by
the
limit.
The
laboratory
that
conducts
such
analyses
must
be
able
to
demonstrate
that
its
detection
or
quantitation
limits
are
low
enough
to
assure
reliable
measurements.

3­
10
February
2003
Chapter
3
Thus,
even
where
a
method
describes
the
sensitivity
measured
or
estimated
by
the
developer
or
the
organization
that
published
the
method,
some
means
are
needed
to
demonstrate
that
a
given
laboratory
can
achieve
sufficient
sensitivity
to
satisfy
the
regulatory
decision
(
e.
g.,
monitoring
compliance).

The
EPA
MDL
procedure
provides
a
means
for
verifying
laboratory
performance
and
has
long
been
used
in
this
fashion
by
EPA
and
various
other
Federal
and
state
agencies.
Other
procedures
may
be
employed,
including
analysis
of
reference
materials
containing
the
analytes
of
interest
at
concentrations
that
are
at
or
below
the
regulatory
limits
of
interest,
spiked
samples
that
are
similarly
prepared
(
e.
g.,
matrix
spikes),
or
performance
evaluation
(
PE)
samples
such
as
those
used
in
laboratory
accreditation
studies.

The
IDE
and
IQE
were
advanced
by
the
regulated
industry
and
subsequently
approved
by
ASTM
International
as
a
means
of
characterizing
the
performance
of
a
method
in
laboratories
that
participate
in
an
interlaboratory
study.
The
idea
in
developing
these
approaches
was
to
establish
detection
and
quantitation
limits
that
could
be
met
by
any
laboratory
that
participated
in
the
study.
An
advantage
of
this
approach
is
that
individual
laboratories
do
not
have
to
demonstrate
sensitivity.
However,
potential
disadvantages
also
exist.
For
example,
it
may
not
be
possible
to
develop
a
realistic
IDE
or
IQE
for
a
new
method
involving
a
highly
innovative
technique
because
there
may
not
be
a
sufficient
number
of
laboratories
practicing
the
technique
to
allow
development
of
an
IDE/
IQE.
Also,
establishing
detection
and
quantitation
limits
that
can
be
met
by
all
laboratories
that
practice
methods
that
are
in
widespread
use
can
potentially
lead
to
worst­
case
limits
that
are
significantly
higher
than
limits
that
can
be
achieved
by
many
commercial
laboratories.

Developers
of
the
IDE/
IQE
have
recognized
that
an
analogous
approach
is
desirable
for
single­
laboratory
application
and
have
begun
work
on
a
within­
laboratory
detection
estimate
(
WDE),
to
be
followed
by
a
within­
laboratory
quantitation
estimate
(
WQE).
As
with
the
IDE/
IQE,
these
approaches
will
capture
a
wide
range
of
sources
of
variability
such
as
temporal
variability,
and
will
include
a
prediction
or
tolerance
limit
(
or
both),
but
will
not
include
interlaboratory
variability.
EPA
would
consider
such
single
laboratory
approaches
if
and
when
they
are
adopted
by
ASTM
International.

3.2.1.3
National
Pollutant
Discharge
Elimination
System
The
National
Pollutant
Discharge
Elimination
System
(
NPDES)
serves
as
the
primary
means
by
which
EPA,
States,
and
Tribes
control
point
source
releases
into
the
nation s
waters.
Under
this
system,
individual
facilities
are
issued
NPDES
permits
that
provide
limitations
on
the
type,
concentration,
and
volume
of
pollutants
that
may
be
legally
discharged.
Typically,
these
pollutant
controls
are
based
on
technology­
based
standards.
If,
however,
these
technology­
based
controls
are
not
adequate
to
protect
the
water­
quality
standard
designated
for
the
facility's
receiving
water,
stricter
controls
are
warranted.
In
such
cases,
NPDES
permits
generally
contain
water
quality­
based
controls.

Development
and
Implementation
of
Technology­
based
Controls
(
Effluent
Guidelines)

EPA
promulgates
national
effluent
limitations
guidelines
and
standards
under
the
authority
of
Clean
Water
Act
Sections
301,
304,
306,
307,
308,
and
501.
The
regulations
allow
the
discharge
of
pollutants
from
normal
industrial
processes
when
the
discharges
have
been
treated
using
various
levels
of
available
treatment
technologies
that
are
affordable.
Functionally,
these
industry­
specific
guidelines
establish
standards
for
the
quality
of
wastewater
discharges
to
waters
of
the
United
States.
They
are
generally
stated
in
the
form
of
concentration­
based
limits
for
selected
substances
that
are
not
to
be
exceeded.
For
example,
the
maximum
oil
concentration
in
wastewater
separated
from
oil
pumped
out
of
an
offshore
well
and
discharged
on
any
single
day
shall
not
exceed
42
milligrams
per
liter
(
mg/
L).
This
form
is
called
a
numeric
effluent
guideline
limit
or
numeric
limit.

February
2003
3­
11
Assessment
of
Detection
and
Quantitation
Approaches
Development
and
Implementation
of
Water
Quality­
based
Controls
States
designate
water­
quality
standards
for
various
bodies
of
water
within
their
boundaries.
Each
standard
consists
of
a
designated
use,
criteria
to
support
that
designated
use,
and
an
anti­
degradation
policy.
Examples
of
designated
uses
include
public
water
supply,
recreation,
and
propagation
of
fish
and
wildlife.
When
the
water­
quality
standard
is
not
met,
waste­
load
allocations
are
developed
to
indicate
the
maximum
amount
of
a
substance
that
can
be
discharged
to
a
particular
water
body
without
impairing
the
designated
use.
EPA
and
authorized
states
calculate
water
quality­
based
effluent
limits
based
on
the
waste­
load
allocation
and
the
variability
of
the
substance
in
the
wastewater
discharge.
The
concept
is
to
prohibit
discharge
of
a
substance
beyond
the
level
at
which
a
designated
use
would
be
impaired.
Water
quality­
based
permits
generally
specify
the
use
of
measurement
methods
promulgated
at
40
CFR
part
136
under
the
Clean
Water
Act
Section
304(
h).

A
special
case
occurs
when
the
water
quality­
based
effluent
limit
is
less
than
the
detection
limit
of
the
most
sensitive
analytical
method.
This
case
is
addressed
in
Section
3.2.3
below,
on
compliance
evaluation
thresholds.

Permit
Compliance
Monitoring
Under
Clean
Water
Act
Sections
318,
402,
and
405,
NPDES
permits
are
issued
to
owners
of
facilities
that
discharge
wastewater
to
waters
of
the
United
States
(
coastal
areas,
lakes,
rivers,
streams,
certain
wetlands,
etc.).
Specific
discharge
limits
are
established
either
for
individual
facilities
or
for
classes
of
facilities.
Individual
permits
are
established
for
industries
with
many
site­
specific
issues
that
determine
the
substances
discharged,
such
as
the
pharmaceutical
industry
in
which
the
specific
drugs
produced
could
influence
the
water
quality.
General
permits
are
issued
when
the
substances
discharged
do
not
vary
widely
among
facilities
(
e.
g.,
coastal
oil
and
gas
extraction
industry
facilities).
The
permit
limits
are
typically
established
using
technology­
based
effluent
guidelines,
unless
the
facility
is
discharging
into
a
water
body
that
does
not
meet
its
designated
use
or
that
will
not
meet
the
designated
use
if
a
technology­
based
limit
is
permitted.

Detection
plays
a
role
in
compliance
monitoring
because
of
concerns
with
measurement
results
at
the
low
end
of
any
measurement
method.
All
measurement
results
are
variable.
At
the
low
end
of
most
measurement
methods,
there
comes
a
point
at
which
a
particular
measurement
result
is
unacceptably
likely
(
a
policy
decision)
to
have
come
from
a
sample
in
which
the
substance
of
interest
is
absent
(
zero
concentration).
Such
a
measurement
result
would
be
below
the
critical
value
defined
by
Currie
(
1995)
and
in
common
usage
it
would
be
called
below
detection.
In
practice,
the
reporting
limit
may
be
set
equal
to
a
critical
value,
detection
limit,
or
quantitation
limit.
Assuming
that
the
reporting
limit
is
a
detection
limit
of
1
mg/
L
of
oil
and
grease,
the
measurement
result
would
be
reported
as
 
less
than
1
mg/
L
of
oil
and
grease. 

3.2.1.4
Non­
Regulatory
Studies
and
Monitoring
EPA
conducts
a
variety
of
non­
regulatory
studies
and
monitoring
activities
to
support
its
Clean
Water
Act
programs.
These
activities
range
from
long
term
surveys,
such
as
the
Great
Lakes
Water
Quality
Surveys
that
are
conducted
each
spring
and
summer
to
monitor
trends
in
water
quality
against
established
baselines,
to
short­
term
studies
that
are
used
to
establish
baselines,
model
pollutant
cycles,
and
guide
direction
for
future
study
and
policy.
Examples
of
such
studies
include
the
National
Study
of
Chemical
Residues
in
Fish
that
was
conducted
in
the
late
1980s
(
a
follow­
up
to
that
study
is
currently
underway),
and
the
Lake
Michigan
Mass
Balance
Study
conducted
in
the
early
1990s.

3­
12
February
2003
Chapter
3
When
designing
a
study
or
monitoring
program,
EPA
uses
information
about
detection
and
quantitation
limits,
along
with
information
on
the
risks
associated
with
the
pollutant(
s)
of
interest
and
the
cost
of
measurement,
to
select
an
appropriate
method
for
measuring
the
pollutant.
Accepting
all
positively
valued
measurement
results
and
selecting
a
measurement
method
with
a
detection
limit
lower
than
the
level
of
concern
for
the
substance
being
measured
would
provide
some
assurance
that
measurement
results
associated
with
that
concentration
would
be
positively
valued.
Selecting
a
measurement
method
with
a
quantitation
limit
lower
than
the
level
of
concern
for
the
substance
being
measured
would
generate
measurement
results
that
are
easier
to
explain
to
the
data
user
and
the
general
public.

3.2.2
Descriptive
versus
Prescriptive
Uses
of
Lower
Limits
to
Measurement
The
literature
on
detection
and
quantitation
generally
assumes
that
these
procedures
are
descriptive,
as
opposed
to
prescriptive.
In
other
words,
detection
and
quantitation
studies
are
described
as
characterizing
the
current
performance
of
a
laboratory
or
laboratories
using
a
method
to
measure
a
substance.
Two
possible
reasons
for
this
treatment
are:
(
1)
the
intended
audience
includes
laboratory
staff
and
measurement
methods
developers
who
wish
to
make
new
methods
useable
to
as
many
laboratories
as
possible,
and
(
2)
the
author
may
have
an
institutional
reason
for
not
attempting
to
control
variability
and
thus
lower
detection
and
quantitation
limits.
On
the
other
hand,
the
technology­
based
and
water
quality­
based
effluent
limitations
programs
administered
by
EPA s
Office
of
Water
have
an
institutional
goal
of
protecting
human
health
and
the
environment.
Providing
this
protection
requires
that
the
Agency
be
able
to
measure
pollutants
at
ever
lower
concentrations.
Establishing
stringent
standards
and
a
compliance
scheme
for
laboratories
is
one
way
to
more
rapidly
develop
the
ability
to
measure
at
these
concentrations.
A
prescriptive
strategy
concerning
detection
and
quantitation
limits
would
be
to:

 
Determine
the
detection
and
quantitation
limits
at
multiple
laboratories.
 
Establish
a
detection
limit
and
a
quantitation
limit
for
the
method
that
is
based
on
some
performance
of
these
laboratories.
These
limits
could
be
established
as
the
limits
reported
by
the
mean
or
median
laboratory,
or
by
some
other
criterion,
such
as
the
pooled
value
of
the
limits
achieved
by
all
laboratories,
or
the
limits
that
are
met
by
a
certain
percentage
of
the
laboratories.
 
Use
the
established
detection
and
quantitation
limits
as
performance
standards
that
must
be
demonstrated
by
laboratories
that
practice
the
method.

Such
an
approach
is
consistent
with
other
performance
standards
included
in
EPA
methods,
such
as
standards
for
instrument
calibration,
recovery
of
spiked
reference
and
matrix
samples,
etc.

The
use
of
such
an
approach
would
help
ensure
that
prescriptive
detection
and
quantitation
limits
(
i.
e.,
performance
standards)
reflect
the
capabilities
of
multiple
laboratories,
rather
than
a
single
state­
of­
the­
art
research
laboratory.
Of
course,
it
is
possible
that
even
when
multiple
laboratories
are
used
to
establish
performance
standards
for
detection
and
quantitation,
some
laboratories
may
not
be
able
to
achieve
these
standards
using
their
current
operations.
However,
most
laboratories
facing
this
problem
should
be
able
to
achieve
these
standards
by
investing
in
staff
training,
improved
equipment,
a
stronger
quality
assurance
program,
or
higher
quality
maintenance
and
operations.

There
is
of
course,
a
risk
that
some
members
of
the
laboratory
community
will
not
be
able
to
meet
the
standard,
either
because
they
are
not
willing
to
invest
the
resources
necessary
to
do
so,
or
for
other
reasons.
That
risk
should
be
considered
when
using
a
prescriptive
approach
to
detection
and
quantitation
(
i.
e.,
establishing
limits
that
act
as
performance
standards).
Conversely,
the
risk
of
using
a
descriptive
approach
is
that
it
can
result
in
detection
and
quantitation
limits
that
reflect
a
broad
community
of
laboratories,
including
those
that
have
made
little
if
any
effort
to
control
variability
at
these
levels,
thus
raising
detection
and
quantitation
limits
to
a
level
that
is
higher
than
desired.

February
2003
3­
13
Assessment
of
Detection
and
Quantitation
Approaches
3.2.3
Compliance
Evaluation
Thresholds
3.2.3.1
Compliance
Evaluation
Thresholds
When
technology­
based
effluent
limitations
are
developed,
the
limits
are
expressed
as
being
at
or
above
the
quantitative
measurement
capabilities
(
e.
g.,
the
ML)
of
one
or
more
analytical
methods
that
are
available
to
support
compliance
monitoring.
Therefore,
it
is
possible
to
monitor
and
evaluate
permit
compliance
at
concentrations
with
an
accepted
degree
of
measurement
certainty.

A
situation
that
arises
frequently
in
addressing
water
quality­
based
limits
is
the
setting
of
the
permit
limit
below
the
detection
or
quantitation
limit
of
the
most
sensitive,
approved
analytical
method.
This
subject
was
addressed
in
EPA's
draft
National
Guidance
for
the
Permitting,
Monitoring,
and
Enforcement
of
Water
Quality­
based
Effluent
Limitations
Set
Below
Analytical
Detection/
Quantitation
Levels
(
WQBEL
guidance).
The
WQBEL
guidance
suggested
use
of
the
minimum
level
of
quantitation
(
ML)
as
the
compliance
evaluation
threshold
(
CET)
when
the
water
quality­
based
effluent
limit
(
WQBEL)
is
below
the
detection
or
quantitation
limit
of
the
most
sensitive,
approved
analytical
method.
In
comments
on
the
WQBEL
guidance,
the
regulated
industry
objected
to
the
CET,
claiming
that
it
did
not
include
interlaboratory
variability
and
other
sources
of
variability.
States
objected
to
the
CET,
claiming
that
it
would
not
allow
them
to
be
as
protective
as
if
the
detection
limit
were
used.
(
This
1994
draft
guidance
document
was
never
finalized
due
to
the
controversy.)

From
a
technical
standpoint,
a
one­
sided
limit
that
addresses
false
positives
only,
such
as
Currie's
critical
value
or
EPA's
MDL,
is
the
most
appropriate
approach
for
producing
a
CET
for
the
situation
in
which
the
WQBEL
is
less
than
detection
limit
in
the
most
sensitive
analytical
method
because
the
one­
sided
limit
allows
measurement
to
the
lowest
possible
level
while
protecting
a
discharger
from
the
risk
of
a
false
violation.
For
example,
consider
the
situation
in
which
2,3,7,8­
tetrachlorodibenzo­
p­
dioxin
(
dioxin)
is
to
be
evaluated
against
the
ambient
water
quality
criterion
of
13
parts­
per­
quintillion
(
ppqt).
The
most
sensitive
analytical
method
approved
at
40
CFR
part
136
is
EPA
Method
1613,
with
an
MDL
of
4
parts­
per­
quadrillion
(
ppq)
and
an
ML
of
10
ppq.
The
MDL
is
more
than
300
times
greater
than
the
ambient
criterion.
Therefore,
if
dioxin
is
detected
in
the
receiving
water
as
a
result
of
a
discharge
(
i.
e.,
the
measurement
result
is
greater
than
the
MDL
of
4
ppq),
there
must
have
been
an
exceedance
of
the
ambient
criterion.
In
the
WQBEL
guidance,
EPA
suggested
use
of
the
ML
because
it
was
the
point
at
which
the
measurement
could
be
considered
reliable.
However,
from
a
purely
technical
standpoint,
the
MDL
(
or
Currie's
critical
value)
is
most
appropriate
if
the
goal
is
to
protect
the
receiving
water.
It
is
important
to
note,
however,
that
because
individual
states
are
responsible
for
implementation
and
enforcement
of
NPDES
permits,
use
of
the
MDL
and
ML
as
regulatory
reporting
and
compliance
evaluation
thresholds
varies
among
the
states.

Detection
and
quantitation
limits
have
been
used
to
establish
CETs
and
permit
limits.
For
example,
see
application
of
the
ML
to
establishment
of
permit
limits
in
Procedure
8
of
Appendix
F
of
Water
Quality
Guidance
for
the
Great
Lakes
System
at
40
CFR
part
132.
However,
EPA
believes
that
the
decision
to
establish
CETs
based
on
a
detection
or
quantitation
limit
is
a
separate
issue
from
the
question
of
which
detection
and
quantitation
limit
approach
is
most
valid.
The
objective
of
the
assessment
described
in
this
document
is
to
evaluate
the
merits
of
each
approach.

3.2.4
Accepting
the
Procedures
of
Voluntary
Consensus
Standards
Bodies
In
February
1996,
Congress
enacted
Public
Law
104­
113
(
15
USC
3701),
the
National
Technology
Transfer
and
Advancement
Act
(
NTTAA).
This
act
directs
"
federal
agencies
to
focus
upon
increasing
their
use
of
(
voluntary
consensus)
standards
whenever
possible,
thus
reducing
federal
procurement
and
operating
costs."
The
Act
gives
Federal
agencies
discretion
to
use
other
standards
3­
14
February
2003
Chapter
3
where
the
use
of
voluntary
consensus
standards
would
be
"
inconsistent
with
applicable
law
or
otherwise
impractical."

The
NTTAA
is
implemented
by
Federal
agencies
based
on
the
policies
described
in
Circular
A­
119
from
the
Office
of
Management
and
Budget
(
OMB).
The
current
version
of
this
OMB
circular
was
published
in
the
Federal
Register
on
February
19,
1998
(
63
FR
8546).

Neither
the
NTTAA
nor
Circular
A­
119
require
that
agencies
replace
existing
government
standards
with
standards
from
a
voluntary
consensus
standard
body
(
VCSB).
In
other
words,
if
EPA
already
has
standards
in
place
for
detection
and
quantitation
approaches,
EPA
is
not
obligated
by
NTTAA
to
replace
these
with
VCSB
standards.

Circular
A­
119
also
discusses
the
effect
of
the
policy
on
the
regulatory
authorities
and
responsibilities
of
Federal
agencies.
The
circular
states
that:

"
This
policy
does
not
preempt
or
restrict
agencies'
authorities
and
responsibilities
to
make
regulatory
decisions
authorized
by
statute.
Such
regulatory
authorities
and
responsibilities
include
determining
the
level
of
acceptable
risk;
setting
the
level
of
protection;
and
balancing
risk,
cost,
and
availability
of
technology
in
establishing
regulatory
standards.
However,
to
determine
whether
established
regulatory
limits
or
targets
have
been
met,
agencies
should
use
voluntary
consensus
standards
for
test
methods,
sampling
procedures,
or
protocols."

Thus,
EPA
is
responsible
for
establishing
the
levels
of
risk
and
protection,
not
only
for
the
regulatory
limits
applied
to
discharges,
but
also
to
the
risks
of
decision
errors
(
e.
g.,
false
negatives
or
false
positives)
in
the
detection
and
quantitation
approaches
applicable
under
the
Clean
Water
Act.

Finally,
Circular
A­
119
describes
two
types
of
technical
standards:
performance
standards
and
prescriptive
standards.
A
performance
standard
is
defined
as:

"
a
standard
...
that
states
requirements
in
terms
of
required
results
with
criteria
for
verifying
compliance
but
without
stating
the
methods
for
achieving
required
results."
In
contrast,
a
prescriptive
standard
is
one
"
which
may
specify
design
requirements,
such
as
materials
to
be
used,
how
a
requirement
is
to
be
achieved,
or
how
an
item
is
to
be
fabricated
or
constructed."

Neither
the
NTTAA
nor
Circular
A­
119
direct
agencies
to
favor
performance
standards
over
prescriptive
standards,
or
vice
versa.
EPA
believes
that
the
current
MDL
procedure
is
a
prescriptive
standard,
in
that
it
specifies
both
the
design
of
the
MDL
study
and
how
the
requirement
to
establish
method
sensitivity
be
achieved.
There
is
some
obvious
flexibility
or
opportunity
for
judgement
in
employing
the
MDL
procedure,
and
much
of
the
historical
debate
over
the
utility
of
the
MDL
procedure
would
suggest
that
it
may
not
be
prescriptive
enough.
The
alternative
approaches
for
establishing
detection
and
quantitation
are
also
prescriptive
standards,
rather
than
performance
standards.

One
option
that
EPA
may
consider
is
to
employ
a
performance­
based
approach
to
establishing
detection
and
quantitation
limits
in
which
method
developers,
laboratories,
and
others
would
be
free
to
use
any
one
of
a
variety
of
approaches
to
establishing
these
limits,
including
the
existing
MDL
procedure
or
a
VCSB
standard.
Thus,
establishing
method
sensitivity
could
be
considered
a
performance
standard
under
NTTAA
and
Circular
A­
119,
rather
than
a
prescriptive
standard.
The
fact
that
different
approaches
(
prescriptive
standards)
yield
different
answers
would
be
immaterial
if
EPA
evaluates
the
answers
relative
February
2003
3­
15
Assessment
of
Detection
and
Quantitation
Approaches
to
a
specific
decision.
That
evaluation
should
not
be
divorced
from
knowledge
of
the
decision
to
be
made
(
e.
g.,
the
regulatory
limit
for
a
given
pollutant).

3.2.5
National
versus
Local
Standards
for
Measurement
In
accordance
with
the
Settlement
Agreement,
EPA
is
re­
examining
the
approaches
of
detection
and
quantitation
used
with
methods
approved
for
use
at
40
CFR
part
136.
The
Clean
Water
Act
authorizes
states
and
local
governments
to
implement
permits,
with
the
requirement
that
they
be
at
least
as
protective
(
stringent)
as
the
national
standards
established
by
EPA.
Thus,
EPA
must
take
into
account
the
impact
of
any
revised
or
new
detection/
quantitation
limit
approaches
and
procedures
on
state
and
local
governments,
as
well
as
on
those
affected
by
state
and
local
requirements.
EPA
also
is
aware
that
some
states
have
implemented
approaches
to
detection
and
quantitation
that
are
either
specific
to
that
state,
result
in
lower
numeric
limits
in
discharge
permits,
or
both.
Given
the
ability
of
state
and
local
governments
to
use
more
stringent
approaches,
any
decision
by
EPA
with
regard
to
this
re­
evaluation
of
detection
and
quantitation
approaches
may
not
have
an
effect
on
those
states
and
local
governments.

3.2.6
Cost
and
Implementation
Issues
Detection
and
quantitation
limit
procedures
are
typically
employed
by
organizations
that
develop
methods
and
by
laboratories
that
use
the
methods.
Method
developers
typically
include
governmental
organizations
such
as
EPA,
NOAA,
USGS,
and
DOE,
or
voluntary
consensus
standards
bodies
(
VCSBs)
such
as
the
American
Public
Health
Association
(
APHA),
ASTM
International,
AOAC­
International,
and
ISO/
IUPAC.
Method
developers
also
may
include
manufacturers
of
instruments
or
supplies
used
in
testing.
Methods
users
generally
are
the
laboratories
performing
tests
to
assess
and
assure
product
quality,
to
support
regulatory
compliance
monitoring,
or
to
support
scientific
studies.

Method
development
requires
a
more
diverse
set
of
skills
than
method
use
because
such
development
generally
demands
an
understanding
of
quality
systems,
statistics,
and
analytical
technologies.
Staff
working
for
the
method
developer
will
usually
include
a
project
manager,
measurement
analysts,
and
statisticians.
Method
use
requires
a
focus
on
obtaining
reliable
results
in
the
analysis
of
a
given
sample.
Staff
working
for
laboratory
typically
include
the
manager
and
measurement
analysts.

3.2.6.1
Implementation
of
a
Detection/
Quantitation
Limit
Procedure
by
a
Method
Developer
The
basic
resources
available
to
the
method
developer
are
time,
money,
and
the
technical
skills
of
its
staff.
The
fundamental
decision
for
implementing
a
detection
or
quantitation
procedure
is
whether
that
procedure
is
intended
to
characterize
the
performance
of
the
method
at
a
well­
performing
laboratory
or
if
it
is
intended
to
characterize
the
performance
of
the
method
across
a
group
of
laboratories.
If
the
procedure
is
intended
to
characterize
the
performance
of
the
method
across
a
group
of
laboratories,
it
is
also
necessary
to
decide
if
there
will
be
some
way
to
compare
the
performance
of
individual
laboratories
to
the
group
performance
standard.
There
are
serious
time,
cost,
and
skill
issues
with
each
of
these
decisions.
Ordering
these
decisions
from
the
least
resource
intensive
to
the
most,
they
are
characterizing
the
performance
of
the
method:
(
1)
at
a
well­
performing
laboratory,
(
2)
at
a
group
of
laboratories,
or
(
3)
at
a
group
of
laboratories
with
comparisons
of
individual
laboratories.
Other
costs
for
the
method
developer
could
include
planning,
data
management,
reference
laboratory
services,
and
whether
laboratories
are
willing
to
volunteer
for
the
study
or
if
their
services
must
be
purchased.

An
independent
decision
is
whether
to
assume
a
simple
model
for
measurement
variability
and
limit
the
number
of
test
concentrations,
iterate
assuming
a
simple
model,
or
to
design
a
study
of
the
relationship
between
measurement
variation
and
the
concentrations
of
the
substances
measured
by
the
3­
16
February
2003
Chapter
3
method.
This
decision
will
greatly
influence
the
number
of
samples
measured
in
the
study.
If
the
laboratories
do
not
volunteer
for
the
study,
then
the
direct
cost
for
measuring
these
samples
or
blanks
ranges
from
a
few
dollars
per
sample
to
more
than
$
1,000
per
sample
for
some
analytes.
Until
such
time
as
the
relationship
between
measurement
results
and
standard
concentrations
becomes
well
known,
such
studies
will
require
the
active
participation
of
professional
statisticians
in
design,
implementation,
and
analysis.

3.2.6.2
Implementation
of
a
Detection/
Quantitation
Limit
Procedure
by
a
Laboratory
A
laboratory
may
implement
detection
or
quantitation
procedures
for
its
own
quality
control
purposes,
because
of
regulatory
requirements,
or
as
part
of
the
study
of
a
method
by
some
other
organization.
When
participating
in
the
study
of
another
organization,
the
laboratory
may
voluntarily
accept
some
cost
of
the
study
for
marketing
purposes,
professional
development,
or
to
benchmark
the
performance
of
the
laboratory.

In
each
case,
a
detection
or
quantitation
limit
approach
will
be
of
little
utility
if
it
is
not
capable
of
being
implemented
by
the
laboratory.
An
advantage
of
straightforward
approaches
such
as
the
EPA
MDL,
the
ACS
limit
of
detection,
and
the
ISO/
IUPAC
critical
value
is
that
they
can,
in
principle,
be
understood
by
analysts
expected
to
use
the
approach.
Likewise,
the
procedures
described
for
implementing
the
MDL
approach
are
straightforward
and
have
been
implemented
by
thousands
of
laboratories.
In
contrast,
the
ASTM
IDE
and
IQE
procedures
are
highly
complex
and,
as
a
consequence,
are
beyond
the
capability
of
nearly
all
environmental
testing
laboratories.

Another
disadvantage
of
highly
complex
procedures
is
that
they
are
usually
more
costly
to
implement
than
simple
procedures.
As
noted
in
Section
3.1.5,
method
performance
generally
improves
over
time,
and
EPA
believes
that
a
detection
and
quantitation
limit
approach
should
be
supported
by
procedures
that
will
allow
individual
laboratories
and
other
organizations
to
affordably
characterize
such
improvement.
A
significant
shortcoming
of
interlaboratory
procedures
is
that
small
laboratories
that
develop
new
techniques
or
modify
existing
techniques
to
achieve
improved
measurement
sensitivity
would
have
to
rely
on,
and
perhaps
even
pay,
other
laboratories
to
demonstrate
the
sensitivity
of
their
procedures.
Such
a
limitation
has
the
effect
of
hindering
method
development
and
improvement.

3.2.7
Use
of
a
pair
of
related
detection
and
quantitation
procedures
in
all
Clean
Water
Act
applications.

In
Section
3.2.1,
we
discussed
several
different
applications
for
detection
and
quantitation
limits
under
the
Clean
Water
Act.
To
review,
these
applications
are:

 
Method
development
and
promulgation,
 
Method
performance
verification
at
a
laboratory,
 
Technology­
based
effluent
guidelines
development,
 
Water
quality­
based
effluent
limits
development,
 
Permit
compliance
monitoring,
and
 
Non­
regulatory
studies
and
monitoring.

Although
EPA
could
develop
a
separate
detection
and
quantitation
approach
for
each
of
these
applications
and
attempt
to
define
and
evaluate
each
of
these
approaches
in
our
re­
examination
of
detection
and
quantitation
approaches,
the
resulting
matrix
of
applications
and
approaches
would
cause
confusion
for
regulators,
permittees,
and
the
laboratory
community.
Further,
when
proposed,
each
element
of
the
matrix
of
approaches
and
applications
would,
individually,
be
subject
to
contention
and
second­
guessing,
and
it
is
likely
that
the
outcome
would
be
nearly
the
same
as
if
a
single
pair
of
February
2003
3­
17
Assessment
of
Detection
and
Quantitation
Approaches
approaches
is
selected.
EPA
prefers
to
avoid
this
confusion
by
using
a
single
pair
of
related
detection
and
quantitation
procedures
to
meet
any
the
needs
of
all
Clean
Water
Act
applications.

3.2.8
Alternative
Procedures
One
of
the
peer
reviewers
who
evaluated
a
draft
version
of
this
assessment
document
noted
that:

"
EPA
has
stated
in
the
TSD
that
one
primary
procedure
is
needed
for
clarity
and
to
avoid
confusion
among
stakeholders.
If
alternate
procedures
are
needed,
the
EPA
Clean
Air
Act
system
of
reference
and
equivalent
methods
has
worked
well,
and
could
be
a
model
for
EPA
to
follow
under
the
Clean
Water
Act."
(
Cooke,
2002)

The
system
of
reference
methods
used
under
the
Clean
Air
Act
is
similar
to
the
existing
"
alternate
test
procedure"
(
ATP)
program
for
analytical
methods
currently
used
within
the
Office
of
Water.
The
difference
between
the
ATP
program
and
the
case
of
the
procedures
for
establishing
detection
and
quantitation
limits
is
that
in
an
ATP
program,
the
goal
is
clear
and
agreed
upon,
whereas
there
remain
fundamental
theoretical
issues
surrounding
detection
and
quantitation.

For
example,
when
a
test
procedure
is
developed
for
use
in
the
Clean
Air
Act
or
Clean
Water
Act
programs,
the
reference
method
is
designed
to
measure
Analyte
X,
in
Matrix
Y,
at
some
concentration
related
to
a
regulatory
need
(
i.
e.,
a
compliance
limit).
Alternative
procedures
may
be
capable
of
making
measurements
of
Analyte
X
in
Matrix
Y,
at
the
level
of
concern,
using
completely
different
instrumentation.
Thus,
the
demonstration
of
equivalency
between
the
reference
method
and
a
possible
alternative
method
is
judged
using
a
metric
that
consists
of
Analyte
X,
Matrix
Y,
and
the
level
of
concern,
as
well
as
other
aspects
of
method
performance.

In
contrast,
the
primary
differences
between
the
EPA
MDL/
ML
approaches
and
potential
alternatives
such
as
the
ASTM
IDE
and
IQE
are
related
to
the
philosophical
differences
of
how
detection
and
quantitation
limits
should
be
derived
and
applied.
These
differences
are
described
at
length
in
this
chapter
and
the
rest
of
the
Assessment
Document.
Therefore,
EPA
does
not
believe
that
a
variant
of
existing
ATP
programs
is
likely
to
be
an
effective
model
for
assessing
other
detection
and
quantitation
procedures.

What
EPA
would
be
willing
to
consider
is
that
an
analytical
method
from
a
VCSB
or
other
source
may
be
acceptable
for
approval
at
40
CFR
Part
136
and
use
in
Clean
Water
Act
programs
even
if
it
employs
an
alternative
procedure
for
establishing
method
sensitivity.
For
example,
consider
the
theoretical
situation
of
an
ASTM
method
for
the
determination
of
an
analyte
regulated
under
the
NPDES
program
that
uses
the
IDE
or
IQE
to
describe
method
sensitivity
and
for
which
the
value
of
the
IDE
or
IQE
was
below
the
relevant
regulatory
limit.
EPA
would
evaluate
the
overall
performance
of
such
a
method
for
approval
at
40
CFR
Part
136,
despite
the
fact
that
the
method
did
not
contain
an
MDL
determined
using
the
Appendix
B
procedure.

3.3
Statistical
Issues
The
goal
of
this
section
is
to
provide
a
brief
explanation
of
the
key
statistical
issues
involved
in
the
development
of
detection
and
quantitation
limits.

3­
18
February
2003
Chapter
3
3.3.1
Sources
of
Variability
Various
known
and
unknown
sources
of
variability
will
influence
measurements
made
by
a
laboratory
using
a
specific
method.
These
sources
may
include
random
measurement
error,
differences
in
analysts,
variations
between
different
equipment
manufacturers
and
models,
variations
in
analytical
standards,
routine
fluctuations
in
equipment
performance,
and
variations
in
facility
conditions
(
e.
g.,
varying
levels
of
background
contributions).

There
are
a
number
of
ways
in
which
variability
can
be
controlled.
One
is
a
strong
quality
assurance
(
QA)
program
that
includes
use
of:
1)
trained
and
qualified
staff,
2)
properly
maintained
equipment,
3)
standards
that
are
fresh
and
properly
prepared
and
stored,
4)
written
standard
operating
procedures
and
methods
for
all
sample
handling,
analysis,
and
data
reduction/
reporting
activities,
5)
procedures
for
monitoring
ongoing
laboratory
performance
and
6)
quality
control
(
QC)
samples
and
QC
acceptance
criteria
to
ensure
that
the
laboratory
systems
are
in
control.
The
EPA
methods
promulgated
at
40
CFR
part
136
require
the
use
of
qualified
staff,
appropriately
cleaned
and
calibrated
equipment,
and
properly
prepared
standards.
Each
method
also
provides
detailed
steps
for
performing
all
sampling
handling
and
analysis
activities.

Even
when
prescribed
EPA
requirements
are
implemented,
however,
it
is
not
possible
to
completely
eliminate
all
variability
within
or
between
laboratories.
The
potential
effects
of
sources
of
variability
should
be
considered
when
establishing
detection
and
quantitation
limits.
Even
with
procedures
in
place
to
control
quality
and
reduce
variability,
it
should
be
recognized
that
some
laboratories
may
achieve
lower
detection
and
quantitation
limits
than
others.
Ultimately,
some
laboratories
may
not
be
capable
of
meeting
low­
level
measurement
requirements
without
some
effort
to
improve
operations.

One
of
the
peer
reviewers
asked
to
evaluate
EPA's
reassessment
pointed
out
that
EPA
should
consider
added
sources
of
variability,
such
as
matrix
interferences,
when
developing
detection
and
quantitation
limits
(
Wait,
2002).
Such
sources
of
variability
are
considered
in
establishing
detection
and
quantitation
limits
for
analytical
methods
under
EPA's
Clean
Water
Act
programs
because
these
detection
and
quantitation
limits
are
established
in
a
single­
laboratory
study,
verified
in
multiple
single­
laboratories,
and,
where
necessary,
further
verified
in
an
interlaboratory
study.
If
matrix
effects
surface
in
these
studies,
either
procedures
are
put
in
place
to
overcome
matrix
interferences,
or
the
detection
and
quantitation
limits
are
adjusted
upward
to
account
for
these
matrix
effects.
However,
it
has
been
EPA's
experience
that
concern
over
matrix
effects
may
be
somewhat
overblown.
For
example,
in
EPA's
interlaboratory
validation
studies
of
the
600­
series
wastewater
methods,
the
recoveries
of
some
organic
analytes
from
some
real­
world
matrices
were
closer
to
100%
than
from
a
reagent
water
matrix.
This
effect
is
thought
to
be
attributable
to
dissolved
solids
in
the
real­
world
matrix
that,
in
effect,
"
salt
out"
the
organic
compounds.

What
EPA
has
not
done,
and
does
not
believe
is
appropriate,
is
to
aggressively
pursue
matrix
effects
in
method
development
or
in
establishing
detection
and
quantitation
limits
(
i.
e.,
EPA
has
not
attempted
to
find
worst­
case
matrices
in
order
to
maximally
exacerbate
matrix
effects).
EPA
has
not
pursued
this
approach
because,
for
any
method,
it
is
possible
to
contrive
and
synthesize
a
sample
matrix
that
would
render
the
method
unusable
(
e.
g.,
by
saturating
an
sample
in
which
organic
analytes
are
to
be
determined
with
a
wide
range
of
polymeric
materials,
both
water
soluble
and
water
insoluble).
Rather,
EPA
considers
the
type
of
matrix
that
would
be
encountered
in
a
wastewater
discharge
and
that
would
be
regulated
under
the
Clean
Water
Act
(
e.
g.,
the
effluents
that
are
discharged
from
properly
designed
and
operated
secondary
treatment
plants).

February
2003
3­
19
Assessment
of
Detection
and
Quantitation
Approaches
A
source
of
variability
that
is
not
considered
in
any
of
the
detection
and
quantitation
limits
is
the
variability
that
is
associated
with
the
sample
itself.
Detection
and
quantitation
limits
focus
exclusively
on
the
capabilities
of
the
measurement
process.
However,
measurements
are
only
as
reliable
as
the
sample
being
measured.
If
the
sample
is
not
truly
representative
of
the
population
from
which
it
was
collected,
then
the
variability
associated
with
measurements
made
in
the
region
of
detection
or
quantitation
may
be
immaterial.

For
example,
EPA s
Technology
Innovation
Office
conducted
a
study
to
characterize
the
effects
of
sampling
variability
on
measured
results.
In
that
study,
results
from
seven
discrete
samples
collected
within
a
two­
foot
distance
of
one
another
were
evaluated.
Each
sample
was
analyzed
for
the
explosive
TNT
on­
site,
using
a
colorimetric
test
kit,
and
in
a
traditional
laboratory
using
EPA
SW­
846
Method
8330
(
high­
performance
liquid
chromatography).
Analysis
of
the
results
from
these
measurements
indicated
that
95%
of
the
total
variability
was
due
to
sampling
location
and
only
5%
was
due
to
differences
between
the
analytical
methods.
Put
another
way,
differences
in
sampling
location
caused
19
times
more
uncertainty
in
the
data
results
than
did
the
choice
of
analytical
method,
over
a
distance
of
only
2
feet
(
Crumbling,
2002).
While
EPA
does
not
wish
to
diminish
the
importance
of
understanding
measurement
error
in
the
region
of
detection
and
quantitation,
EPA
believes
it
is
equally
important
to
understand
it
in
the
context
of
the
overall
sampling
and
analysis
error.

3.3.2
Censoring
Measurement
Results
Measurement
results
are
often
reported
as
less
than
some
detection,
quantitation,
or
reporting
limit
(
see
Section
3.2.1.3,
Permit
Compliance
Monitoring)
without
providing
a
single
best
estimate
for
the
numeric
result.
For
example,
if
a
direct
reading
of
the
measurement
results
would
indicate
a
concentration
of
3
mg/
L
and
the
reporting
limit
for
the
substance
is
5
mg/
L,
the
laboratory
may
only
report
that
the
measurement
result
is
less
than
5
mg/
L.
Statisticians
call
this
process
of
suppression
of
results
less
than
a
specified
amount
 
censoring. 
Reasons
for
the
practice
of
censoring
relate
directly
to
issues
surrounding
the
development
of
detection
and
quantitation
limits
(
i.
e.,
the
premise
that
measurement
results
below
certain
low
levels
may
not
useable
for
certain
purposes).

In
order
to
evaluate
low­
level
variability,
EPA
conducted
a
comprehensive
study
of
results
from
1/
10th
of
the
MDL
to
concentrations
into
the
usual
quantitation
range.
Ten
different
analytical
techniques
were
evaluated
in
the
study
(
see
Appendix
B,
Characterizing
Measurement
Variability
as
a
Function
of
Analyte
Concentration
for
a
Variety
of
Analytical
Techniques).
Data
from
this
study
indicate
that
measurement
results
may
be
generated
at
low
concentrations
which
are
quite
variable
in
relation
to
the
true
concentration
in
the
sample.
While
this
observation
has
not
been
demonstrated
with
every
substance
measured,
it
is
suggested
by
plots
of
data
from
most
of
the
measurement
techniques
observed
in
the
study.
An
example
is
Ammonia
as
Nitrogen
by
Method
350.3.
Plotting
measurement
results
versus
concentrations
spiked
into
reagent
water
samples
(
Figure
3­
1),
we
see
the
strong
relationship
between
measurement
results
and
spike
concentrations
that
would
be
expected.
However,
it
is
difficult
to
see
what
is
going
on
at
low
concentrations
in
a
graphic
that
covers
measurement
results
over
several
orders
of
magnitude.

3­
20
February
2003
Chapter
3
1.2
1.0
0.8
0.6
0.4
0.2
0.0
Classicals,
Method
350.3Classicals,
350.3
AMMONIA
AS
NITROGEN,
MG/
LAMMONIA
L
Log­
10
of
Measured
Concentration
Measured
Concentration
0.0
0.2
0.4
0.6
0.8
1.0
Spike
Concentration
Figure
3­
1
By
plotting
the
measurement
results
versus
the
spike
concentrations
on
log
scales
(
Figure
3­
2),
we
would
expect
to
see
an
expansion
of
variability
at
low
concentrations,
a
contraction
of
variability
at
high
concentrations
and
points
mostly
plotted
along
a
45
°
angle
to
indicate
that
measurement
results
are
approximately
equal
to
the
spike
concentrations.
However,
we
only
see
the
45
°
angle
at
higher
concentrations.
Measurement
results
in
the
lowest
order
of
magnitude
appear
to
have
reached
a
plateau
below
which
they
do
not
go.
ASTM
Committee
E­
1
has
termed
the
model
that
describes
the
general
pattern
displayed
by
these
data
as
the
"
General
Analytical
Error
Model."

1.00
0.50
0.10
0.05
0.001
0.005
0.010
0.050
0.100
0.500
1.000
Classicals,
Method
350.3Classicals,
350.3
AMMONIA
AS
NITROGEN,
MG/
LAMMONIA
L
Log­
10
of
Spike
Concentration
Figure
3­
2
February
2003
3­
21
Assessment
of
Detection
and
Quantitation
Approaches
Despite
such
evidence
that
results
at
low
concentrations
can
be
quite
variable,
some
data
users
prefer
to
use
the
actual
measurement
results
(
even
if
they
are
negative
values),
rather
than
to
censor
the
results
at
a
reporting
or
detection
limit,
because
censoring
data
at
such
a
limit
can
introduce
bias
into
the
data
set.
If
all
low
values
are
eliminated,
then
the
mean
of
the
remaining
data
would
have
a
positive
bias.
In
other
words,
while
negative
or
extremely
low
values
may
be
considered
problematic
by
some,
they
are
of
value
to
statisticians
and
modelers,
especially
when
dealing
with
analytes
that
are
highly
toxic
and/
or
environmentally
significant.

Some
programs,
such
as
EPA's
Superfund
Contract
Laboratory
Program,
require
laboratories
to
report
the
measurement
result
obtained
from
the
analysis
in
conjunction
with
a
qualifier
that
the
result
is
below
a
specified
detection,
quantitation,
or
reporting
level.
Going
back
to
the
example
in
the
first
paragraph,
the
laboratory
might
report
both
a
measured
value
of
3
mg/
L
and
a
reporting
limit
of
5
mg/
L.
Under
certain
assumptions,
measurement
results
below
the
specified
level
could
then
be
used
to
calculate
averages
and
statistical
estimates
that
would
be
superior
to
estimates
calculated
using
censored
data.

EPA
believes
that
such
an
approach
provides
the
greatest
degree
of
flexibility
for
data
users,
but
also
believes
that
it
should
be
used
with
care.
First,
data
users
who
choose
to
use
values
reported
below
a
detection
or
quantitation
limit
need
to
have
a
firm
understanding
of
the
limitations
of
those
data.
Second,
and
as
noted
in
Section
3.2.1.3,
Permit
Compliance
Monitoring,
reporting
data
below
a
detection
or
quantitation
limit
can
lead
to
misinterpretation.

One
of
the
peer
reviewers
that
evaluated
EPA s
assessment
of
detection
and
quantitation
limit
approaches
noted
that
European
Union
(
EU)
has
adopted
another
variant
for
reporting
or
censoring
data.

"
In
this
case,
the
EU
has
adopted
EPA
Method
1613B
(
for
analysis
of
dioxins
and
furans)
as
well
as
EPA s
MDL
approach.
However,
the
EU
has
further
specified
that
the
MDL
be
used
as
an
Upper
Bound
reporting
limit
where
all
non­
detects
are
found
in
the
analysis
of
human
or
animal
foodstuff.
This
forces
laboratories
to
achieve
levels
available
with
modern
instrumentation,
otherwise,
the
Upper
Bound
reporting
level
is
above
the
regulatory
compliance
level,
and
the
data
(
or
foodstuffs)
are
rejected"
(
Cooke,
2002).

EPA
agrees
that
this
approach,
which
yields
a
?
worst­
case 
(
or
highest
possible)
estimate
of
the
pollutant
concentration,
can
serve
as
a
useful
regulatory
tool
for
encouraging
the
analytical
and
regulated
community
to
pursue
measurements
at
the
lowest
levels
necessary
to
protect
human
and
ecological
health.
However,
EPA
also
cautions
that
this
approach
also
should
be
recognized
as
a
regulatory
strategy
that
effectively
censors
measurements
made
below
the
MDL.

EPA
believes
that
while
the
issue
of
censoring
is
important,
it
should
not
be
a
consideration
when
selecting
a
detection
and
quantitation
limit
approach.
The
decision
to
censor
data
is
a
data
reporting
and
data
use
issue,
rather
than
a
detection
and
quantitation
issue.
This
issue
will
apply
regardless
of
what
detection
or
quantitation
limit
approach
is
used.
The
EU
approach
reflects
a
similar
point
of
view,
in
that
it
relies
on
the
MDL
as
a
detection
approach,
and
also
establishes
this
limit
as
the
reporting
level
for
non­
detects
in
order
to
encourage
development
of
lower
MDLs.

3.3.3
Outliers
Outliers
are
extreme
or
aberrant
measurement
values
that,
on
inspection,
do
not
follow
the
characteristics
of
a
set
of
data.
Outliers
may
be
generated
by
a
number
of
causes,
such
as
errors
in
following
an
analytical
procedure,
errors
in
recording
numerical
results,
or
the
result
of
extreme
random
variation
in
a
properly
operating
process.
For
example,
if
a
new
measurement
method
is
being
tested
but
3­
22
February
2003
Chapter
3
the
laboratory
fails
to
follow
the
procedure
correctly
with
some
samples,
the
associated
measurement
results
may
stand
out
as
outliers.
A
graphic
example
is
provided
in
Figure
3­
3,
which
shows
measurement
results
for
aluminum,
determined
using
EPA
Method
1620,
versus
concentration.
At
a
spike
concentration
of
250
:
g/
L,
one
of
the
measured
values
is
about
750
:
g/
L,
and
visually
stands
out
from
the
rest
of
the
values.
This
result
may
turn
out
to
be
an
outlier.

A
common
process
for
identifying
potential
outliers
is
to
apply
one
or
more
statistical
procedures
for
identifying
values
far
from
the
mean
(
average)
of
the
data.
An
example
of
such
a
procedure
is
ASTM
Practice
D­
2777.
Because
extreme
values
can
be
expected
to
occur
on
occasion,
it
is
not
necessarily
appropriate
to
exclude
them
from
the
measurement
results
used
to
develop
detection
or
quantitation
values.
As
recommended
in
the
ASTM
procedure,
a
review
of
the
analyst's
records
associated
with
the
measurement
may
establish
whether
the
extreme
value
was
caused
by
failure
to
follow
the
method
or
by
some
rare
event
associated
with
the
method.
If
the
method
under
study
was
not
followed,
it
is
appropriate
to
exclude
the
measurement
result
from
the
detection
or
quantitation
analysis.
If
the
measurement
result
is
a
rare
event
associated
with
the
method
under
study
it
may
also
be
appropriate
to
exclude
the
measurement
result
from
the
results
in
the
study.

Measured
Concentration
1500
1000
500
0
Metals,
Method
1620Metals,
1620
ALUMINUM,
UG/
LALUMINUM,
L
0
200
400
600
800
1000
1200
Spike
Concentration
Figure
3­
3
Influential
early
work
in
using
ranking
procedures
to
help
identify
outlying
laboratories
in
studies
was
conducted
by
Youden
(
Youden,
W.
J.
and
E.
H.
Steiner,
Statistical
Manual
of
the
Association
of
Official
Analytical
Chemists,
1975).

February
2003
3­
23
Assessment
of
Detection
and
Quantitation
Approaches
3.3.4
Criteria
for
the
Selection
and
Appropriate
Use
of
Statistical
Models
Detection
and
quantitation
limits
may
be
based
on
statistical
models
of
the
relationship
between
measurement
variation
and
the
concentration
of
a
substance
in
the
sample.
Results
are
produced
by
adding
varying
known
amounts
of
the
substance
to
the
sample
( 
spiking ),
making
replicate
measurements
at
each
concentration,
and
modeling
the
variability
of
the
results
as
a
function
of
concentration.
This
section
summarizes
the
history
of
modeling
variability
versus
concentration,
considers
criteria
for
selecting
models,
and
discusses
current
practices
with
regard
to
available
data.

3.3.4.1
Short
History
of
Modeling
Measurement
Results
Over
time,
a
number
of
different
models
have
been
used
to
estimate
measurement
variation.
Currie
(
1968)
modeled
variation
in
radiochemical
measurement
methods
using
a
procedure
associated
with
counting
large
numbers
of
distinct
objects
which
are
appropriately
modeled
with
the
Poisson
distribution.
However,
he
relied
on
large
sample
sizes
and
standard
normal
distributions
to
describe
all
other
types
of
measurement
methods.
Hubaux
and
Vos
(
1970)
developed
a
procedure
based
on
an
estimated
calibration
relationship
that
uses
smaller
sample
sizes
to
estimate
Currie s
detection
and
quantitation
limits.
Again,
measurement
results
were
assumed
to
follow
standard
normal
distributions,
but
it
was
also
assumed
that
measurement
variation
was
constant
throughout
the
range
of
interest.
Similarly,
Glaser
et
al.
(
1981)
suggested
that
measurement
variation
increases
linearly
with
concentration,
but
they
did
not
provide
estimators
under
this
theory
because
they
believed
that
measurement
variation
is
usually
approximately
constant
in
the
range
of
detection.
Glaser
et
al.
(
1981)
did
suggest
that,
when
appropriate
data
were
available,
a
linear
regression
analysis
of
the
relationship
over
the
analytical
range
be
performed.
Clayton
et
al.
(
1987)
discussed
transforming
the
measurement
results
(
using
logarithms
or
square
root
functions).
Gibbons
et
al.
(
1991)
suggested
that
measurement
variability
may
be
proportional
to
concentration.
Rocke
and
Lorenzato
(
1995)
proposed
a
model
motivated
by
physical
characteristics
of
measurement
processes,
in
which
measurement
variability
is
approximately
constant
at
low
concentrations,
but
changes
in
a
continuous
mathematical
manner
to
a
relationship
where
variability
increases
as
concentration
increases.

Figure
3­
4
illustrates
the
fundamental
analytical
measurement
models
in
linear
and
logarithmic
domains.
The
models
are
applicable
to
nearly
all
analytical
measurements;
we
will
not
deal
with
the
exceptions
because
they
represent
a
small
percentage
of
cases.
As
can
be
seen
from
the
top
two
graphs,
response
is
a
linear
function
of
concentration
in
both
the
linear
and
log
domains.
The
middle
two
graphs
and
the
bottom
two
graphs
are
those
most
pertinent
to
the
discussion
detection
and
quantitation.

3­
24
February
2003
Chapter
3
Response
vs.
Concentration
Response
vs.
Concentration
Linear
Domain
Log­
Log
Domain
100
100
80
10
60
1
40
0.1
20
0.01
0
0.001
0
20
40
60
80
100
0.01
0.1
1
10
100
1000
Concentration
Concentration
SD
vs.
Concentration
SD
vs.
Concentration
Linear
Domain
Log­
Log
Domain
10
100
8
10
6
1
4
2
0.1
0
0.01
0
20
40
60
80
100
0.01
0.1
1
10
100
1000
Concentration
Concentration
RSD
vs.
Concentration
RSD
vs.
Concentration
Linear
Domain
Log­
Log
Domain
100
1000
80
100
60
40
10
20
0
1
0
20
40
60
80
100
0.01
0.1
1
10
100
1000
Concentration
Concentration
Figure
3­
4
3.3.4.1.1
Detection
Limits
Using
Variability
at
Low
Concentrations
The
middle
two
graphs
in
Figure
3­
4
show
variability
versus
concentration
and
show
the
model
postulated
by
Rocke
and
Lorenzato.
The
flat
(
constant)
portion
of
the
graph
in
the
linear
domain
is
difficult
to
see
because
it
occurs
near
the
origin,
but
it
can
be
seen
easily
in
the
log
domain.
Most
detection
approaches
(
e.
g.,
Currie's
critical
value
and
detection
limit;
EPA's
MDL;
the
ACS
LOD)
are
constructed
assuming
that
the
flat
(
constant)
region
of
the
variability
versus
concentration
relationship
in
the
graph
holds
true,
although
the
graph
is
rarely
displayed
(
a
horizontal
line
would
be
singularly
uninteresting).
Detection
approaches
such
as
Currie's
critical
value,
detection
limit,
LOD,
and
MDL
are
constructed
by
multiplying
the
standard
deviation
in
the
flat
region
by
some
constant.

February
2003
3­
25
RSD
SD
Response
RSD
SD
Response
Assessment
of
Detection
and
Quantitation
Approaches
Contention
and
differences
of
opinion
occur
in
determining
how
to
arrive
at
an
"
appropriate"
standard
deviation
and
what
to
do
with
the
standard
deviation
when
you
have
it.
Currie's
critical
value
and
EPA's
MDL
use
a
multiple
of
the
standard
deviation
in
a
similar
manner
(
a
t­
statistic
adjusted
for
the
number
of
replicates
used
for
Currie's
critical
value;
3.14
for
7
replicates
in
EPA's
MDL).
The
IDE
uses
an
additional
upward
adjustment
based
on
a
statistical
tolerance
limit
calculation.

3.3.4.1.2
Quantitation
Limits
Using
Standard
Deviation
Multiples
and
Models
of
Standard
Deviation
versus
Concentration
and
RSD
versus
Concentration
The
limit
of
quantitation
(
LOQ)
advanced
by
Currie
and
the
American
Chemical
Society's
Committee
on
Environmental
Improvement,
and
EPA's
minimum
level
of
quantitation
(
ML)
result
from
multiplication
of
the
standard
deviation
by
a
factor
of
10,
again
assuming
a
flat
portion
of
the
variability
versus
concentration
graph.
This
factor
of
10
is
directed
at
achieving
a
relative
standard
deviation
(
RSD)
of
10
percent.
An
advantage
of
this
approach
is
that
a
quantitation
limit
is
produced,
regardless
of
what
the
RSD
turns
out
to
be.

For
example,
it
is
known
that
the
determination
of
2,4­
dinitrophenol
by
EPA
Method
625
produces
highly
variable
results
and
that
10
percent
RSD
cannot
be
achieved
for
this
compound.
Multiplying
the
standard
deviation
of
replicate
measurements
of
low­
level
samples
results
in
a
quantitation
limit
that
is
considerably
higher
than
the
quantitation
limits
for
other
compounds
analyzed
by
Method
625.
The
RSD
at
this
quantitation
limit
could
be
30,
50,
or
70
percent.
Arbitrarily
limiting
the
quantitation
limit
to
some
value
(
e.
g.,
30%,
as
with
the
ASTM
IQE)
could
prohibit
the
use
of
EPA
Method
625
for
determination
of
2,4­
dinitrophenol.
If
2,4­
dinitrophenol
were
present
at
high
concentration
in
a
discharge,
it
would
not
be
reported.
Although
it
could
be
argued
that
a
more
precise
method
should
be
used
for
determination
of
2,4­
dinitrophenol,
determination
of
pollutants
by
a
large
suite
of
different
methods
would
be
quite
costly
with
little
meaningful
benefit.
Increasing
precision
(
i.
e.,
decreasing
measurement
error)
would
be
critical
only
if
the
concentration
at
issue
was
near
a
compliance
limit.

Another
means
of
arriving
at
a
limiting
RSD
is
to
graph
RSD
versus
concentration,
as
shown
in
the
bottom
two
graphs
of
Figure
3­
3.
This
approach
is
used
by
the
ASTM
IQE.
It
has
the
advantage
that
a
model
is
fit
to
data,
rather
than
using
a
point
estimate
such
as
the
Currie
and
ACS
LOD
or
the
EPA
ML.
However,
this
approach
requires
considerably
more
data
than
are
necessary
for
approaches
based
on
point
estimates.
In
addition,
how
a
model
is
selected
can
play
a
major
role
in
the
outcome.

3.3.4.2
Criteria
for
Selecting
Models
Both
statistical
and
graphical
procedures
have
been
proposed
for
selecting
between
models
for
measurement
results
versus
spike
concentrations.

Statistical
Criteria
While
statistical
criteria
are
available
for
choosing
between
models
of
similar
types,
the
currently
available
criteria
are
not
satisfactory
for
choosing
between
the
wide
variety
of
models
considered
for
the
relationship
between
measurement
variation
and
spike
concentration,
based
on
EPA's
studies.
More
technically,
statistical
criteria
include
using:
(
1)
the
simplest
model
to
obtain
statistical
significance,
(
2)
the
model
with
the
smallest
estimated
variability,
and
(
3)
the
model
with
the
smallest
likelihood
ratio.
Given
the
wide
variety
of
models
considered
for
detection
and
quantitation,
there
are
problems
associated
with
each
of
these
procedures.
Data
that
obviously
do
not
follow
the
model
may
produce
statistically
significant
results,
variability
may
be
estimated
with
weights
that
make
the
various
estimates
incomparable,
and
the
likelihood
function
may
not
be
comparable
between
models.

3­
26
February
2003
Chapter
3
Graphical
Criteria
Graphical
criteria
may
be
susceptible
to
some
subjectivity
in
their
application,
but
they
are
currently
the
best
available
method
for
choosing
between
models.
At
the
most
basic
level,
the
primary
graphical
criteria
is
for
the
form
of
the
model
to
be
suggested
by
the
available
data.
To
consider
the
quality
of
the
graphical
analysis,
it
is
useful
to
see
if
some
small
number
of
data
are
overly
influential
in
determining
if
a
model
does
or
does
not
fit.
Given
the
ability
of
the
human
eye
to
discern
deviations
from
a
straight
line
rather
than
from
a
curved
line,
a
useful
technique
is
to
plot
the
data
so
that
they
will
indicate
a
straight
line
if
they
follow
the
model
of
interest.

3.3.4.3
Assessment
of
Current
Models
EPA
graphed
variability
versus
concentration
data
with
regard
to
how
real
data
from
measurement
methods
used
under
the
Clean
Water
Act
would
conform
to
a
number
of
different
models.
For
details
of
how
data
sets
were
selected
and
how
data
were
collected
within
the
data
sets,
see
Appendix
B,
Characterizing
Measurement
Variability
as
a
Function
of
Analyte
Concentration
for
a
Variety
of
Analytical
Techniques.
Four
sets
of
composite
scatter
plots
for
all
combinations
of
analytical
technique
by
analyte
by
study
were
produced.
These
sets
include:

1.
Measurement
versus
Spike
Concentration,
2.
Log
Measurement
versus
Log
Spike
Concentration,
3.
Observed
Standard
Deviation
versus
Spike
Concentration,
4.
Log
Standard
Deviation
versus
Log
Spike
Concentration,
and
5.
Relative
Standard
Deviation
(
RSD)
versus
Log
Spike
Concentration.

There
are
hundreds
of
scatter
plots
in
each
set,
sorted
by
the
source,
measurement
technique,
and
study.
The
first
set
of
scatter
plots
can
be
used
to
evaluate
how
well
measurement
results
match
the
spiked
concentration
in
the
water.
If
the
assumed
straight
line
model
is
true,
then
the
relationship
outlined
by
the
plotted
data
will
be
approximately
linear.
These
relationships
are
plotted
using
log­
log
plots
so
that
small
deviations
from
the
line
can
be
easily
visualized.
All
the
graphs
are
contained
in
attachments
to
Appendix
B
of
this
Assessment
Document.

The
plot
of
observed
standard
deviations
versus
spike
concentrations
can
be
used
to
evaluate
the
reasonableness
of
the
constant
variation
and/
or
linearly
increasing
variability
models
(
Currie,
1968,
Hubaux
and
Vos,
1970,
and
Glaser
et
al.,
1981).
If
the
constant
model
for
standard
deviation
is
true,
there
would
be
no
apparent
relationship
between
the
standard
deviation
and
spike
concentration.
If
the
straight­
line
model
for
standard
deviation
is
true,
plots
are
expected
to
indicate
an
approximately
linear
relationship.
Analogously,
the
standard
deviation/
spike
concentration
versus
spike
concentration
is
expected
to
show
a
straight­
line
relationship
when
variability
is
proportional
to
the
spike
concentration
(
Gibbons
et
al.,
1991).
The
log­
log
plots
of
standard
deviation
versus
spike
concentration
are
expected
to
indicate
if
log
or
square
root
transformations
may
be
appropriate
(
Clayton
et
al.,
1987)
or
to
display
a
shape
that
approximates
a
"
hockey
stick"
when
it
is
appropriate
to
use
the
model
proposed
by
Rocke
and
Lorenzato
(
1995).
With
the
Rocke
and
Lorenzato
model,
variability
near
zero
will
be
approximately
constant,
but
variability
will
increase
proportionally
with
concentration
in
the
higher
concentration
range.

The
large
number
of
plots
make
it
difficult
to
draw
general
conclusions.
For
the
most
part,
conclusions
must
be
considered
on
a
case­
by­
case
basis.
One
somewhat
general
observation
is
that
measurement
variability
over
low
concentrations
does
not
appear
to
fit
a
particular
curvilinear
shape,
and
thus
may
be
considered
to
be
approximately
constant
in
this
range
for
a
large
number
of
analytical
techniques.

February
2003
3­
27
Assessment
of
Detection
and
Quantitation
Approaches
3.3.5
Methodology
for
Parameter
Estimation
Along
with
various
approaches
of
detection
and
quantitation
and
models
for
measurement,
a
number
of
specific
procedures
have
been
suggested
for
estimating
model
parameters.
Maximum
likelihood
and
least
squares
are
two
generally
applicable
statistical
methods
that
can
be
used
in
estimating
model
parameters.
There
are
advantages
and
disadvantages
to
both
that
must
be
weighed
in
particular
cases.
A
standard
statistical
practice
for
evaluating
the
quality
of
an
estimation
procedure
is
to
calculate
the
precision
and
bias,
usually
best
understood
by
examining
a
plot
of
residuals
from
a
fit
to
a
function.
All
else
being
equal,
the
estimation
procedure
with
the
greatest
precision
and
least
bias
is
preferred.
In
some
cases,
precision
and
bias
can
be
calculated
based
on
the
assumptions
behind
the
estimation
procedure.
In
other
cases,
it
is
either
necessary
or
convenient
to
estimate
precision
and
bias
using
simulations.
From
a
general
theoretical
perspective,
the
maximum
likelihood
estimation
methodology
is
preferable
because
it
generates
estimates
that
are
generally
best
with
regard
to
properties
of
precision
and
bias
(
especially
for
larger
sample
sizes),
while
also
being
approximately
normally
distributed.
Unfortunately,
maximum
likelihood
methodology
sometimes
can
be
problematic
because
the
method
requires
the
solution
of
complex
equations.
Least
squares
estimation
is
generally
more
tractable,
and
thus
is
more
generally
applicable,
although
the
estimates
that
result
may
not
be
as
desirable
from
a
theoretical
statistical
perspective.

What
can
sometimes
be
overlooked
in
considering
estimating
for
model
fitting
is
that
direct
measurement
of
variation
of
the
blank
or
low
level
concentration
may
be
the
most
cost­
effective
and
least
difficult
method
to
implement.
The
loss
in
statistical
efficiency
in
comparison
to
more
elaborate
estimation
and
model
fitting
methodology
would
be
offset
by
the
relative
ease
and
lower
cost.

3.3.6
False
Positives
and
False
Negatives
In
this
section,
we
discuss
the
impact
of
detection,
quantitation,
and
reporting
levels
on
false
positive
measurement
results
and
false
negative
measurement
results.
The
definitions
of
false
positives
and
false
negatives
are
directly
related
to
the
concepts
of
critical
value
and
detection
limit
used
by
Currie
(
1995).
These
terms
were
adapted
from
statistical
decision
theory
to
establish
the
framework
for
decision
making
with
regard
to
detection
of
analytes.
The
critical
value,
as
defined
by
Currie,
is
the
point
at
which
the
detection
decision
is
made.
That
is,
measured
values
that
are
less
than
the
critical
value
are
judged
to
be
"
not
detected."
Measured
values
that
exceed
the
critical
value
are
judged
to
be
"
detected."

The
critical
value
is
defined
such
that
when
the
analyte
is
not
present
in
a
sample,
there
is
a
small
possibility
that
a
measurement
will
exceed
the
critical
value.
A
measurement
that
indicates
the
critical
value
has
been
exceeded
is,
therefore,
the
result
of
one
of
two
circumstances:
(
i)
the
analyte
is
present
in
the
sample;
or
(
ii)
the
analyte
is
not
present
in
the
sample
and,
by
chance,
the
measurement
has
exceeded
the
critical
value.
The
occurrence
of
(
ii)
is
defined
as
the
"
false
positive"
situation.
A
measurement
that
is
less
than
the
critical
value
occurs
when:
(
iii)
the
analyte
is
not
present
in
the
sample;
or
(
iv)
the
analyte
is
contained
in
the
sample
but
the
measurement
procedure
fails
to
indicate
its
presence.
The
occurrence
of
(
iv)
is
defined
as
the
"
false
negative"
situation.

3­
28
February
2003
Chapter
3
The
following
table
summarizes
the
two
possible
outcomes
for
each
decision:
"
detected"
or
"
not
detected."

Decision
State
of
the
Sample
Analyte
Present
Analyte
Not
Present
Detected
Correct
(
i)
False
Positive
(
ii)

Not
Detected
False
Negative
(
iv)
Correct
(
iii)

As
formulated
by
Currie,
the
Detection
Limit
is
a
value
greater
than
the
Critical
Value
that
is
used
to
evaluate
the
capabilities
of
analytical
procedures.
In
the
terminology
of
statistical
decision
theory,
the
Detection
Limit
corresponds
to
a
true
value
referred
to
as
the
?
Alternative 
(
see,
e.
g.
Introduction
to
Mathematical
Statistics,
by
Hogg
and
Craig,
5th
edition,
[
1995]).
The
Detection
Limit
is
not
a
part
of
the
detection
decision
process
that
is
applied
to
individual
sample
results.
The
Detection
Limit
is
defined
such
that
when
the
analyte
is
present
in
the
sample
at
a
value
equal
to
the
Detection
Limit,
there
is
a
small
probability
that
a
measured
value
will
be
less
than
the
Critical
Value,
and
thereby
result
in
the
false
negative
decision
of
?
Not
Detected. 

A
common
error
in
many
published
discussions
of
false
negatives
in
relation
to
detection
and
quantitation
(
such
as
the
ASTM
IDE)
is
the
claim
that
using
Currie's
detection
limit
as
a
reporting
limit
or
action
level
will
somehow
?
control 
false
negatives.
That
claim
is
both
false
and
counter­
productive.
To
illustrate
the
problem
with
this
error,
consider
the
scenario
in
which
the
true
concentration
for
the
substance
of
interest
in
a
sample
is
equal
to
the
critical
value.
Also
assume
that
measurement
variability
is
approximately
normal
in
distribution
throughout
the
region
of
concern.
The
critical
level
(
alpha
level)
is
set
to
0.01
or
1%
throughout
the
remainder
of
this
discussion.

If
the
reporting
limit
is
set
equal
to
the
critical
value,
then
given
a
large
number
of
measurements
on
the
sample,
about
half
of
the
results
will
be
reported
as
being
measured
above
the
reporting
limit,
and
about
half
of
the
measurement
results
will
be
reported
as
being
measured
below
the
reporting
limit.
Measurement
results
below
the
reporting
limit
are
treated
as
if
there
is
no
analyte
in
the
sample.
These
are
false
negative
measurements,
and
the
false
negative
rate
is
50%.

Now
set
the
reporting
limit
to
Currie's
detection
limit.
Recall
that
the
true
concentration
in
the
sample
is
equal
to
the
critical
value.
Given
a
large
number
of
measurements
on
the
sample,
about
1%
of
the
measurement
results
will
be
reported
as
being
measured
above
the
reporting
limit,
and
99%
of
the
measurement
results
will
be
reported
as
being
measured
below
the
reporting
limit.
This
is
antithetical
to
Currie's
formulation
of
detection.

To
illustrate
the
intent
of
Currie's
detection
limit,
set
the
reporting
limit
equal
to
Currie's
critical
value,
and
create
a
sample
with
a
true
concentration
equal
to
Currie's
detection
limit.
Given
a
large
number
of
measurements
on
this
sample,
about
99%
of
the
measurement
results
will
be
reported
as
being
measured
above
the
reporting
limit,
and
1%
of
the
measurement
results
will
be
reported
as
being
measured
below
the
reporting
limit.
Knowledge
of
Currie's
detection
limit
can
be
used
to
determine
if
the
measurement
method
meets
the
needs
of
a
study.
For
instance,
a
study
concerned
with
a
wastewater
treatment
technology
that
is
not
expected
to
be
effective
at
concentrations
below
10
mg/
L
may
call
for
a
relatively
inexpensive
measurement
method
capable
of
detecting
the
analyte
at
10
mg/
L,
rather
than
a
more
expensive
measurement
method
capable
of
measuring
a
hundred
times
lower.

February
2003
3­
29
Assessment
of
Detection
and
Quantitation
Approaches
3.3.7
Statistical
Prediction
and
Tolerance
When
we
define
a
critical
value,
detection
limit,
or
quantitation
limit,
different
descriptive
terminology
will
dictate
differences
in
the
numerical
value
of
the
limit.
We
will
use
a
critical
value
as
an
example,
but
the
questions
motivating
detection
and
quantitation
limits
can
be
phrased
in
similar
fashion.
Do
we
want
a
critical
value
that
tells
us
how
likely
it
is
that:

1.
A
measurement
result
was
produced
by
measuring
a
blank
sample,
2.
The
next
measurement
result
will
be
produced
by
measuring
a
blank
sample,
or
3.
The
next
[
pick
any
number]
of
measurement
results
will
be
produced
by
measuring
a
blank
sample?

In
statistical
terms,
these
three
objectives
may
be
addressed,
respectively,
by
application
of
methodology
for
determining:

1.
Percentiles;
2.
Prediction
intervals;
and
3.
Tolerance
intervals.

Percentiles
are
fairly
straight
forward
to
interpret,
i.
e.,
they
specify
the
percentage
of
a
distribution
that
falls
below
a
given
percentile
value.
Prediction
and
tolerance
intervals
are,
in
effect,
confidence
intervals
on
percentiles
and
can
be
somewhat
more
difficult
to
understand
and
apply.
There
are
many
excellent
textbook
and
literature
references
that
present
the
theory
and
application
of
prediction
and
tolerance
intervals
such
as
Hahn
and
Meeker,
Statistical
Intervals,
Wiley,
1991,
and
Pratt
and
Gibbons,
Concepts
of
Non­
parametric
Theory,
Springer­
Verlag,
1981.
Hahn
and
Meeker
describe
at
length
the
different
statistical
intervals
including
their
properties,
applications,
methodology
for
constructing
the
intervals.
Pratt
and
Gibbons
have
an
excellent
discussion
of
tolerance
intervals
that
is
general
in
application
due
to
the
non­
parametric
perspective,
i.
e.,
no
distributional
assumptions
are
required
for
the
results
to
be
valid.

One
of
the
peer
reviewers
of
EPA's
reassessment
states:

 
Tolerance
intervals
are
inappropriate
for
environmental
monitoring.
The
main
issues
here
are
1)
is
the
true
concentration
greater
than
some
specified
safe
of
action
level,
with
sufficient
confidence,
and
2)
what
interval
of
possible
concentrations
is
consistent
with
one
or
a
series
of
measurements,
with
a
specified
degree
of
confidence?
Both
are
statements
about
a
given
sample
or
series
of
samples,
and
not
about
the
hypothetical
variability
of
future
estimates.
Suppose
that
one
has
a
sample
of
10
observations
with
mean
concentration
of
1
ppb
and
standard
deviation
of
0.5
ppb.
Then
the
estimated
99%
critical
level
is
(
2.326)(
0.5)
=
1.2
ppb.
One
may
choose
to
use
a
t­
score
instead
of
a
normal
score
so
that
the
chance
that
a
future
observation
will
exceed
this
level
is
in
fact
99%.
In
this
case,
the
critical
level
estimate
would
be
(
3.250)(
0.5)
=
1.6
ppb.
This
does
actually
correspond
to
a
prediction
interval
for
future
observations
from
a
zero
concentration
sample.

 
If
one
asked
instead
for
a
95%
confidence
interval
for
the
.99
percentage
point
of
the
true
distribution
of
measurements
(
assuming
normality)
when
the
true
quantity
is
zero,
this
can
be
calculated
approximately
using
a
chi­
squared
distribution
and
covers
the
interval
(
0.9
ppb,
2.4
ppb).
It
does
not,
however,
make
sense
to
use
2.4
ppb
as
a
threshold,
since
the
chance
of
a
future
observation
exceeding
2.4
ppb
when
the
true
mean
concentration
is
0
is
about
.0005,
far
smaller
than
the
intended
false­
positive
limit
of
.01. 
(
Rocke,
2002)

3­
30
February
2003
Chapter
3
Another
of
the
peer
reviewers
of
EPA's
reassessment
states:

"
the
operational
definition
as
taken
from
pp.
5­
2/
5­
3
of
MDL
=
t
0.99
(
df)
S
does
not
correspond
to
a
confidence
statement
that
I
can
interpret....
replaced,
although
I
agree
that
a
number
of
statistical
quantities
could
be
used;
where
the
 
fray 
seems
to
be
most
boisterous.
more
careful
in
the
use
of
statistical
terminology.
e
both
refer
often
to
confidence
 
intervals, 
when
in
fact
the
quantity
of
interest
is
a
confidence
limit
 
or
tolerance
limit,
etc.
 
on
some
underlying
parametric
quantity.)...

"
If
we
accept
the
TSD s
argument
on
p.
3­
25
that
the
practical
value
of
tolerance
limits
is
limited,
then
the
MDL
should
be
viewed
as
a
prediction
limit.
an
additional
term
as
per
Gibbons
(
1994,
p.
98):

"
Also,
to
reemphasize,
the
single
most
problematic
issue
when
developing
a
detection
limit
is
correction
for
false
negatives.
emphasis
on
LC­
type
values
such
as
the
MDL
[
when
correctly
calculated,
as
in
(
1)],
as
motivated
by
an
underlying
sort
of
practical/
environmental
conservatism
that
essentially
removes
false
negatives
from
the
estimator s
development.
interpretation.
ontinue,
however,
since
there
seems
to
be
a
fair
amount
of
confusion
on
the
issue
in
the
analytical
chemistry
literature.
from
my
reading
of
the
TSD
is
that,
in
effect,
we
are
calculating
an
LC,
but
using
terminology
that
makes
some
readers
think
it s
an
LD.
false
negative
errors
are
not
the
critical
issue
here,
and
hence
that
the
approach
is
reasonable
(
once
correct
calculations
are
undertaken).
an
effort
to
overcome
this
confusion
in
terminology.
in
reply
I d
suggest
emphasizing
that
an
LC
calculation
is
a
form
of
decision
limit,
not
a
detection
limit.
sers
will
still
confuse
the
terms,
or
reverse
their
meaning,
or
not
see
the
difference,
or
who
knows
what
else?
this
battle
is...)

"
One
caveat:
on
limit
argument
is
acceptable,
if
the
use
of
tolerance
limits
rather
than
prediction
limits
is
in
fact
desired,
then
Gibbons 
(
1994,
p.
99)
presentation
or
an
equivalent
approach
should
be
used
instead
to
correct
the
MDL
calculation."
(
Piegorsch,
2002)

Similarly,
Hahn
and
Meeker
describe
situations
in
which
the
various
intervals
or
limits
are
appropriate
to
use.
the
peer
reviewer,
the
terms
?
intervals 
and
?
limits 
are
sometimes
used
interchangeably).
also
give
examples
of
the
sort
of
applications
that
are
suitable
for
each
type
of
This
should
be
this
is
(
By
the
way,
the
TSD,
and
I,
should
be
W
And
if
so,
it
must
contain
I
took
from
the
TSD
(
in
§
3.3.6)
an
implicit
I
am
willing
to
accept
this
I
suspect
the
fray
will
c
The
bottom
line
I
can
accept
the
argument
that
But,
the
Agency
should
put
forth
(
I
expect
they
will
ask
me
how,
and
But
here
I
suspect
many
u
I
don t
know
how
winnable
although
I
think
the
predicti
(
As
noted
by
They
limit
although
the
decision
to
use
a
particular
type
of
limit
in
a
given
application
is
not
determined
strictly
by
theoretical
considerations
but
is
also
a
matter
of
judgment.

February
2003
3­
31
Assessment
of
Detection
and
Quantitation
Approaches
Prediction
intervals
contain
results
of
future
samples
from
a
previously
sampled
population
with
a
specified
level
of
confidence.
Prediction
limits
are
not
estimators
of
parameters
such
as
means
or
percentiles.
For
example,
a
prediction
interval
may
be
constructed
to
contain
future
sampling
results
expressed
as
a
mean
or
standard
deviation
of
a
future
sample
or
all
of
a
certain
number
of
individual
future
sampling
results.

Therefore,
EPA
agrees
with
the
first
peer
reviewer
that
the
use
of
tolerance
intervals
in
environmental
monitoring
is
inappropriate.
EPA
also
agrees
with
the
second
peer
reviewer
that
there
is
considerable
confusion
in
the
terminology.

3.3.7.1
Tolerance
Intervals
Tolerance
intervals
contain
a
specified
proportion
of
a
population
of
measured
values
with
a
given
statistical
confidence
level.
For
example,
we
say
that
a
proportion,
P,
of
a
population
is
contained
within
the
intervals
(
L1
,
L2
)
with
(
1­")
100%
confidence.
The
lower
and
upper
ends
of
the
interval,
L1
and
L2
,
respectively,
are
referred
to
as
tolerance
intervals.
A
tolerance
interval
is
therefore
the
endpoint
of
an
interval
of
random
length
that
is
determined
on
the
basis
of
having
a
specified
probability
of
1­"
that
its
coverage
of
the
population
is
at
least
equal
to
a
specified
value
P.
The
quantity
1­"
is
referred
to
as
the
confidence
level
for
the
interval
and
P
is
the
minimum
proportion
of
the
population
contained
in
the
interval.
Tolerance
intervals
are
not
estimators
of
values
such
as
a
mean
or
a
percentile
but
rather
values
that
are
always
guaranteed
to
be
either
greater
than
or
less
than
the
desired
value
at
some
level
of
statistical
confidence.
Pratt
and
Gibbons
discuss
this
and
other
properties
that
affect
the
utility
of
tolerance
intervals
and
create
difficulties
in
the
interpretation
and
application
of
tolerance
intervals.

In
effect,
the
determination
of
what,
if
any,
interval
to
use
is
a
policy
decision.
The
choice
of
which
kind
of
interval
to
use
should
consider
how
easy
it
is
to
estimate
the
interval
you
want
under
the
conditions
that
exist.
As
Pratt
and
Gibbons
point
out,
the
interpretation
of
tolerance
intervals
(
and
analogously,
prediction
intervals)
can
be
problematic,
especially
when
issues
of
sample
size
and
the
choice
of
confidence
level
come
into
play.
Pratt
and
Gibbons
cite
examples
where
the
interplay
of
sample
size
and
high
percentile
and
confidence
level
make
tolerance
intervals
useless.

3.3.7.2
Use
of
Tolerance
and
Prediction
in
Setting
Detection
and
Quantitation
Limits
Statistical
intervals
can,
and
have
by
a
number
of
authors,
be
adapted
for
use
in
setting
detection
and
quantitation
limits.
The
basic
approach
requires
a
functional
definition
of
detection
or
quantitation
that
includes
a
statistical
term
or
terms.
An
interval
could
then
be
constructed
about
the
statistical
term
which
could
be
used
to
assess
the
detection
or
quantitation
limit,
or
make
an
adjustment
to
a
calculated
value
that
would
result
in
the
detection
or
quantitation
limit.
For
example,
most
detection
limit
estimators
are
functionally
dependent
on
an
estimate
of
standard
deviation
of
measurement
error.
A
statistical
interval
could
be
constructed
about
the
standard
deviation
and
the
length
of
the
interval
could
be
used
to
assess
the
detection
limit.
The
end
points
of
the
interval
could
be
used
as
the
basis
for
an
adjustment
(
upward
or
downward)
in
the
calculated
limit.

However,
the
use
of
prediction
and/
or
tolerance
limits
in
setting
detection
and
quantitation
limits
is
not
an
absolute
requirement
and
should
be
evaluated
in
the
context
of
specific
applications
and
policy
considerations.
In
practice,
the
effect
of
adjustment
of
detection
and
quantitation
limits
by
use
of
prediction
and
tolerance
intervals
can
be
quite
large,
depending
on
the
amount
of
data
available
and
the
choices
of
percentiles
and
confidence
levels.

3­
32
February
2003
Chapter
3
3.3.8
Design
of
Detection
and
Quantitation
Studies
The
issues
associated
with
the
design
of
detection
and
quantitation
studies
include:
how
well
a
selection
of
spike
concentrations
can
be
used
to
differentiate
between
different
models
for
the
relationship
between
measurement
results
and
spike
concentrations,
how
the
distance
between
spike
concentrations
can
impact
estimates
of
detection
and
quantitation
limits,
how
to
reduce
the
influence
of
uncontrollable
factors
in
the
measurement
process
(
probability
design),
how
complete
to
make
the
design
factors
in
terms
of
the
physical
measurement
process,
and
how
flexible
to
make
the
design
factors
in
terms
of
the
physical
measurement
process.

3.3.8.1
Spike
Concentrations
and
Modeling
If
a
model
under
consideration
cannot
be
described
by
the
number
of
spike
concentrations
in
the
design,
then
it
is
not
possible
to
tell
if
the
model
is
appropriate.
To
take
the
simplest
example,
it
is
not
possible
to
describe
the
slope
of
a
line
associated
with
linearly
increasing
variation
from
a
single
spike
concentration.
Two
well­
spaced
spike
concentrations
would
allow
you
to
estimate
a
slope,
but
provide
you
with
no
idea
of
the
variability
of
the
estimate.
Three
well­
spaced
spike
concentrations
represent
the
minimum
requirement
for
estimating
the
linear
relationship
and
the
variability
of
that
relationship.

Clayton
et
al.
(
1987)
describe
the
relationship
between
the
spread
of
the
spike
concentrations,
the
number
of
spike
concentrations,
and
the
number
of
replicate
measurements
with
regard
to
estimated
variability
when
a
linear
model
is
used.
While
the
specific
equation
used
in
their
paper
does
not
apply
to
all
models,
it
indicates
principles
that
do
apply.
Increasing
the
number
of
replicate
measurements,
increasing
the
number
of
spike
concentrations,
and
reducing
the
spread
of
the
spike
concentrations
are
all
expected
to
reduce
estimated
variability
along
with
the
associated
detection
and
quantitation
limits.
However,
one
of
the
components
of
variability
associated
with
detection
and
quantitation
is
that
associated
with
estimating
the
calibration
relationship.
To
account
for
this
source
of
variation,
it
may
be
appropriate
to
cover
the
entire
calibration
range.
On
the
other
hand,
many
replicates
at
a
high
concentration
may
improperly
weight
the
data
in
favor
of
high
detection
and
quantitation
estimates.

3.3.8.2
Probability
Design
The
process
known
as
randomization
is
an
important
statistical
consideration
in
the
design
and
interpretation
of
experimental
studies.
Randomization
involves
the
allocation
of
experimental
units
to
factors
and
treatments
under
study
according
a
design
determined
by
probability.
Randomization
avoids
bias
and
systematic
errors
that
can
occur
in
studies
where
randomization
is
not
used.
Randomization
is
discussed
in
classic
texts
such
as
Statistics
for
Experimenters,
by
Box,
Hunter,
and
Hunter,
Wiley,
1978.

In
studies
of
measurement
methods,
randomization
can
be
used
in
the
process
of
creating
spike
concentration
solutions
and
the
ordering
of
analyses.
However,
randomization
has
practical
drawbacks,
particularly
with
regard
to
studies
designed
to
establish
detection
or
quantitation
limits.
For
example,
consider
a
simple
study
involving
the
analyses
of
samples
spiked
at
five
concentrations
of
the
analyte
of
interest,
with
five
replicates
of
each
sample
analyzed.
A
total
of
25
analyses
are
required
for
the
study,
and
the
analyses
of
the
samples
can
be
organized
in
a
5
by
5
matrix.
A
random
number
is
assigned
to
each
block
in
the
matrix,
as
a
means
of
randomizing
the
order
of
the
replicates
at
each
concentration.

By
virtue
of
this
randomized
design,
a
sample
with
a
high
concentration
of
the
analyte
of
interest
may
end
up
being
analyzed
immediately
prior
to
a
sample
with
a
very
low
concentration
of
the
analyte.
Unfortunately,
this
can
lead
to
problems
that
result
from
the
"
carry­
over"
of
analyte
within
the
instrumentation
from
one
analysis
to
the
next.
When
carry­
over
occurs,
the
apparent
concentration
of
the
low­
concentration
sample
can
be
inflated
because
some
of
the
high­
concentration
sample
l
may
be
carried
February
2003
3­
33
Assessment
of
Detection
and
Quantitation
Approaches
into
the
low­
concentration
sample
2.
In
the
context
of
a
study
designed
to
establish
?
how
low
you
can
go 
(
i.
e.,
establishing
a
detection
limit),
carry­
over
of
the
analyte
into
a
low­
concentration
sample
may
compromise
the
results
by
inflating
the
result
for
low­
concentration
sample
2,
but
not
inflating
the
results
for
other
low­
concentration
samples
because
the
randomized
design
did
not
cause
them
to
be
analyzed
immediately
following
a
high­
concentration
sample.

Analysts
are
aware
of
the
potential
for
carry­
over
and
generally
take
steps
during
routine
analyses
to
minimize
the
chance
that
it
will
occur.
Examples
of
steps
that
can
minimize
carry­
over
problems
include
analyzing
?
cleaner 
samples
before
?
dirtier 
samples,
and
interspersing
?
blanks 
between
samples
when
possible.
Obviously,
the
intentional
segregation
of
low
and
high
concentration
samples
defeats
the
purpose
of
the
randomized
design.
Interspersing
blanks
between
the
samples
can
be
effective,
as
well
as
blocking
similar
concentrations
together
and
randomizing
blocks.
But
in
order
to
ensure
that
the
blanks
do
not
have
other
effects
on
the
results,
blanks
would
be
needed
between
each
sample
or
block
analysis,
and
this
would
greatly
increase
the
cost
of
the
study
(
e.
g.,
25
samples
and
24
blanks
would
be
required
in
case
of
pure
randomization).
Therefore,
despite
the
statistical
benefits,
in
practice,
randomization
of
the
sample
analysis
sequence
can
be
difficult
to
apply
in
detection
and
quantitation
limit
studies.

In
the
Agency s
studies
of
variability
as
a
function
of
concentration
discussed
in
Sections
1.3.2.1
­
1.3.2.3
of
this
document,
EPA
chose
to
use
a
non­
random
design
to
avoid
carry­
over
problems
and
to
limit
the
potential
difficulties
with
measurements
at
very
low
concentrations.
For
example,
if
there
was
no
instrument
response
at
concentration
X,
then
it
would
be
unlikely
that
there
would
be
a
response
at
a
concentration
of
X/
2.
In
the
non­
random
design,
EPA
permitted
the
analyst
to
stop
analyses
of
ever­
lower
concentrations,
whereas
a
randomized
design
would
have
required
that
all
the
samples
be
analyzed,
even
when
there
was
no
instrumental
response
for
many
of
those
samples.

One
of
the
peer
reviewers
evaluating
EPA s
draft
version
of
this
Assessment
Document
commented
that
the
effects
of
carry­
over
could
have
been
mitigated
by
studying
variability
around
the
calibration
line
rather
than
the
mean
of
the
replicates.
However,
carry­
over
affects
subsequent
samples
differently.
The
effect
of
the
carry­
over
cannot
be
mitigated,
regardless
of
whether
variability
is
studied
around
the
calibration
line
or
the
mean
of
the
replicates,
unless
the
amount
of
carry­
over
is
known
and
can
be
subtracted
from
the
affected
(
low­
concentration)
sample.
This
subtraction
has
limitations
because
of
error
accumulation
and
because
the
amount
of
carry­
over
cannot
be
determined
precisely
without
extensive
studies
at
multiple
concentrations.

3.3.8.3
Completeness
The
physical
measurement
process
can
be
studied
using
rough
approximations
or
it
can
be
studied
more
rigorously.
A
rough
approximation
could
use
the
available
components
of
a
method
as
applied
to
convenient
samples.
A
more
rigorous
study
would
use
a
complete,
specific,
and
well­
defined
measurement
method
with
all
sample
processing
steps.
The
appropriate
level
of
study
will
probably
depend
on
the
purpose
of
the
study.

Measurement
procedures
(
methods)
may
be
more
or
less
strictly
designed.
Variability
in
what
is
allowed
in
the
procedures
may
add
to
variability
in
the
measurement
results.
To
the
extent
that
permutations
of
a
method s
procedures
are
not
expected
to
be
used
in
a
particular
detection
or
quantitation
study,
EPA
recommends
that
this
information
be
included
in
the
report
on
the
study
results.
While
there
may
be
physical/
chemical
reasons
for
extrapolating
the
results
of
a
variability
study
on
one
set
of
procedures
to
permutations
of
those
procedures,
there
is
no
statistical
basis
for
making
such
an
extrapolation.
Statistical
theory
by
itself
is
only
able
to
describe
conditions
that
have
been
observed.
On
the
other
hand,
a
knowledge
of
the
underlying
physics
of
the
measurement
process
can
guide
the
completeness
of
the
modeling
process
when
statistical
procedures
fail.
For
example,
the
Rocke
and
3­
34
February
2003
Chapter
3
Lorenzato
model
in
the
linear
or
log­
log
domain
may
be
the
best
general
characterization
of
a
physical
measurement
process.
Therefore,
this
model
can
be
applied
to
data
to
produce
a
complete
answer
when
statistical
procedures
fail
to
deduce
the
"
correct"
model.

February
2003
3­
35
Table
3­
1.
Summary
of
Issues
Considered
Issue
Considered
Description
Assessment/
Relevance
to
Detection
and
Quantitation
Level
Approaches
Blank
versus
zero
concentration
It
is
not
possible
to
measure
 
zero 
concentration
of
a
substance.

As
chemists
push
detection
capabilities
closer
to
 
zero ,

analysis
of
blanks
is
needed
establish
if
the
result
measured
is
a
true
concentration
or
background
contamination.
Useful
approaches
should
address
the
potential
contribution
of
the
blank,
both
through
the
design
of
studies
that
generate
detection
and
quantitation
limits,
and
through
evaluation
of
the
study
results.

Lack
of
instrument
response
Instruments
do
not
always
produce
a
response
(
i.
e.,
sometimes
they
are
not
capable
of
yielding
a
result
at
the
known
concentration
of
a
pollutant.)
Operational
systems
for
detection
and
quantitation
need
to
take
instrument
non­

response
into
account.

Matrix
effects
If
not
properly
controlled,
substances
in
a
particular
matrix
can
interfere
with
measurements
of
pollutants,
resulting
in
high
or
low
bias
in
the
measurement.
Procedures
for
determining
detection
and
quantitation
limits
should
allow
for
evaluation
of
data
in
specific
matrices
of
concern
when
all
efforts
to
resolve
the
matrix
interferences
have
been
exhausted.

Bias
(
recovery)

correction
Some
detection
and
quantitation
approaches
apply
a
correction
for
bias
based
on
the
slope
of
recovery
versus
concentration
line.
The
typical
relationships
between
recovery,
precision,
and
concentration
in
analytical
chemistry
show
that
precision
worsens
as
recovery
falls.
Therefore,

increasing
detection
and
quantitation
limits
to
allow
for
lowered
recovery
may,
in
fact,
provide
a
double
correction.
Preliminary
tests
have
shown
that
negative
detection
limits
can
be
produced
when
a
recovery
model
has
a
negative
slope.

Measurement
quality
over
the
life
of
a
method
Measurements
made
by
a
given
method
may
improve
in
quality
over
time
as
analysts
gain
more
experience
and/
or
as
improvements
are
made
in
the
underlying
analytical
technology.
Detection
and
quantitation
limit
approaches
should
be
supported
by
procedures
that
will
allow
individual
laboratories
and
other
organizations
to
affordably
characterize
the
improvements.

Method
development
and
promulgation
CWA
Section
304(
h)
requires
EPA
to
promulgate
test
methods
that
are
used
in
certain
certification
and
permitting
activities
under
the
Act.
If
necessary,
EPA
must
develop
these
methods.

EPA
also
can
promulgate
methods
developed
by
other
organizations
if
they
meet
EPA
needs.
However,
organizations
presently
use
a
variety
of
differing
approaches
for
detection
and
quantitation.
There
are
real
benefits
to
standardization
of
detection
and
quantitation
limit
approaches,
but
it
is
impractical
to
force
outside
organizations
that
develop
methods
to
accept
standardization
adopted
by
EPA.

Laboratory­
specific
performance
verification
Laboratories
that
use
a
particular
method
must
be
capable
of
demonstrating
they
can
achieve
the
required
detection
limit.
This
objective
can
be
achieved
by
establishing
a
reasonable
detection
limit
based
on
tests
in
an
experienced
laboratory
and
requiring
individual
laboratories
to
confirm
they
achieve
this
detection
limit.

3­
36
3­
37
Table
3­
1.
Summary
of
Issues
Considered
Issue
Considered
Description
Assessment/
Relevance
to
Detection
and
Quantitation
Level
Approaches
Effluent
guideline
development
CWA
requires
EPA
to
develop
technology­
based
guidelines
and
standards
concerning
the
treatment
and
discharge
of
pollutants
into
US
waters.
Effluent
guidelines
and
standards
are
based
on
existing
technological
capabilities.

Because
pollutant
limits
are
derived
from
statistical
analysis
of
data
that
captures
all
sources
of
variability
in
the
industrial
treatment
process,
including
analytical
variability,
facilities
employing
well­
designed
and
operated
treatment
should
be
capable
of
achieving
technology­
based
limits.

Development
of
water
quality­
based
controls
CWA
requires
EPA
and
authorized
States
to
implement
water
quality­
based
controls
when
technology­
based
controls
will
not
sufficiently
protect
designated
uses
for
a
water
body.
Detection
and
quantitation
limit
capabilities
are
not
a
consideration
when
water
quality
criteria
are
established,
meaning
that
water
quality
criteria
may
be
lower
than
levels
that
can
be
reliably
measured
using
available
technology.
Ideally,
detection
and
quantitation
limit
approaches
will
include
procedures
that
encourage
the
lowering
of
detection
and
quantitation
limits
so
that
human
health
and
the
environment
are
protected.

Compliance
monitoring
CWA
requires
EPA
and
authorized
States
to
issue
NPDES
permits
to
facilities
that
discharge
into
waters
of
the
U.
S.,
and
requires
municipalities
to
issue
pre­
treatment
permits
to
facilities
that
discharge
into
publicly
operated
treatment
systems.
Legal
issues
must
be
considered
when
establishing
permit
limits.
Compliance
limits
must
reflect
realistic
measurement
capabilities.

Non­
regulatory
studies
and
monitoring
EPA
conducts
a
variety
of
non­
regulatory
studies
and
monitoring
activities
the
goals
of
the
CWA.
Ideally,
detection
and
quantitation
limits
should
allow
reliable
detection
and
measurement
of
pollutants
at
levels
that
could
become
of
environmental
concern.

Descriptive
versus
prescriptive
uses
of
detection
and
quantitation
limits
Descriptive
approaches
characterize
the
current
performance
of
a
laboratory
or
laboratories
that
might
use
a
method
to
analyze
pollutants.
Prescriptive
approaches
define
a
limit
that
laboratories
must
demonstrate
they
can
achieve
before
practicing
a
method.
The
prescriptive
approach
to
detection
and
quantitation
is
consistent
with
EPA s
use
of
other
prescriptive
performance
standards
that
laboratories
must
achieve.
The
prescriptive
approach
requires
specification
of
reasonably
attainable
detection
limits.
Even
so,
a
prescriptive
approach
could
result
in
exclusion
of
laboratories
that
cannot
achieve
the
limit.

A
descriptive
approach
can
reflect
capabilities
of
poor
laboratories
and
can,

therefore,
have
the
effect
of
raising
detection
and
quantitation
limits
to
levels
that
are
higher
than
desired
and
not
consistent
with
the
performance
of
laboratories
that
make
reasonable
efforts
to
control
variability.

Compliance
evaluation
thresholds
Water
quality­
based
permitting
can
suggest
permit
limits
that
are
below
the
detection
or
quantitation
limits
of
the
most
sensitive,
approved
method.
EPA s
draft
guidance
suggested
that
the
permit
limit
be
established
from
the
water
quality
standard
but
that
compliance
be
evaluated
at
the
quantitation
limit
of
the
most
sensitive
method.
Permit
writers
should
have
the
flexibility
to
use
the
detection
limit,
quantitation
limit,
or
other
limit
as
the
compliance
evaluation
threshold
so
that
the
environment
is
protected.
3­
38
Table
3­
1.
Summary
of
Issues
Considered
Issue
Considered
Description
Assessment/
Relevance
to
Detection
and
Quantitation
Level
Approaches
Accepting
the
procedures
of
VCSBs
The
National
Technology
Transfer
Advancement
Act
directs
EPA
to
focus
on
increasing
their
use
of
standards
published
by
VCSBs
such
as
ASTM,
ISO,
etc.,
when
it
is
consistent
with
the
Agency s
mission.
VCSBs
use
differing
approaches
to
detection
and
quantitation
needs
to
be
considered.
EPA
would
continue
to
accept
analytical
methods
from
VCSBs
that
may
use
other
detection
and
quantitation
approaches
when
the
methods
meet
EPA s
regulatory
needs.

National
versus
local
standards
for
measurement
CWA
authorizes
State
and
local
governments
to
implement
provisions
of
the
Act,
as
long
as
they
do
so
in
a
way
that
is
at
least
as
protective
of
the
environment
as
the
national
standards
established
by
EPA.
This
assessment
must
recognize
the
impact
of
any
new
or
revised
detection
or
quantitation
approaches
on
State
and
local
governments
and
requirements.
Given
that
state
and
local
governments
may
use
more
stringent
approaches,
adoption
of
new
or
modified
approaches
by
EPA
as
a
result
of
this
assessment
may
have
no
practical
impact
if
states
choose
to
continue
use
of
their
existing
approaches.

Cost
and
implementation
issues
The
financial
and
technical
resources
required
to
determine
detection
limit
approaches
vary
widely
according
the
complexity
of
the
procedures
involved.
Organizations
that
develop
methods
typically
have
greater
resources
available
for
determining
limits
than
do
organizations
that
use
the
methods.
EPA
must
be
sensitive
the
capabilities
of
the
organizations
that
develop
and
that
use
methods.
Data
from
EPA
studies
indicate
that
the
true
detection/
quantitation
limits
at
a
given
point
in
time
can
only
be
arrived
at
by
running
hundreds
of
replicates.
A
better
alternative
would
be
to
identify
a
simple
procedure
that
yields
a
reproducible
estimate
and
to
allow
laboratory­
specific
adjustment
based
on
actual
conditions
in
the
laboratory.

Use
of
multiple
approaches
or
a
single
pair
of
approaches
Analytical
methods
are
used
to
support
multiple
applications
under
CWA,
including
development
of
regulatory
requirements,

compliance
monitoring
and
enforcement,
and
non­
regulatory
studies
and
monitoring.
EPA
could
adopt
multiple
detection
and
quantitation
level
approaches,
choosing
for
each
application
an
approach
that
is
best
suited
to
that
need.
Use
of
a
single
pair
of
approaches
that
meets
the
needs
of
all
CWA
programs
is
preferable
to
the
adoption
of
multiple,
application­
specific
approaches.
Selection
of
multiple
approaches
would
likely
yield
a
matrix
of
approaches
and
applications
that
would
cause
confusion
and
frustration
among
regulators,
permittees,
and
the
laboratory
community.
It
is
also
likely
that
the
suitability
of
each
approach
to
each
designated
application
would
be
subject
to
the
same
level
of
contention
that
has
been
applied
to
nearly
all
existing
approaches.

Sources
of
variability
Various
known
and
unknown
sources
of
variability
can
impact
laboratory
results.
Steps
can
be
taken
to
control
known
sources
of
variability.
When
such
steps
are
taken,
some
variability
will
still
exist,
and
it
can
be
expected
that
interlaboratory
variability
will
be
greater
than
intralaboratory
variability.
The
potential
impacts
of
interlaboratory
variability
must
be
considered
when
selecting
detection
and
quantitation
limit
approaches.
Even
if
prescriptive
measures
are
used
to
control
variability,
it
should
be
recognized
that
some
laboratories
may
achieve
lower
detection
and
quantitation
limits
than
others.
3­
39
Table
3­
1.
Summary
of
Issues
Considered
Issue
Considered
Description
Assessment/
Relevance
to
Detection
and
Quantitation
Level
Approaches
Censoring
measurement
results
Measurement
results
are
often
reported
as
less
than
some
detection,
quantitation,
or
reporting
limit
(
i.
e.,
they
are
censored
below
a
designated
reporting
threshold).
The
primary
reason
for
censoring
is
to
avoid
reporting
results
with
a
higher
degree
of
uncertainty,
based
on
a
policy
decision.
Although
such
results
may
not
have
the
desired
level
of
certainty
for
most
applications,
they
may
be
of
value
to
statisticians
and
modelers
who
handle
large
volumes
of
data.
Although
the
issue
of
censoring
is
important,
it
should
not
be
a
consideration
when
selecting
a
detection
and
quantitation
limit
approach.
The
decision
to
censor
data
is
a
data
reporting
and
data
use
issue,
and
it
will
apply
regardless
of
how
the
detection
or
quantitation
limit
is
established.

Outliers
Outliers
are
measurement
results
that
are
inconsistent
with
the
vast
majority
of
results.
They
may
arise
from
random
variation
or
some
deviation
in
the
measurement
process.
Data
sets
used
for
development
of
detection
and
quantitation
limits,
for
QC
acceptance
criteria,
and
for
other
purposes,
should
continue
to
be
screened
for
outliers.

Selection
of
statistical
models
Method
sensitivity
is
usually
established
based
on
measurement
variation.
Nearly
all
analytical
techniques
produce
results
that
can
generally
be
classified
according
to
one
of
three
basic
models.
Approaches
that
rely
on
collection
of
large
data
sets
and
selection
of
appropriate
models
based
on
graphical
analysis
of
the
results
have
been
proposed
as
a
more
accurate
means
of
determining
detection
and
quantitation
limit
approaches
than
simple
models
based
on
the
collection
of
small
data
sets.
Regardless
of
which
model
is
selected,
it
should
yield
a
reasonable
and
reproducible
estimates
of
detection
and
quantitation.
Differences
in
the
cost
and
complexity
of
these
approaches
must
be
weighed
against
differences
in
observed
outcome
using
either
approach.

Methodology
for
parameter
estimation
Methods
for
estimating
the
goodness
of
fit
of
a
set
of
data
to
a
particular
model
include
use
of
statistical
estimation
procedures
for
precision
and
bias
and
graphical
displays.
However,
the
exact
tests
to
be
used
may
not
be
detailed
well
enough
in
the
detection/
quantitation
limit
approach.
Details
of
estimating
goodness
of
fit
must
be
specified
and
must
be
based
on
actual
tests
of
real­
world
variability
versus
concentration
data.

False
positives
and
false
negatives
If
a
pollutant
is
present
in
a
sample,
but
not
measured,
the
reported
result
is
a
 
false
negative. 
If
a
pollutant
is
not
present
in
a
sample,
but
a
positive
result
is
measured
and
reported,
the
reported
result
is
a
 
false
positive. 
The
issue
of
allowing
for
false
negatives
is
contentious.
A
common
error
in
discussions
of
detection
and
quantitation
limit
approaches
is
that
reporting
limits
can
be
used
to
control
both
the
false
positive
and
false
negative
rates.
As
long
as
the
reporting
limit
is
the
only
tool
for
controlling
false
positive
and
false
negatives,
setting
the
reporting
limit
higher
only
reduces
the
probability
of
a
false
positive
at
the
expense
of
the
false
negative
rate.
3­
40
Table
3­
1.
Summary
of
Issues
Considered
Issue
Considered
Description
Assessment/
Relevance
to
Detection
and
Quantitation
Level
Approaches
Statistical
prediction
and
tolerance
Currie s
original
approaches
are
based
on
confidence
limits.

More
recently,
others
have
suggested
that
detection
and
quantitation
approaches
should
be
based
on
statistical
prediction
or
tolerance
intervals
because
compliance
measurements
are
future
samples,
whereas
the
limits
are
based
on
available
measurements.
Statistical
intervals
can
be,
and
have
been,
adapted
for
use
in
setting
detection
and
quantitation
limits
by
a
number
of
authors.
However,
the
use
of
prediction
and/
or
tolerance
limits
in
setting
detection
and
quantitation
limits
is
not
a
requirement
and
should
be
evaluated
in
the
context
of
specific
applications
and
policy
considerations.

Design
of
detection
and
quantitation
studies
Studies
designed
to
characterize
detection
and
quantitation
limits
can
be
affected
by
the
selection
of
concentrations
studied,

how
well
uncontrollable
factors
in
the
measurement
process
are
reduced,
the
degree
to
which
entire
measurement
process
is
studied,
and
the
flexibility
of
the
design
factors
in
terms
of
the
physical
measurement
process.
Resources
may
be
insufficient
to
support
detection/
quantitation
limit
approaches
that
model
variability
versus
concentration
because
the
selection
of
concentrations
may
require
iteration
when
results
do
not
meet
their
respective
criteria.
