FSIS Comparative 

Risk Assessment for 

Listeria monocytogenes in Ready-to-eat Meat and Poultry Deli Meats

Prepared by

Risk Assessment Division

Office of Public Health Science

Food Safety and Inspection Service

United States Department of Agriculture

March 2009

Contributors

Heather Hicks Quesenberry

Daniel Gallagher

Sarah Endrikat2

David LaBarre1

Eric Ebel1

Carl Schroeder1

Janell Kause1

Acknowledgements

Many people within and outside the Food Safety and Inspection Service,
U.S. Department of Agriculture contributed to the development of this
risk assessment, including Daniel Engeljohn, Carol Maczka, Regis
Pouillot, James Lindsey, Flora Tsui, and researchers of the National
Alliance for Food Safety and Security. 

Contents

  TOC \o "1-3" \h \z \u    HYPERLINK \l "_Toc226941188"  Contributors	 
PAGEREF _Toc226941188 \h  ii  

  HYPERLINK \l "_Toc226941189"  Acknowledgements	  PAGEREF _Toc226941189
\h  iii  

  HYPERLINK \l "_Toc226941190"  Executive Summary	  PAGEREF
_Toc226941190 \h  1  

  HYPERLINK \l "_Toc226941191"  Background	  PAGEREF _Toc226941191 \h  1
 

  HYPERLINK \l "_Toc226941192"  Results	  PAGEREF _Toc226941192 \h  2  

  HYPERLINK \l "_Toc226941193"  Conclusions	  PAGEREF _Toc226941193 \h 
2  

  HYPERLINK \l "_Toc226941194"  Introduction	  PAGEREF _Toc226941194 \h 
3  

  HYPERLINK \l "_Toc226941195"  Methods	  PAGEREF _Toc226941195 \h  4  

  HYPERLINK \l "_Toc226941196"  stage I: Prevalence and Level of L.
monocytogenes in RTE Meat and Poultry deli meats at retail	  PAGEREF
_Toc226941196 \h  5  

  HYPERLINK \l "_Toc226941197"  stage II: Growth of L. monocytogenes
from retail purchase to consumption	  PAGEREF _Toc226941197 \h  13  

  HYPERLINK \l "_Toc226941198"  Stage III: deli meat Consumption	 
PAGEREF _Toc226941198 \h  16  

  HYPERLINK \l "_Toc226941199"  Stage IV: L. monocytogenes Dose-response
Relationship	  PAGEREF _Toc226941199 \h  17  

  HYPERLINK \l "_Toc226941200"  Results	  PAGEREF _Toc226941200 \h  17  

  HYPERLINK \l "_Toc226941201"  Conclusions	  PAGEREF _Toc226941201 \h 
23  

  HYPERLINK \l "_Toc226941202"  Appendix I: L. monocytogenes in
Ready-to-eat Meat and Poultry Deli meat	  PAGEREF _Toc226941202 \h  24  

  HYPERLINK \l "_Toc226941203"  Data Collection Methods	  PAGEREF
_Toc226941203 \h  24  

  HYPERLINK \l "_Toc226941204"  Study Results	  PAGEREF _Toc226941204 \h
 26  

  HYPERLINK \l "_Toc226941205"  Prevalence and Number of Samples	 
PAGEREF _Toc226941205 \h  26  

  HYPERLINK \l "_Toc226941206"  Logistic Regression	  PAGEREF
_Toc226941206 \h  35  

  HYPERLINK \l "_Toc226941207"  Comparison of Findings of the National
Alliance for Food Safety and Security with those of the Food
Processors’ Association	  PAGEREF _Toc226941207 \h  37  

  HYPERLINK \l "_Toc226941208"  Conclusions	  PAGEREF _Toc226941208 \h 
38  

  HYPERLINK \l "_Toc226941209"  Appendix II: Sensitivity Analysis	 
PAGEREF _Toc226941209 \h  39  

  HYPERLINK \l "_Toc226941210"  STORAGE TIME-TEMPERATURE SENSITIVITY	 
PAGEREF _Toc226941210 \h  39  

  HYPERLINK \l "_Toc226941211"  SHELF LIFE SENSITIVITY	  PAGEREF
_Toc226941211 \h  47  

  HYPERLINK \l "_Toc226941212"  RELATIVE IMPACTS OF MODEL VARIABLES	 
PAGEREF _Toc226941212 \h  48  

  HYPERLINK \l "_Toc226941213"  References	  PAGEREF _Toc226941213 \h 
52  

 Figures

  TOC \h \z \c "Figure"    HYPERLINK \l "_Toc215306753"  Figure 1. A
conceptual model of the stages in this risk assessment and the critical
inputs considered within each stage.	  PAGEREF _Toc215306753 \h  4  

  HYPERLINK \l "_Toc215306754"  Figure 2. Cumulative density functions
for the amount of L. monocytogenes in prepackaged compared to
retail-sliced RTE deli meats.	  PAGEREF _Toc215306754 \h  6  

  HYPERLINK \l "_Toc215306755"  Figure 3. Probability plots for fitted
lognormal (base 10) distribution to observed levels for retail-sliced
and prepackaged deli meat.	  PAGEREF _Toc215306755 \h  10  

  HYPERLINK \l "_Toc215306756"  Figure 4. Fitted cumulative distribution
plots and observed retail data for L. monocytogenes levels in
retail-sliced versus prepackaged deli meats	  PAGEREF _Toc215306756 \h 
11  

  HYPERLINK \l "_Toc215306757"  Figure 5. Quantile-quantile plot of
fitted distributions for L. monocytogenes levels from retail-sliced and
prepackaged deli meat	  PAGEREF _Toc215306757 \h  12  

  HYPERLINK \l "_Toc215306758"  Figure 6. Histograms of estimated number
of deaths per annum for retail-sliced and prepackaged product based on
the 4,000 dose-response simulations of the FDA-FSIS model.	  PAGEREF
_Toc215306758 \h  21  

  HYPERLINK \l "_Toc215306759"  Figure 7. Cumulative density plots
estimated number of deaths per annum for retail-sliced and prepackaged
product based on the 4,000 dose-response simulations of the FDA-FSIS
model.	  PAGEREF _Toc215306759 \h  22  

  HYPERLINK \l "_Toc215306760"  Figure 8. Number of RTE samples by
location of slicing (source).	  PAGEREF _Toc215306760 \h  29  

  HYPERLINK \l "_Toc215306761"  Figure 9. Prevalence of L. monocytogenes
in deli meat by location of slicing.	  PAGEREF _Toc215306761 \h  30  

  HYPERLINK \l "_Toc215306762"  Figure 10. Number of deli meat samples
collected per store	  PAGEREF _Toc215306762 \h  31  

  HYPERLINK \l "_Toc215306763"  Figure 11. Number of RTE samples by deli
meat type	  PAGEREF _Toc215306763 \h  32  

  HYPERLINK \l "_Toc215306764"  Figure 12. Prevalence of L.
monocytogenes in RTE deli meats by deli meat type	  PAGEREF
_Toc215306764 \h  33  

  HYPERLINK \l "_Toc215306765"  Figure 13. Prevalence of L.
monocytogenes in RTE deli meats samples sliced at retail.	  PAGEREF
_Toc215306765 \h  34  

  HYPERLINK \l "_Toc215306766"  Figure 14. Graphical display of logistic
regression results using deli meat sample prevalence at individuals
stores as the dependent variable	  PAGEREF _Toc215306766 \h  37  

  HYPERLINK \l "_Toc215306767"  Figure 15. Relative ratio of deaths
differing storage times between retail-sliced and prepackaged product.	 
PAGEREF _Toc215306767 \h  40  

  HYPERLINK \l "_Toc215306768"  Figure 16. Relative frequency of storage
time for retail-sliced versus prepackaged product.	  PAGEREF
_Toc215306768 \h  42  

  HYPERLINK \l "_Toc215306769"  Figure 17. Storage time for
retail-sliced and prepackaged product.	  PAGEREF _Toc215306769 \h  43  

  HYPERLINK \l "_Toc215306770"  Figure 18. Relative frequencies for
storage temperature.	  PAGEREF _Toc215306770 \h  44  

  HYPERLINK \l "_Toc215306771"  Figure 19. Relative ratio of deaths
between retail-sliced versus prepackaged product for differing
definitions of shelf life.	  PAGEREF _Toc215306771 \h  48  

  HYPERLINK \l "_Toc215306772"  Figure 20. Recursive partitioning and
regression tree.	  PAGEREF _Toc215306772 \h  49  

  HYPERLINK \l "_Toc215306773"  Figure 21. Box plots for each deli meat
category by age group.	  PAGEREF _Toc215306773 \h  50  

  HYPERLINK \l "_Toc215306774"  Figure 22. Interaction plots comparing
the effect of growth inhibitor (GI) use and slicing location on the mean
number of deaths from listeriosis.	  PAGEREF _Toc215306774 \h  51  

 Tables

  TOC \h \z \c "Table"    HYPERLINK \l "_Toc215306775"  Table 1.
Survival analysis input for statistical distribution fitting for the
level of L. monocytogenes in deli meats at retail.	  PAGEREF
_Toc215306775 \h  7  

  HYPERLINK \l "_Toc215306776"  Table 2. Best fit maximum likelihood
results and probability plot distribution parameters for retail-sliced
deli meat.	  PAGEREF _Toc215306776 \h  8  

  HYPERLINK \l "_Toc215306777"  Table 3. Best fit maximum likelihood
results and probability plot distribution parameters for prepackaged
deli meat.	  PAGEREF _Toc215306777 \h  8  

  HYPERLINK \l "_Toc215306778"  Table 4. Quantiles from fitted lognormal
distributions for retail-sliced and prepackaged L. monocytogenes levels.
  PAGEREF _Toc215306778 \h  12  

  HYPERLINK \l "_Toc215306779"  Table 5. Plant distribution and
estimated fraction of production prior to the Interim Final Rule.	 
PAGEREF _Toc215306779 \h  14  

  HYPERLINK \l "_Toc215306780"  Table 6. Fraction of deli meat
production by slicing location and antimicrobial growth inhibitor use
during July 2007.	  PAGEREF _Toc215306780 \h  16  

  HYPERLINK \l "_Toc215306781"  Table 7. Estimated mean number of deaths
per year from L. monocytogenes in deli meat among three populations
stratified by age and four deli meat categories.	  PAGEREF _Toc215306781
\h  19  

  HYPERLINK \l "_Toc215306782"  Table 8. Estimated mean number of
illnesses from L. monocytogenes in deli meat per year among three
populations stratified by age and four deli meat categories.	  PAGEREF
_Toc215306782 \h  20  

  HYPERLINK \l "_Toc215306783"  Table 9. Prevalence of product samples
and stores visited based on sampling locations.	  PAGEREF _Toc215306783
\h  26  

  HYPERLINK \l "_Toc215306784"  Table 10. Prevalence of product samples
(retail-sliced, prepackaged) and stores visited based on quarter of
year.	  PAGEREF _Toc215306784 \h  27  

  HYPERLINK \l "_Toc215306785"  Table 11. Prevalence of product samples
(retail-sliced, prepackaged) and stores visited based on time of day (AM
versus PM).	  PAGEREF _Toc215306785 \h  27  

  HYPERLINK \l "_Toc215306786"  Table 12. Prevalence of only
retail-sliced product and stores visited based on time of day (AM versus
PM).	  PAGEREF _Toc215306786 \h  27  

  HYPERLINK \l "_Toc215306787"  Table 13. Prevalence of product samples
(retail-sliced, prepackaged, and chubs) and stores visited based on
store type (major grocery chain versus other grocers).	  PAGEREF
_Toc215306787 \h  28  

  HYPERLINK \l "_Toc215306788"  Table 14. Prevalence of L. monocytogenes
in retail-sliced and prepackaged deli meat by site. The number of
positive samples and the total number of samples are shown in
parentheses.	  PAGEREF _Toc215306788 \h  30  

  HYPERLINK \l "_Toc215306789"  Table 15. Results of logistic regression
for store prevalence as function of processing type, store type, and
time of day indicator variables	  PAGEREF _Toc215306789 \h  35  

  HYPERLINK \l "_Toc215306790"  Table 16. Prevalence of L. monocytogenes
in sliced deli meat by site and processing type from the Food Products
Association.	  PAGEREF _Toc215306790 \h  37  

  HYPERLINK \l "_Toc215306791"  Table 17. Overall results of statistical
tests for prevalence of L. monocytogenes on RTE meat and poultry deli
meats by location, season, time of day for slicing at retail, and by
deli meat type.	  PAGEREF _Toc215306791 \h  38  

  HYPERLINK \l "_Toc215306792"  Table 18. Estimated mean number of
deaths and illnesses per annum by fraction of consumer storage time.	 
PAGEREF _Toc215306792 \h  39  

  HYPERLINK \l "_Toc215306793"  Table 19. Fitted Weibull distributions
according to the deli meat category	  PAGEREF _Toc215306793 \h  41  

  HYPERLINK \l "_Toc215306794"  Table 20. Storage times for
retail-sliced and prepackaged product.	  PAGEREF _Toc215306794 \h  42  

  HYPERLINK \l "_Toc215306795"  Table 21. Estimated mean number of
deaths per year from L. monocytogenes in deli meat among three
populations stratified by age and four deli meat categories using the
alternative storage time-temperature data.	  PAGEREF _Toc215306795 \h 
45  

  HYPERLINK \l "_Toc215306796"  Table 22. Estimated mean number of
illnesses per year from L. monocytogenes in deli meat among three
populations stratified by age and four deli meat categories.	  PAGEREF
_Toc215306796 \h  46  

  HYPERLINK \l "_Toc215306797"  Table 23. EGR for product with and
without growth inhibitor by shelf life.	  PAGEREF _Toc215306797 \h  47  

  HYPERLINK \l "_Toc215306798"  Table 24. Mean number of deaths and
illnesses per annum by shelf life.	  PAGEREF _Toc215306798 \h  47  

 

Executive Summary

DRAFT FSIS Comparative Risk Assessment for Listeria monocytogenes in
Ready-to-eat Meat and Poultry Deli Meats

March 2009

Background

Listeria monocytogenes (L. monocytogenes) is an important foodborne
pathogen, estimated to cause approximately 2,500 illnesses, 2,300
hospitalizations, and 500 deaths each year in the United States. In an
effort to understand better the sources of foodborne Listeria infection,
the Food and Drug Administration (FDA) and the Food Safety Inspection
Service (FSIS), working collaboratively, developed a quantitative risk
assessment for L. monocytogenes that compared the risk of listeriosis
among twenty-three categories of ready-to-eat (RTE) foods. The results
of the risk assessment, completed in 2003, indicated deli meats pose the
greatest risk for listeriosis, accounting for approximately 1,600
illnesses per year.

Based on these findings, FDA and FSIS conducted a preliminary analysis
using the 2003 L. monocytogenes risk assessment to evaluate the relative
risk of illness from Listeria on deli meat sliced and packaged at
federally-inspected processing establishments (prepackaged deli meat)
compared to deli meat sliced at retail facilities. This risk assessment
contained industry data for L. monocytogenes on retail deli meat from
delicatessens in California and Maryland (Gombas et al. 2003). The
results of this risk assessment indicated a high percentage of
listeriosis cases related to deli meats were associated with those
sliced at retail. Because these results, however, were based on limited
retail L. monocytogenes contamination data for deli meats, FSIS sought
to gather additional data specifically to examine the relative risk of
illness from prepackaged deli meat compared to deli meat sliced at
retail facilities more closely.

Therefore, the U.S. Department of Agriculture, Agricultural Research
Service funded the National Alliance for Food Safety and Security
(NAFSS) – a consortium of twenty-five research universities – to
conduct a four-state study in which prepackaged deli meat and deli meat
sliced and packaged at retail were analyzed for the prevalence and level
of L. monocytogenes (Draughon 2006). Methods

Data from the NAFSS study, described in Appendix I of the risk
assessment report, were used as inputs to the deli meat exposure pathway
developed as part of the abovementioned 2003 FDA-FSIS risk assessment
for Listeria in ready-to-eat foods. The pathway consists of four
distinct stages. The Retail Stage determines the level of L.
monocytogenes in prepackaged deli meats and in deli meats sliced at
retail. The Growth Stage uses an exponential growth rate function to
model growth of L. monocytogenes in deli meat between purchase at retail
and consumption. The Consumption Stage uses information about deli meat
serving sizes and the number of servings consumed to estimate consumer
exposure to L. monocytogenes in deli meat. Lastly, by integrating the
predicted exposure with a dose-response relationship, the Dose-response
Stage predicts the probability of death from consuming L. monocytogenes
on deli meat. 

Two distinct consumer storage time-temperature distributions were used
for the risk assessment. The first analysis used the same storage
time-temperature distributions as the 2003 FDA-FSIS risk assessment. The
storage times were taken from an American Meat Institute (AMI) 2001
survey and consumer storage temperatures were taken from an Audits
International 1999 survey. For this first analysis, the storage times
for both prepackaged deli meat and deli meat sliced at retail used the
same values. A second analysis, described in Appendix II, was performed
using the consumer survey conducted by RTI International, Tennessee
State University, and Kansas State University (Cates et al. 2006). The
results of the survey indicated prepackaged deli meat was stored for
statistically significant longer periods than deli meat sliced at
retail. The survey did not find any difference for storage temperature.
This second analysis thus used different storage time distributions for
prepackaged versus retail sliced product.

Results

This risk assessment, using current retail contamination data for deli
meat (Draughon 2006; NAFSS) and current consumer behavior data for deli
meats (Cates et al. 2006; RTI) indicates that of those listeriosis cases
and deaths attributed to deli meats, approximately 83%  are associated
with deli meats sliced at retail. The estimated mean number of illnesses
per year from prepackaged deli meats was 188.6 with a 95% confidence
interval (CI) of 184.7 – 192.4. The estimated mean number of deaths
per year from L. monocytogenes prepackaged deli meats was 34.1 (95% CI:
33.4 – 34.9). In contrast, the estimated mean number of illnesses per
year from retail-sliced deli meats was 919.6 (95 CI: 686.4 – 932.4).
The estimated mean number of deaths per year from L. monocytogenes in
retail-sliced deli meats was 166.9 (95% CI: 164.5 – 169.3). 

Conclusions

Of those illnesses and deaths from L. monocytogenes from deli meat
consumption, a large percentage is attributed to deli meat sliced at
retail facilities. The remainder is from prepackaged deli meat. Studies
are needed to determine how contamination of deli meat at retail occurs
and to design effective mitigations for reducing listeriosis associated
with the consumption of deli meat sliced at retail.

Introduction

In 2000, the Food and Drug Administration (FDA) and the U.S. Department
of Agriculture’s Food Safety and Inspection Service (USDA/FSIS) began
a risk assessment to identify which ready-to-eat (RTE) foods pose the
greatest risk for listeriosis in the U.S. (FDA-FSIS 2003). Deli meat was
found to pose the greatest risk of listeriosis among all RTE food
categories. Based on these results and in response to public comments on
the FSIS proposed rule Performance Standards for the Production of
Processed Meat and Poultry Products (66 FR 12589), FSIS developed a risk
assessment for L. monocytogenes in RTE meat and poultry products (FSIS
2003). The risk assessment model predicted that the use of
post-lethality interventions and antimicrobial growth inhibitors
significantly lowered the public health risk of listeriosis compared to
either control if used independently. Post-processing lethality
treatments that reduced L. monocytogenes in products formulated or
processed to inhibit the growth of any remaining L. monocytogenes were
predicted to be the most effective in protecting public health. Both the
2003 FDA-FSIS and 2003 FSIS Listeria risk assessments served as the
scientific basis for FSIS’ interim final rule for the control of L.
monocytogenes during processing (“Control of Listeria Monocytogenes in
Ready-to-Eat Meat and Poultry Products,” 68 FR 34208; June 6, 2003
(revised January 1, 2006); 9 CFR 430).

Subsequently, in 2004, FDA and FSIS did a preliminary analysis using L.
monocytogenes contamination data for retail deli meat from California
and Maryland (Gombas et al. 2003) to estimate the relative risk of
listeriosis from deli meat sliced and packaged in FSIS-inspected
processing establishments (hereafter termed prepackaged) versus those
sliced and packaged at retail facilities. Results suggested that deli
meat sliced and packaged at retail posed the greater risk, accounting
for approximately 80% of all listeriosis cases from deli meat.

In 2006, researchers with the National Alliance for Food Safety and
Security (NAFSS) – a consortium of 25 research universities –
completed a study of L. monocytogenes contamination in prepackaged RTE
meat and poultry deli meats and those sliced and packaged at retail from
California, Maryland, Georgia, and Tennessee (Draughon 2006). FSIS
adapted the FDA-FSIS (2003) risk assessment model to examine data from
the NAFSS study and reanalyze the comparative risk of listeriosis from
prepackaged RTE deli meat versus RTE deli meat sliced and packaged at
retail. This report describes the analysis and its findings. 

Lastly, while doing this risk assessment, a consumer survey done by RTI
International, Tennessee State University, and Kansas State University
was released (Cates et al. 2006; survey available at
http://www.foodrisk.org) indicating prepackaged deli meat may be stored
for statistically significant longer periods than deli meat sliced at
retail. Therefore, we did a sensitivity analysis by rerunning the risk
assessment model using these new survey results and compared the output
to that from using the AMI survey results, as described in Appendix II
of this report.

Methods

The analysis uses the deli meat exposure pathway from the risk
assessment model developed and utilized in a previous L. monocytogenes
risk assessment (FDA-FSIS, 2003) that estimated risk of death
attributable to 23 ready-to-eat (RTE) food categories. This analysis
separates the deli meat category into prepackaged deli meats and those
sliced at retail establishments. Because of increased use of
antimicrobial growth inhibitors in deli meat, each deli meat type is
divided into those with or without antimicrobial growth inhibitors.
Consistent with the FDA-FSIS (2003) risk assessment model, this analysis
considers four conceptual stages (  REF _Ref197160429  \* MERGEFORMAT 
Figure 1 ). 

Figure   SEQ Figure \* ARABIC  1 . A conceptual model of the stages in
this risk assessment and the critical inputs considered within each
stage.

 

The Retail Stage determines the presence and level of L. monocytogenes
in the two deli meat types.

 

The Growth Stage uses an exponential growth rate modified to account for
antimicrobial growth inhibitor usage to predict growth of L.
monocytogenes in deli meat between retail and consumption. 

The Consumption Stage predicts the L. monocytogenes exposure dose
consumed in servings of deli meats, which is a consequence of serving
size and the number of servings. 

The Dose-response Stage predicts the probability of death from L.
monocytogenes per serving by integrating the predicted exposure
distribution with a dose-response relationship. 

The output of these four stages in this risk assessment is the annual
number of illnesses and deaths (and the corresponding risk of illness or
death per serving) from Listeria deli meat. While these four stages are
updated in the FDA-FSIS (2003) model for the deli meat food category,
all other food categories remain as originally parameterized. This risk
assessment describes the analysis used to parameterize each of the four
updated stages in more detail below.

stage I: Prevalence and Level of L. monocytogenes in RTE Meat and
Poultry deli meats at retail

The prevalence and level of L. monocytogenes in RTE meat and poultry
deli meats at retail establishments were determined using data from a
National Alliance for Food Safety and Security (NAFSS) study in which 6
of 3,522 (0.17%) samples and 49 of 3,518 (1.39%) samples tested positive
for L. monocytogenes from prepackaged and retail-sliced deli meats,
respectively. This difference was statistically significant (p <0.05).
Analyses of these data are described in depth in Appendix I. Of the six
positive samples from prepackaged deli meat, all had L. monocytogenes
levels (0.3 MPN/gram. Of the 49 positive samples from deli meat sliced
and packaged at retail, L. monocytogenes levels ranged from <0.3 to (110
MPN/gram. Cumulative density plots, assuming a detection limit of 0.008
MPN/gram (i.e. 1 MPN/125 g), are shown in   REF _Ref197161057  \*
MERGEFORMAT  Figure 2 .

Figure   SEQ Figure \* ARABIC  2 . Cumulative density functions for the
amount of L. monocytogenes in prepackaged compared to retail-sliced RTE
deli meats.

Data for the prevalence and level of L. monocytogenes in deli meat
sampled at retail were fitted to probability distributions as inputs to
the FDA-FSIS (2003) model. Because there were few positive samples,
distribution fits were considered approximate. The survival analysis
module of Number Cruncher Statistical Systems (NCSS: 
http://www.ncss.com/) was used to fit an appropriate statistical
distribution to prepackaged and retail-sliced deli meat separately.
Survival analysis allows incorporation of left and right censoring into
distribution fitting. Left censoring indicates that the true level of L.
monocytogenes in deli meat is less than reported. Right censoring
indicates that the true level of L. monocytogenes in deli meat is higher
than reported. Interval censoring indicates that the true value is
between two fixed values. To be conservative, all but one of the
observed L. monocytogenes positive samples of deli meat with a level of
( 0.3 MPN/gram were treated as having a level of 0.3 MPN/gram. The
remaining positive sample was treated as an interval measurement between
0.008 MPN/gram and 0.3 MPN/gram. Negative samples were assumed to have
L. monocytogenes levels ( 0.008 MPN/gram (i.e. ( 1 MPN/125 gram). The
inputs to the survival analysis are shown in   REF _Ref197161103  \*
MERGEFORMAT  Table 1 . The comparison of maximum likelihood fit to
various probability distributions is provided below for retail-sliced
(Table 2) and prepackaged (Table 3) deli meat. The parameters for each
distribution were determined by least-squares regression fit to the
corresponding probability plot. 

Table   SEQ Table \* ARABIC  1 . Survival analysis input for statistical
distribution fitting for the level of L. monocytogenes in deli meats at
retail.

Retail (deli) sliced

Prepackaged

No. Samples1	L. monocytogenes level (MPN/gram)1	Censor Type2

No. Samples	L. monocytogenes level (MPN/gram)	Censor Type2

3,469	(0.008	L

3,516	( 0.008	L

1	Between 0.008 and 0.3	I

1	Between 0.008 and 0.3	I

29	0.3	F

5	0.3	F

3	0.92	F





1	0.93	F





1	0.94	F





3	2.3	F





1	15.3	F





1	24	F





1	46	F





3	( 110	R





1 L. monocytogenes levels were not given for five positive retail-sliced
deli meat samples. These data were thus not used in the distribution
fitting.

2 Censor type refers to the censoring used by the survival analysis fit.
L indicates left censoring (actual value is less than observed); I
indicates interval censoring (actual value is between two known values).
F indicates actual value is observed level. R indicates right censoring
(actual value is greater than observed).

Table   SEQ Table \* ARABIC  2 . Best fit maximum likelihood results
and probability plot distribution parameters for retail-sliced deli
meat.

Distribution	Log Likelihood	Shape1	Scale1

Weibull	-315.606	NA2	NA2

Lognormal	-316.634	-25.6314	9.309884

Lognormal10	-316.634	-11.1316	4.043231

Loglogistic	-318.041	-19.0915	3.277907

Logistic	-375.906	-13.795	3.168052

Extreme Value	-396.146	-22.3905	15.19124

Exponential	-13046.5	         1	0.012057

Normal	    NA2	       NA2	NA2

1 The interpretation of these parameters varies depending on the
distribution. For most distributions, the shape is the mean of the
distribution and the scale is the standard deviation.

2 The probability plot estimate could not be calculated for these
parameters.

Table   SEQ Table \* ARABIC  3 . Best fit maximum likelihood results and
probability plot distribution parameters for prepackaged deli meat.

Distribution	Log Likelihood	Shape1	Scale1

Extreme Value	-43.1107	-2.38335	1.303234

Normal	-43.2915	-1.80981	0.628146

Logistic	-43.5157	-1.14005	0.183313

Weibull	-49.6842	NA2	NA2

Lognormal10	-49.865	-27.3912	7.79663

Lognormal	-49.865	-11.8958	3.386033

Loglogistic	-50.0892	-19.078	2.275309

Exponential	-715.407	1.00E+00	1.37E-03

1 The interpretation of these parameters varies depending on the
distribution. For most distributions, the shape is the mean of the
distribution and the scale is the standard deviation.

2 The estimate could not be calculated for these parameters.

Though the Weibull and extreme value distributions are suggested as best
fitting these data (based on the maximum likelihood criterion), the
lognormal distribution was selected as the most appropriate. The
lognormal fit to the distribution of the level of L. monocytogenes in
retail-sliced deli meat is statistically no different from the Weibull
distribution. It is preferred that both retail-sliced and prepackaged
distributions are modeled as the same type. Environmental contaminants
such as bacterial levels are often fit to a lognormal distribution and
this distribution has theoretic justification. The probability plots and
the resulting fit for both retail-sliced and prepackaged deli meat are
shown on the following page in Figure 3. The fitted cumulative density
plots and observed data points are shown in Figure 4a. The fit for the
retail-sliced deli meat appears adequate. The distribution fit for the
prepackaged deli meat is uncertain because of only two data points.
Figure 4b extrapolates the cumulative density curves to lower levels.
The deli meat exposure assessment model uses levels as low as 10-8
MPN/gram as inputs.

Figure   SEQ Figure \* ARABIC  3 . Probability plots for fitted
lognormal (base 10) distribution to observed levels for retail-sliced
(a) and prepackaged deli meat (b).

Figure   SEQ Figure \* ARABIC  4 . Fitted cumulative distribution plots
and observed retail data for L. monocytogenes levels in retail-sliced
versus prepackaged deli meats. Illustration (a) is over the range of
observed L. monocytogenes levels. Illustration (b) is over the entire
range of L. monocytogenes levels in deli meats used as an input into
deli meat exposure pathway of the 2003 FDA-FSIS risk assessment model.

The quantile-quantile plot of the two fitted distributions is shown in  
REF _Ref197225148  \* MERGEFORMAT  Figure 5 . Because the same
distribution shape (lognormal) was selected for both retail-sliced and
prepackaged, the quantile-quantile plot is a straight line. The
quantile-quantile line is below the 1:1 reference line, indicating, as
expected, that for a given percentile, the prepackaged L. monocytogenes
level is lower than the L. monocytogenes retail-sliced level over the
range of interest. The quantile-quantile line parallel to the reference
line, indicating that the difference between the two distributions is
greater at the extreme upper tails of the distributions.

Figure   SEQ Figure \* ARABIC  5 . Quantile-quantile plot of fitted
distributions for L. monocytogenes levels from retail-sliced and
prepackaged deli meat. (A 1:1 reference line is included for visual
comparison).

A fixed number of quantiles from the distribution of L. monocytogenes
levels in deli meats serve as inputs to the 2003 FDA-FSIS exposure
assessment model. Based on the fitted parameters shown in Table 2 and
Table 3, the quantiles needed for the exposure assessment model were
determined using free statistical software, R
(http://www.r-project.org/). These quantiles are given below in   REF
_Ref197482261  \* MERGEFORMAT  Table 4 .

Table   SEQ Table \* ARABIC  4 . Quantiles from fitted lognormal
distributions for retail-sliced and prepackaged L. monocytogenes levels.


Cumulative Fraction	Retail-sliced L. Monocytogenes level (MPN/gram)
Prepackaged L. Monocytogenes level (MPN/gram)

0.8	1.87E-08	8.99E-10

0.85	1.15E-07	4.11E-09

0.9	1.12E-06	2.78E-08

0.95	3.30E-05	4.72E-07

0.99	1.88E-02	9.58E-05

0.995	1.92E-01	6.70E-04

0.999	2.31E+01	3.70E-02

0.9999	8.04E+03	4.98E+00

Max1	8.03E+06	6.16E+03

1Based on simulation of 1,000,000 random numbers from the appropriate
fitted distribution.

The risk assessment analysis used in this report assumed independence
among samples. This assumption may not be met for these data, however,
because the samples collected from the same retail location are likely
to be correlated. Cross-contamination or poor hygienic conditions within
a retail location may result in the clustering of positive L.
monocytogenes findings by store; therefore, analyzing the data by store
location may provide a more accurate estimate of the relative risk ratio
for retail-sliced versus prepackaged products. However, due to the
blinding process used during sample collection, individual store
identifiers were removed. Without these store identifiers, store visits
can only be estimated based on time and date of sample collection. Also,
sample collection times were not provided for samples from Minnesota, so
determining individual store visits was not possible. As a result of
these data limitations, all individual samples were treated as
independent for this risk assessment analysis. A comparison of the
results based on the assumption of independence of samples versus
samples grouped by approximate store visit may be found in the Appendix.

stage II: Growth of L. monocytogenes from retail purchase to consumption

To assess consumer exposures, the growth of L. monocytogenes from retail
purchase to consumption was modeled. Given regulatory changes subsequent
to the development of the FDA-FSIS (2003) risk assessment, the model’s
predicted growth for deli meats needed adjustment. Nevertheless, the
storage time and temperature distributions were left unchanged from the
FDA-FSIS (2003) risk assessment model and the same time and temperature
distributions were used for both prepackaged and retail-sliced deli
meat. 

FSIS provides three alternatives for establishments producing certain
RTE meat and poultry deli meats to control for L. monocytogenes (9 CFR
430, 2003): 

Alternative 1: Employ both a post-lethality treatment and an
antimicrobial growth inhibitor for Listeria monocytogenes on RTE deli
meats. 

Alternative 2: Employ either (a) a post-lethality treatment or (b) an
antimicrobial growth inhibitor for the pathogen on RTE deli meats. 

Alternative 3: Employ sanitation measures only (uses neither a
post-lethality treatment nor an antimicrobial growth inhibitor).

Deli meat that uses an antimicrobial growth inhibitor is expected to
have lower growth rates of L. monocytogenes than deli meat that does not
use antimicrobial growth inhibitor. Data on production volumes for each
category were used to estimate the use of antimicrobial growth
inhibitors in RTE deli meat, and current regulations were used to
estimate conservative maximum allowable growth rates. 

To qualify as using an antimicrobial growth inhibitor under the Interim
Final Rule 9 CFR 430, the growth of L. monocytogenes may not exceed two
logs during the shelf life of the product. This information was used to
modify the existing FDA-FSIS risk assessment model to account for
different growth in deli meat with and without antimicrobial growth
inhibitors. The comparison of retail-sliced versus prepackaged was
calculated by splitting the deli meat category into four separate
categories based on two factors: where the deli meat was sliced and
whether antimicrobial growth inhibitor was used. Exponential growth
rates for L. monocytogenes were calculated for product with and without
antimicrobial growth inhibitors using data from the 2003 FDA-FSIS risk
assessment and the estimated fraction of deli meat in each category
prior to the implementation of the Interim Final Rule. These older data
were used for calculating growth rates because it better matched the
period of reported growth rates in the FDA-FSIS risk assessment model.
Once the growth rates were determined, this risk assessment used more
current manufacturer production volume data to calculate the fraction of
deli meat in each category. 

Prior to the Interim Final Rule, fewer plants employed alternatives that
used antimicrobial growth inhibitors therefore less product was
formulated with an antimicrobial growth inhibitor when compared to
current conditions. The overall growth rate of L. monocytogenes should
be lower after the implementation of the Interim Final Rule because the
composition of the product is different – a greater fraction of
product contains antimicrobial growth inhibitors. 

To calculate the relative growth rates for deli meat with and without
antimicrobial growth inhibitor, the fraction of deli meat using
antimicrobial growth inhibitor prior to the Interim Final Rule was
needed. The number of establishments using each L. monocytogenes control
alternative (1, 2 (a or b), or 3) was estimated by FSIS economists. This
is shown in Table 5. The fraction of production was estimated by
assuming that each plant within a Pathogen Reduction/Hazard Analysis and
Critical Control Point (PR/HACCP) size category produced the same
volume, and that the total fraction of production was 48%, 48%, and 4%
for large, small, and very small plants (FSIS 2003) respectively.

Table   SEQ Table \* ARABIC  5 . Plant distribution and estimated
fraction of production prior to the Interim Final Rule.

Lm Control

Alternative	PR/HACCP Size Category1

	Large	Small	Very Small	Total

	No.2	Fraction3	No.2	Fraction3	No.2	Fraction3	No.2	Fraction3



(1) Both post processing lethality and antimicrobial growth inhibitor	

7	

0.018	

20	

0.007	

15	

0.000	

42	

0.026



(2a) Post processing lethality only	

15	

0.039	

79	

0.029	

49	

0.001	

143	

0.068



(2b) Antimicrobial growth inhibitor only	

40	

0.104	

122	

0.044	

65	

0.001	

227	

0.149



(2) Neither post processing lethality nor antimicrobial growth inhibitor


123	

0.319	

1107	

0.400	

2072	

0.038	

3302	

0.757

Total	185	0.480	1328	0.480	2201	0.040	3714	1.000

1 Based on PR/HACCP classification. 

2 No. is the number of plants.

3Fraction is the fraction of production = number of plants within size
and alternative / total number of plants within size * total fraction by
size.

Based on this analysis, it was estimated that 17.5% (2.6% + 14.9%) of
deli meat used antimicrobial growth inhibitors prior to the
implementation of the Interim Final Rule. The percentage of deli meat
using antimicrobial growth inhibitors was assumed the same for
prepackaged and retail-sliced deli meat.

The exposure assessment portion of the 2003 FDA-FSIS model was adjusted
to account for possible use of antimicrobial growth inhibitor by
adjusting the exponential growth rate (EGR) of L. monocytogenes among
RTE meat and poultry deli meats. The FDA-FSIS model estimated that the
mean EGR at 5oC was 0.282 log10 CFU/gram/day. The model treats this as a
stochastic parameter and adjusts for stochastic storage time,
temperature, and a correlation between the two. Appendix 8 in the 2003
FDA-FSIS risk assessment report lists the references used to calculate
this value: 15 published articles with 23 reported growth rates across a
range of deli meat products. Most of these reference data are from the
late 1980’s to early 1990’s, which is why the use of production data
from prior to the implementation of the Interim Final Rule was deemed
appropriate.

FSIS L. monocytogenes Compliance Guidelines
(http://www.fsis.usda.gov/OPPDE/rdad/

FRPubs/97-013F/Lm_Rule_Compliance_Guidelines_May_2006.pdf ) state that
to qualify as utilizing one of two most stringent alternative L.
monocytogenes control options (Alternative 1 or 2) in the Interim Final
Rule, no more than 2 log10 growth is allowed over the entire shelf life
of the product. No temperature is specified during this shelf life, nor
is the shelf life itself specified. If this standard is interpreted to
be 2 log10 growth over 14 days at 5oC, the exponential growth rate is 2
log10/gram/14 days = 0.143 log10 CFU/gram/day. Using this calculation,
as the product shelf life is reduced, the calculated EGR would increase
because the same 2-log10 growth would occur in a shorter time. 

For comparison, consumer storage time is available based on an American
Meat Institute (AMI 2001) survey. Results of the survey suggest
approximately 40% of ready-to-eat product is stored for less than 3
days, and another 45% of product is stored from 4 to 7 days. A total of
96% of product is stored for less than 14 days. While consumer storage
time is not the same as shelf life, the 14-day assumption appears
reasonable. A sensitivity analysis of this shelf life assumption is
provided in Appendix II.

If the exponential growth rate (EGR) for product with antimicrobial
growth inhibitor (GI) is based on the regulation, then to calculate the
EGR for product without GI:

fGI x EGRwith + (1 - fGI) x EGRwithout = EGRFDA

0.175 x 0.143 log10 CFU/gram/d + 0.825 x EGRwithout = 0.282 log10
CFU/gram/day

EGRwithout =0.311 log10 CFU/gram/day

The EGR for product both with and without antimicrobial growth inhibitor
are within the observed range for the 23 literature values noted
previously and within one standard deviation of the mean EGR.

The maximum L. monocytogenes level that can occur in product can also be
adjusted. As there are no additional data for this parameter, it was
left unchanged from the existing FDA-FSIS model. 

To adjust the growth rates in the deli meat exposure pathway of the 2003
FDA-FSIS risk assessment model, an additional multiplier based on
adjusting the mean EGR was added. If the product did not have GI, the
stochastic EGR for each iteration was multiplied by 0.311/0.282 = 1.104.
If the product did have GI, the stochastic EGR for each iteration was
multiplied by 0.143/0.282 = 0.507. Note that the EGR for product with GI
is calculated based on FSIS regulation, not on actual industry
performance, which may be greater.

Stage III: deli meat Consumption

Serving sizes and total number of deli meat servings annually consumed
that were estimated for the 2003 FDA-FSIS risk assessment were unchanged
for this analysis. Nevertheless, total servings of deli meats needed to
be proportioned among (i) prepackaged deli meats with antimicrobial
growth inhibitors; (ii) prepackaged deli meats without antimicrobial
growth inhibitors; (iii) retail-sliced deli meats with antimicrobial
growth inhibitors; and (iv) retail-sliced deli meats without
antimicrobial growth inhibitors.

The fraction of servings for each of the four deli meat categories was
estimated from  industry survey data from USDA/FSIS Form 10,240-1,
Production Information on Post-Lethality Exposed Ready-to-Eat Products,
gathered in July 2007 in accordance with 9 CFR 430.4(d). For example,
32.2% of servings were calculated to be prepackaged product with
antimicrobial growth inhibitor. Overall, approximately 47% of deli meat
is sold prepackaged, and 53% is retail-sliced (Table 6).

Table   SEQ Table \* ARABIC  6 . Fraction of deli meat production by
slicing location and antimicrobial growth inhibitor use during July
2007.

Alternative	Prepackaged (sliced at plant)	Retail-Sliced 	Total

With antimicrobial growth inhibitor	0.322	0.267	0.589

Without antimicrobial growth inhibitor	0.144	0.267	0.411

Total	0.466	0.534	1.000

Stage IV: L. monocytogenes Dose-response Relationship

In the 2003 FDA-FSIS risk assessment, there are three age-specific
dose-response relationships that have been developed – one for those
60 years of age or older (referred to as “elderly” in the 2003
FDA-FSIS risk assessment), those who are more than 30 days old to 60
years of age (“intermediate” age population), and fetuses and
neonates from 16 weeks after fertilization to 30 days old (perinatal
subpopulation). The methods used in the 2003 FDA-FSIS risk assessment
are the same as those used here.

The dose-response model was run in calibrated mode. In calibrated mode,
a scaling factor was used for each of the 4,000 simulations to adjust
the dose-response curve from the mouse model to meet a specified number
of deaths in humans. For this analysis, the comparative risk assessment
was calibrated to the number of deaths attributed to deli meats, based
on data from the Centers for Disease Control and Prevention, used in the
2003 FDA-FSIS risk assessment. Given the increased implementation of
Listeria control procedures at the processing plant and antimicrobial
growth inhibitor use in the product, these values are likely to
overstate estimated deaths under current conditions. Thus, the estimated
deaths are meant for comparative purposes only, but do not affect
accurate estimates of the “relative risk” of listeriosis and
subsequent death associated with deli meats sliced that are
retail-sliced compared to those that are prepackaged.

Results

The original deli meat category in the 2003 FDA-FSIS Listeria risk
assessment was split into the four separate categories such that
exposure distributions were estimated (using that model) for (i)
prepackaged deli meats with antimicrobial growth inhibitors; (ii)
prepackaged deli meats without antimicrobial growth inhibitors; (iii)
retail-sliced deli meats with antimicrobial growth inhibitors; and (iv)
retail-sliced deli meats without antimicrobial growth inhibitors. These
exposure distributions were generated from the two contamination
distributions at retail (i.e., one for prepackaged deli meat and another
for retail-sliced deli meat). The growth predictions applied to each of
these distributions predicted the effects of variable storage times and
temperatures on the number of L. monocytogenes per gram of deli meat
depending on whether it included antimicrobial growth inhibitors or not.
The exposure distribution finally determined the variability in dose per
serving by considering the (stochastic) number of grams constituting a
serving of deli meat. In the FDA-FSIS (2003) risk assessment model,
these exposure distributions were integrated with the FDA-FSIS L.
monocytogenes dose-response models (one for each of the three
age-specific subpopulations) to predict the annual number of deaths
attributed to each of the four categories. The estimated mean numbers of
deaths per year are given in Table 7 below. Clearly, the use of
antimicrobial growth inhibitors reduces the number of estimated deaths.
This is most notable for the retail-sliced product, which starts with a
higher level at retail. Also notable is the impact that the lower L.
monocytogenes starting distribution has on lowering the number of deaths
from prepackaged products. 

The estimated mean number of deaths per year associated with prepackaged
product was 13.8 (4.4+9.4), and the estimated mean number of deaths per
annum associated with retail-sliced product was 125.5 (23.5+102.0), with
an estimated total annual number of deaths equal to 139.3. Ten percent
of the estimated per annum deaths (13.8/139.3 = 9.89%) are attributable
to prepackaged product, while the remaining 90% are attributable to
retail-sliced product (125.6/139.3 = 90.11%). The relative risk on a per
annum basis for deli meats sliced at retail versus sliced at plant is
thus 125.6/13.8 = 9.1. These results are almost identical to the earlier
2003 FDA-FSIS Listeria risk assessment findings, which used NFPA retail
data (Gombas et al. 2003). A similar analysis was conducted for
illnesses. The 2003 FDA-FSIS model assumes a constant illness to
mortality ratio by age group of 3.7, 11.3, and 12.7 for elderly,
intermediate, and perinatal age groups respectively. Estimated illnesses
from L. monocytogenes in deli meat are shown in   REF _Ref198950948  \*
MERGEFORMAT  Table 8 . A mean of 698.0 illnesses were attributed to
retail-sliced product and a mean of 76.8 illnesses were attributed to
prepackaged product, for a relative risk ratio of 9.1. 

Table   SEQ Table \* ARABIC  7 . Estimated mean number of deaths per
year from L. monocytogenes in deli meat among three populations
stratified by age and four deli meat categories.

Deli Meat Category	Elderly

(95% CI)	Intermediate Age

(95% CI)	Perinatal

(95% CI)	All Age Groups (95% CI)

Prepackaged with growth inhibitor	3.4

(3.3 - 3.5)	0.8

(0.8 - 0.8)	0.2

(0.2 - 0.2)	4.4

(4.3 - 4.5)

Prepackaged without growth inhibitor 	7.2

(7.0 - 7.3)	1.8

(1.7 - 1.8)	0.5

(0.4 - 0.5)	9.4

(9.1 - 9.6)

Retail-sliced with growth inhibitor	18.0

(17.5 - 18.4)	4.4

(4.3 - 4.6)	1.1

(1.1 - 1.2)	23.5

(23.0 - 24.1)

Retail-sliced without growth inhibitor	78.0

(76.5 - 79.6)	18.9

(18.5 - 19.3)	5.1

(5.0 - 5.2)	102.0

(100.1 - 104.0)

Subtotal: Prepackaged	10.5

(10.3 - 10.8)	2.6

(2.5 - 2.6)	0.7

(0.7 - 0.7)	13.8

(13.5 - 14.1)

Subtotal: Retail-sliced	96.0

(94.3 - 97.7)	23.3

(22.9 - 23.7)	6.2

(6.1 - 6.3)	125.6

(123.4 - 127.7)

Subtotal: With growth inhibitor	21.3

(20.8 - 21.8)	5.3

(5.1 - 5.4)	1.4

(1.3 - 1.4)	27.9

(27.3 - 28.6)

Subtotal: Without growth inhibitor	85.2

(83.6 - 86.8)	20.7

(20.3 - 21.0)	5.6

(5.5 - 5.6)	111.4

(109.4 - 113.4)

Total	106.5

(104.7 - 108.3)	25.9

(25.5 - 26.3)	6.9

(6.8 - 7.0)	139.3

(137.1 - 141.6)



Table   SEQ Table \* ARABIC  8 . Estimated mean number of illnesses
from L. monocytogenes in deli meat per year among three populations
stratified by age and four deli meat categories.

Deli Meat Category	Elderly

(95% CI)	Intermediate Age (95% CI)	Neonatal

(95%CI)	All Age Groups

(95% CI)

Prepackaged with growth inhibitor	12.4

(12.1 - 12.8)	9.3

(9.0 - 9.6)	2.8

(2.7 - 2.8)	24.5

(23.9 - 25.2)

Prepackaged without growth inhibitor	26.5

(25.8 - 27.2)	19.9

(19.4 - 20.5)	5.8

(5.7 - 5.9)	52.2

(50.9 - 53.5)

Retail-sliced with growth inhibitor	66.5

(64.8 - 68.1)	50.1

(48.8 - 51.4)	14.4

(14.2 - 14.7)	131.0

(127.9 - 134.2)

Retail-sliced without growth inhibitor	288.7

(283.0 - 294.4)	213.6

(209.5 - 217.7)	64.7

(63.8 - 65.6)	567.0

(556.5 - 577.6)

Subtotal: Prepackaged	38.9

(38.1 - 39.8)	29.2

(28.6 - 29.9)	8.6

(8.5 - 8.7)	76.8

(75.1 - 78.4)

Subtotal: Retail-sliced	355.2

(348.9 - 361.5)	263.7

(259.3 - 268.2)	79.1

(78.1 - 80.2)	698.0

(686.4 - 709.6)

Subtotal: With growth inhibitor	78.9

(77.1 - 80.7)	59.4

(58.0 - 60.8)	17.2

(16.9 - 17.5)	155.5

(152.1 - 159.0)

Subtotal: Without growth inhibitor	315.2

(309.3 - 321.1)	233.5

(229.3 - 237.8)	70.5

(69.5 - 71.5)	619.2

(608.2 - 630.2)

Total	394.1

(387.4 - 400.8)	292.9

(288.3 - 297.6)	87.7

(86.6 - 88.8)	774.8

(762.5 - 787.1)



The estimated number of deaths was summed across each age group for
each simulation. A histogram and cumulative density plot of the
estimated number of deaths between retail-sliced and prepackaged product
are shown in   REF _Ref197486738  \* MERGEFORMAT  Figure 6  and   REF
_Ref197486755  \* MERGEFORMAT  Figure 7 , respectively.

Figure   SEQ Figure \* ARABIC  6 . Histograms of estimated number of
deaths per annum for retail-sliced and prepackaged product based on the
4,000 dose-response simulations of the FDA-FSIS model.

Figure   SEQ Figure \* ARABIC  7 . Cumulative density plots estimated
number of deaths per annum for retail-sliced and prepackaged product
based on the 4,000 dose-response simulations of the FDA-FSIS model.

To evaluate better if the estimated mean number of deaths among the
different scenarios were statistically different, a bootstrap analysis
comparing the means of the scenarios was undertaken. One hundred
thousand samples (with replacement) were sampled from the 4,000
simulations of each specified scenario. The mean of each of these
100,000 samples was then calculated. This process was repeated 100,000
times to generate a distribution of means. The mean and 95% confidence
interval from this distribution was then obtained. (Sensitivity analysis
indicated that even the 2.5th and 97.5th % quantiles had stabilized with
100,000 runs.) Recall that these simulations were based on the starting
L. monocytogenes distributions at retail for either the retail-sliced or
prepackaged. Uncertainty about these distributions was not included.
Thus, the resulting confidence intervals are narrower and more likely to
find statistical differences than if the initial distributions included
uncertainty as well. These results are presented in Table 18. 

Table 18. Statistical comparison of mean number of estimated deaths by
processing type.

Scenario	Mean	LCL* (2.5%)	UCL** (97.5%)

Prepackaged	13.8	13.5	14.1

Retail-sliced	125.6	123.4	127.7

Difference in means (retail-sliced – prepackaged)	111.8	109.6	114.0

*LCL=lower confidence level about the mean.

**UCL=upper confidence level about the mean.

 

The 95% confidence interval for the difference in means does not include
0. Thus, the difference in means is statistically significant from 0 at
95% confidence. Using the fractions of each product in Table 16 and an
annual number of deli meat servings of 2.84x109, 1.78x1010, and 5.95x106
for elderly, intermediate, and perinatal, the estimated deaths per
serving are shown in Table 19. The number of servings is taken from
FDA-FSIS (2003). The perinatal values are based on the intermediate
number of servings corrected for a pregnancy rate of 0.0174 and an
exposure period of 7 days / 365 days.

Table 19. Estimated mean number of deaths per serving among the three
age groups and four deli meat categories.

Food Category	Elderly	Intermediate	Perinatal

Prepackaged with antimicrobial growth inhibitor	3.67x10-9	1.43x10-10
1.15x10-7

Prepackaged without antimicrobial growth inhibitor	1.75x10-8	6.88x10-10
5.33x10-7

Retail-sliced with antimicrobial growth inhibitor	2.37x10-8	9.33x10-10
7.16x10-7

Retail-sliced without antimicrobial growth inhibitor	1.03x10-7	3.97x10-9
3.21x10-6



Conclusions

Based on this analysis, RTE meat and poultry products sliced at retail
are approximately 9 times more risky on an annual basis than prepackaged
product. If consumers store retail-sliced product for shorter periods
than prepackaged product, this ratio might be overstated. 

The analysis described in this report indicates the need for two types
of data. First, environmental/ecological data are needed to indicate
occurrence and origin of L. monocytogenes at retail. Second, consumer
handling data are needed for how consumers treat RTE product sliced at
retail versus prepackaged product sliced at the manufacturing facility. 

Appendix I: L. monocytogenes in Ready-to-eat Meat and Poultry Deli meat

The presence and level of L. monocytogenes in ready-to-eat (RTE) meat
and poultry products at was determined using data from a study conducted
by the National Alliance for Food Safety and Security (NAFFS) (Draughon,
2006).

Data Collection Methods

The sampling group was comprised of four designated sites in the
Foodborne Disease Active Surveillance Network (FoodNet). These were
Northern California (CA), Georgia (GA), Minnesota (MN), and Tennessee
(TN). Sampling was weighted by the populations in counties
(http://www.census.gov) so that exposure could be estimated.
Approximately 75% of shopping is done at major supermarket chains and
25% is done at other grocers, such as independent retailers (Gombas et
al. 2003). The number of samples collected from supermarkets versus
independent retailers was weighted accordingly. Also, >50% of consumers
purchase RTE meat products that are sliced at delicatessens with the
remainder purchasing sliced prepackaged products. USDA data suggest that
approximately 47% of RTE deli meat is sliced at the processing plant and
prepackaged. The relative number of samples between prepackaged and
retail-sliced was therefore kept approximately equal as part of the
sampling design. Sample data were encoded by the researchers to prevent
identification of the store. 

Approximately 2,000 samples (125 grams each) were analyzed from each of
the four designated sites, with approximately equal numbers of samples
sliced at retail versus sliced at a processing establishment and a
smaller number of intact product samples. The sampling protocol was
designed to allow for statistically valid comparisons among sites, RTE
products type, and retail-sliced versus prepackaged, assuming an α =
0.05 and a 90% power of detecting a difference of 2% in the comparison
of binomial proportions. 

The following product types were sampled: cured poultry, uncured
poultry, pork, and beef. Analysis of approximately 1,000 samples of each
product type was done to support conclusions at the desired level of
certainty. Use of any antimicrobial or growth inhibiting agents was
noted at the time of sample collection. 

Intact samples were collected by purchasing whole, intact hams, roast
beefs, turkey rolls, etc. These large pieces of cooked meat are commonly
referred to as “logs” or “chubs.” Each chub tends to weigh
between 10 and 20 pounds. A random number table was used to choose five
35-gram core samples from each intact chub. The core samples were then
tested for presence and level of L. monocytogenes.

Specific instructions were provided for sample collectors, including the
product category, the number of samples of each type of product to be
obtained, size of the sample to be purchased, and how to choose,
collect, hold and transport the sample. Sample collection was
standardized to maintain consistency. 

Sampling and laboratory analyses followed standard laboratory practices.
This included temperature monitoring during shipment, chain of custody
documentation, aseptic transfer and handling within the laboratory, and
initiating analyses within 24 hours of receipt of sample. The
laboratories were instructed to discard any sample with package damage
such that the microbiological integrity of the sample was not
compromised. Samples not meeting quality control requirements were noted
and discarded. The FSIS standard laboratory method for L. monocytogenes
detection was implemented by the laboratories for use in this study.
Presence/absence for L. monocytogenes was determined by inoculation in
UVM broth followed by Fraser broth then modified Oxford (MOX) agar.
Positive samples were quantified using a FSIS protocol 9-tube Most
Probable Number (MPN) method with a reported detection limit of 0.3
MPN/gram.

NAFSS research laboratories, as approved by FSIS, were experienced in
detecting L. monocytogenes in food. Samples were assigned codes and the
following product information recorded: sampling location (FoodNet site
along with producer information, retailer’s name, and location of
purchase), date of receipt at the laboratory, whether the sample
appeared to be packaged in-store or prepackaged, and the use-by or
sell-by date. Any store information or identifiers were removed prior to
transfer to FSIS.

Statistical Analyses

Statistical analyses were performed using Number Cruncher Statistical
Systems (NCSS) 2001 (Hintz, 2001) and R version 2.6.1 (R Development
Core Team, 2007). For statistical tests, p values less then 0.05 were
considered statistically significant, and p values between 0.05 and 0.10
were considered marginally significant.

Data were analyzed in a variety of ways. The prevalence of L.
monocytogenes among product samples sliced at retail and those that were
prepackaged were analyzed by sampling site, product type, store type,
time of day (morning or afternoon), and quarter of the year using tests
of proportions. The null hypothesis for this test is that all the
prevalences are equal. The alternative hypothesis is that at least one
prevalence differs from some other. This type of statistical test
assumes independence among the samples, an assumption that is not likely
met for these data. Because multiple samples were collected at the same
store, multiple positive L. monocytogenes findings are likely to be
correlated because of cross-contamination and poor hygienic conditions
at the store. Statistical tests with correlated positive samples would,
on average, claim to find statistically significant results more
commonly than intended. 

Tests of proportions were also conducted at the retail store level. A
store was considered positive for retail-sliced or prepackaged if any of
the samples for that category were found positive for L. monocytogenes.
Stores are much more likely to be independent, but serious problems
arise from this approach as well. Store identifiers (even arbitrary
labels) were removed from data provided prior to submittal to FSIS as
part of the data encoding and blinding process. Store visits were
therefore estimated based on date and time of sampling collection. A
second problem was that sample collection times were not provided for
samples from Minnesota, thus the number of stores available was much
smaller than the number of samples. Statistical tests based on only a
few hundred samples lack sufficient statistical power and are unlikely
to detect small differences in prevalence rates at reasonable levels of
confidence. Finally, this approach does not directly incorporate the
number of samples collected at each store.

The final approach used was a logistic regression that predicts the
store prevalence for retail-sliced and prepackaged product as a function
of indicator variables: where the product was sliced, the store type,
and the time of day the sample was collected. Because it is based on
store prevalence, this approach is not subject to the correlation
problem. The regression was weighted by the number of samples taken at
the store, and evaluated more than one explanatory variable
simultaneously. 

Study Results

Prevalence and Number of Samples

Fifty-seven samples were found to be positive for L. monocytogenes
resulting in an overall prevalence rate of 0.76%. Two of these positives
were found in chub samples, six were found in prepackaged samples, and
the remaining 49 positives were found in retail-sliced samples. The
number of prepackaged and retail samples across the four FoodNet sites
is shown in Table 9. 

Table   SEQ Table \* ARABIC  9 . Prevalence of product samples1 and
stores visited based on sampling locations. 

Category	Sampling Locations (Site)

	CA	GA	MN	TN

Number of product samples2	0.74%

(10/1360)	0.60%

(12/2000)	0.95%

(16/1685)	0.85%

(17/1995)

Estimated number of stores sampled3	6.98%

(6/86)	4.93%

(7/142)	n/a3	10.23%

(9/88)

1  Product samples from each store include those sliced and packaged at
retail and those sliced and packaged by the manufacturer.

2  Chub data are not included. Number of positive samples and total
number of samples are given in parentheses.

3   Store visit estimated based on similar sampling date and time. No
sample times were provided for MN, so estimate of stores sampled was not
available.

Slightly fewer product samples were taken in CA than other sites. More
stores were sampled in GA than other sites. In addition to prepackaged
and retail-sliced product samples, 105 and 300 additional chub samples
were collected in MN and TN respectively. Assuming independence, a test
of proportions indicated no statistically significant difference for the
prevalence within product samples among the four sites (p = 0.75).
Neither was there any statistical difference for the store prevalence
across the sites (p = 0.31). This allowed for pooling of the data for
purposes of discussing total prevalence.

The number and prevalence for retail-sliced and prepackaged samples by
quarter of the year is shown in Table 10. More product samples and more
stores were visited in the 3rd quarter than in other quarters. Assuming
independence, a test of proportions indicated a statistically
significant difference for the prevalence within product samples (p =
0.01) but not store prevalence (p = 0.31).

Table   SEQ Table \* ARABIC  10 . Prevalence of product samples
(retail-sliced, prepackaged) and stores visited based on quarter of
year. 

Category	Quarter of Year

	1st 	2nd 	3rd 	4th 

Number of product samples	0.16%

(2/1275)	0.74%

(13/1746)	1.15%

(28/2430)	0.76%

(12/1589)

Estimated number of stores sampled1	2.63%

(2/76)	7.37%

(7/95)	5.34%

(7/131)	10.00%

(6/60)

1Store visit estimated based on similar sampling date and time. No
sample times were provided for MN, so product samples include MN but
stores sampled do not. Chub data are not included.

The number and prevalence for retail-sliced and prepackaged samples by
time of day is shown in Table 11. Slightly more product samples and
stores were sampled in the afternoon. Assuming independence, a test of
proportions indicated a statistically significant difference for the
prevalence within product samples (p = 0.04) but not store prevalence (p
= 0.75).

Table   SEQ Table \* ARABIC  11 . Prevalence of product samples
(retail-sliced, prepackaged) and stores visited based on time of day (AM
versus PM). 

Category	Time of Day

	AM	PM

Number of product samples1	0.51%

(13/2540)	1.04%

(32/3060)

Estimated number of stores sampled1	5.42%

(9/166)	6.81%

(13/191)

1Store visit estimated based on similar sampling date and time. No
sample times were provided for MN, so neither product samples nor stores
sampled include MN. Chub data are not included.

The more interesting time of day analysis looked solely at retail-sliced
product as shown in Table 12. Retail-sliced product samples collected in
the afternoon were more than twice as likely to test positive for L.
monocytogenes – 1.92% versus 0.92%. Assuming independence, this
difference was statistically significant (p = 0.04). While the store
prevalences were also higher in the afternoon (7.83% versus 5.80%), the
differences were not statistically significant (p = 0.64).

Table   SEQ Table \* ARABIC  12 . Prevalence of only retail-sliced
product and stores visited based on time of day (AM versus PM).

Category	Time of Day

	AM	PM

Number of product samples1	0.92%

(12/1307)	1.92%

(31/1612)

Estimated number of stores sampled1	5.80%

(8/138)	7.83%

(13/166)

1Store visit estimated based on similar sampling date and time. No
sample times were provided for MN, so neither product samples nor stores
sampled include MN.

The number and prevalence for retail-sliced and prepackaged samples is
shown in Table 13. As designed, more product samples were collected at
major grocery chains. Assuming independence, a test of proportions found
a marginal statistically significant difference for the prevalence
within product samples (p = 0.07) but not store prevalence (p = 0.82).

Table   SEQ Table \* ARABIC  13 . Prevalence of product samples
(retail-sliced, prepackaged, and chubs) and stores visited based on
store type (major grocery chain versus other grocers).

Category	Store Type2

	A	B

Number of product samples1	0.64%

(31/4801)	1.10%

(24/2186)

Estimated number of stores sampled1	5.58%

(11/197)	6.71%

(11/164)

1Store visit estimated based on similar sampling date and time. No
sample times were provided for MN, so product samples include MN but
stores sampled do not.

2A represents major grocery chains. B represents other grocers.

Product samples were collected from prepackaged product, from product
sliced at retail delis, and a limited number from intact chubs collected
at retail. The number of RTE product samples by location of slicing is
shown in Figure 8. A total of 3,518 retail-sliced samples, 3,522
prepackaged samples, and 405 chub samples were collected. A given chub
may have been sampled multiple times making the number of unique chubs
uncertain. 

Figure   SEQ Figure \* ARABIC  8 . Number of RTE samples by location of
slicing (source).

The data also indicate that deli meat sliced at retail is more likely to
be contaminated than prepackaged deli meat (1.39% versus 0.17%). The
results are shown in Figure 9. Assuming independence, a test of
proportions between retail and prepackaged prevalence indicated
retail-sliced deli meat had a statistically significant higher
prevalence (p < 0.0001). 

Figure   SEQ Figure \* ARABIC  9 . Prevalence of L. monocytogenes in
deli meat by location of slicing.

The site and slicing location results for sliced deli meat only are
shown in Table 14. Chub results are not included. The striking
difference in prevalence between retail-sliced versus prepackaged is
evident at all sites. Differences among the sites are relatively minor.

Table   SEQ Table \* ARABIC  14 . Prevalence of L. monocytogenes in
retail-sliced and prepackaged deli meat by site. The number of positive
samples and the total number of samples are shown in parentheses. 

	Site

	CA	GA	MN	TN	Overall

         Processing	

Retail-sliced	1.3%

(12/929)	1.4%

(10/731)	1.4%

(12/841)	1.5%

(15/1017)	1.4%

(49/3518)

	

Prepackaged	0.0%

(0/1071)	0.0%

(0/629)	0.5%

(4/844)	0.2%

(2/978)	0.2%

(6/3522)

	

Overall	0.6%

(12/2000)	0.7%

(10/1360)	0.9%

(16/1685)	0.9%

(17/1995)	0.8%

(55/7040)

Note:  Chub data are not included.

For the 362 stores identified across the three sites available (CA, GA,
TN) retail-sliced deli meat was sampled at 308 stores and prepackaged
deli meat was sampled at 313 stores. For most stores, both types of deli
meat was collected – 259 of these stores had both retail-sliced and
prepackaged samples collected, 49 had only retail-sliced samples
collected, and 54 had only prepackaged sliced samples collected. The
testing results showed that only one store had positives samples for
both retail-sliced and prepackaged deli meat. An additional 20 of the
stores had positive retailed-sliced samples, and one store had positive
prepackaged deli meat only.

Histograms of the number of retail-sliced and prepackaged deli meat
samples taken at each store are shown in Figure 10. For retail-sliced
deli meat, the number of deli meat samples per store ranged from 1 to
30, with a median of 8. The 25th and 75th% quantiles were 6 and 10
respectively. For prepackaged deli meat, the number of deli meat samples
ranged from 1 to 24, with a median of 9. The 25th and 75th% quantiles
were 6 and 11, respectively.

Figure   SEQ Figure \* ARABIC  10 . Number of deli meat samples
collected per store. MN data are not included because stores could not
be identified.

Some differences existed among the different sites for labeling types of
deli meats. After correcting for obvious misspellings and accounting for
multiple orderings, the types of deli meats listed in the data were:
beef, beef/chicken/pork, beef/chicken/turkey, beef/pork,
beef/pork/turkey, bologna, chicken, chicken/pork, chicken/turkey/pork,
ham, mixed,  pork, pork/turkey, poultry, poultry (chicken), poultry
(chicken/pork), poultry (chicken/pork/beef), poultry (turkey), poultry
(turkey/pork), and roast beef.

 

Many categories of deli meat types had very few samples. For purposes of
this analysis, these categories were combined into 5: beef, bologna,
pork, poultry, or mixed. Deli meat labeled as “bologna” was
classified into different product types. If labeled by the sampler as
“beef bologna,” it was categorized as beef. If labeled with mixed
components, it was categorized as mixed. If labeled simply as bologna,
it was categorized as bologna. Deli meat listed as poultry but
containing mixed components was categorized as mixed. For example, the
samples labeled “poultry (chicken/pork)” were categorized as mixed.
Based on this categorization, the counts by product type are given in
Figure 11.

Figure   SEQ Figure \* ARABIC  11 . Number of RTE samples by deli meat
type. Chub data are not included. One sample (not shown) did not include
any listing for deli meat type.

The prevalence of L. monocytogenes across the different deli meat types
is shown in Figure 12. Although it appears that beef has a slightly
higher prevalence, the differences were not statistically significant
based on a test of proportions (p = 0.22) among the five different deli
meat types (beef, bologna, mixed, pork, poultry). The corresponding L.
monocytogenes prevalence for beef, bologna, mixed meat, pork, poultry
deli meats were 1.28%, 0.31%, 0.44%, 0.87%, and 0.65%, respectively.
There does not appear to be any difference in the prevalence of L.
monocytogenes based on whether the deli meat was cured or uncured. A
similar test was conducted for retail-sliced only deli meat samples with
similar results. Overall, there was no statistically significant
difference in the prevalence of L. monocytogenes among the different
deli meat types (p = 0.43)

Figure   SEQ Figure \* ARABIC  12 . Prevalence of L. monocytogenes in
RTE deli meats by deli meat type. Chubs were not included.

Samplers were asked to identify if the sample included an antimicrobial
formulation. Of the 7,446 samples, 51 were identified as using an
antimicrobial agent, 1,008 did not use an antimicrobial agent, and 6,387
were blank. Antimicrobial agents listed included potassium lactate,
sodium diacetate, sodium erythorbate, calcium lactate, sodium phosphate,
sodium benzoate, ascorbic acid, sodium citrate, and citric acid. Of the
57 samples positive for L. monocytogenes, 6 listed sodium erythorbate
use, 1 listed sodium lactate/sodium diacetate use, and 50 were blank. 

There is an indication that positive retail-sliced samples were
clustered by store when positive L. monocytogenes results were found.
Figure 13 illustrates the deli meat sample prevalence for retail-sliced
deli meat among the 21 stores with at least one positive result. Three
of these stores had 50% or greater prevalence, and six of these stores
had greater than 30% prevalence. Of the 308 identified stores sampled
for retail-sliced deli meat, 37 L. monocytogenes positive deli meat
samples were found among 22 stores. The remaining positive samples were
from MN, where individual stores could not be identified. Six of these
stores accounted for 21 of the 37 positive samples found. Thus, it
appears that a few retail stores accounted for most of the positive deli
meat samples found. This finding is indicative of cross contamination at
the retail establishment. It is also the reason that the independence
assumption of the test of proportions for deli meat samples is likely
not completely valid.

Figure   SEQ Figure \* ARABIC  13 . Prevalence of L. monocytogenes in
RTE deli meats samples sliced at retail. The estimated store visit was
based on similar sampling date and time. No sample times were provided
for MN; thus, MN data not included. Thirty-seven total deli meat samples
are shown.

Logistic Regression

To overcome the limitations with the test of proportions used above
(non-independence for deli meat samples and small sample size for store
samples), a logistic regression was performed. Logistic regression is
appropriate when the dependent variable represents a proportion of
positive results such as the deli meat prevalence for retail-sliced deli
meat at an individual store. The assumptions for standard linear
regression are not valid not here: the dependent variable is bounded to
fall between 0 and 1, the errors are not normally distributed, and the
regression must be weighted by the sample size used to calculate the
prevalence. Logistic regression transforms the prevalence to a scale
more suitable for regression. The analysis was performed in R using the
generalized linear model (glm). In the language of R, a binomial family
was specified which used the logit transformation as the link function.

The prevalence of retail-sliced and prepackaged deli meat was calculated
separately for each store. This prevalence was regressed against several
indicator variables: processing type (retail-sliced versus prepackaged),
time of day, and store type. Retail-sliced and prepackaged prevalences
from the same store were treated as independent. Given that only one
store had both processing types found positive, this seemed a reasonable
approach. The number of samples of each type was used to weight the
regression. (Thus, store prevalences with only one sample received less
weight than store samples with 30 samples.) The logistic regression
approach also had the advantage that all three explanatory variables
were included simultaneously.

The regression function was

 

where: logit() = the logit transformation function; prevalence = the
deli meat sample prevalence for each store and processing type
(retail-sliced versus prepackaged); processing type = 0/1 indicator
variable with 0 for prepackaged and 1 for retail-sliced; store type =
0/1 indicator variable with 0 for type A stores (major grocery chains)
and 1 for type B stores (other grocery stores); and time of day = 0/1
indicator variable with 0 for AM and 1 for PM. 

The number of data points used in the regression was 613. This is less
than twice the number of individual stores sampled (2*362=724) because
not all stores had both retail-sliced and prepackaged samples collected.

 

The results for the parameter estimates are given in Table 15. The
variables processing type and store type are statistically significant.
The time of day the sample was collected is marginally significant.

Table   SEQ Table \* ARABIC  15 . Results of logistic regression for
store prevalence as function of processing type, store type, and time of
day indicator variables. Data for MN not included. N=613.

Parameter	Estimate	Standard Error	Z value	p

Intercept	-7.96	0.76	-10.39	<0.0001

Processing type	2.90	0.73	4.00	<0.0001

Store type	0.99	0.33	3.03	0.002

Time of day	0.59	0.35	1.68	0.093



As expected from examining the data, whether the sample was prepackaged
versus retail-sliced was strongly statistically significant. This is
consistent with the test of proportions for deli meat samples. The
result for time of day is consistent with the deli meat sample test of
proportions for time of day. Both results indicate marginal statistical
significance.

Figure 14 illustrates the results using a logistic regressions based on
one explanatory variable at a time as the explanatory variable. Because
the vast majority of points had 0 prevalence and only two values (0/1)
were used for the explanatory variables, a small random number was added
to the (x,y) coordinate for each point in order to better illustrate the
density of points at 0 prevalences. 

Figure   SEQ Figure \* ARABIC  14 . Graphical display of logistic
regression results using deli meat sample prevalence at individuals
stores as the dependent variable. MN data not included.

Comparison of Findings of the National Alliance for Food Safety and
Security with those of the Food Processors’ Association

A comparison of NAFSS retail contamination findings with those of the
National Food Processors Association (now Food Products Association) 
(Gombas et al. 2003) is enlightening, although keep in mind that sample
collection methods, sample sizes and analyses methods differed and these
can all affect the results. The total number of deli meat samples was
roughly equivalent:  Gombas et al. sampled approximately 9,000 deli meat
samples compared to about 7,000 (excluding chubs) for this research. The
split between retail-sliced and prepackaged was somewhat different
however. Approximately 77% of the samples from Gombas et al. were
prepackaged, versus approximately 50% for this work. USDA/FSIS data
suggest that approximately 47% of RTE deli meat is sliced at the
processing plant and prepackaged.

Gombas et al. found retail-sliced and prepackaged prevalences of 2.7%
and 0.4% respectively. This research found prevalences lower by about a
factor of 2: 1.4% and 0.2% respectively. This may indicate improvements
in deli meat handling, increased use of post-processing lethality and
antimicrobial growth inhibitor, or other improvements at the processing
plant or retail between when the studies were conducted. 

The earlier research found a difference in prevalence between their two
sampled sites.   REF _Ref197160220  \* MERGEFORMAT  Table 16  below
shows the derived results. Compare these data to the corresponding Table
6 above for the more recent data. Whereas this work found a consistent
prevalence across all sites and a significant difference between
retail-sliced versus prepackaged, the earlier work found no difference
in processing type at one site and a statistically significant
difference at another. 

Table   SEQ Table \* ARABIC  16 . Prevalence of L. monocytogenes in
sliced deli meat by site and processing type from the Food Products
Association (Gombas et al. 2003). 

	Site

	CA	MD	Overall

         Processing1	Retail-sliced	0.70%	4.2%	2.7%

	Prepackaged	0.55%	0.19%	0.4%

	Overall	0.6%

(28/4600)	1.2%

(54/4599)	0.9%

(82/9199)

The number of positive samples and the total number of samples are shown
in parentheses where available.

Gombas et al. also found that the prevalence was higher for
retail-sliced deli meat, but that the levels for positives were actually
higher for prepackaged deli meat. This current work found consistently
that both the prevalence and levels were higher for retail-sliced deli
meat compared to prepackaged. 

Conclusions

  REF _Ref197160152 \h  \* MERGEFORMAT  Table 17  summarizes the results
of all the statistical testing. RTE deli meat is more contaminated with
L. monocytogenes, both in terms of prevalence and level, when sliced at
retail than when prepackaged. The marginal statistical link between
positive results and time of day as well as the clustering according to
the store where the sample was collected is an indication that cross
contamination within retail establishments is occurring. There was no
significant difference in prevalence of L. monocytogenes among the
various four FoodNet sites.

Table   SEQ Table \* ARABIC  17 . Overall results of statistical tests
for prevalence of L. monocytogenes on RTE meat and poultry deli meats by
location, season, time of day for slicing at retail, and by deli meat
type.

Variable	Statistical Test1

	Deli meat samples2	Stores3	Logistic regression4

Geographic location	N

(p=0.75)	N

(p=0.31)

	Quarter of year	Y

(p=0.01)	N

(p=0.31)

	Time of Day	Y

(p=0.04)	N

(p=0.75)	M

(p=0.093)

Time of day (retail-sliced only)	Y

(p=0.04)	N

(p=0.64)

	Store Type	M

(p=0.07)	N

(p=0.82)	Y

(p=0.002)

Prepackaged versus retail-sliced	Y

(p<0.0001)

Y

(p<0.0001)

Deli meat Type

	N

(p=0.22)



Deli meat Type (retail-sliced only)	N

(p=0.43)



1 Chub data were not included in any of the analyses. Statistical test
results were considered statistically significant if ( < 0.05 and
marginal if 0.05 ( ( ( 0.10. A “Y” indicates the differences were
statistically significant; an N” indicates that they were not; an
“M” indicates that the differences were marginally significant. The
exact p values for the test result are given in parentheses below.

2 Deli meat samples were assumed independent for the purposes of the
test of proportions. In practice, because multiple samples were
collected from the same store, samples were not independent. Thus, the
test of proportions is more likely to erroneously claim a statistically
significant result than the choice of ( would indicate.

3 A store was considered positive if at least one of the deli meat
samples collected at the store was positive for L. monocytogenes.

4 All three explanatory variables were included simultaneously.

Appendix II: Sensitivity Analysis

This appendix reports the results of conducting sensitivity analyses on
two major model inputs: the time-temperature storage by consumers for
retail-sliced versus prepackaged deli meat and the interpretation of
“shelf life” for regulatory purposes under the Interim Final Rule.
The shelf life time affects the growth rates of Listeria between deli
meats with and without growth inhibitors. All analyses were conducted on
calibrated mode for the dose-response modeling for 390 deaths across all
food groups.

STORAGE TIME-TEMPERATURE SENSITIVITY

One of the identified concerns for the comparative risk assessment was
the assumption that consumers treat plant-sliced and retail sliced
product similarly in the home. This assumption has been analyzed two
ways. The first simply assumes that the storage time for retail-sliced
product was some fraction of the storage time distribution used in the
FDA 2003 model.  

Consumer storage times used in the exposure assessment model were taken
from a consumer survey conducted by the American Meat Institute (AMI)
(2001). Results of the survey suggest that approximately 40% of
ready-to-eat product is stored for less than 3 days, and another 45% of
product is stored from 4 to 7 days. A total of 96% of product is stored
for less than 14 days. The previous analysis used this same storage time
distribution for both retail-sliced and prepackaged product. Consumers
may store retail-sliced deli meats for shorter periods than prepackaged
deli meats. Thus, to assess the effect of a reduced consumer storage
time, the storage time distribution in the retail exposure model was
adjusted by arbitrary factors of 0.25, 0.50, and 0.75. The results in
terms of the number of deaths and illnesses are shown in Table 18. The
ratio of deaths caused by retail-sliced versus prepackaged product is
shown in Figure 15. The comparative risk ratio decreased as the consumer
storage times for the retail-sliced meats decreased; however,
retail-sliced product is estimated to cause 1.7 times more deaths than
prepackaged product even when stored for a quarter of the time. All else
being equal, if consumers store retail sliced deli meat for only 25% of
the time that they store prepackaged deli meat, retail sliced product
still causes a greater number of deaths than prepackaged product.

Table   SEQ Table \* ARABIC  18 . Estimated mean number of deaths and
illnesses per annum by fraction of consumer storage time.

Storage Time Fraction	25%	50%	75%	100%

Deaths	70.7	105.5	127.1	139.3

Illnesses	397.8	589.9	708.0	774.7

Ratio of Deaths, Retail-sliced: Prepackaged	1.7	3.7	5.4	9.1



 

Figure   SEQ Figure \* ARABIC  15 . Relative ratio of deaths differing
storage times between retail-sliced and prepackaged product.

The second time/temperature analysis was based on a national survey of
U.S. adults using a Web-enabled panel survey approach. The survey was
conducted by RTI International, Tennessee State University, and Kansas
State University. The purpose of the survey was to characterize home
storage and refrigeration practices for a variety of refrigerated
ready-to-eat (RTE) foods and consumers' knowledge and use of open date
statements among pregnant women, seniors, and the remaining population.
A description of the survey and an analysis of the data are given by
Cates et al. (2006). The study design, the survey questionnaire, the
data dictionary and row data (Microsoft Excel format) are available at  
HYPERLINK "http://www.foodrisk.org/"  http://www.foodrisk.org/ . Note
that the survey asked consumers how long the packaged was stored until
the product was consumed. The reported storage times represent the time
for the last serving, but some product would normally be consumed prior
to this. Because the same question was used for both prepackaged and
retail sliced product, and because longer storage times represent the
greater risk, the reported storage times for the last serving were used
to compare the two product types.

For storage time, this analysis found a statistically significant
difference between retail-sliced versus prepackaged product. Both
storage time distributions could be fit by Weibull distributions as
shown in Table 19.  

Table   SEQ Table \* ARABIC  19 . Fitted Weibull distributions according
to the deli meat category

Deli Meat Category	N	Shape	Scale

Retail-sliced	443	1.830	7.777

Prepackaged	387	1.137	             18.390



These different distributions are shown graphically in Figure 16. Note
the long tail for the prepackaged product storage time.

Figure   SEQ Figure \* ARABIC  16 . Relative frequency of storage time
for retail-sliced versus prepackaged product.

These distributions were used to calculate the storage time for the
cumulative probabilities used in the FDA-FSIS model. The results, along
with the existing FDA-FSIS times, are shown in Table 20 and Figure 17.

Table   SEQ Table \* ARABIC  20 . Storage times for retail-sliced and
prepackaged product.

Cumulative Probability	Consumer Storage Time (d)

	FDA-FSIS	Retail-Sliced	Prepackaged

0	0.0	0.00	0.00

0.39	2.0	5.29	9.89

0.84	5.5	10.83	31.33

0.91	9.0	12.57	39.83

0.96	12.5	14.73	51.42

0.97	18.0	15.44	55.44

0.99	26.0	17.92	70.46

0.999	45.0	22.36	100.64



 

Figure   SEQ Figure \* ARABIC  17 . Storage time for retail-sliced and
prepackaged product.

Two points stand out from this analysis. The first is that storage time
for retail-sliced product is significantly shorter than for prepackaged
product. The median storage time for retail-sliced product is 6.4 days,
while for prepackaged product is 13.3 days. The second point is that
both storage distributions are longer than the distribution used in the
original FDA-FSIS model.  

The same survey data were used to analyze storage temperatures. These
data were fit by a logistic distribution with location parameter of
40.15 and scale parameter of 3.193. The sample size was 2,037. The
FDA-FSIS model uses 939 temperature measurements as inputs, not a
probability distribution. The temperature for a given run is then
sampled from these values. To run the model with the new temperature
distribution, R was used to generate 939 random numbers from the fitted
logistic distribution. The relative frequencies are shown in Figure 18.
The newer logistic distribution is slightly less peaked and has a longer
tail toward higher temperatures. Storage temperatures were assumed the
same regardless of whether the product was retail-sliced or prepackaged.

Figure   SEQ Figure \* ARABIC  18 . Relative frequencies for storage
temperature.

The estimated mean number of deaths per year associated with prepackaged
product was 34.1, and the estimated mean number of deaths per year
associated with retail-sliced product was 166.9, with an estimated total
annual number of deaths equal to 201.0 (Table 21). All of these values
are higher than the corresponding numbers for the original analysis
because the storage times were longer. There were 139.3 deaths estimated
for the original analysis. Seventeen percent of the estimated per annum
deaths (34.1/201.0 = 16.96%) are attributable to prepackaged product,
while the remaining 83% are attributable to retail-sliced product
(166.9/201.0 = 83.03%). The relative risk on a per annum basis for deli
meats sliced at retail versus sliced in plants is thus 166.9/34.1 =
4.89. Corresponding results for estimated illnesses are in Table 22. 

Table   SEQ Table \* ARABIC  21 . Estimated mean number of deaths per
year from L. monocytogenes in deli meat among three populations
stratified by age and four deli meat categories using the alternative
storage time-temperature data.

Deli Meat Category	Elderly

(95% CI)	Intermediate Age

(95% CI)	Perinatal

(95% CI)	All Age Groups (95% CI)

Prepackaged with growth inhibitor	8.1

(7.9, 8.3)	1.9

(1.9, 2.0)	0.5

(0.5, 0.5)	10.5

(10.3, 10.8)

Prepackaged without growth inhibitor 	18.1

(17.7, 18.6)	4.4

(4.3, 4.5)	1.1

(1.1, 1.1)	23.6

(23.0, 24.2)

Retail-sliced with growth inhibitor	20.4

(19.9, 20.9)	4.9

(4.7, 5.0)	1.3

(1.2, 1.3)	26.5

(25.9, 27.2)

Retail-sliced without growth inhibitor	108.2

(106.4, 109.9)	25.4

(25.0, 25.8)	6.7

(6.7, 6.8)	140.3

(138.1, 142.6)

Subtotal: Prepackaged	26.2

(25.7, 26.8)	6.3

(6.2, 6.4)	1.6

(1.6, 1.6)	34.1

(33.4, 34.9)

Subtotal: Retail-sliced	128.6

(126.7, 130.5)	30.3

(29.9, 30.7)	8.0

(7.9, 8.1)	166.9

(164.5, 169.3)

Subtotal: With growth inhibitor	28.5

(27.9, 29.1)	6.8

(6.6, 6.9)	1.8

(1.7, 1.8)	37.1

(36.3, 37.8)

Subtotal: Without growth inhibitor	126.3

(124.4, 128.1)	29.8

(29.4, 30.3)	7.8

(7.7, 7.9)	163.9

(161.6, 166.3)

Total	154.8

(152.7, 156.9)	36.6

(36.2, 37.1)	9.6

(9.5, 9.7)	201.0

(198.4, 203.6)



Table   SEQ Table \* ARABIC  22 . Estimated mean number of illnesses
per year from L. monocytogenes in deli meat among three populations
stratified by age and four deli meat categories.

Deli Meat Category	Elderly

(95% CI)	Intermediate Age

(95% CI)	Perinatal

(95% CI)	All Age Groups (95% CI)

Prepackaged with growth inhibitor	30.0

(29.2, 30.8)	21.7

(21.1, 22.3)	6.5

(6.4, 6.6)	58.2

(56.7, 59.7)

Prepackaged without growth inhibitor 	67.1

(65.4, 68.7)	49.6

(48.4, 50.9)	13.7

(13.4, 13.9)	130.4

(127.3, 133.5)

Retail-sliced with growth inhibitor	75.5

(73.7, 77.3)	54.9

(53.6, 56.3)	16.0

(15.7, 16.2)	146.4

(143.0, 149.8)

Retail-sliced without growth inhibitor	400.2

(393.6, 406.8)	287.5

(282.8, 292.1)	85.6

(84.5, 86.6)	773.2

(761.1, 785.4)

Subtotal: Prepackaged	97.1

(95.1, 99.1)	71.3

(69.8, 72.9)	20.2

(19.8, 20.5)	188.6

(184.7, 192.4)

Subtotal: Retail-sliced	475.7

(468.7, 482.7)	342.4

(337.6, 347.2)	101.5

(100.4, 102.6)	919.6

(906.8, 932.4)

Subtotal: With growth inhibitor	105.5

(103.4, 107.7)	76.6

(75.0, 78.2)	22.5

(22.1, 22.8)	204.6

(200.5, 208.6)

Subtotal: Without growth inhibitor	467.3

(460.3, 474.2)	337.1

(332.3, 341.9)	99.2

(98.1, 100.3)	903.6

(890.9, 916.4)

Total	572.8

(565.1, 580.5)	413.7

(408.5, 418.9)	121.7

(120.5, 122.9)	1108.2

(1094.4, 1122.1)

SHELF LIFE SENSITIVITY

The second sensitivity analysis was conducted on the definition of shelf
life under the Interim Final rule. This allows RTE producers to qualify
for one of the categories as using growth inhibitors, and is used within
the risk assessment to define the growth rates between product with and
without growth inhibitor.

Maintaining the FDA-FSIS Listeria model’s assumed consumer storage
distribution, the exponential growth rate (EGR) of Listeria on deli meat
was then changed for both retail-sliced and prepackaged product based on
a shelf life of 10, 14, and 21 days. The resulting EGRs are shown in
Table 23. Increasing the shelf life decreased the number of deaths
(Table 24). Recall that a maximum two-log growth is allowed during the
shelf life to qualify to growth inhibitor classification. Thus, a longer
shelf life requires a lower growth rate for product with growth
inhibitor. The change in shelf life from 10 days to 14 days resulted in
a 10% reduction in the mean number of deaths. A weeklong extension of
shelf life from 14 days to 21 days resulted in a 5% reduction in the
number of deaths. This suggests that the assumption of a 14-day shelf
life may be adequate for predicting the number of deaths or illnesses
due to Listeria. 

Table   SEQ Table \* ARABIC  23 . EGR for product with and without
growth inhibitor by shelf life.

Shelf Life	10 day	14 day	21 day

With growth inhibitor	0.20	0.14	0.10

Without growth inhibitor	0.30	0.31	0.32



Table   SEQ Table \* ARABIC  24 . Mean number of deaths and illnesses
per annum by shelf life.

Shelf Life	10 day	14 day	21 day

Deaths	155.1	139.3	131.7

Illnesses	861.1	774.7	732.9

Ratio of Deaths, Retail-sliced: Prepackaged	8.1	9.1	7.9



The comparative risk ratios (Table 23 and Figure 19) exhibited no
definitive correlation with the change in shelf life; however, the EGR
for product with growth inhibitor consistently decreased, while the EGR
for product without growth inhibitor increased as the shelf life
increased (Table 23). The differences in the comparative risk may be a
result of the iterative process used to adjust the dose-response curve
and may not necessarily indicate a true difference in the relative risk.


 

Figure   SEQ Figure \* ARABIC  19 . Relative ratio of deaths between
retail-sliced versus prepackaged product for differing definitions of
shelf life.

RELATIVE IMPACTS OF MODEL VARIABLES

Using the original comparative risk assessment model results,
statistical analyses were conducted to elucidate the relative importance
of model inputs. A recursive partitioning and regression tree was
generated in R to determine which factor (age, slicing location, or
growth inhibitor use) had the greatest effect on the number of resulting
deaths (Figure 20). The first division in the tree indicates that age is
the most important factor and that the elderly are more likely to die
from listeriosis than either the neonatal or the intermediate
population. Following the tree along the elderly branch, the next
division is by slicing location. The tree indicates that retail-sliced
product is at greater risk for causing listeriosis than prepackaged
product. Finally, the retail-sliced product is divided according the
growth inhibitor use.

 

Figure   SEQ Figure \* ARABIC  20 . Recursive partitioning and
regression tree.

Using the data from all 4,000 simulations, box plots were generated for
each deli meat category by age group (Figure 21). The box plots
reemphasize the effect of age on the risk of death from listeriosis,
with the elderly population having the highest number of deaths for each
of the deli meat categories. Within each age group, growth inhibitor
reduced the number of deaths; however, the box plots show that even with
the use of growth inhibitor, retail-sliced deli meats result in a
greater risk of death due to listeriosis than prepackaged meats. 

 

Figure   SEQ Figure \* ARABIC  21 . Box plots for each deli meat
category by age group. Prepack = prepackaged, Retail = retail-sliced, W
= with growth inhibitor, WO = without growth inhibitor.

As seen in the box plots, each of the four deli meat categories follows
a similar trend, with the elderly age group at the highest risk for
death. An interaction plot for the elderly age group was created to
compare the effect of growth inhibitor use and product slicing location
on the mean number of deaths. There is a significant difference between
the mean number of deaths resulting from retail-sliced product when
compared to prepackaged product (Figure 22a). While the use of growth
inhibitor greatly decreased the mean number of deaths resulting from
retail sliced product, prepackaged product without growth inhibitor
results in fewer deaths than retail sliced product with growth inhibitor
(Figure 22b). 

 

Figure   SEQ Figure \* ARABIC  22 . Interaction plots comparing the
effect of growth inhibitor (GI) use and slicing location on the mean
number of deaths from listeriosis.

References

American Meat Institute. 2001. Consumer Handling of RTE meats.
(Unpublished data submitted to Docket No. 99N-1168 as part of the
FDA-FSIS risk assessment.)

Cates, S.C., R.A. Morales, S.A. Karns, L.A. Jaykus, K.M. Kosa, T.
Teneyck, C.M. Moore, P. Cowen. 2006. Consumer knowledge, storing, and
handling practices regarding Listeria in frankfurters and deli meats:
results of a Web-based survey. Journal of Food Protection.
69(7):1630-39.

CDC. 2007. Preliminary FoodNet Data on the Incidence of Infection with
Pathogens Transmitted Commonly Through Food --- 10 States, 2006.
Morbidity and Mortality Weekly Report 56(14):336-339. 

Draughon, A.F. 2006. A collaborative analysis/risk assessment of
Listeria monocytogenes in ready-to-eat processed meat and poultry
collected in four FoodNet states. Symposium S-16: Contamination of
Ready-to-Eat Foods: Transfer and Risk: Listeria monocytogenes and Other
Microorganisms. International Association for Food Protection 93rd
Annual Meeting, Calgary, Alberta. August 13-16.

FDA-FSIS. 2003. Quantitative assessment of relative risk to public
health from foodborne Listeria monocytogenes among selected categories
of ready-to-eat foods. Washington, DC. Available at
http://www.foodsafety.gov/~dms/lmr2-toc.html.

FSIS. 2003. Risk assessment for Listeria monocytogenes in deli meat.
Washington, DC. Available at: 
http://www.fsis.usda.gov/PDF/Lm_Deli_Risk_Assess_Final_2003.pdf. 

Gombas, D.E., Y. Chen, R.S. Clavero, V.N. Scott. 2003. Survey of
Listeria monocytogenes in ready-to-eat foods. Journal of Food Protection
66:559-69.

Hintz, J. 2001. NCSS and Pass. Number Cruncher Statistical Systems.
Kaysville, UT.

Lianou, A., and J.N. Sofos. 2007. A review of the incidence and
transmission of Listeria monocytogenes in ready-to-eat products in
retail and food service environments. Journal of Food Protection
70:2172-2198.

Mead, P.S., Slutsker, L., Dietz, V., et al. 1999. Food-related illness
and death in the United States. Emerging Infectious Disease 5:607–625.

R Development Core Team. R: A language and environment for statistical
computing. R Foundation for Statistical Computing, Vienna, Austria.
2007. ISBN 3-900051-07-0. Available at: http://www.R-project.org.

van Belle, G. 2002. Statistical rules of thumb. Wiley-Interscience, New
York.

 Risk Assessment Division, Office of Public Health Science, Food Safety
and Inspection Service, U.S. Department of Agriculture, Washington DC.

 Department of Civil & Environmental Engineering, Virginia Tech
University, Blacksburg, VA.

 For our purpose, meat and poultry are considered together when
discussing deli products (i.e., deli meat refers to any product
containing beef, pork and/or poultry).

 Note that the lognormal and lognormal (base 10) are equivalent.

 See, for example, van Belle 2002.

 FSIS’ interim final rule for the control of L. monocytogenes during
processing (“Control of Listeria Monocytogenes in Ready-to-Eat Meat
and Poultry Products,” 68 FR 34208; June 6, 2003 (revised January 1,
2006); 9 CFR 430).

 9 CFR 430 provides requirements for the FSIS’ interim final rule,
“Control of Listeria Monocytogenes in Ready-to-Eat Meat and Poultry
Products,” (68 FR 34208; June 6, 2003 (revised January 1, 2006)).  

 The risk assessment calibration mode used was set to 390 deaths across
all food groups. This number may be lower today given the increased use
of post-processing lethality and antimicrobial growth inhibitors
compared to the when the original FDA-FSIS model was developed.

 Estimated based on industry survey data collected with USDA/FSIS Form
10,240-1, Production Information on Post-Lethality Exposed Ready-to-Eat
Products, gathered in July 2007 in accordance with 9 CFR 430.4(d). See  
REF _Ref197225400  \* MERGEFORMAT  Table 6  above for details.

 Chub data were not included in this test of proportions.

 Because of the large number of blanks, this antimicrobial formulation
data was not used as part of the risk assessment described below.
Instead, USDA data on current industry practices were used to estimate
the fraction of product with antimicrobial usage.

 The statistical term for this is jitter.

 Estimated based on industry survey data collected with USDA/FSIS Form
10,240-1, Production Information on Post-Lethality Exposed Ready-to-Eat
Products, gathered in July 2007 in accordance with 9 CFR 430.4(d). See  
REF _Ref197225400  \* MERGEFORMAT  Table 6  above for details.

DRAFT FSIS Comparative Risk Assessment for Listeria monocytogenes in
Ready-to-eat Meat and Poultry Deli Meats Report

 PAGE   48 

 PAGE   19 

This information is distributed solely for the purpose of
pre-dissemination peer review under applicable information quality
guidelines. It has not been formally disseminated by FSIS. It does not
represent and should not be construed to represent any Agency
determination or policy.

Draft – Do not Cite or Quote

DRAFT FSIS Comparative Risk Assessment for Listeria monocytogenes in
Ready-to-eat Meat and Poultry Deli Meats Report

This information is distributed solely for the purpose of
pre-dissemination peer review under applicable information quality
guidelines. It has not been formally disseminated by FSIS. It does not
represent and should not be construed to represent any Agency
determination or policy.

						 PAGE   20 

		

DRAFT FSIS Comparative Risk Assessment for Listeria monocytogenes in
Ready-to-eat Meat and Poultry Deli Meats Report

 PAGE   52 

„š

^„š

„š

^„š

h("

h("

h("

h("

„@

„Ð^„@

„@

^„@

m

Â

J

”

-



h”

h”

h”

h”

&

;

<

=

>

J

K

L

f

g

h

j

k

l

m

n

o

‹

Œ

Ž

ž

Ÿ

 

º

»

¼

¿

À

h”

 h”

h”

h”

h”

h”

h”

 h”

h”

h”

h”

h”

h”

 h”

h”

h”

h”

(

)

*

D

E

F

G

H

I

J

K

L

h

i

h”

h”

 h”

ᄀ脈栖ඔqࡕ洁H渄H甄Ĉᄔ脈栖ඔq䡭Ѐ䡮Ѐࡵᴁi

j

k

r

s

t

Ž

‘

’

“

”

•

–

²

³

´

µ

Ö

ç

h”

 h”

h”

h”

h”

h”

h”

h”

h”

 h”

h”

h”

h”

h”

h”

h”

j¬

 h”

h”

h”

h”

h”

h”

h”

h”

 h”

h”

h”

h”

h”

h”

 h”

h”

h”

h”

h”

h”

 h”

h”

h”

h”

h”

h”

 h”

h”

h”

 h”

h”

h”

h”

h”

 hqo

hÚL

hn

h:

 h:

h:

h:

h:

 h:

h:

h:

h:

h:

h…

h…

h

Ć

Ć

Ć

$

$

$

$

h

$

$

$

$

$

$

$

$

$

$

$

$

$

$

$

$

$

$

hW

Ȁ沖ԀᣖČ

Č

Ȁ沖ԀᣖČ

Č

Ȁ沖ԀᣖČ

Č

Ȁ沖ԀᣖČ

Č

Ȁ沖ԀᣖČ

Č

Ȁ沖ԀᣖČ

Č

Ȁ沖ԀᣖČ

Č

Ȁ沖ԀᣖČ

Č

Ȁ沖ԀᣖČ

Č

hW

Ȁ沖ԀᣖČ

Č

Ȁ沖ԀᣖČ

Č

Ȁ沖ԀᣖČ

Č

Ȁ沖ԀᣖČ

Č

Ȁ沖ԀᣖČ

Č

Ȁ沖ԀᣖČ

Č

Ȁ沖ԀᣖČ

Č

Ȁ沖ԀᣖČ

Č

Ȁ沖ԀᣖČ

Č

h

Ȁ沖ԀᣖČ

Č

鐇ÿ혈F鐃쟿﬊⼕!㌦

㐦

㐦

Ȁ沖ԀᣖČ

Č

鐇ÿ혈F鐃쟿﬊⼕!㌦

㐦

㐦

Ȁ沖ԀᣖČ

Č

鐇ÿ혈F鐃쟿﬊⼕!㌦

㐦

&4

Ȁ沖ԀᣖČ

Č

鐇ÿ혈F鐃쟿﬊⼕!㌦

㐦

㐦

Ȁ沖ԀᣖČ

Č

鐇ÿ혈F鐃쟿﬊⼕!㌦

㐦

㐦

Ȁ沖ԀᣖČ

Č

鐇ÿ혈F鐃쟿﬊⼕!㌦

㐦

&4

Ȁ沖ԀᣖČ

Č

鐇ÿ혈F鐃쟿﬊⼕!㌦

㐦

㐦

Ȁ沖ԀᣖČ

Č

鐇ÿ혈F鐃쟿﬊⼕!㌦

㐦

㐦

Ȁ沖ԀᣖČ

Č

鐇ÿ혈F鐃쟿﬊⼕!㌦

㐦

&4

Ȁ沖ԀᣖČ

Č

鐇ÿ혈F鐃쟿﬊⼕!㌦

㐦

㐦

kd

׺⎂٠׺Č

Ć

؀ᶈČ

Ć

\

$

 hn

 hn

Ȁ沖ԀᣖČ

Č

Ȁ沖ԀᣖČ

Č

Ȁ沖ԀᣖČ

Č

Ȁ沖ԀᣖČ

Č

hn

hn

hn

 h'

h'

h'

h'

h'

Ȁ沖ԀᣖČ

Č

鐇ÿ혈r鐅姿똗訡鴫2씦

Ȁ沖ԀᣖČ

Č

鐇ÿ혈r鐅姿똗訡鴫2씦

Ȁ沖ԀᣖČ

Č

鐇ÿ혈r鐅姿똗訡鴫2씦

Ȁ沖ԀᣖČ

Č

鐇ÿ혈r鐅姿똗訡鴫2씦

Ȁ沖ԀᣖČ

Č

鐇ÿ혈r鐅姿똗訡鴫2씦

Ȁ沖ԀᣖČ

Č

鐇ÿ혈r鐅姿똗訡鴫2씦

Ȁ沖ԀᣖČ

Č

鐇ÿ혈r鐅姿똗訡鴫2씦

Ȁ沖ԀᣖČ

Č

鐇ÿ혈r鐅姿똗訡鴫2씦

Ȁ沖ԀᣖČ

Č

鐇ÿ혈r鐅姿똗訡鴫2씦

Ȁ沖ԀᣖČ

Č

鐇ÿ혈r鐅姿똗訡鴫2씦

”ÿY

&Å

”ÿY

&Å

”ÿY

&Å

”ÿY

&Å

”ÿY

&Å

”ÿY

&Å

”ÿY

&Å

”ÿY

&Å





”ÿY

&Å

Ć

Ć

 hi&

" hq

& hi&

# hq

Ć

Ć

  hq

# hi&

  hi&

# hq

 hq

ô

ô

ô

ô

  hq

# hq

 hq

 hi&

 hq

" hq

}

}

阂l㐃ԁᣖČ

Č

紀섋ؑ䬘逞․紆

Ȁ沖ԀᣖČ

Č

紀섋ؑ䬘逞$紆

Ȁ沖ԀᣖČ

Č

紀섋ؑ䬘逞$紆

" hq

# hq

  hq

# hq

 hq

 hq

阂l㐃ԁᣖČ

Č

Ȁ沖ԀᣖČ

Č

$

  hq

# hq

 hq

 hq

" hq

Ȁ沖ԀᣖČ

Č

$

  hq

 

  hq

  hq

# hq

 hq

 hq

% hq

" hq

Ȁ沖ԀᣖČ

Č

$

u

v

x

܀ 

r

v

w

x



€

–

—

˜

™

š

¥

¨

Ç

Ð

Ò

Ó

,

-

8

9

<

=

x

Ò

Ó

;

<

>

¥

¦

=

>

E

F

\

]

^

_

`

o

€

ƒ

Œ

¤

¥

×

à

ù

# hq

& hq

& hq

ù

û

 hq

 hq

Ć

Ć

Ȁ沖ԀᣖČ

Č

Ȁ沖ԀᣖČ

Č

Ȁ沖ԀᣖČ

Č

Ȁ沖ԀᣖČ

Č

$

Ȁ沖ԀᣖČ

Č

Ć

Ć

Ȁ

Ć

Ć

h-

Ć

Ć

Ć

Ć

Ć

h-

h-

”ÿY

&Å

Ć

”ÿY

&Å

”ÿY

&Å

”ÿY

&Å

”ÿY

&Å





”ÿY

&Å

”ÿY

&Å

”ÿY

&Å

”ÿY

&Å

”ÿY

&Å

”ÿY

&Å

”ÿY

&Å

”ÿY

&Å

”ÿY

&Å





”ÿY

&Å

”ÿY

&Å

”ÿY

&Å

”ÿY

&Å

ЀL摧皔%

Ć

Ć

Ȁ沖ԀᣖČ

Č

Ȁ沖ԀᣖČ

Č

Ȁ沖ԀᣖČ

Č

Ȁ沖ԀᣖČ

Č

h("

 hn

 hn

h("

  not represent and should not be construed to represent any Agency
determination or policy.

Outputs

■ Annual Illnesses and Deaths

