Meteorological Model Performance for Annual 2011 WRF v3.4 Simulation

1. 	INTRODUCTION

The Weather Research and Forecasting model (WRF) was applied for the
entire year of 2011 to generate meteorological data to support emissions
and photochemical modeling applications for this year. The WRF
meteorological fields will be converted to air quality modeling input
data and used to support assessments of ozone, PM2.5, visibility, and a
variety of toxics. 

The WRF model was applied to a 36 km continental United States scale
domain (36US) and a 12 km continental United States scale domain
(12US2). Both model simulations were initialized directly from
meteorological analysis data. Model parameterizations and options
outlined in this document were chosen based on a series of sensitivity
runs performed by U.S. Environmental Protection Agency (USEPA) Office of
Research and Development that provided an optimal configuration based on
temperature, mixing ratio, and wind field. All WRF simulations were done
by Computer Sciences Corporation (CSC) under contract to the USEPA.

2. 	MODEL CONFIGURATION

2.1 	Configuration of the 36US Domain

Version 3.4 of the WRF model, Advanced Research WRF (ARW) core
(Skamarock, 2008) was used for generating the 2011 simulations. Selected
physics options include Pleim-Xiu land surface model, Asymmetric
Convective Model version 2 planetary boundary layer scheme, Kain-Fritsch
cumulus parameterization utilizing the moisture-advection trigger (Ma
and Tan, 2009), Morrison double moment microphysics, and RRTMG longwave
and shortwave radiation schemes (Gilliam and Pleim, 2010).

The WRF model was initialized using the 12km North American Model
(12NAM) analysis product provided by National Climatic Data Center
(NCDC). Where 12NAM data was unavailable, the 40km Eta Data Assimilation
System (EDAS) analysis (ds609.2) from the National Center for
Atmospheric Research (NCAR) was used. Analysis nudging for temperature,
wind, and moisture was applied above the boundary layer only. The model
simulations were conducted in 5.5 day blocks with soil moisture and
temperature carried from one block to the next via the ipxwrf program
(Gilliam and Pleim, 2010). Landuse and land cover data were based on the
1992-1993 U.S. Geological Survey (USGS) data. 

Figure 2.1 shows the 36US domain, which utilizes a Lambert conformal
projection centered at  (-97,40) with true latitudes of 33 and 45
degrees north. The domain contains 164 cells in the X direction and 128
cells in the Y direction. All cells are 36 km2. As shown in Table 2.1,
the atmosphere is resolved with 35 vertical layers up to 50 mb, with the
thinnest layers being nearest the surface to better resolve the
planetary boundary layer (PBL).

WRF Layer	Height (m)	Pressure (mb)	Sigma

35	17,556	5000	0.000

34	14,780	9750	0.050

33	12,822	14500	0.100

32	11,282	19250	0.150

31	10,002	24000	0.200

30	8,901	28750	0.250

29	7,932	33500	0.300

28	7,064	38250	0.350

27	6,275	43000	0.400

26	5,553	47750	0.450

25	4,885	52500	0.500

24	4,264	57250	0.550

23	3,683	62000	0.600

22	3,136	66750	0.650

21	2,619	71500	0.700

20	2,226	75300	0.740

19	1,941	78150	0.770

18	1,665	81000	0.800

17	1,485	82900	0.820

16	1,308	84800	0.840

15	1,134	86700	0.860

14	964	88600	0.880

13	797	90500	0.900

12	714	91450	0.910

11	632	92400	0.920

10	551	93350	0.930

9	470	94300	0.940

8	390	95250	0.950

7	311	96200	0.960

6	232	97150	0.970

5	154	98100	0.980

4	115	98575	0.985

3	77	99050	0.990

2	38	99525	0.995

1	19	99763	0.9975

Surface	0	100000	1.000



Table 2.1 WRF layers and their approximate height above ground level.

Figure 2.1 Map of WRF model domain: 36US

2.2 	Configuration of the 12US2 Domain

The 12km configuration is the same as the 36km domain with two
exceptions: the Group for High Resolution Sea Surface Temperatures
(GHRSST) (Stammer et al., 2003) 1km SST data was used to provide more
resolved information compared to the more coarse data in the NAM
analysis. Additionally, landuse and land cover data were based on the
National Land Cover Database 2006 (NLCD 2006). Analysis nudging for
temperature, wind, and moisture was applied above the boundary layer
only. The model simulations were conducted in 5.5 day blocks with soil
moisture and temperature carried from one block to the next via the
ipxwrf program (Gilliam and Pleim, 2010). 

Figure 2.2 shows the 12US2 domain, which utilizes a Lambert conformal
projection centered at (-97,40) with true latitudes of 33 and 45 degrees
north. The domain contains 396 cells in the X direction and 246 cells in
the Y direction. All cells are 12 km2. The atmosphere is resolved with
35 vertical layers up to 50 mb (see table 2.1), with the thinnest layers
being nearest the surface to better resolve the PBL.

Figure 2.2 Map of WRF model domain: 12US2.

3	MODEL PERFORMANCE DESCRIPTION

The WRF model simulations were evaluated to determine whether the output
fields represent a reasonable approximation of the actual meteorology
that occurred during the modeling period. Identifying and quantifying
these output fields allows for a downstream assessment of how the air
quality modeling results are impacted by the meteorological data. For
the purposes of this assessment, 2-meter temperature and mixing ratio,
10-meter wind speed and direction, and shortwave radiation are
quantitatively evaluated. A qualitative evaluation of precipitation is
also provided.

The observation database for surface-based temperature, wind speed and
direction, and mixing ratio is based on measurements made at United
States (i.e., National Weather Service) and Canadian (i.e., Environment
Canada) airports. The observational dataset (ds472 network) is available
from NCAR. Monitors used for evaluation are shown in Figure 3.1.

 

Figure 3.1 Stations used for model performance: ds472 network.

Shortwave downward radiation measurements are taken at Surface Radiation
Budget Network (SURFRAD) ( HYPERLINK "http://www.srrb.noaa.gov/surfrad"
http://www.srrb.noaa.gov/surfrad ) and Integrated Surface Irradiance
Study (ISIS) ( HYPERLINK "http://www.srrb.noaa.gov/isis/index.html"
http://www.srrb.noaa.gov/isis/index.html ) monitor locations. The
SURFRAD network consists of 7 sites and the ISIS network consists of 9
sites across the United States (see Figure 3.2). Both networks are
operated by the National Oceanic and Atmospheric Administration (NOAA),
with SURFRAD sites existing as a subset of ISIS monitors that provide
higher level radiation information not used in this evaluation.

Figure 3.2. Location of ISIS and SURFRAD radiation monitors.

Rainfall amounts are estimated by the Parameter-elevation Relationships
on Independent Slopes Model (PRISM) model, which uses an elevation-based
regression model to analyze precipitation. PRISM’s horizontal
resolution is approximately 2 to 4 km and is re-projected to the WRF
modeling domain for direct qualitative comparison to model estimates.
The rainfall analysis is limited to the contiguous United States as the
model utilizes elevation and measured precipitation data at automated
weather stations.

Model performance (i.e., temperature, wind speed, and mixing ratio) is
described using quantitative metrics: mean bias, mean (gross) error,
fractional bias, and fractional error (Boylan and Russell, 2006). These
metrics are useful because they describe model performance in the
measured units of the meteorological variable and as a normalized
percentage. Since wind direction is reported in compass degrees,
estimating performance metrics for wind direction is problematic as
modeled and observed northerly winds may be similar but differences
would result in a very large artificial bias. For example, the absolute
difference in a northerly wind direction measured in compass degrees of
1° and 359° is 358° when the actual difference is only 2°. To
address this issue, wind field displacement, or the difference in the U
and V vectors between modeled (M) and observed (O) values, is used to
assess wind vector performance (Equation 1). Performance is best when
these metrics approach 0. 

(1) 	Wind displacement (km) = (UM – UO + VM – VO)*(1 km/1000
m)*(3600 s/hr)*(1 hr)

Rainfall performance is examined spatially using side-by-side
comparisons of monthly total rainfall plots. The WRF model outputs
predictions approximately 15 meters above the surface while observations
are at 10 meters. WRF generates output at near instantaneous values (90
second time step) as opposed to longer averaging times taken at monitor
stations. This should be considered when interpreting model performance
metrics. 

3.1	Model Performance for Winds

WRF-predicted wind speed estimates are compared to surface-based
measurements made in the ds472 network described earlier. The results
for the 36US (Figure 3.1.1) and 12US2 (Figure 3.1.2) domains are shown
below. 

At 36km, wind speeds are generally overpredicted across most hours of
the day for all seasons, in terms of mean bias. In general, performance
improves at 12km, with less overprediction. However, at 12km WRF tends
to slightly overpredict wind speeds in the early morning and afternoon
hours, while slightly underpredicting wind speeds in the late evening
and overnight hours. There is no significant seasonal variability at
either resolution in terms of wind speed.

The monthly spatial distributions of the wind speed biases (m/s) for all
hours (Figures 3.1.3-3.1.6) and daytime hours (Figures 3.1.7-3.1.10) are
also presented. No appreciable difference is observed in the biases for
daytime hours versus all hours. However, WRF tends to slightly
overpredict wind speeds for areas in the eastern US and underpredicts
wind speeds in the western US, particularly southern California. As
noted above, these biases persist regardless of changes in season.

 

Figure 3.1.1. Distribution of hourly bias by hour and hourly bias,
error, fractional bias, and fractional error for wind speed by month for
36US domain.

Figure 3.1.2. Distribution of hourly bias by hour and hourly bias,
error, fractional bias, and fractional error for wind speed by month for
12US2 domain.

Figure 3.1.3. Spatial distribution of wind speed bias (m/s) across all
hours for the months of January, February, and March (top to bottom) for
the 12US2 domain.

Figure 3.1.4. Spatial distribution of wind speed bias (m/s) across all
hours for the months of April, May, and June (top to bottom) for the
12US2 domain.

Figure 3.1.5. Spatial distribution of wind speed bias (m/s) across all
hours for the months of July, August, and September (top to bottom) for
the 12US2 domain.

Figure 3.1.6. Spatial distribution of wind speed bias (m/s) across all
hours for the months of October, November, and December (top to bottom)
for the 12US2 domain.

Figure 3.1.7. Spatial distribution of wind speed bias (m/s) across
daytime hours for the months of January, February, and March (top to
bottom) for the 12US2 domain.

Figure 3.1.8. Spatial distribution of wind speed bias (m/s) across
daytime hours for the months of April, May, and June (top to bottom) for
the 12US2 domain.

Figure 3.1.9. Spatial distribution of wind speed bias (m/s) across
daytime hours for the months of July, August, and September (top to
bottom) for the 12US2 domain.

Figure 3.1.10. Spatial distribution of wind speed bias (m/s) across
daytime hours for the months of October, November, and December (top to
bottom) for the 12US2 domain.

Wind vector displacement (km) is presented below for the 36US (Figure
3.1.11) and 12US2 (Figure 3.1.12) domains utilizing the ds472
observation network described earlier. These plots show the entire
distribution of hourly wind displacement by month and by hour of the
day. Overall, model performance is adequate in terms of wind vector
differences. Both the 36- and 12-km simulations have a mean wind
displacement of around 5km. Since this difference is less than the
horizontal resolution, negligible impacts due to wind displacement are
expected.

 Figure 3.1.11. Distribution of hourly wind displacement (km) by hour
and month for the 36US domain.

Figure 3.1.12. Distribution of hourly wind displacement (km) by hour and
month for the 12US2 domain.

3.2 	Temperature

Temperature estimates are compared to the ds472 observation network
described earlier and are presented below for the 36US (Figure 3.2.1)
and 12US2 (Figure 3.2.2) domains.

Overall, WRF slightly underpredicts surface temperature at both 36- and
12-km for most hours, with a slight overprediction in the early morning
hours. In the summer months (June, July, and August), there appears to
be less variability in both simulations, with the inner quartile range
(IQR) more closely centered around zero in both simulations. Overall,
with an average IQR of +/- 2 degrees Celsius (C), this is considered
good model performance.

In Figures 3.2.3-3.2.6 and 3.2.7-3.2.10 the monthly spatial
distributions of the temperature bias for the 12km simulation is
presented for all hours and daytime only, respectively. Overall, a
persistent slight underprediction of temperature is noted for most
months. During daytime hours, a more significant underprediction of
temperature is noted across much of the central and eastern US when
compared to all hours. In areas of the western US, there is a persistent
slight overprediction of temperature, regardless of season.

 

Figure 3.2.1. Distribution of hourly bias by hour and hourly bias,
error, fractional bias, and fractional error for temperature by month
for the 36US domain.

Figure 3.2.2. Distribution of hourly bias by hour and hourly bias,
error, fractional bias, and fractional error for temperature by month
for the 12US2 domain.

Figure 3.2.3. Spatial distribution of temperature bias (C) across all
hours for the months of January, February, and March (top to bottom) for
the 12US2 domain.

Figure 3.2.4. Spatial distribution of temperature bias (C) across all
hours for the months of April, May, and June (top to bottom) for the
12US2 domain.

Figure 3.2.5. Spatial distribution of temperature bias (C) across all
hours for the months of July, August, and September (top to bottom) for
the 12US2 domain.

Figure 3.2.6. Spatial distribution of temperature bias (C) across all
hours for the months of October, November, and December (top to bottom)
for the 12US2 domain.

Figure 3.2.7. Spatial distribution of temperature bias (C) across
daytime hours for the months of January, February, and March (top to
bottom) for the 12US2 domain.

Figure 3.2.8. Spatial distribution of temperature bias (C) across
daytime hours for the months of April, May, and June (top to bottom) for
the 12US2 domain.

Figure 3.2.9. Spatial distribution of temperature bias (C) across
daytime hours for the months of July, August, and September (top to
bottom) for the 12US2 domain.

Figure 3.2.10. Spatial distribution of temperature bias (C) across
daytime hours for the months of October, November, and December (top to
bottom) for the 12US2 domain.

3.3 	Mixing Ratio

Water mixing ratio estimates are compared to the ds472 observation
network described earlier and are presented below for the 36US (Figure
3.3.1) and 12US2 (Figure 3.3.2) domains. 

In either simulation, no significant positive or negative bias is
observed. However, WRF tends to be slightly drier in the early afternoon
hours relative to the rest of the day. Additionally, there is more
uncertainty in model predictions during the spring and summer months.
This increase in error is explained by the increased convective activity
and influx of moist air masses that are typical of that time of year. In
general, WRF performance was adequate for water vapor mixing ratio.

The monthly spatial distributions of the mixing ratio bias for the 12km
simulation are shown in Figures 3.3.3-3.3.6 (all hours) and 3.3.7-3.3.10
(daytime). Little appreciable difference is observed in the biases
either across all hours or just daytime. This is to be expected since
water vapor mixing ratio has less temporal variability when compared to
other variables (i.e., temperature). In the central and western US,
mixing ratio is generally underpredicted across all months of the year,
whereas a slight overprediction is observed in the eastern states for
most months. In October and November, WRF exhibits a general
underprediction for most locations across the country.

 

Figure 3.3.1. Distribution of hourly bias by hour and hourly bias,
error, fractional bias, and fractional error for water vapor mixing
ratio by month for the 36US domain.

Figure 3.3.2. Distribution of hourly bias by hour and hourly bias,
error, fractional bias, and fractional error for water vapor mixing
ratio by month for the 12US2 domain.

Figure 3.3.3. Spatial distribution of water vapor mixing ratio bias
(g/kg) across all hours for the months of January, February, and March
(top to bottom) for the 12US2 domain.

Figure 3.3.4. Spatial distribution of water vapor mixing ratio bias
(g/kg) across all hours for the months of April, May, and June (top to
bottom) for the 12US2 domain.

Figure 3.3.5. Spatial distribution of water vapor mixing ratio bias
(g/kg) across all hours for the months of July, August, and September
(top to bottom) for the 12US2 domain.

Figure 3.3.6. Spatial distribution of water vapor mixing ratio bias
(g/kg) across all hours for the months of October, November, and
December (top to bottom) for the 12US2 domain.

Figure 3.3.7. Spatial distribution of water vapor mixing ratio bias
(g/kg) across daytime hours for the months of January, February, and
March (top to bottom) for the 12US2 domain.

Figure 3.3.8. Spatial distribution of water vapor mixing ratio bias
(g/kg) across daytime hours for the months of April, May, and June (top
to bottom) for the 12US2 domain.

Figure 3.3.9. Spatial distribution of water vapor mixing ratio bias
(g/kg) across daytime hours for the months of July, August, and
September (top to bottom) for the 12US2 domain.

Figure 3.3.10. Spatial distribution of water vapor mixing ratio bias
(g/kg) across daytime hours for the months of October, November, and
December (top to bottom) for the 12US2 domain.

3.4 	Precipitation

Monthly total rainfall is plotted for each grid cell to assess how well
the model captures the spatial variability and magnitude of convective
and non-convective rainfall. As described earlier, the PRISM estimations
for rainfall are only within the continental United States. WRF rainfall
estimates by month are shown for all grid cells in the domain. Monthly
total estimates are shown for the 36US domain (Figures 3.4.1 through
3.4.4) and 12US2 domain (Figures 3.4.5 through 3.4.8).

In general, WRF performs adequately in terms of the spatial patterns and
magnitude of precipitation across the US throughout the year. Both
simulations, however, tend to overestimate precipitation in elevated
terrain (e.g., northern CA and the Pacific Northwest). The 12km
simulation tends to generate slightly higher precipitation amounts
compared to the 36km simulation, and at times generates amounts higher
than observed values (e.g., the southeast in July). Isolated amounts of
overpredicted or underpredicted precipitation in the summer months
relative to the observations is likely due to uncertainty in the
convective parameterization scheme utilized.

Figure 3.4.1. PRISM analysis (left) and WRF (right) estimated monthly
total rainfall (in) for January, February, and March.

3.4.2. PRISM analysis (left) and WRF (right) estimated monthly total
rainfall (in) for April, May, and June.

Figure 3.4.3. PRISM analysis (left) and WRF (right) estimated monthly
total rainfall (in) for July, August, and September.

Figure 3.4.4. PRISM analysis (left) and WRF (right) estimated monthly
total rainfall (in) for October, November, and December.

Figure 3.4.5. PRISM analysis (left) and WRF (right) estimated monthly
total rainfall (in) for January, February, and March.

Figure 3.4.6. PRISM analysis (left) and WRF (right) estimated monthly
total rainfall (in) for April, May, and June.

	

Figure 3.4.7. PRISM analysis (left) and WRF (right) estimated monthly
total rainfall (in) for July, August, and September.

	

Figure 3.4.8. PRISM analysis (left) and WRF (right) estimated monthly
total rainfall (in) for October, November, and December.

3.5 	Solar Radiation

Photosynthetically activated radiation (PAR) is a fraction of shortwave
downward radiation and is an important input for the biogenic emissions
model for estimating isoprene (Carlton and Baker, 2011). Isoprene
emissions are important for regional ozone chemistry and play a role in
secondary organic aerosol formation. Radiation performance evaluation
also gives an indirect assessment of how well the model captures cloud
formation during daylight hours.

Shortwave downward radiation estimates are compared to surface based
measurements made at SURFRAD and ISIS network monitors for the 36US
(Figure 3.6.1) 12US2 (Figure 3.6.2) domains. 

Overall, both the 36- and 12km simulations show WRF has little bias in
shortwave radiation predictions during the fall and winter months.
Biases tend to grow during the spring and peak in the summer, though the
spread in overpredictions tends to be less than 100 W/m2 on average,
with a median bias close to zero. 

More variability is noted on an hourly basis. WRF tends to overpredict
early morning to early afternoon shortwave radiation, while
underpredicting the late afternoon and early evening values. The median
overprediction at the time of greatest incoming solar radiation is near
100 W/m2. In the late afternoon and evening hours, the median bias is
close to -50 W/m2. These errors are likely attributable to the model
being unable to accurately simulate cloud features at subgrid (<12km)
scales. This assumption is based on the slight improvement in
predictions at 12km versus 36km. 

Figure 3.5.1. Distribution of hourly bias for shortwave radiation (W/m2)
by month (top) and by hour of the day (bottom) for the 36US domain.

Figure 3.5.2. Distribution of hourly bias for shortwave radiation (W/m2)
by month (top) and by hour of the day (bottom) for the 12US2 domain.

4	CLIMATE REPRESENTATIVENESS OF 2011

Figures 4.1 and 4.2 show the divisional rankings for observed
temperatures across the US for 2011. A climatic representation of the
precipitation for 2011 is shown in Figures 4.3 and 4.4. These plots are
useful in determining the representativeness of 2011 in terms of certain
climatological variables compared to historical averages. 

Temperatures in 2011 were average to above average for most of the
central and eastern US throughout the spring and summer months. Some
areas in the southern and southeastern portions of the country exhibited
near record warmth from June through August. Conversely, February
through July tended to be cooler than average for the western US with
near-record cold exhibited in the Pacific Northwest.

The spring and summer months experienced below average precipitation for
much of the southern and southeastern US, whereas wetter conditions than
average were experienced across the northern tier states. During the
fall and winter months, most of the eastern US experienced average to
above average precipitation, while the western and southern US remained
generally below average. 

Figure 4.1 Climatic temperature rankings by climate division: January to
June 2011.  HYPERLINK
"http://www.ncdc.noaa.gov/temp-and-precip/maps.php"
http://www.ncdc.noaa.gov/temp-and-precip/maps.php 

Figure 4.2 Climatic temperature rankings by climate division: July to
December 2011.  HYPERLINK
"http://www.ncdc.noaa.gov/temp-and-precip/maps.php"
http://www.ncdc.noaa.gov/temp-and-precip/maps.php 

Figure 4.3 Climatic rainfall rankings by climate division: January to
June 2011.  HYPERLINK
"http://www.ncdc.noaa.gov/temp-and-precip/maps.php"
http://www.ncdc.noaa.gov/temp-and-precip/maps.php 

Figure 4.4 Climatic rainfall rankings by climate division: July to
December 2011.  HYPERLINK
"http://www.ncdc.noaa.gov/temp-and-precip/maps.php"
http://www.ncdc.noaa.gov/temp-and-precip/maps.php 

5	REFERENCES

Boylan, J.W., Russell, A.G., 2006. PM and light extinction model
performance metrics, goals, and criteria for three-dimensional air
quality models. Atmospheric Environment 40, 4946-4959.

Carlton, A.G., Baker, K.R., 2011. Photochemical Modeling of the Ozark
Isoprene Volcano: MEGAN, BEIS, and Their Impacts on Air Quality
Predictions. Environmental Science & Technology 45, 4438-4445.

Cooper, O.R., Stohl, A., Hubler, G., Hsie, E.Y., Parrish, D.D., Tuck,
A.F., Kiladis, G.N., Oltmans, S.J., Johnson, B.J., Shapiro, M., Moody,
J.L., Lefohn, A.S., 2005. Direct Transport of Midlatitude Stratospheric
Ozone into the Lower Troposphere and Marine Boundary Layer of the
Pacific Ocean. Journal of Geophysical Research – Atmospheres 110,
D23310, doi:10.1029/2005JD005783.

ENVIRON, 2008. User's Guide Comprehensive Air Quality Model with
Extensions. ENVIRON International Corporation, Novato.

Gilliam, R.C., Pleim, J.E., 2010. Performance Assessment of New Land
Surface and Planetary Boundary Layer Physics in the WRF-ARW. Journal of
Applied Meteorology and Climatology 49, 760-774.

Langford, A.O., Reid, S.J., 1998. Dissipation and Mixing of a
Small-Scale Stratospheric Intrusion in the Upper Troposphere. Journal of
Geophysical Research 103, 31265-31276.

Otte, T.L., Pleim, J.E., 2010. The Meteorology-Chemistry Interface
Processor (MCIP) for the CMAQ modeling system: updates through
MCIPv3.4.1. Geoscientific Model Development 3, 243-256.

Skamarock, W.C., Klemp, J.B., Dudhia, J., Gill, D.O., Barker, D.M.,
Duda, M.G., Huang, X., Wang, W., Powers, J.G., 2008. A Description of
the Advanced Research WRF Version 3.

Stammer, D., F.J. Wentz, and C.L. Gentemann, 2003, Validation of
Microwave Sea Surface Temperature Measurements for Climate Purposes, J.
Climate, 16, 73-87.

 Version 3.4 was the most current version of WRF at the time the 2011
meteorological model simulations were performed.

 1200 GMT to 0000 GMT

  PAGE   \* MERGEFORMAT  10 

