											6 February 09

DRAFT CHARGE FOR MID-CYCLE REVIEW   

SCIENCE AND TECHNOLOGY FOR SUSTAINABILITY

BOSC SUBCOMMITTEE 

1.0      Objectives.  The objectives of this mid-cycle review are: 

To evaluate the progress made by the Office of Research and Development
(ORD) Science and Technology for Sustainability Research Program towards
completing the commitments it made in response to the initial BOSC
program review (April 25-26, 2007), and 

To evaluate and obtain advice on key future directions for the research
program which have been developed and other potential areas that could
be considered. 

2.0      Background Information.    Independent expert review is used
extensively in industry, federal agencies, Congressional committees, and
academia.  The National Academy of Science has recommended this approach
for evaluating federal research programs.

For the Agency’s environmental research programs, periodic independent
reviews are conducted at intervals of four or five years to characterize
research progress, to identify when clients are applying research to
strengthen environmental decisions, and to evaluate client feedback
about the research.  Mid-cycle evaluations are an important part of this
program review process.  Scheduled midway through the review cycle,
these independent assessments give ORD an opportunity to gauge the
program’s progress relative to the commitments it made following its
last review. 

For the upcoming mid-cycle review, the Science and Technology for
Sustainability Research Program is preparing a progress report that will
provide the context for discussions during the meeting.  The report will
identify progress the program has made towards its long-term goals, and
changes implemented by the program in response to BOSC’s major
recommendations from the 2007 review.

The STS Program has undergone significant changes since the initial BOSC
review.  The changes are based upon (1) the BOSC 2007 recommendations,
(2) feedback from the 2006 OMB PART review, (3) significant emerging
issues in the sustainability arena, and (4) budget and organizational
changes in EPA.  As a result, ORD has initiated work on a few key
emerging areas of science, e.g., biofuels and green building issues.

Several documents will be provided to the Subcommittee to use in
addressing the charge questions.  ORD will provide two tables that
summarize the changes to the overall program.  Additional documents
include, but are not limited to, the latest versions of the STS MYP, the
US EPA Biofuels Strategy. This review is not intended to be the
in-depth technical evaluation of a full program review.  Presentation
time will be minimized in favor of discussion.  

3.0 	Charge Questions for ORD’s Science and Technology Sustainability
Research Program.  

ORD is interested in receiving feedback concerning the following
questions:

How responsive has the Science and Technology for Sustainability (STS)
Research Program been to the recommendations made in the April 2007 BOSC
program review report?  The subcommittee will evaluate progress made
regarding “commitments” to the BOSC recommendations as outlined in
ORD’s response.  Specifically, the BOSC will evaluate the
accomplishments and effectiveness of the funded research.

How clear is the rationale for the Science and Technology for
Sustainability Research Program as described in the documents provided
to the Subcommittee?  Is it consistent with the advice previously given
by the BOSC?

If needed, what additional performance metrics (e.g., quality and impact
of publications, timeliness of completing goals) might be appropriate
for the Science and Technology for Sustainability Research Program?

What changes could the program make to enhance the impact of the
research products and complement other existing efforts in the field? 
How should products be delivered to ensure they are used within and
outside the Agency, thereby contributing the greatest value?

Given the need to strategically focus the STS program on national
environmental priorities, is the initial focus on Biofuels appropriate
considering the STS Long Term Goals?  If so, is the proposed program
appropriately designed to address these important issues?

Are there any other areas of national significance in the near term that
the program should address?

4.0	Summary Assessment

In developing a short report that responds to the above charge
questions, the BOSC Mid-cycle Subcommittee should provide a summary
assessment, including a single qualitative rating, which reflects the
extent to which the program is making progress in response to the BOSC
review of 2007.  The rating should be in the form of one of the
adjectives defined below, which are intended to promote consistency
among BOSC program reviews.  The adjective should be used as part of a
narrative summary of the review, so that the context of the rating and
the rationale for selecting a particular rating will be transparent. 
For mid-cycle reviews, the rating should be based on the quality, speed,
and success of the program's actions in addressing previous BOSC
recommendations.  The adjectives to describe progress are:  

o	Exceptional:  indicates that the program is meeting all and exceeding
some of its goals, both in the quality of the science being produced and
the speed at which research result tools and methods are being produced.
 An exceptional rating also indicates that the program is addressing the
right questions to achieve its goals.  The review should be specific as
to which aspects of the program’s performance have been exceptional.

o	Exceeds Expectations: indicates that the program is meeting all of its
goals.  It addresses the appropriate scientific questions to meet its
goals, and the science is competent or better.  It exceeds expectations
for either the high quality of the science or for the speed at which
work products are being produced and milestones met.

o	Meets Expectations: indicates that the program is meeting most of its
goals.  Programs meet expectations in terms of addressing the
appropriate scientific questions to meet their goals, and work products
are being produced and milestones are being reached in a timely manner.
The quality of the science being done is competent or better.

o	Not Satisfactory: indicates that the program is failing to meet a
substantial fraction of its goals, or if meeting them, that the
achievement of milestones is significantly delayed, or that the
questions being addressed are inappropriate or insufficient to meet the
intended purpose.  Questionable science is also a reason for rating a
program as unsatisfactory for a particular long-term goal.  The review
should be specific as to which aspects of a program’s performance have
been inadequate.

 

5.0	Subcommittee Approach for Mid-Cycle Review

Hold one (1) combined administrative and technical (public)
teleconference prior to the face-	to-face meeting.

Allows subcommittee to become familiar with Federal Advisory Committee
Act (FACA) meeting requirements.

Allows the subcommittee Chair to make review and writing assignments.

Allows the ORD to present background and other relevant materials to the
subcommittee.

Allows the subcommittee to ask clarifying questions.

EPA shall distribute background materials and documents requested by the
Subcommittee in advance of the teleconference calls.

Hold a one-day face-to-face meeting for the mid-cycle review.

The meeting will include brief ORD presentations on program progress.

Members of the Science and Technology for Sustainability Mid-Cycle
Subcommittee will ask questions and discuss the progress with ORD
representatives.

Subcommittee members will draft portions of the short report.

Hold one (1) teleconference call within one month following the
face-to-face meeting to finalize the draft short report.

 Evaluating Federal Research under the Government Performance and
Results Act (National Research Council, 1999).

A Federal Advisory Committee for the U.S. Environmental Protection
Agency’s Office of Research and Development

 PAGE   

 PAGE   1 

            

		A Federal Advisory Committee for the U.S. Environmental Protection
Agency’s Office of Research and Development

Draft Charge for March 12, 2009 Science and Technology for
Sustainability Mid-Cycle Subcommittee Meeting

