Welcome

 

A Message to Our Readers
Featured Article

 

The Skinny on PART
Insights

 

Acquisition Management
Program Management
Strategic Management
Research Update
Events
Links
Contact Us
The Skinny on PART
1  Introduction

Integration of budget and performance, the fifth element of the President's Management Agenda (PMA), is now formally implemented. The Program Assessment Rating Tool (PART) is in its third year of evaluating a program's purpose, design, planning, management, results, and accountability to determine its overall effectiveness.

With 60% of the federal budget evaluated via PART, the Administration has shown a lasting commitment to improving federal program performance and accountability. The question for agencies dealing with PART is what to expect next, and specifically how much of scant analytical resources should be applied to PART, especially compared to other traditional avenues for program and budget justification. This white paper is a cheat sheet on PART, including what to expect next from PART and best practices for agencies responding to PART.

2  What is PART Anyway?

Achieving results oriented government a central policy goal for the Bush administration. To achieve this goal, the administration established the PMA, comprised of 5 initiatives including Budget and Performance Integration (BPI). A central element in this BPI is the Office of Management and Budget's (OMB) PART, a diagnostic tool meant to provide a consistent approach to evaluating programs across the Federal government

PART's goal is to develop information that will be useful to the budget process. OMB and individual agencies go through yes/no questions that evaluate program performance in four areas. This evaluation is translated into a numeric score for each area and overall program effectiveness (OMB weighting):

  • Purpose (20%)
  • Strategic Planning (10%)
  • Program Management (20%)
  • Program Results (50%)

 

OMB also hopes PART can help establish performance criteria for good government management and will be used in conjunction with annual performance plans and reports required by the Government Performance and Results Act (GPRA).

The PART effort started in 2002 and OMB and individual agencies are currently working together to review 20% of federal programs under the PART process each year until all programs are evaluated. In addition, the PART review process revisits the programs previously evaluated each year until 100% of the federal budget is assessed via PART. The schedule of reviews looks like this:

  • CY02: 20% (FY 2004 budget)
  • CY03: new 20% + previous 20% (FY 2005 budget)
  • CY04: new 20% + previous 40% (FY 2006 budget)
  • CY05: new 20% + previous 60% (FY 2007 budget)
  • CY06: new 20% + previous 80% (FY 2008 budget)

To date, 399 Programs representing $903B of funding have been evaluated via PART. These programs have been judged against relative a high standard of clear purpose, solid planning, strong management, and demonstrable results:

3 How PART Scores Relate To Funding

There is some correlation between PART and funding. 80% of discretionary programs rated effective by PART received funding increases in FY03-FY04. Funding was cut for 67% of discretionary programs rated ineffective by PART in FY03-FY04. At this same time, the connection is not absolute and additional factors are likely involved. Please contact Pivotal for more detailed analysis on this topic.

4  What The Agencies Are Saying About PART

While no one can argue with the need and value of establishing a connection between results and investments. Determining performance, especially in government, is a difficult process fraught with ambiguity. Linking performance evaluations with budget decisions brings into play the complex web of influences and goals that are inherent in government.

Now in its third year of implementation, several things stand out about PART and results oriented government. Here is what some Agencies1 are saying about PART:

What people think
about
Detailed comments
the validity of PART
  • While there is some correlation between PART scoring and budgets received, it is not the determining factor in budget allocation.
  • PART scores do not necessarily relate to program performance. We are improving scores by improving the paperwork and the descriptions of the programs (not all bad).
  • The link between the PPBS process and PART is unrealistic. Budget decisions are often made for reasons not captured in PART (e.g. politics, world events).
…how PART is designed
  • The PART unit of analysis (what programs are chosen and how programs are defined) needs refinement.
  • How is statutory design isolated in PART? An Agency could effectively administer a program but can't control what Congress wants.
  • PART is budget focused. GPRA is Agency focused. There is still a need for a good system for measuring execution and an effective way to relate budget, execution and Agency strategy.
  • The PART tool implies a level of accuracy (1-100 by 1) that is not meaningful. PART is probably no more accurate than a 5 point (best) or 10 point (maybe) scale. This creates several problems:
    • Management takes the scores seriously, thus significant management attention and time is wasted trying to justify the difference of a few points.
    • A change in examiners can result in differences in scoring that is probably not materially significant, yet show a change in effectiveness.
    • Linked questions within PART can result in large point swings that have little relation to the reality of incremental program improvements.
...responding to PART
  • Hard to correlate PART and GPRA, specifically integrating measures from the agency to sub-program level. OMB seems to focus on PART to the exclusion of GPRA. (Sometimes unclear if there should be two or one performance plans)
  • PART, especially with GPRA, overburdens Agency management staff. For example, Performance Accountability Reports (PARs) GPRA Requirement) and PART usually fall on the same sub-organization. PART should replace or be integrated with PARs.
  • It seems we are reinventing the wheel - no clear source for PART best practices.

1Based on discussions with government executives responsible for PART at 10 different agencies.

5 What to Expect Next

To address these challenges, PART continues to evolve. Moving forward, expect the following:

  • Congress paying more attention to PART. PART has effectively raised the stakes for performance management in government. OMB and Authorizers will increasingly use PART to inform agency and OMB decisions for management, legislative, or regulatory improvements, and budget decisions. Appropriators will use PART as a tool to objectively justify budget cuts or increases. Under this Administration, more often than not it will be used to reduce spending.

    In addition, Rep Platts (R-PA) and Rep Davis (R-VA) introduced legislation in late February amending GPRA to require regular evaluation of all federal programs. While not mandating PART, legislation would require OMB to assess each federal program's performance at least once every 5 years using a cross-cutting program evaluation strategy, assessing in the same year programs with similar goals.

  • PART starting earlier in the budget process: OMB released the FY06 PART guidance 2 months earlier than in FY05. OMB ideally would like to get PART done early in the year to know the results when the first draft of the justification is written for bureau heads in the summer.
  • Capacity becoming an issue with PART: Agencies have reported considerable PART workload, and reassessing programs is consuming more time than anticipated. This is situation is exacerbated by the lack of coordination between PART and GPRA. Also, OMB has no system for monitoring recommendations or evaluating performance. Especially as the number of programs evaluated by PART increases, Agencies and OMB will have challenges in allocating scarce analytical resources. As such, look for OMB to try to focus their efforts moving forward as shown below.

6  PART Best Practices

Pivotal has identified several best practices to help agencies approach and respond to PART. These best practices are not prescriptive, but are techniques and perspectives that have proven successful in dealing with PART. PART is an evolving tool and not every organization or situation is alike. Some practices will work better than others depending on the specific situation.

  • Make PART consistent with your Budget. Eliminate confusion and the need for reconciliation between how your organization approaches the budget and PART. Submitting PARTs at the same level as you allocate performance commitments allows clean mapping of budget to performance and makes PART easier to address. Also, dealing with PARTs at the agency level provides the opportunity to develop common approaches to problems, rather than sub-optimize for individual programs. Re-align the budget into centers consistent with the PART definition of a program. Make resources flexible within the centers to accomplish its objectives and manage as a portfolio of projects.
  • Partner with OMB. Agencies must be proactive and involved in the process. Partner with your OMB examiner. If he or she is not working with you, your scores will not improve no matter what you change. Be sure to provide clear, well organized, documentation for all of your PART answers. The spreadsheet provided by OMB is too space-restrictive to cover everything your examiner needs, make it as easy as possible for him or her, and try to think like your auditor.
  • Adopt a best practices approach. Get your OMB Examiner to help you identify best practices for each question. Then compare and consider adopting the best practices or altering your processes to adopt some of the key aspects of the best practices. By adopting/using identified best practices, PART discussions are shifted to execution (i.e. are your effectively applying/following these practices, verified how?) and away from justification (i.e., prove you are funding the right projects at the right levels). The latter is often an impossible task.
7  The Big Picture: The Value Is In The Journey

The government's goal is to provide high quality services at a reasonable cost. Stated differently, the government should strive to achieve outcomes that represent the best value for the taxpayer. Establishing this connection between results and investment, or ROI, is a significant challenge. While PART raises the stakes for program performance management, it is just the latest in a series of government management focuses designed to measure results and justify funding decisions. Legislatures and oversight bodies instinctively desire to guide improvement. Although PART seems to have some staying power, especially given recent legislative activity, there is no guarantee that PART will not give way to a new initiative.

While PART may survive or be replaced, the concern for taxpayers' ROI and attempts to link an agency's budget with the agency's performance is here to stay. Each individual initiative will have mixed success and ultimately make way for a new acronym, but the focus will remain constant. For Agencies dealing with PART or its successor, the questions deal with how to react, embrace, comply, ignore, or resist? The challenge and opportunity for Agencies dealing with PART is to recast compliance, responding to the requirements of a law or directive, into the context of self-directed and self-motivated improvement. Your Agency's work is too important to be left to chance or to be defined by random data calls and oversight requests. Responses are also less painful, more effective, and happen faster if driven internally. While PART's goal of establishing and maximizing taxpayer ROI will likely remain elusive, the real value of efforts like PART is its ability to inspire agency leaders to focus on results and drive improvement from within.

 

 

©2006 Pivotal Insight, LLC. All rights reserved.