The "Fear" of Performance Management

In a discussion with a colleague recently the topic of why more governments don't have an active performance management program came up. I will admit that the discussion was more speculative than scientific, but we generally agreed that many jurisdictions and agencies likely don't implement performance management programs out of fear of both what they might find as well as how the data that is reported might be used against them (the remainder of non-practitioners likely have no idea what it means!). There are stories of early meetings of CompStat in New York city in which supervisors were skewered based on crime stats in their area (although later accounts suggested a softening in the tone), and perhaps this is what government managers reference when thinking about reasons not to further performance management programs. But anyone focused on that aspect is ignoring the second part of that story which is the potentially positive impact of programs like CompStat. 

New York city has one of the lowest violent crime rates and the lowest property crime rate of large cities in the US. This is in stark contrast to the late 80s when it had one of the highest. The rates dropped precipitously throughout the 1990s, around the same time that CompStat came into being. While this relationship may very well be spurious, I imagine that the new management style in the police department had at least SOME effect on crime outcomes. I use CompStat as an isolated case of one agency's efforts, but it would be an interesting exercise to look at cities and states with advanced performance management programs (Atlanta, Albaquerque, and North Carolina spring to mind) and analyze the short and long term impacts of those programs both socially and politically. The reason this might be helpful is that it would be informative to cities who "fear" such programs in demonstrating the long term value of these programs. Certainly there are painful short term realizations of inefficiencies that would be made from better data and analysis but several case studies on improved outputs might put those fears to rest.

Citizen Awareness of Government Performance Measures

One aspect of the performance reporting cycle that I don't see discussed much is the role of the citizen in holding governments accountable for tax dollars. There are often waves of citizen anger over an isolated project or issue that is a lightning rod for criticism of government waste, but few groups seem to focus on the need for evaluating the performance of governments as a whole. A quick google search of "citizens for government accountability" did not seem to yield much in the way of citizens interested in government performance metrics (but did reveal a lot of anger about other issues!). There are some organizations that I have mentioned in previous posts with an interest in this area, but they are more academic than citizen-focused (I forgot to mention in previous posts the Performance Institute who offers a tremendous array of workshops and forums on government performance).

Ultimately I believe that the success or failure of government performance management in the public sector will be attributable to citizen engagement in the topic. Many people have grown used to easily finding out about crime statistics in a given neighborhood and would certainly notice if the resource was taken away. It will be interesting to see if that will translate to ensuring that their trash was collected, that equipment at the local park is full operational, or that the number of potholes in the streets are going down and not up. A notable paper by the Urban Institute addressed this point back in 2000 but it would be nice to see an update in this area. Most experts will admit that performance measurement has improved a great deal in the last 10 years, but the real question might be whether or not the average citizen cares, and if not, what will the impact on this area of government be in the future?

(as an aside, I'm simultaneously writing this post and watching a special on Federal attempts to clean up the Chesapeake and other waterways. It led me to the EPAStat quarterly report which unfortunately doesn't tell me if the fish or crabs are coming back to the estuary, but does tell me how many Chesapeake Bay Significant Discharge Permits were issued. With uninformative data like this there's no wonder it's hard to engage people in performance management.)

Will new Federal CIO, CTO Change Performance

There are several new faces in DC and it will be interesting to see what their impact on Federal performance management will be. The previous announcement of the Vivek Kundra as the nation's CIO was followed up with this week's announcement that Aneesh Chopra will be the nation's CTO and Jeffrey Zients will be the nation's Chief Performance Officer. This is a high-powered team of professionals that has worked in the city, state, and private sector and have been tasked to work together on bringing accountability back to the Federal government. My hope is that they will take a performance metric approach to accountability and ensure that agencies in the Federal government have a comprehensive performance plan which will be used to analyze effectiveness. I will post articles and comments on their progress moving forward.

Benchmarking Within a State

I mentioned in a previous post the attempts by ASCA to come up with benchmarking measures within the Corrections industry. There are several other groups attempting to benchmark data points within an industry (see HSRI Core Indicators for one example), but a couple of states have focused on a more intra-state approach to benchmarking across all government areas, but within the state. North Carolina and Florida have both embarked on projects to define metrics across jurisdictions in their states for the purpose of benchmarking against each other over time. It will be interesting to see if there is any consolidation among the varying groups attempting to come up with common performance metrics, be they state, industry, or professional organizations/commissions (such as the Performance Management Commission). While it may make sense to take from each group relevant measures, it may not be feasible to incorporate all of them for fear of drowning agencies in performance measurement reporting.

Comparison of Federal Performance Plans: HHS vs. DOJ

Browsing through several different Federal agency performance reports online there are some clear leaders in terms of quality of measures and others that have some challenges. In municipal performance reporting, police departments have often led the way in statistical performance data, largely because crime information is well-defined and one of the most monitored by citizens. Social service agencies typically lag behind their public safety counterparts with respect to municipal performance metrics. At the Federal level, it appears that the opposite is true.

The HHS Administration for Children and Families website has the agency performance reports going back to 2000. A look at the HHS ACF 2008 performance report shows some decent measures (many are outcome-oriented, a break from the output-focused measures of many agencies) that have been created by ACF as well as what looks like honest reporting of those measures. I assume honest reporting based on the fact that the agency has met less of its targets over the last several years. The measures appear well thought out and given the honest reporting of missed targets, the data appears to be reliable. Additionally, the reports get shorter each year since 2000. That's probably a good thing with respect to the public actually reading the document, and the agency focusing more intensely an a narrow set of measures and goals. If you can't explain a measure on a cocktail napkin, it's less likely to be reported and recorded accurately over time.

On the other side of the public spectrum is the DOJ Performance and Accountability Report for 2008. The report is a combination of both performance and financial data and comes just under the whopping-300 page mark. The parts that focus on performance metrics are lacking in a number of ways. Most are output-oriented and don't inform management or the public as to the effectiveness of the agency (One of the measures, "Terrorist Acts Committed by Foreign Nationals Against U.S. Interests", is zero in most years with 2001 the notable exception and would almost certainly be known without needing to be included in the performance report. The measure is neither informative nor a helpful management tool). While the municipal law enforcement agencies have made great strides in performance reporting, the Federal level agencies seem to just be getting their feet under them. One reason may be because of the difficulty in attributing crime and arrest rates to a Federal agency whose jurisdiction is the entire country. Most responsibilities of the agency are shared with state and local jurisdictions, yet the DOJ has little or no control over those agencies. Regardless, there are almost certainly some performance measures that the DOJ could come up with that are more worthwhile. If not, how can their effectiveness truly be measured? A good start would be to narrow the report into something more digestible that might actually be used in agency management.

State Correctional Group Leading the Way In Performance Measurement

An excellent example of a national group attempting to standardize measures across an industry is the Association of State Correctional Administrators. They have an ambitious project titled the Performance Based Measures system that seeks to standardize measures within corrections through strict definitions and counting measures for actions such as assaults, accidents, health-related measures and many others. Information on the project can be found here.

I was fortunate to attend a training session for the program and I have to say that it was really impressive. The counting rules and definitions manual is thorough and fairly clear (and as thick as a dictionary) and can be found here. They even have a web-based tool for jurisdictions to enter their data. The biggest problem that I could tell from their efforts was the general lack of sophistication of a number of state and local corrections agencies. Several of the jurisdictions I talked to simply didn't have the data to fill out the requirements of the PBMS system.

Still, this could be a great starting point for other areas of government such as health care, human services, parks and recreation, etc. to set up their own set of performance measures. It will not be easy, though. The PBMS system was started in 2001 and took more than 6 years to finalize and get a production application running.

For public data on performance metrics and management, check out these cities:

Washington DC: http://capstat.oca.dc.gov/

Baltimore: http://www.ci.baltimore.md.us/government/citistat/

New York City:

http://www.nyc.gov/html/nypd/downloads/pdf/crime_statistics/cscity.pdf