Dashboards for Public and Private Sector Use

Dashboards seem to be all the rage these days and it's easy to understand why. In a world where instant and voluminous information is the standard, data dashboards allow users to take in a large amount of data in a short amount of time. My own company recently developed a new Dashboard tool because that seems to be where basic data analysis is headed. But a quick Google search on "data dashboards" reveals a myriad of tools and articles for various dashboard solutions. One question I'd like to pose is what makes for a good data dashboard and how can that be used in the context of an overall performance management program? Can dashboards be used to better inform an agency on performance? I'll follow up with a few postings on this topic but I look forward to any comments.

Measurement and Managing

You can't manage what you don't measure! If an organization believes that adage and decides to measure, it must also leverage software to increase its chances to manage successfully. I recommend that organizations take a three-pronged approach to ensure that they are not only measuring, but measuring with confidence!

The first ‘prong’ is what I call, “Identification”. What is being identified? What the organization should be measuring. It should measure those activities or results that are important to successfully achieving its goals. Once these are identified, they will be defined and measured to ensure that progress is made toward the organization’s goals.

Software does not currently offer much help in identifying what needs to be measured, but it does provide a mechanism to capture and document the metrics that will be measured. It also provides a central repository for this information as well as a search engine for locating them.

The second ‘prong’ is collection. This is the entering of raw data that happens at a defined interval: Weekly, Monthly, Yearly, etc. The data is typically entered by organization workers and prone to error. Workers can also enter descriptions of missed targets as well as narratives that provide insight into improvements.

Software offers a lot of help here. It can provide a clean web based interface to allow the organization workers to enter the data. It can offer a myriad of validations to reduce error rates. It can even import the data from existing systems so that data entry is minimized and/or eliminated.

The final ‘prong’ is where transparency is realized and provided to the organization’s constituents. I call it, “Dissemination”. There is an expectation that the data be tallied and distributed to interested parties allowing them to have a more complete picture of the activities and performance of the organization.

This is where the right software really shines! It can provide visually appealing, clear, informative reports that allow user interaction, filtering, sorting, exporting, etc. Since a picture is worth a thousand words, it can also provide graphical representations of the data so that trends can easily be spotted. Management can use these visual outcomes to assess performance and set goals for improvement.

Without the data gathered from performance measures—particularly those tied to the organization’s goals—it can be difficult to judge if an organization is being managed efficiently. Organizations should use software to take a three-pronged approach to measurement: Identification, Collection, and Dissemination. When they do, they will be well on their way to measuring with confidence!

Performance Management in Africa

I recently came across a seedling of performance management in Africa of all places through the Africa Public Service Performance Monitoring and Evaluation Forum. The only presence that they currently have is on Facebook but I encourage anyone interested to follow them online and assist in informing the discussion. It would be interesting if Africa became a hotbed of performance management discussion.

Software as a Service (SaaS) for Performance Management - Part 1

This is the first of a two-part post about Software as a Service (SaaS) for performance management. In the first part I'll provide an introduction to SaaS, define what it is and touch on some of its applications to performance management. In the second part I'll detail the various benefits of SaaS including but not limited to cost-effectiveness, reliability, and ease of deployment.

The race is on for Government to focus on results, but the race is long and difficult because the path is rot with challenges. New technologies promise transformation of performance management processes, a great opportunity, indeed, but often at a high cost with a long implementation timeframe. SaaS solutions offer a cost effective way to simpler performance management implementation.

Municipalities, large and small, need to discover Software as a Service (SaaS) — referred to by some as applications that live in “the cloud”, Web-based, online, or on-demand as a way to save time and money without sacrificing the basic tenants of performance management: Results, Relevancy, Transparency, Timeliness, and Accuracy. SaaS also brings applications under a single platform, making it easier to implement performance management programs as well as an improved ability to benchmark across jurisdictions. SaaS is a technology trend that promises to provide a simpler solution for implementing performance management programs.

SaaS Defined

SaaS, on-demand, and cloud computing have become common terms in the technology world. In one way or another, they refer to the growing trend of software vendors providing their applications over the Web as a service, rather than as a set of code to install on a local server or desktop computer. Some actually provide both solutions. With SaaS, customers tap into one code base that is refined and enhanced (producing better results for the users) based on feedback from all users. The whole is indeed greater than the sum of its parts. Organizations and/or individuals subscribe to the service and access it using a computer, laptop or even their mobile phone. These applications are known as on-demand, Cloud-based, Web-based, or online. Cloud computing is used by many SaaS providers and refers to massive server farms that host applications online for many customers. Cloud computing enables flexible processing power and storage capacity to scale up or down, depending on actual usage.

Almost everyone who uses the Internet today has used SaaS. Email programs such as Gmail, Hotmail, and Yahoo! Mail are examples of SaaS. What is really nice is that there is nothing to download or install, users simply create an account and log in through a Web browser. Popular social networking sites such as LinkedIn, Facebook, and Twitter operate using a SaaS model.

SaaS as a Government Performance Management Tool

SaaS is ideal for government seeking technology solutions to help them implement performance management programs. Because these programs often encompass many disparate agencies/divisions that can geographically span hundreds or thousands of miles, it is useful to have a centralized, common technology platform that many users can share. In the event that some users are outside of your network, SaaS allows users to access your performance management data and application from anywhere. SaaS offers both simplicity and a cost-effective solution for government users.

The Future of Performance Management

I was at an AGA breakfast a few months ago and the point was raised that people in government had been talking about performance management for "some time", with few results. It got me to thinking abut whether there is a future in performance management and what it might look like. A couple of publications I recently read struck me as two different thoughts on where we're headed in this field, one being an "academic" piece on performance management and the other more grounded in practical approaches. The first was a point/counterpoint look at the history of performance management (largely using examples from New York city) and how it might inform better practices in municipal management around the country. The counterpoint in the article focuses on some of the failures at the Federal level in creating a useful performance measurement framework. The authors, Dennis Smith and Beryl Radin, have a largely academic debate which at times gets muddled in teasing out the nuances of performance "management" vs. "measurement". But generally speaking it's a good article with a few highlights that I took from it:

1. Cities like New York took decades getting to an effective performance management program so we shouldn't be too hard on cities that don't get it right away.
2. In the case of New York, moving from an input/output focused metrics to outcome based measures seem to be the turning point where measuring performance becomes effective.
3. Local efforts seem to have been more effective than national efforts in this space.

There's a lot more to the article and it must be purchased from JPAM, but it's an excellent piece.

The other is a series published by the Urban Institute called "Legislating for Results" and offers a framework not only for performance management within individual agencies, but also a more broad plan for managing an entire jurisdiction. What I liked about the series is that it's a practical, step-by-step guide to getting results from government (in contrast to more academic approaches in other pieces). One of the nuances that Urban employs in the series is using the term "information" synonymously with metrics/measures. In fact, what we're looking for from performance measures is actually nothing more than information, and the Urban plan emphasizes providing quality information to government decisions makers in the hope of improving outcomes. My main problem with the piece is that there are many moving parts and it incorporates budgeting, communicating to the media, etc. into the framework. As a whole the series is a little too much, but I see most of the value from the three pieces on getting the right information, getting quality information, and using that information for planning purposes.

From the two pieces I got that there are clearly failures in performance management, but also some successes. Additionally, even though some academics have been talking about performance measures and management for "some time", the approaches are under constant revision and dare I say, improving. The important thing is to continue to push for jurisdictions to use performance measures, and to assist those with existing performance management programs to constantly improve them.

Where Are the Performance Metrics in the Recovery Act?

About a month and a half ago, OMB released the Federal reporting requirements for the American Recovery and Reinvestment Act. It's taken me a little while to look through the requirements, and I had read that the focus was on job creation, but I was still a little surprised at the lack of Federal interest in other performance metrics. It pretty much boiled down to "# of jobs created" and a fairly standard set of financial data points entailing how the funds were spent. The Federal government has always been a laggard in requiring its own agencies to adopt strong performance management practices, but it didn't seem out of the question that in doling out hundreds of billions of dollars they would put together a few other metrics to make jurisdictions more accountable. At the very least it would have gotten towns, cities, and counties in the habit of reporting performance data, and possibly start some of them down the road of a more substantial performance management program. Lets hope that future rounds of government reporting requirements contain a more diverse set of measures, or at least let jurisdictions name their own. It's one step toward a more ingrained culture of government performance management.

Using Performance Metrics to Manage

In the final installment on a performance management framework, we'll look at using performance metrics and analysis in order to effectively manage agencies and their programs by using remediation and corrective actions. We've already covered the first and second steps, and will focus on the final two in this posting (all four are listed just below):

1. Report performance metric data on pre-defined schedule.

2. Analyze data for troubling trends or missed targets. Operationally research root cause(s) of problems.

3. Provide corrective action for metrics where target was missed or data is trending in wrong direction.

4. Repeat process for next reporting period.

Assuming that reliable metrics have been gathered and reported, and that data trends have been analyzed, an agency should have a good idea about where it stands operationally. The question then becomes how best to use the new information. For example, if I'm an FEMS agency that knows my emergency response times are trending in the wrong direction, and I know that the problem lies somewhere in my call center, what's the next step? (I'll answer this in a minute.) Given the diversity of agency missions that exist within any government, it would be impossible to give specific guidelines on how to fix troubling trends. For the purpose of our framework, however, the important thing is that the information is used to formulate some plan of action, and that the plan of action is clear, has timelines, and is documented for future consideration. Maintaining documentation of attempted corrective actions can be particularly helpful when there are several options for remediation. Each option can be tried over a given reporting period and performance data can be tracked. If there is some improvement in the numbers, the corrective action was likely effective; if there is little or no improvement according to the data, then another option on the list may be your best bet. The important thing in documenting the remediation is not to spin your wheels by proposing the same corrective action repeatedly and expecting a different outcome with each successive attempt.

Going back to our emergency response example in which we assume that the call center has been identified as the source for deteriorating response times, there may be multiple options to improve performance, including additional training, process re-engineering, etc. There may not be an obvious "best" remedial option, but the important thing is to pick one and continue to track response times. If additional training was implemented but the trend is not reversed in response times, then lack of training can be eliminated as both the cause of the problem as well as a corrective action. Continue the cycle of capturing and reporting the metrics, but with a different corrective action this time. Perhaps the response process is streamlined or adjusted and overall times improve. We then have some indication that our proposed solution had a positive effect on the operations that we are tracking. Through trial and error in the corrective action process, while concurrently continuing to track and report data, any agency can improve effectiveness in its operations.

The important takeaway from this exercise is that in order to demonstrate marked improvement in any public sector operation or program, all steps in the framework that we've outlined here (and in past postings) must be followed. Tracking and reporting metrics without proposing and documenting remediation in trouble spots won't bring about the change in negative outcomes that most agencies are seeking. The feedback loop of track-report-remediate-repeat is the fundamental process behind our performance management framework, and is essential in solving government inefficiencies.

Will new Federal CIO, CTO Change Performance

There are several new faces in DC and it will be interesting to see what their impact on Federal performance management will be. The previous announcement of the Vivek Kundra as the nation's CIO was followed up with this week's announcement that Aneesh Chopra will be the nation's CTO and Jeffrey Zients will be the nation's Chief Performance Officer. This is a high-powered team of professionals that has worked in the city, state, and private sector and have been tasked to work together on bringing accountability back to the Federal government. My hope is that they will take a performance metric approach to accountability and ensure that agencies in the Federal government have a comprehensive performance plan which will be used to analyze effectiveness. I will post articles and comments on their progress moving forward.

Comparison of Federal Performance Plans: HHS vs. DOJ

Browsing through several different Federal agency performance reports online there are some clear leaders in terms of quality of measures and others that have some challenges. In municipal performance reporting, police departments have often led the way in statistical performance data, largely because crime information is well-defined and one of the most monitored by citizens. Social service agencies typically lag behind their public safety counterparts with respect to municipal performance metrics. At the Federal level, it appears that the opposite is true.

The HHS Administration for Children and Families website has the agency performance reports going back to 2000. A look at the HHS ACF 2008 performance report shows some decent measures (many are outcome-oriented, a break from the output-focused measures of many agencies) that have been created by ACF as well as what looks like honest reporting of those measures. I assume honest reporting based on the fact that the agency has met less of its targets over the last several years. The measures appear well thought out and given the honest reporting of missed targets, the data appears to be reliable. Additionally, the reports get shorter each year since 2000. That's probably a good thing with respect to the public actually reading the document, and the agency focusing more intensely an a narrow set of measures and goals. If you can't explain a measure on a cocktail napkin, it's less likely to be reported and recorded accurately over time.

On the other side of the public spectrum is the DOJ Performance and Accountability Report for 2008. The report is a combination of both performance and financial data and comes just under the whopping-300 page mark. The parts that focus on performance metrics are lacking in a number of ways. Most are output-oriented and don't inform management or the public as to the effectiveness of the agency (One of the measures, "Terrorist Acts Committed by Foreign Nationals Against U.S. Interests", is zero in most years with 2001 the notable exception and would almost certainly be known without needing to be included in the performance report. The measure is neither informative nor a helpful management tool). While the municipal law enforcement agencies have made great strides in performance reporting, the Federal level agencies seem to just be getting their feet under them. One reason may be because of the difficulty in attributing crime and arrest rates to a Federal agency whose jurisdiction is the entire country. Most responsibilities of the agency are shared with state and local jurisdictions, yet the DOJ has little or no control over those agencies. Regardless, there are almost certainly some performance measures that the DOJ could come up with that are more worthwhile. If not, how can their effectiveness truly be measured? A good start would be to narrow the report into something more digestible that might actually be used in agency management.