Measure Behavior - Measure Success!

Authors : David A. Army, CMRP, Strategic Asset Management Inc. and Gwendolyn Army, LCPC, Foresight Consulting

The Perspective

Many times, maintenance professionals have been involved in changes to processes or systems where they have relied on seat of the pants knowledge to determine whether or not they were successful. More often than not, the initiative flounders once their attention has been turned to other endeavors.

There has been a great deal of debate over the years revolving around how maintenance or, more importantly, maintenance processes should and should not be measured. The theories vary far and wide, but in essence, most of the debate is centered on how to measure both the current status and ultimately, the end result. Should it be measured on maintenance cost per unit output, on cost as a percent of asset replacement value, on equipment uptime, etcetera, and etcetera. The debate continues to rage, but we contend that the debate is focused on the wrong timeframe and perhaps the wrong framework. All of these measurements, as stated, concern themselves with “outcomes”, or after the fact measurements of changing variables.
Times have changed significantly and there are three primary reasons why new performance measures are required:

Traditional accounting and measures are no longer relevant to a company moving toward a world class operating environment, although they portray a certain reality
Customers are requiring higher standards; competition has increased, which in turn requires metrics that relate how well the organization is meeting those standards in relation to the competition
Management techniques, technology and reporting mechanisms used in plants have changed significantly
Behavior change is now recognized as a key contributor to the success of any process initiative

Leading vs. Lagging Indicators

The most compelling reason for the need for a change in approach is that the measures stated previously are based on lagging or outcome indicators. These are after the fact results, unchangeable once the time period of measurement has been completed. Many decisions are now pushed down to the shop floor, and for those individuals, and for this level of focus; we find the old high level outcome measures inadequate. We desire measures that are meaningful to the entire organizational hierarchy. We can then use those measures, and others to monitor and promote particular behaviors by our employees. Using outcome indicators is like looking out the back window of a car to see where you’ve gone.
Today’s environment requires measurements that can predict, determine, and influence desired outcomes. We need to be able to affect the final outcomes for whatever period we are measuring by developing and monitoring interim indicators. We would like to use an analogy. Measuring maintenance is like investing in the stock market. Our investments should be geared to high value returns that are predictable. A common strategy is to look at leading “market” indicators to judge how well the investments are going to payback, which is the lagging indicator

If the leading indicators are “Bearish”, you have time to correct your actions
If the leading indicators are “Bullish”, you know that your efforts (investments) are going to produce the required return

Managing the trends of the leading indicators is our view of successfully managing your investment in maintenance. So the premise is to measure both leading and lagging indicators; but it must be done in some context, some overall process that integrates tactical with your strategic direction. We call such a process the “Managing System.” This forms the basis for integrating people and processes within a common framework. The “Managing System” is the umbrella process used to guide the organization on a day-to-day basis and is considered a foundational process. The managing system integrates the overall operational strategy for the assets to a series of cascaded goals and objectives that are linked down through the organization. It is then used to establish the measurements of these goals at each level, and the key process indicators needed to ensure that the processes remain healthy. These measures are a mixed set of the proper leading and lagging indicators.

This is the mechanics of setting the system up; the key is in utilizing the system. The system is used as the primary vehicle to review how well we are performing against what we said we would do. It’s the “Plan, Do, Review, Act” model. To make this work, it should be used at all levels of the organization, have the reviews on a scheduled frequency, publish the results of the measurements, and hold people accountable for the end results. By using a mix of leading and lagging indicators, as we review the leading indicators on a weekly basis, we have the ability and time to correct deviations from expectations by the time we review the lagging indicators at the end of the month. As we assess industrial operations, we see a semblance of this system in place, but it’s often disjointed and based on lagging indicators that do not tie the strategic direction to tactics used at the floor level. In most cases, the measurements used are wrong and don’t relate to the behaviors that you want to produce; so it doesn’t provide a vehicle for change management.

It’s important to measure, we all know that, but it’s imperative to measure the right things! Maintenance measurements are a part of a global set of indicators that gauge your facilities viability. So it is important to ensure that we measure the right sets of information.

What’s Missing?

The fact is lagging measures are reflective end results of what people are doing at the front end of the process. For example, there really is no such thing as wrench time without acknowledging that people are turning the wrenches. Time has to do with people planning to be at a certain location, at a specific time and having the necessary tools and materials. In order to have the tools and materials, someone has to identify and order these ahead of time. Someone else has to make the equipment available. Only then can we complete the work and increase the amount of time spent turning the wrench. Planning, scheduling, making equipment available, showing up when requested, and performing that work in a timely manner are all behaviors.

Most of us don’t tend to think of metrics this way. In fact, the way we label results takes the people out of the equation. Well, with one major exception, reprimands upon failure to meet targets. For us to truly manage a process we have to place people and their behavior back into the equation. We must conceptualize a managing system that includes people’s behavior.

Are Performance Indicators the Answer – Why Not?

If indicators were taken and used in the spirit for which they were created, would be one thing. However, human beings have a natural aversion to being measured and held accountable for performance. And therein lies the rub. No matter how well intentioned the indicators that we’ve set up are, the people on the shop floor will find a way to circumvent them. In fact, we have found clients that we’ve worked with to be quite ingenious when it comes to finding ways to “beat the system.” So while these indicators track the apparent success of the process, they don’t tell the whole story. The key to achieving results and sustaining the process is to combine process indicators with behavioral indicators.

We have found it extremely beneficial to focus on behaviors as part of any initiative. In the past we used to talk about “Best Practices” we now talk specifically about “Best Behaviors.” With any initiative, we often spend a great deal of time identifying those indicators that will give us reassurance that the process is working. If we’re really on the ball, we’ll not only develop results (or lagging) indicators, but often process (leading) indicators. Tied into other systems these provide quantitative evidence of success or lack thereof. What happens when the pressure is reduced i.e., consultants or external help goes away? Often organizations revert to the old and comfortable ways, or we find that the quantitative evidence has been creatively dealt with and results aren’t what we think or wish.

We have discovered that it is not enough to just manage the numbers. What is of the most value is that along with developing the process is to develop a list of behaviors we want the organization to exhibit. Then develop behavioral metrics that are aligned with the desired behaviors. After process installation, or hard wiring, we then program the organization by coaching and facilitating to those desired behaviors and then provide qualitative measures.

How Do You Establish Best Behaviors?

The human psyche is broken down into three main elements for the sake of Change Management. Changing beliefs, knowledge and vision is the intellectual or cognitive component. Changing what is done, how it is done, and what is gained is the action component. How we respond to the success, failure or stress of the endeavor is the emotional aspect and not one to be ignored. All three elements are interdependent.

Assessment of current behaviors and beliefs is important to establish when doing baseline “as is” metrics and indicators. An example of this is the typical belief that “we are heroes if we drop everything to correct breakdowns.” In a reactive plant environment this is the norm. There is a rush or sense of pride and accomplishment. “See how quickly we responded and got production back on line.” This belief is often reinforced by promotions and pats on the back.

In most Work Management process improvements, our desire is to change the reactive belief to one that stresses zero breakdowns and planned maintenance. This leads to more profitability for the company, paying off for the individual by maintaining employment, providing a different level of satisfaction, and removing the chaos from the day. The new belief is one that states, responding to breakdowns means that the process has failed, and that if not corrected could lead to the demise of the company.

Interviewing all levels of the workforce to find their beliefs and how they go about their jobs is important for establishing baselines. This is used to identify how the organization has moved, once the process improvement begins. It can also be used to establish scorecard “red light” behaviors.

It is extremely important that prior to commencing any installation or implementation activities that the new desired behaviors are identified. Can you think of some? Well, here are a few:

People attend planning meetings and are prepared to make decisions
A craftsperson knows what he or she will be working on during the next week or day
A craftsperson is confident that he or she will be allowed to perform that identified work
An operator knows what equipment will be taken out of service tomorrow
A planner understands the importance of clear and concise work instructions
Work is not permitted to begin until the parts are available
Feedback is provided on Preventive Maintenance activities

And the list goes on.

Determining the desired proactive behaviors and beliefs becomes the basis for the behavioral. Observation and self reports of sometimes doing things the “new way” but occasionally reverting to the old behaviors are yellow light conditions. It is important to praise this transition phase and refrain from focusing on “not good or fast enough” to encourage continuation to green light behaviors and beliefs. Reward and reinforcement create the desired behavioral change. Punishment only causes resentment and resistive behavior.

Why Behavior Focus is the Key

Modified and new behaviors are the key to any change initiative. Towards the end of a recent engagement, we were asked by the client for a way to measure the success of the engagement. The immediate answer was, “We have met all of the targets.” But we were then challenged to assure him that once left alone, the process would continue and not revert to the past, what we refer to as sustainability. This caused us to pause for a moment and think about what would be a prime indicator of continued progress. Well, we had the metrics and the behavioral tool, but they were measured differently. Metrics were measured quantitatively while the behaviors were measured qualitatively. In the end we came up with the idea that a combination of both would provide an ideal vehicle to Certify the organization “competent”, or “sustaining” for the process that was being modified. We would measure them on two axes; performance and behavior. Using a normalized scoring methodology we were then able to rate the organization on a 1 to 15 scale.

At the conclusion of an engagement we can certify an organization to both the performance metrics and the behavioral metrics. One cannot achieve sustainability without strong evidence of the presence of both elements. Simply set forth, we measure the client against normalized assessment points for both performance and behavior. We then grade or graph against two axes and if the client passes a set point that was initially agreed upon by the client then we can declare the organization at either a competent, sustainable, or high performing category (Figure 1). It isn’t quite important at this point that we worry about the three grades. What is important is that the questions are dependent upon the process chosen, and are used to evaluate performance (quantitative) and behaviors (qualitative). They can be used at the front end for baseline determination and at the back end for a measure of organization movement.

Behavior-Performance Matrix

Figure 1: Behavior/Performance Matrix

Without going into the boring details, the Certification tool has ended up being a tremendous Change management tool. In one organization, the leading business unit demanded to be “Certified.” All of the scorecards were green and all of the behavioral metrics (as measured by the business unit, not independently) were on track (meeting all established targets for performance). For the certification activity we used business unit personnel and consultants. The consultants took the lead position and lo and behold, the business unit, much to everyone’s chagrin, did not pass. Why? It was the fact that the observed behaviors of the individuals at both craft and management levels did not support the process or engage in the targeted new behaviors. After much soul searching, the manager in charge participated in determining the path forward and corrective actions. Three months later they tried it again, and passed. What we discovered was the Certification process became a tremendous focusing tool for the organization. It made them go back and reflect on those desired behaviors and galvanized them to get it right the next time.

Thus, from outward appearances, one might assume that the process is firmly in place and sustaining. This is especially true with early performance gains. However, without these requisite behaviors, starting at the very highest levels of the organization, firmly in place, one can expect a return to the status quo, once the training wheels have been removed.