Email: September 10, 2009 7:20 AM
I found it an immensely useful seminar and I left feeling most excited about RBA and its potential in a NZ social service context. I have had the privilege of presenting it a number of times. But … alas … I seem to have encountered a bit of a hitch. My colleagues and I can’t agree on how to make the most of some of the quadrants, and with some of the things we have presented, we have confused those we are supposed to be training. Can you help please? I’ve had a look at the RA Guide, and it seems to support one position, but we can’t understand how it actually fits. We are looking at the Quadrant model, particularly the Upper Right Quadrant. The fact that it is an upper quadrant means that it is concerned with our effort, the things that we do. So if, in the Upper Left Quadrant we have among our activities listed there the fact that we are delivering services to 12 families, we could put in the Upper Right Quadrant information about the fact that our staff have relevant qualifications, are up-to-date with their service standards, deliver services in a timely manner, etc. All these things give us an indication of the quality of our effort. It tells us that we have gone to the most appropriate lengths to get the job done to best effect. But what about survey forms or questionnaires about the quality of our service? These do not appear to be things that relate to what we do, but what our customers feel about what we have done. Should these go in the Lower Quadrants? And if we were running courses, would the details around the number of people completing the course go in the Upper Right or the Lower Right? We can’t agree. Some of us say it should go in the Upper Right as the fact that X customers completed the course tell us how well we did the work. However, others among us are saying that the number of people completing the course is an effect and should go in the Lower Quadrants–it relates more to the effect of our actions than the effort of our actions. IN the RAG, section 3.8, number (5)(c), we are having difficulty understanding why a “circumstance where numbers get better, but customers are not better off” would go in the Upper Right Quadrant? Wouldn’t this still go in the Lower right, but indicate the level to which someone is better off? It just doesn’t seem to fit in an “effort” quadrant. Sorry for the lengthy pleas for help, but can you be of assistance? Regards,David Booth Regional Relationship Manager Family and Community Services (FACS) Ministry of Social Development
The upper right quadrant “How well did we do it” category can contain measures about customers. Consider the measures % of classes delivered on time and % of customers who say classes were delivered on time. They are almost identical measures. Customer data can often tell us how well services were delivered (How well did we do it?). The fundamental question to ask when thinking about whether a customer measure is upper or lower right quadrant is this: Could we do well on this measure and customers might still not be better off? For example, the % of customers who think the class is delivered on time is 95%. But the class itself is terrible and no one learns anything.
This same principle is illustrated by how we categorize the completion rate for a service or training course. The mere completion of a service or training course usually tells us little or nothing about whether people got anything positive out of it. So % of clients completing service or training is usually an upper right quadrant measure. But there are exceptions to this rule. If the training has an established reputation for giving its students new skills, then completion can be a lower right quadrant measure. Life saving courses, where earning a certificate really means something, is a good example.
This principle can be carried further to the general question of how to classify different measures of customer satisfaction. Customer satisfaction measures fall into two broad categories: Did we treat you well? and Did we help you with your problems? Customers can say they were treated well, that they liked the counselor, thought the building was comfortable, accessible etc. but still not be helped with their problems. So “Did we treat you well” is an upper right measure and “Did we help you with your problems” is lower right.
The deepest underlying principle here is that measures are classified not on the basis of some intrinsic characteristic of the measure, but rather on the basis of what question the measure helps to answer. If measures help answer How well did we do it? they go in the upper right quadrant. If measures help answer Is anyone better off? they in the lower right. If they could conceivably answer both questions, then pick the stronger answer, or as a last resort put it in both places. This same principle applies to the difference between population indicators and performance measures. Sometimes a given measure can play both roles, so that sometimes it serves as a performance measure, and sometimes as an indicator.
I would encourage you to read Chapter 4 (especially pages 65 to 79 and 99) of “Trying Hard Is Not Good Enough” where these matters are discussed. There can sometimes be legitimate differences of opinion about how to categorize a measure. When this happens, just put is somewhere and move on. Because all the measures in both upper right and lower right quadrants will be considered in Step 4 of the 5 step process for identifying the most important performance measures (See Appendix G).
Finally, the reference to “circumstance” is specifically about the client or customer’s circumstance. Are they in stable housing?. Do they have a living wage job? It is not about a more general definition of circumstances. For this reason it always goes in the lower right quadrant?
