Unusual Subroutines

The blog and musings of Christopher Allen-Poole

How not to get useful information… an unfortunately useless tool for evaluating people

There is this nasty tendency to believe that sliders are good for, well, anything when it comes to getting user feedback. In order to disaffect you of this notion, I submit the following: two-dimensional graph

These don't even have questions associated. They're just under the title 2D self-assessment by {{name}}. Theoretically they are supposed to represent:

  • Rate your behavior.
  • Rate your results

I've taken the liberty of superimposing several of these charts on top of each other. Now, my basic question: what's the difference between the different balls? If this is a scale between 1-7, which one represents 6.7 x 6.8 and which one represents 6.8 x 6.6? Is it possible to determine which one is which? How do I know that someone didn't slip with the mouse and move .3 either way?

Now, before someone says "this chart isn't designed for that" or make the argument, "You can't boil this down to that many significant digits" my answer is that you can't boil this down to ANY number of significant digits. It is a completely context free diagram and so provides no useful information. I can't imagine what Tufte would say about something so utterly useless.

A ratings scale with more than seven distinct steps starts becoming useless. This is why these should be in sets of 1-5's. That also moves the content into something which can easily be broken down into a set of verbal equivalencies. For example:

Rate your performance:

  • Very poorly
  • Poorly
  • Average
  • Well
  • Very well

If I'd like, I could even do something crazy like define "Very well" as "single-handedly saved the company a billion dollars while teaching orphans about the dangers of tooth decay." That's a ridiculous standard, but it is something anyone should be able to look at and say, "I did that. I did that while playing a virtuosic piano recital with four broken fingers" or "I didn't do anything even close to that."

One of the chief points of reviews is to allow managers and the people who report to them the opportunity to sit together and ask what the employee's done well, what he's done badly, and how to improve. And, to be honest, I have no idea what 5 mm. means. The entire thing means, "I did well, but could to better," which is true for all but messianic level employees.

And at the end of the day I, as a manager, am looking for growth, and I'm looking to see how I can help people grow. That is my job. Depending on who you ask, that job is t least as important as being able to provide people with excellent software in my own right. A simple five-part question means that I can accomplish that goal. A vague 2D chart which is drag-and-drop does not let me do that. Especially since EVERYONE who reports to me has a bullet above. Heck, I even have a bullet somewhere in that vicinity when I filled out my own self-assessment.

So… what do I make of the above? A waste of all of our time. It makes me sad.

blog comments powered by Disqus