Wednesday, March 31

My rating system.

I have a rating system I've come to use when discussing just about anything worth rating. It went through many iterations before I settled on my current, simple one.

It may seem pretty harsh to some people, but the world has become used to rating everything they like "5 stars" and stuff they don't as "1 star," with the middle ratings seldom used. This is unfortunate, as "5 stars" now tells me almost nothing about a book, CD, or film except that to that person, it didn't suck. And if something gets 3 or 4 stars, instead of thinking, "ah! better than average!" I now think, "what's wrong with it?"

But let's back up. First, in any rating system, there's the question of granularity. Just how many gradations are needed to rate something? Well, that's a subject that deserves a treatment all its own, but in general, I believe five is enough. There are situation where more gradations, say ten, would avoid the dreaded "three and a half" syndrome, but as a rule I don't think human beings are very adept at placing things into ten boxes meaningfully (never mind more). Five seems convenient - it's the number of fingers on the hand you're not holding your baby, cat, or miniature dog in, and it's easily grasped by the mind (who psychologists say can store roughly five items in its cache, otherwise known as instant or ultra-short-term memory, like being able to identify groups of objects immediately by their shape; think of small group of stones or blocks... it's easy to tell how many are in a group without counting, up to five, at which point it gets much harder or for some people, impossible (again, without counting)). Five also provides what is in my mind the minimum number of gradations necessary for any sort of meaningful rating. Two (good, bad) is too rudimentary, as is three. Four is just weird, to me. There should be some central point (this may be one of the few occasions in which I favor odd numbers over even). And as I described, after five I believe you start to get too much complexity and diminishing returns. What exactly is the subjective difference between a book that earns a 7/10 versus one which rates 8/10? That's too granular.

So, we've settled on five. Now I want to do my part to re-normalize the whole endeavor and get 3 back to being the true middle, and push everything down a bit so that anything less than 5 doesn't automatically connote suckage, and so that all five numbers have a distinct meaning. Particularly, I want to reserve 5 for things that are truly great, not just things "I like quite a lot." Maybe 4 can be the new 5. It's a bit like the Michelin star system, only not THAT harsh. If you are a restaurant or hotel and you get any Michelin stars, you're pretty awesome. Three and you're world class. My system won't be like that any more than it will be like the Amazon system, in which the overall aggregate rating for nearly every product with a sufficient number of reviews is 4.

Without further ado, here's what I feel is a logical system. Think of stars, bullet points, numbers, tiny clown heads, or whatever you want - I'll just use numbers to make it easier.

1 - Skip it
2 - Mediocre
3 - Average
4 - Good
5 - Excellent

Though, I've gone a step harsher at times (just to be a contrarian) and leaned more toward this:

1 - Bad or below average
2 - Average; mediocre
3 - Good; worthwhile
4 - Very good to excellent
5 - An all-time classic

The problem with this system is that it makes 2 average, which is weird. It does have one good thing going for it though: it reserves 5 for the truly great things; an honor which should only be handed out very occasionally (like for The Empire Strikes Back or Hamlet). You ever see those reviews on 'zon where people say "I wish I could give this SIX stars! It's changed my life!" Well, usually they're wrong, and while I don't recommend a six-star system for Amazon (it would take less than a week before every started using it casually, and if a prospective buyer didn't see lots of 6-star ratings, they'd think "What's wrong with it?"), I do however think there is some merit in a "transcendent score," if you will, a holy 6th star that must not be given out unless the item being reviewed is truly one for the ages (like Krull), and not just for that year, or that week.

So I've come to amend my system to this final, spectacularly clever one, which I will now use in all future blogs and any reviews I may write up:

0 - One of the worst things ever
1 - Awful; Skip It, Please
2 - Very Weak to Below Average
3 - Average Fare; Par For the Course
4 - Above Average to Very Good
5 - Excellent; Get/Read/See/Hear It!
6 - An all-time classic

For most items, you can pretend the 0th and 6th "star" don't even exist. Getting a rating of 5 is NOT a slight at all, nor is 1 to be considered a step above anything - it's still awful. That's why I don't want to call my system a "7-star rating system." Because really, it's not. The frequency distribution for say, a thousand items reviewed at random ideally should describe a bell curve with the highest level at 3, lower levels at 2 and 4, and an even smaller but still decent number of items scoring a 1 or 5. On the other hand, 0 and 6 should only have been assigned to a few items each, if that (below what a true bell curve would predict).

Basically I want to make room for Tolkien to surpass other 5-star books I love and for things that have actually shaken me with how horrific they are (like anything by L. Ron Hubbard) to be tagged somehow below the ordinary turkeys we come across and just stop reading or playing, or walk out on. The 0 and 6 ratings give me a special way to do this. If I use symbols, I may even use a completely different symbol for them. Imagine one through five bullet points for the regular scale; a 0 may be represented by a hollow-point (white like a O rather than filled in with black), and a 6 may get a little circled 'C' for "classic" (something like a copyright symbol, but obviously not exactly like one or I may confuse people).

So in essence it's still a 5-star system but with the seldom-to-be-used option of a pitchfork and a harp on either end for when the thing being reviewed just begs for it and the usual system simply won't do. Actually, I kinda like the pitchfork and harp idea :)

What do my thousands of readers think? Is this a good system? Makes sense? Too much ado about nothing? I'd be interested in your comments, as always. Even suggestions for symbols!


billybytedoc said...

Very good discussion, and a needed one. I don't think you need the "Zero" because I ( and I think most ) wouldn't go for something rated 1 or 2. I do like the 6 or "C". But as you say, "C" must be used rarely!!

Now good luck getting people to use it!

Metamatician said...

Thanks for the encouragement!

Soul Survivor said...

I like it. The return of the exclusivity of 5 stars is exciting to me.

Metamatician said...


Hans said...

I like the idea of symbols, especially for the worst and very best. Without getting gross - maybe a bulldozer or dunce hat for the worst ever and a medal or shooting star for the very best.

Metamatician said...

Good ideas! I like the shooting star.

Archived Posts

Search The Meta-Plane