For those unfamiliar with the workings of the contest, each book is scored by five judges who rate it from 1 to 9, with 9 being a perfect score and 1 meaning (not that RWA actually says this in its judging guidelines) that the novel was written by a moron, edited by a chimpanzee, and printed by an inebriated individual on a broken press that left out every third page. Books that score in the top ten percent of their categories--up to a maximum of eight books in each category--move on to the final round of judging.
Yesterday I received my RITA scores in the mail. Here's how the five judges scored my book (bear with me--this will get really interesting in a minute):
8
9
9
8.8
6.5
As you can see, my book made a good showing. Three judges thought it was pretty darn amazing, one it was fairly amazing, and one thought it was better than average. Because there were eight finalists in my category (Inspirational Romance), we know there were at least 80 books entered. According to my score sheet, the distribution of total scores for the books entered in my category was as follows:
Lower half (about 40 entrants): total score of 35.7 or below.
Second quarter (about 20 entrants): 35.8 - 38.6.
Top quarter (about 12 entrants plus the 8 who finaled): 38.7 and higher.
My final score was 41.3. RWA doesn't reveal what I'd have needed to displace the lowest-scoring finalist, but if the judge who awarded my book 6.5 points had instead given it a 1 or a 2 (or possibly even a 3--I stink at probability and statistics), there's an excellent chance I would now be one of the happy eight.
See? I told you this was going to get interesting.
Here's what the RITA score sheet says:
RWA applies a standard deviation method for determining finalists, which means that if the lowest score for this work was found to be outside the range limits, the lowest score was replaced by the average score to determine the final score for this manuscript.
Let's imagine that my 6.5 was a 1 or a 2. Because that score would be found outside the range limits, it would be tossed out and the other scores would be averaged. The low score would be then be replaced by a tidy 8.7, bringing my total score up to 43.5--which, again, might very well have tossed me into the finalists' circle.
I've been thinking about this since exchanging e-mails yesterday with an author friend who mentioned that one RITA judge had given her book a 1. Because my friend's other scores were much higher, the 1 was thrown out and replaced, as described above.
I'd like to believe that giving the lowest possible score to a book by an author of my friend's calibre was some kind of mistake, but my friend thinks that judge hoped to scuttle her chances of becoming a RITA finalist. If that's true, the judge wasn't terribly clever. Rather than dragging down my friend's total score, the nasty judge boosted it.
I know a lot of very good writers who have found ones and twos on their books' score sheets. It has happened to me, as well, although my first book didn't receive high enough scores from the other four judges to have finaled even after the "nasty" score was dropped. But in a scoring system of 1-9, shouldn't a 1 be reserved for those rare cases when a book is so amazingly awful that you wonder who in the world made the unfortunate decision to publish it? Sure, judging books is a subjective business, but there's a difference between a boring or disappointing book and one that shocks us because it is so poorly written.
Why are so many RITA entrants receiving these low scores? Could some RITA judges be attempting to covertly punish authors they hate? I'd like to think we're all behaving like professionals, but the judges remain anonymous, so who knows?
5 comments:
Isn't it possible that the judges are trying to boost the entrant's score, knowing that their low score will be replaced by the average?
Either way, it seems a bizarre way of working out a final score. It seems to me to be saying that if a book is marked particularly low (for a high-scoring book) or high (for a low-scoring book), then that mark must surely be a mistake, so let's replace it with the average. So, for a given book, the view of the majority of judges is considered to be the view of all the judges, with little leeway for individual opinion. Which kind of stinks, doesn't it?
Isn't it possible that the judges are trying to boost the entrant's score, knowing that their low score will be replaced by the average?
Well, yeah. But I don't think that's occurred to anyone yet, which is why I blogged about it today. I've certainly never seen it discussed in any of the writers forums. And Neal, I'm surprised you didn't point out that any judge wanting to do damage would better accomplish her goal by giving a score of 5, not 1. With four nines and a five, it would be impossible for a book to do well.
No, it's not a perfect system, but it's the best RWA has come up with so far.
Bottom line, there's no way to ensure that only the very best books win these awards. First of all, not every eligible book is entered. Second, a single cranky judge could keep the world's best book from finaling or winning. And third, what the heck is a "great book," anyway? Don't we all have different ideas about that?
What the RITA awards do--and I think they do this pretty well--is represent the best books in our genre. While it's true that the very best books don't necessarily win RITAs, it's also true that mediocre books don't win. Even if the five judges in the preliminary round just happen to be my five best friends and they all give me nines because they think I'm cute, I'll have to impress another panel of five judges in the final round. And what are the odds that they, too, will love me and give me nines just for being cute? The RITAs have sometimes been disparaged as "popularity contests," but judges and books are matched randomly, and the judges remain anonymous. So I can't see how money or friendship could buy anyone a RITA.
As I said, it's not a perfect system, but it does spotlight some very good books. So I'm impressed by RITA wins and I'll probably enter my next book in the contest, too.
Personally, I think it would be fairer if the contest threw out the lowest and the highest score. That would make more sense.
I don't get that if you scored SUPER LOW you should be rewarded with an average. That makes no sense. But whatever.
I can tell you right now, I have never bought or read a book because it was a RITA winner. In fact, I have heard it said that most readers don't even know what the RITA is.
It is one of those useless awards that is only meaningful to a small group of people. It has no wider meaning beyond that group.
So, I guess what I'm saying is, who cares? It's a dumb scoring system for a contest that doesn't mean much for sales or anything....
Personally, I think it would be fairer if the contest threw out the lowest and the highest score. That would make more sense.
Sounds good to me. But that might be less fair for some reason that we're not seeing. I do know that RWA has studied the problem long and hard, and this is the solution they've come up with.
You're right that most romance readers have never heard of the RITA awards. But many of the die-hard fans do know about them. And they're a very big deal inside the industry--editors and agents urge their authors to enter the contest.
Does a RITA win translate to higher book sales? Most authors would say no, but there's really no way to measure that.
Brenda -
Congratulations on what is obviously a very well written book!
I think your facts are just slightly off though. I believe the average that is employed is based on all five scores - including the low score. At least, that's the only way I've been able to derive the total listed on my score sheets. Had you received a one instead of a six.five, the average would still have been higher than your current score, but not by much.
However, I like to think that the judges are not that manipulative. (grin) Congratulations again!
Post a Comment