Monday, April 20, 2009

New Rules for Netflix Ratings

I am a big fan of Netflix. I put some thought into how I use the service in order to get the most for my money. I'm fairly happy with the results I get, but sometimes I have to tweak my usage to serve myself better. How I rate movies helps me remember how I felt about a movie and it helps the system suggest more movies, or predict how much I'll like a movie. After a few years of rating movies one way I have decided to change.

The Old Way

My old system for rating a movie was to try to rate it as objectively as possible. I focussed heavily on the merit of the movie, acting, script, and direction. I would then combine that with my preferences and come up with a rating. This introduced some personal bias, but I think most of the ratings were pretty fair. The exceptions were a few movies that I either loved greatly or hated completely, at which point I would typically let my emotions get the better of my objectivity and rate generally loved movies poorly or generally disliked movies highly.

The problem with this is that I was trying to be objective and not allow my bias to influence the ratings too greatly. This would be great if I were the only one reviewing these movies, or if the rating data wasn't being used for other purposes. Neither of those conditions are true, though. In short, I was being unfair to myself out of some sort of misguided attempt at journalistic integrity, even though I'm no journalist.

Other oddities happened because of this as well. I stopped trusting my own ratings. When someone asks me what I thought of a movie I will look that movie up on Netflix and use that rating to stir up the long term memories associated with that movie. It works great because I have the movie box, description, and my rating all there on one screen. I found that, increasingly as of late, I was having to mentally adjust my ratings based on whether I thought they were skewed for objectivity when I made them.

The New Rules
With my new system I will not change my baseline ratings. Instead, I will allow my bias to more significantly influence my ratings. After I have my final number doing this I will review it to make sure it accurately reflects how I interpret and feel about a movie. Then I'll click the little star that matches.

Basically, everything starts out the same as above. I get a rating number by thinking about how well made the movie was and whether it's worth watching. Then I allow myself to modify that rating by zero or more stars depending on how I felt about the movie and how strongly I felt it. If I have no strong emotions either way then a three star movie will remain at that rating. If I enjoyed that movie a good bit, I will probably add a star. I may add two stars in some circumstances. I doubt I would ever feel the need to add three. The opposite is true if I genuinely disliked a movie.

A few examples:
I recently rented The Prestige. It was a decent movie that mixes science, fake magic, and real magic. I thought it was beautifully shot and decently acted. It was an okay script. Objectively, I think I would give it four out of five stars. Once I added more of my personal bias into it I reduced it to three stars because I didn't like some of the treatment it gives to science, it was a little over-the-top, and it has an fairly obvious plot twist that seems to be there only for plot-twist addicts.

I also recently saw the import So Close. This is something like a Charlie's Angels flick set in Hong Kong starring the locals. It wreaks of bad acting, it's completely over the top, and it's cheesy as anything. The action scenes are top notch, though. If you enjoyed the Charlie's Angels series and like Jackie Chan movies then you may enjoy this. I objectively gave it two stars out of five. I think in the grand scheme of things that movies like this are largely trash. They are, however, trash I tend to enjoy. I liked the car chases and the Asian culture infused in this. So I bumped the rating up to three stars.

As you can see, two movies on the opposite ends of the quality spectrum now have the same rating. I'm able to be both intellectually and emotionally honest.

Other Rules
I did pluralize the word 'rule' for a reason. I have changed the way I think of a few things related to rating movies. I will no longer rate movies 'Not Interested' unless I have a very good reason. I have re-assessed my category ratings using the new Taste Preferences with a particular focus on emotional honesty.

For 'Not Interested', right now I'm reserving it for series items where I've seen parts of the series, but not all, and I am completely uninterested in watching any more. This means there are only 3 items with this rating so far: Dragon Ball Z, Home Movies, and Survivor Season 1. The first two are cartoons that I don't like, yet they are suggested because I apparently differ from the normal person who watches anime and adult oriented cartoons. The last is just weird. I don't know if the system suggested this for me or not, but I'll leave it there so that it won't suggest any "reality" shows.

I didn't like the effect that too many 'Not Interested' selections had on my suggestions and other ratings. I also don't like that it inflated some of my ratings counts. I've seen enough movies without the ones I haven't seen being counted.

My category ratings were a mixed bag of intellectual ratings, emotional ratings, and shame. Some categories I rated higher not because I like watching those movies, but because the movies themselves tend to be well made. That's great, until you realize that you aren't interested 15 minutes in but watch the whole thing anyway. The emotional ratings are probably the right ones, at least that's my take. Some of the ratings were born of shame, though. I was ashamed that I like anime, seeing that as the last step into hopeless geekdom. Finally, I realized that these ratings were entirely for me to help Netflix know what kind of movies I might enjoy. I'll eventually betray the same information by what I rent and how I rate it, so I should be honest to myself and rate categories as I think I would actually want to watch the movies in them. The good thing about Taste Preferences is that it presents the data in a way that makes this easier to swallow by asking you how often you want to watch such movies instead of forcing you to rate them on a five star scale. My only gripe is that I wish the ratings were more granular instead of never, sometimes, often.

That's a lot of thought put into rating movies. The good news is that I mull over these decisions for so long each time I rent. Rating a movie takes a second or two. I'm just trying to maximize my results.

No comments: