Netflix Prize: Forum

Forum for discussion about the Netflix Prize and dataset.

You are not logged in.

Announcement

Congratulations to team "BellKor's Pragmatic Chaos" for being awarded the $1M Grand Prize on September 21, 2009. This Forum is now read-only.

#1 2006-10-06 16:37:04

prodigious
Member
Registered: 2006-10-06
Posts: 1

A single customer that rated 17,000 movies

Customer # 305344 apparently rated 17,000+ movies. 

Is this really realistic?  Can any of you imagine a customer rating that many movies?  More's the point, if you paid a customer to rate one movie a minute, that would take 7 weeks of full time work to rate that many.

Is some of the data bogus?

Offline

 

#2 2006-10-06 17:20:59

chen
Member
From: USA
Registered: 2006-10-04
Posts: 27

Re: A single customer that rated 17,000 movies

Jim Bennett mentioned this guy (or girl) in passing during his presentation at Recommenders conference (available online). Apparently this person really exists and is a real customer.

He also mentioned customers who always give everything a score of 1, or their polar-opposite brothers who give everything a score of 5. Would you consider these points outliers? It's your call...

Offline

 

#3 2006-10-08 08:03:46

Sigmoid Curve
Member
Registered: 2006-10-07
Posts: 5

Re: A single customer that rated 17,000 movies

The most avid raters prove the most difficult to predict.  I find it difficult to imagine how rational each and every response to 17000 stimuli could be.  Although it may be possible to identify taste preferences that customer 305344 shares with others, using this customer's responses to predict others could be a source of bias.  This customer will probably get the boot in my algorithm.

On the other hand, chen questions whether to consider those who rate everything "1" or "5" as outliers.  Au contraire, I enthusiastically welcome such customers in my dataset -- their responses are quite easy to predict wink

Offline

 

#4 2006-10-08 11:07:47

Barry
Member
Registered: 2006-10-03
Posts: 167
Website

Re: A single customer that rated 17,000 movies

Ratings that aren't 1 or 5 are more likely to be thought out and to indicate something. Maybe we should throw out all the 1 and 5 ratings.

Offline

 

#5 2006-10-08 11:29:43

willakawill
Member
From: Chicago
Registered: 2006-10-04
Posts: 117
Website

Re: A single customer that rated 17,000 movies

What a fascinating piece of info.

So this person must have watched at least 10 hours of movies, 7 days a week for the last 8 or 9 years. Even if it was a club or a huge family that would be impossible. The best plan on offer allows 4 DVDs at a time and this team would need at least 6 per day.

Another interesting fact from the Netflix top 100 movies.
Crash is at #1 (so maybe the Oscars have an influence)
Capote is at #100 (or maybe not...)

Offline

 

#6 2006-10-08 18:33:57

alexoss
Member
Registered: 2006-10-06
Posts: 1

Re: A single customer that rated 17,000 movies

You don't have to have watched a movie to rate it.  They invite you to rate movies you've already seen so they can give you better recommendations...

Offline

 

#7 2006-10-08 20:08:36

willakawill
Member
From: Chicago
Registered: 2006-10-04
Posts: 117
Website

Re: A single customer that rated 17,000 movies

alexoss wrote:

You don't have to have watched a movie to rate it.  They invite you to rate movies you've already seen so they can give you better recommendations...

Did you mean you don't have to rent a movie to rate it?

Offline

 

#8 2006-10-09 06:34:59

vdicarlo
Member
From: Davis, California
Registered: 2006-10-03
Posts: 78

Re: A single customer that rated 17,000 movies

If you have a customer account, you can rate any movie in the collection, regardless of whether you've seen or rented it. The interface makes it convenient to browse through many pages of the collection rating one movie after the other, and invites the user to rate more movies to get better recommendations.

There are currently seven ratings, each grouped below with its explanation from the Netflix website:

5 Stars
Loved it

4 Stars
Really Liked It

3 Stars
Liked It

2 Stars
Didn't Like It

1 Star
Hated It

Not Interested
Not interested in this movie

No Opinon
I don't know enough about this movie to have an opinion

Offline

 

#9 2006-10-09 09:33:37

willakawill
Member
From: Chicago
Registered: 2006-10-04
Posts: 117
Website

Re: A single customer that rated 17,000 movies

So perhaps my algorithm should filter out the ratings of any customer who rated more than say 500 movies and treat that data as erroneous.

Offline

 

#10 2006-10-09 10:53:54

jswanner
Member
Registered: 2006-10-09
Posts: 1

Re: A single customer that rated 17,000 movies

willakawill wrote:

So perhaps my algorithm should filter out the ratings of any customer who rated more than say 500 movies and treat that data as erroneous.

I have currently rated 710 movies, and I have seen all of those movies.  Not that I have rented all of those from Netflix.  The only exceptions would be those movies that I rated as "Not Interested", but I would say that is less than 20 movies.

Offline

 

#11 2006-10-09 11:34:21

willakawill
Member
From: Chicago
Registered: 2006-10-04
Posts: 117
Website

Re: A single customer that rated 17,000 movies

jswanner wrote:

willakawill wrote:

So perhaps my algorithm should filter out the ratings of any customer who rated more than say 500 movies and treat that data as erroneous.

I have currently rated 710 movies, and I have seen all of those movies.  Not that I have rented all of those from Netflix.  The only exceptions would be those movies that I rated as "Not Interested", but I would say that is less than 20 movies.

Great. So do you think that someone has reliably rated 25 times the number of movies that you rated? Will you trust this data?

Offline

 

#12 2006-10-09 12:26:16

petersendidit
Member
Registered: 2006-10-02
Posts: 5

Re: A single customer that rated 17,000 movies

I have rated 2010 movies and have seen every single one.  Now yes many of those I saw before I had netfilx but each an every movie I have watched.  I am only in my 20's too.  It would not be hard for me to believe that a couple in their 70's could have collectively seen 17,000.


Yes I have lived for 192,830 hours and have spent 4,020 of those watching movies.  Its only 2% of my life.

Last edited by petersendidit (2006-10-09 12:33:19)

Offline

 

#13 2006-10-09 13:34:27

willakawill
Member
From: Chicago
Registered: 2006-10-04
Posts: 117
Website

Re: A single customer that rated 17,000 movies

Thank you petersendidit.
Interesting piece of info.
So you are 22 years old. Have you been rating movies that you were viewing since you were 1 year old?
Have your tastes changed since then?
Do the ratings you gave for movies you watched at age 3 still relate to your current ratings?
Did you love 'Bambi' then. Do you love it now or would you trash it?
Is the data you supplied to Netfix a valid predictor of how you will rate a movie you watch today?

Offline

 

#14 2006-10-09 13:55:06

petersendidit
Member
Registered: 2006-10-02
Posts: 5

Re: A single customer that rated 17,000 movies

The only "kid" movies I have rated are ones that I have seen in the last 8 years or so.  I haven't rated stuff like Bambi or stuff like that.

Have my tastes changed? not really.  I have had the favorite few movies for a long time now.  I very very rarely watch a movie twice.

The data that netfilx has for me would be valid on how I would think about a movie today.  I have even gone back and modified the rating I have given a movie if I saw it again and though differently about it.

Offline

 

#15 2006-10-09 13:58:04

petersendidit
Member
Registered: 2006-10-02
Posts: 5

Re: A single customer that rated 17,000 movies

Another thing to think about with the users that has rated 17,000 movies is . . How many movies do you think Roger Ebert has seen and rated.  Who says that there aren't other critics out there that use netfilx and have rated all the movies they have seen.

Offline

 

#16 2006-10-09 15:05:50

willakawill
Member
From: Chicago
Registered: 2006-10-04
Posts: 117
Website

Re: A single customer that rated 17,000 movies

Critics get to see movies before they are released in a theater, not on DVD.
They also get to see them for free plus the usual coffee and donut. (critics never eat popcorn, Something to do with professional pride) smile

So you have watched something like a movie a day since you were 16, with the ocasional day off per week.

Now I want you to know that this is an investigation and not an attack on you. It is easy to misunderstand posts sometimes. You are a living statistic who has valuable information pertinent to the competition. Something that cannot easily be obtained otherwise.

The 70 year old couple that you suggest, assuming they began watching movies in a theater at the age of 16 like you, would, like you, have had to watch a movie every day of the week with one day off for every week of their adult lives. They would have spent, inflation adjusted, over $200,000 without popcorn.

Yes I agree with you that it is not impossible. The question for me just purely in terms of this contest is can I trust this data?

What do you think?

Offline

 

#17 2006-10-09 16:51:46

petersendidit
Member
Registered: 2006-10-02
Posts: 5

Re: A single customer that rated 17,000 movies

A movie a day is pretty dang close.  I have slowed down here the past 5 months or so but yeah.  I was on a first name basis with the guys at blockbuster till netfilx started up. (A movie a day on top of averaging about 70 miles a week on a road bike a week and keeping a girlfriend and school/work)

I think that there is no way of knowing if that data is valid or not without talking to the the person who submitted that data.  The same question could be asked about the people who have rated only 20 movies, do they really understand the rating system or are they just clicking to get it out of their way.  I mean who knows all 17000 movies that that user rated could be dead on with the avarage ranking of each movie.

Offline

 

#18 2006-10-09 23:25:12

Aron
Member
Registered: 2006-10-02
Posts: 186

Re: A single customer that rated 17,000 movies

I think you have to use the ratings of the 17,000 movie rater. One thing I was considering was dropping the 'All 1's' people under the theory that they are being flippant. However, it could quite easily be the case that they specifically rate only the movies they hate. It doesn't mean they don't hate them. They could be taking the strategy of telling Netflix just what NOT to show. It's all still good data as far as you know.

But Netflix has clearly signalled preconditioning the data in this way. I'm just not seeing the logic.

Offline

 

#19 2006-10-09 23:42:55

willakawill
Member
From: Chicago
Registered: 2006-10-04
Posts: 117
Website

Re: A single customer that rated 17,000 movies

Netflix wrote:

Straightforward statistical linear models with a lot of data conditioning

My emphasis added.
If I ever get this thing downloaded this will be my main focus. I think this is pivotal in reducing the RMSE.

We shall see.

Offline

 

#20 2006-10-10 01:22:36

CS1
Member
From: San Jose, CA
Registered: 2006-10-02
Posts: 151

Re: A single customer that rated 17,000 movies

This person could be employed any number of ways.  The folks who assign the MPAA ratings have to watch the movies.  Then there are the folks who give different organizations' approval (think of groups that quantify the violence, or the "family values", in films).

Anyway, when you're receiving the $1 million, you'll probably get introduced to that rater.  Be ready.  big_smile

Offline

 

#21 2006-10-10 14:53:04

Shark
Member
Registered: 2006-10-02
Posts: 23

Re: A single customer that rated 17,000 movies

I finally got a sucessful handle on the data. That user is insane.

Offline

 

#22 2006-10-13 13:52:03

Ummon
Member
Registered: 2006-10-12
Posts: 5

Re: A single customer that rated 17,000 movies

It's a lot of ratings but it's not that hard to believe. What if said user was a professional film critic? 17,000 movies can be watched in about 47 years at a rate of one per day. 23.5 if you watch two per day. I am sure there are critics out there who watch 2 movies every day - after all that's their job!

Offline

 

#23 2006-10-17 04:20:39

in5ane
Member
Registered: 2006-10-16
Posts: 47

Re: A single customer that rated 17,000 movies

This 17k guy annoys me smile

90% of the time he rates below average by two stars, and 50% of his ratings are 1. Yet he only appears seven times in the qualifying set, despite influencing pretty much every movie in the training set. This also means he is a candidate for comparison with other customers pretty much all the time, which is killing my poor little PC sad

Offline

 

#24 2006-10-17 19:26:34

Theorist
Member
Registered: 2006-10-16
Posts: 2

Re: A single customer that rated 17,000 movies

Another possibility is that the 17k guy/girl, perhaps, is physically handicapped.  Sometimes there's not a whole heck of a lot you can do other than watch movies.

Offline

 

#25 2006-10-18 12:35:11

FoCu Programmers
Member
Registered: 2006-10-17
Posts: 7

Re: A single customer that rated 17,000 movies

I personally don't think you have to watch a movie to rate it.  If I am going through the netflix rating system and see some teenie bopper chic flic straight from the Disney channel, I will rate it a 2, maybe even a 1.  I don't have to see it to know I won't like it.  It's very possible that the "17k guy" is rating simply by genre or something, not necessarily by movies he's seen.  Just a thought.

I just had another thought on this 17k guy.  We only have 17770 of Netflix's movies downloaded. So this guy may have rated even more movies that we're not seeing.

Last edited by FoCu Programmers (2006-10-27 06:01:35)

Offline

 

Board footer

Powered by PunBB
© Copyright 2002–2005 Rickard Andersson