Netflix Prize: Forum

Actually, IMDB is available for download free of charge to ANYONE.  The only restriction for licensing is that you may not use it for commercial purposes (without asking/paying for it).  Winning a contest (or attempting to win a contest) is not "commercial purposes" at least, it is not according to three different attorneys that I have spoken to. 

So I do not see how an IMDB employee has an advantage over me (or you).

coz wrote:

Actually, IMDB is available for download free of charge to ANYONE.  The only restriction for licensing is that you may not use it for commercial purposes (without asking/paying for it).  Winning a contest (or attempting to win a contest) is not "commercial purposes" at least, it is not according to three different attorneys that I have spoken to. 

So I do not see how an IMDB employee has an advantage over me (or you).

If you try downloading IMDB, you will find that less than half of the movie titles in the Netflix movie file is in the IMDB genres file.

Furthermore, since most (if not all) of the Netflix movie titles are present on Amazon.com, with the required additional info, I suspect that IMDB probably has this data, but is reserving it for paying customers.

Correct, but Netflix has said that if we use any outside data, they will use their own source for that data, not ours. 

So, if I use a table of titles/actors/genre/director/etc that has been gathered from IMDB, Netflix will create a similar table from thier own databases for use.  They would not use the one I created.

Perhaps a small proportion of teams want additional information, though I doubt that proportion is very high.  Netflix themselves have additional data available to them, but instead of using it they open up this contest. 

I believe that anyone who understands the enormity of the task-in-hand would realise that extra data would only confuse matters.  A film is an incredibly complex thing to quantify.  One person's blood-filled, pillow-hiding horror gore-fest is another persons twisted, dark-humoured romantic comedy.  Someone at IMDB may then simply label that film as horror, even though it does have a vein of black humour throughout.  They try to give the genre that most people will take it as.

Of course we could then add data about the actors.  Perhaps you like films with Clare Danes in, "Shop Girl",  "Romeo and Juliet", "Little Women", so any film with Miss Danes would be marked more positive for certain users.

However, perhaps the user only likes Clare when she plays girlie-girls, so the system would be marking "Terminator 3" wrong, because she plays a different character completely from many of her other roles.  Do we add the actor, and the type of character the actor played too?  Or perhaps we have to add the amount of on-screen time also, for Miss Danes only played a minor part in "Rage at lake Placid" and "U-Turn"

The complications get worse.  The team entering this data will have to then judge how good a persons acting was.  Perhaps you find a film funny when an serious actor acts badly and enjoy the film more, or perhaps the bad acting will make you dislike the film more?

The team editing this data is going to get bigger and bigger as more and more data is needed; they need to ensure they're all annotating to the same specification.  Slapstick humour might make you laugh, and maybe 5% of the annotators, where does that leave the tags?  Sometimes accurate for me, sometimes accurate for you, but never accurate for everyone.  You could label it as humour-slapstick, but someone else may interpret the film as just clumsy acting-no humour.

Getting people to rate a film from 1-5 includes all this.  The stars can mean whatever the user wants to mean, a system with this much data should be able to accurately predict films for you regardless of the stars meaning.  I would think that even if a user inverted the star system (1 star good, 5 stars bad) the system would still be able to predict their choices accurately (as long as enough people used the stars correctly that is).

The genre information, which seems to be the most important, can all be implied from this dataset much more accurately than from IMDB.  All you need to do is process it correctly.

Sorry if this seems a rant, its aimed at everyone chasing extra information.  It really will not help, unless you have an ayschronous neural simulator the size of a building to churn through it.  And even if you had such a device, part of the system would be to assign 5 stars to each film, for each user.  You may as well let the users own neural net' work out the 5 stars for you and just use them!

Exactly, I totally agree with rob and Paul.

What genre is "The Sopranos, Season 1"?  Crime?  Drama?  Comedy?  "Television"?  That's the trouble.

Unfortunately, "all four" is hard to mine from existing sources, which was my point.

I disagree that the contest is unfair, or favors on group over another (although I haven't run a program to see whether or not this is true.)

Even if one has more data on the subject than others, one must be able to know
1. How it relates to the overall data.
2. How to analyse the results of the data.
3. How to write all that 'extra' code before the contest ends.

If this is the case, I submit that whoever has done that is already knee-deep in providing valuable intelligence software to some community, and would probably require Third Party Licensing agreements in order to use it for this contest.

As I have already said, we are an extremely diverse community looking at the same problem from quite a few angles. I believe we all stand a chance.

thats really just untrue.  netflix stands to gain the most from this contest, and they're most likely garunteed to gain it at that.  theres alot of very cheap (relatively) labor going on right now.

probably wrong wrote:

in my view the relevant set of data is actually really small.

I think you are probably wrong.

buddyglass wrote:

What genre is "The Sopranos, Season 1"?  Crime?  Drama?  Comedy?  "Television"?  That's the trouble.

Why not rate it as a linear superposition of ALL genres?  That way, its top-ranking genres would be these four.

Going a step further, you could also utilize dynamic definitions for your genres that are based on specific user ratings.

RGB wrote:

Also, how can the prize admin be confident that an employee from Netflix won't download this data to his computer and hand it off to friends who enter the contest in return for a percentage of the winnings?

Good question... thought that myself.  However, it is from my understanding (according to the post under the Forum FAQ about winning, where it says "If you decide to go ahead with Prize verification, you have one week to both accept the non-exclusive license and provide your code and description for verification.") that the verification process happens AFTER the 30-days "last call".  You then have one (1) week AFTER that to submit your description and source.  If this is the case, then there are no worries about anybody on the "inside" taking your idea or algorithm... because once it's verified that your source & description ACTUALLY produced the results in the TEST & QUIZ set, you'll win.

Again... I'm only SPECULATING that this is how the week submission rule is supposed to work, based on what the post in the "Forum FAQ" states.  Otherwise, if I'm wrong, I also would be REAL reluctant to post my source... especially if nobody had below .88 for MONTHS, then you suddenly post a .82... and within a week of submitting your code, your entry looses to somebody with a .81.  =S