Netflix Prize: Forum

why can we not see any more beyond place 100 in the Leaderboard?

"After three (3) months have elapsed from the start of the Contest, when the RMSE of a submitted prediction set on the quiz subset improves beyond the qualifying RMSE an electronic announcement will inform all registered Participants that they have thirty (30) days to submit additional candidate prediction sets to be considered for judging.

At the end of this period, qualifying submissions will be judged (see Judging below) in order of the largest improvement over the qualifying RMSE on the test subset."


This is confusing wording.  What do they mean by 'order of the largest improvement'?  improvement of the so far winning entry?  does that mean they will not even judge anything that does not exceed 10%?  But also, will they at least tell us how we did if we submit.  For example, if somebody is at 9.4%, and they make a submission that gets to 9.9%, I would hope they would be told since that tells them they made some headway and to continue on that path.

can anybody enlighten?

thanks

The full description of the Term of the contest states:

The Contest entry period commences on October 2, 2006 (00:00:00 UTC) and continues until the Grand Prize is awarded...

So no, there will be no progress prizes awarded after the Grand Prize.

I read where the contest will go on to at least 2011 in the rules.
is this a misprint?
or will it continue after the million dollar prize to see who does best until then?

thanks

I want to ask the hello_computer's question again:

Will the site be up and running after Jule, 26th?

Will we be able to submit our predictions and continue working on the problem as a hobby?

Mayber there is some misunderstanding somewhere. But from what I understand (and I have said this elsewhere recently), once the prize is won I presume Netflix will publish its solution set. And with this in hand:
1. everyone can continue the contest on their own - forever
2. you can make zillions of 'submissions' per day - no cr/hackers needed
Or did I miss something somewhere?

I just wish netflix would be more user friendly.  you know, I called and asked a question of another sort (regarding two usernames, which I did only because I forgot the first one, and never used it) and they told me that they don't know the answer, and that I would have to WRITE to them in California.  they seemed to have overlooked the PR aspect of marketing.

One question.. what happens if neither team achieves >=10% on the verification set? Both teams have had thousands of submissions through which they could have overfit the qualification set and could end up doing poorly on the verification set. If that's the case, does the contest reopen?

Last edited by dpe82 (2009-07-27 00:50:02)