Contests / NEDC Score Projections

Steve asked what the projection for the number 1 score for NEDC5 is, and I got carried away.

 

Obviously, we're only three submissions in - but that's enough to get some rudimentary projections going. (Hey, I'm a medical student - I need a hobby that isn't as variable as the human body.) This is by no means meant to be accurate. It was just fun.

 

So we've got three in; releases 1-3 scoring 56.5, 63.5, and 65.5% respectively. The biggest limit to this process is that our panelists aren't going to follow our equations perfectly. That's alright, predicting the future is never an exact science.

 

WARNING: STATISTICS TERMS AHEAD!

 

Since we're working with an incomplete data set, we have to start simply and see what kind of equation we're looking at. Ideally, the more starting data the better, but here we are.

 

 nedc stats.PNG (5.59KB)
downloads: 3

 

R-Squared Values (We want as close to 1 as possible for the best fit)

Linear - 0.907; Not great, not horrible. But the scores aren't gonna be linear, I can tell you that.

Exponential - 0.894; Not good.

Polynomial - 1; That's amazing, but if we reach release #10 we get a negative value, so we can't use this.

Logarithmic - 0.975; Really good. I initially used this because it's what I thought made the most sense.

Power Scale - 0.97; Also really good. I used this as a comparison.

Moving Average - Not enough data, don't even try.

 

So our best bet is going to be using a logarithmic or power scale fit to our first few submissions. Here's the equations for the statistics nerds;

 

Log: y=56.8+8.39lnx

Power: y=56.8x^0.138

 

Now that we've got that, we estimate our remaining release scores and get our projections. This is where things got interesting.

 

NEDC4's top scoring entry was The Junkyard by alex. That got 80.63%; keep that in mind. Also note that the numbers here only refer to release order and not placing. It's reverse of placing. Did I do that confusing enough? Good.

 

Log Fit

 Log.PNG (17.76KB)
downloads: 4

 

Power Fit

 Power.PNG (18.3KB)
downloads: 7

 

Easy-to-Read-Tables

 Tables.PNG (6.94KB)
downloads: 8

 

Based on the current (tiny) trend, this predicts that we wind up somewhere in the 76-78% range for our NEDC5 winner. There are a lot of limitations to this. I'm not some random news site, I'm going to be upfront with it.

- This is an extremely small sample size

- This doesn't account for many variables; the big thing being that every single entry after this could score 0.0000000001 above the previous for all I know - it's simply the best mathematical guess based on a very tiny amount of predictive data.

 

Okay, let's see how well this works for other NEDC iterations. Special thanks to cam for writing this down.

 

NEDC2

 

Log Fit

 NEDC2 log.PNG (30.11KB)
downloads: 5

% off First Place score: -1.17%

 

Power Fit

 NEDC2 power.PNG (30.14KB)
downloads: 1

% off First Place score: +2.88%

 

NEDC4

 

Log Fit

 nedc4 log.PNG (32.88KB)
downloads: 1

% off First Place score: -0.04%

 

Power Fit

 nedc4 power.PNG (33.46KB)
downloads: 1

% off First Place Score: +5.7%

 

Conclusions:
Not much, because I don't have much data to go off of.

 

From past competitions, it seems that a log fit suits the projection better in general. It's amazingly close to projecting the final score in NEDC4. NEDC2 is much less predictable!

 

As I'm clicking submit, I'm seeing new releases. Time to see how they fit!

 

 

 

 

EDIT: Google sheets screwed up my formatting and I didn't notice, so I edited a few of the charts and calculations.

Quick update factoring in the new releases.

 

Log Fit (Red is actual scores)

 Round 2.PNG (21.48KB)
downloads: 11

 

Power Fit (Red is actual scores)

 Round 2 Power.PNG (26.52KB)
downloads: 10

 

Both actual scores are trending lower so far, but looking at the previous NEDCs, variation from the fit is not unusual and in fact should be expected. Also to note; of course the fit is perfect for the first three, because that's what it uses to project the rest!

you must be reeeally bored. 

This is fantastic

Conclusions:

Not much, because I don't have much data to go off of.

 

 

Polynomial - 1; That's amazing, but if we reach release #10 we get a negative value, so we can't use this.

 

This is actually not really that surprising/amazing considering a 2nd order polynomial (which I assume you used as its the standard setting in excel) will always have an R^2 value of 1 for 3 data points. This is because an N'th order polynomial will always fit exactly through N+1 data points. To really check which method is most accurate I would also try to use the standard error of the estimate in combination with the r^2 value as this will tell you more about the absolute error within your model.

Yeah, it was an exercise of “Steve said something that sparked an idea” but I wouldn’t mind going through and doing that eventually.