Reconsider How Ties Receive Points

[Bobnova]

Maybe the ten year score guy should try harder now that he has all the extra experience and software development? Surely he can beat his old, slow software, score easily enough!

 

 

Personally I like the averaging it out version.

The newcomer to a spot wasn't good enough to beat the previous score, but the previous score doesn't beat the new one, either.

[xxbassplayerxx]

Maybe the ten year score guy should try harder now that he has all the extra experience and software development? Surely he can beat his old, slow software, score easily enough!

 

I think the earlier submissions should be the first on the list, but the points should be the same.

 

i.e. You submit first, I submit second, and Reggie submits third. The order would be like so:

 

1. Bobnova (first in the list denoting the oldest submission)

1. Bassplayer

1. Reggie (third in the list denoting the newest submission)

 

Personally I like the averaging it out version.

The newcomer to a spot wasn't good enough to beat the previous score, but the previous score doesn't beat the new one, either.

 

I'm not sure if there are any benches where widespread ties are common, but with four or five people having the same score, it could get messy. You could be tied for third or fourth with six or seven people and get less points than the guys below you!

[Bobnova]

Should have pushed harder then! You won't ever get less points than the next step down from that tie though. It's an average of the points for those tied people, not splitting the single set of points from the position they're tied with.

[xxbassplayerxx]

Should have pushed harder then! You won't ever get less points than the next step down from that tie though. It's an average of the points for those tied people, not splitting the single set of points from the position they're tied with.

 

Would you believe me if I told you that I'm an engineer?

 

[Antinomy]

Devroush, thank you for clearing.

 

The newcomer to a spot wasn't good enough to beat the previous score, but the previous score doesn't beat the new one, either.

Are you sure, that's how, say world records in sports are counted? Last time I checked it was "first come, first served". A tie didn't mean another man having a world record. You had to break the record to get the prize.

 

Rasparthe, nice point. Suggest an algorithm that treats both cases.

Edited March 4, 2013 by Antinomy

[Bobnova]

Would you believe me if I told you that I'm an engineer?

 

 

Absolutely! Engineers use calculators and web browsers don't have 'em. Y'all have more important things to use the brainpower on after all.

Now if you design bridges I'd like to know which ones

[Antinomy]

Lol

[knopflerbruce]

Devroush, thank you for clearing.

 

Are you sure, that's how, say world records in sports are counted? Last time I checked it was "first come, first served". A tie didn't mean another man having a world record. You had to break the record to get the prize.

 

Rasparthe, nice point. Suggest an algorithm that treats both cases.

 

This is correct, however, if you list the best results in that sport, you'd get a tied 1st spot, not 1st and 2nd if the scores are the same.

 

I feel it's quite inaccurate to assume an older score is at a higher skill level, a result is a result - time is only a variable because we didn't find a better solution at the time. Option 2 is no doubt the most fair one to every bencher - you tie a result, you get the same points for the same skills shown.

[GIPrice]

sorry bassplayer I love you and I thank you for selling me that UD9 but I feel that order of submission is the most fair way.

[Trouffman]

I voted Method 2 , because it is more respectful for people benching. you get a "score" that "rank" X no matter if you are alone with this score or not Your score rank X. Not X+1 because X+1 doesn't egal X.

 

okay maybe not really understandable but I suggest Method 2 if not too difficult to implement.

[ZL1]

actually imo it should be the other way around, efficiency should be first, so smaller mhz = higher score, since its a more efficient run

[Vinster]

actually imo it should be the other way around, efficiency should be first, so smaller mhz = higher score, since its a more efficient run

 

Ya, but massman pointed out in the first page ppl can lower their clock after the fact before the screenie... will want to make ppl lie and it's impossible to enforce...

 

I prefer the way it is now, Older date gets more points.... but if it changes, I'd prefer method 2.

 

what will happen to all the ties in SP1M with folks that have 0.1 points?

 

Vin

[ZL1]

Ya, but massman pointed out in the first page ppl can lower their clock after the fact before the screenie... will want to make ppl lie and it's impossible to enforce...

 

I prefer the way it is now, Older date gets more points.... but if it changes, I'd prefer method 2.

 

what will happen to all the ties in SP1M with folks that have 0.1 points?

 

Vin

 

oh I see, somehow I never think like a cheat bad cheat,but also bad cop I guess

in this case I actually like no1, the avg, motivates you to try for higher and also affects the old score, as it has been matched, meaning it lost from its status no matter what

[Dead Things]

in this case I actually like no1, the avg, motivates you to try for higher and also affects the old score, as it has been matched, meaning it lost from its status no matter what

That's an excellent point. I had been on side with Method 2 throughout the whole thread until ZL1's post, and suddenly I'm 100% convinced Method 1 would be better. With Method 2, bassplayer would receive the same 2nd place points for tying the existing score as he would if he beat it. There should be some degree of reward/incentive for beating an existing score.

 

And now that I'm thinking this way, here's some mental diarrhea... One of the problems with NHL hockey is that some games are worth a combined 3 points (ones that go to overtime) while others are only worth a combined 2 points in the standings (ones settled in regulation time). This intrinsically suggests that closer games are 'more important' than games in which one team is clearly better than the other. The result of this imbalance is that at the end of the season teams are bunched together in the standings, providing less separation between better and worse teams than there would be if all games were worth three points like in soccer or two points like pre-Bettman hockey.

 

In the case of hockey, it's exciting because any team - no matter how good or bad they may be in real life - has a legitimate chance of making the playoffs. And this creates excitement among fans, which is likely the only reason why the broken and imbalanced points system has never been addressed.

 

This relates to the current question because as you get farther down the leaderboard, ties abound in significant quantities for most benchmarks - particularly on the CPU side of things. Method 2 would add quite a large sum of points to the system that previously did not exist. Based on the NHL experience, my concern then is that Method 2 would break the points system by creating an imbalance that would generate greater rewards for mediocre results and lesser rewards for superior results.

 

But hey - like I said, that's just mental diarrhea while I procrastinate at work. Maybe the NHL example has no bearing on this case. Either way, I still feel like there should be a greater reward for beating an existing score than there is for tying it.

[MTP04]

Awarding who can submit the fastest makes sense to me, so I vote to keep it as it is.

 

 

EDIT, fixed my grade 1 spelling.

Edited March 6, 2013 by MTP04

[Movieman]

My thinking is as is current, first in, top points and I'll explain why.

When one guy does an accomplishment and submits it you know that he is the one that worked out how to get those numbers. You don't know this for the second guy or the third. They may have benefitted from info from the first guy.

[Bobnova]

From that standpoint we should freeze rankings after X days, as anybody that beats that first guy may have benefited from him too.

[Bones]

Personally I like Option 1 best.

[Poorya_lion]

I vote for option 1# . There should be a difference between a guy who reach a score sooner (with maybe old hardware or weaker) and other guys who reach the same score (maybe after some years and with more powerful hardware)

[Trouffman]

I was thinking about it yesterday night... and comes out method 2 is okay but could lead to some potential issues and interrogation :

 

* Are we ranking comparable scores or people ? (a score is link t a person but the point are awarded according the score.)

* What about very sall points like 0.1pt : Method is then not possible, because it will lead to have a load of small point awareded and no difference in between very close score or so...

* Sandbagging... I think that in a competition especially, the person to put out the score first deserve to get more point (while i didn`t think about it when voting first.) That lead bac to my first point. What are we ranking precisely. Scores, scores and submission time, Scores and alphabetical names, monkeys, peanuts ? Method 2 will not avoid sangbagging in competition, while the goal of sandbagging is to achieve better score in the rnaking for sure what happen if it'ts exactly the same ? I would then consider 1st person to submit the one thatget the most points.

 

Just my 2 cents.

[I.nfraR.ed]

I voted for Method #2, but on a second thought Method #1 would be better.

First of all, efficiency could not be taken into account, so we're left with some other option which will be a compromise.

 

Let me think about this situation:

Haswell is out and all top guys submit their 1M results, because they had it long time before launch.

Then after 2-3 weeks I'm able to purchase my retail cpu when it gets to Bulgaria.

I push to the max with the best efficiency I can achieve and eventually match some of the inefficient scores, but get lower points, because I was not able to submit earlier. How's that exactly fair?

Efficiency can not be taken into account, we all agree with what Pieter said on the topic.

 

Now, averaging the points for the tied results will eventually make the other guy who submitted first to push harder and earn his points again. And that's a good thing.

 

The thing is older scores (e.g. Athlon64 3200+ 1M rankings) submitted like 5 years ago might not be able to "defend" their position, but I still think it's the better option than the current one.

 

BUT, if there's a significant performance hit for the server, then I'd leave it like it is now. Looking at the 3770K 1M rankings,

there are too many ties between 5-6 guys at every step (5.234, 5.250, 5.281 and so on).

 

Maybe it's a good idea to have that average points in case of a tie for top5 only in popular rankings, because the point difference there is bigger.

For other places it's like 0.5pts max.

[xxbassplayerxx]

I vote for option 1# . There should be a difference between a guy who reach a score sooner (with maybe old hardware or weaker) and other guys who reach the same score (maybe after some years and with more powerful hardware)

 

The problem with this is that the difference can be quite large... 10 points in the case in the OP.

 

What has changed in that instance? Nothing at all... still running the same benchmark, still running a chip released in 2009, still running an X58 board (earliest is using RIIIE, I was using OC--comparable boards), etc.

 

For a 3D benchmark, what you're saying has more merit but does it really deserve more points? Because someone may have been into a hobby earlier? Joining a hobby and competing against old results should not be a detriment. If that was the case, I should sell all of my 8800 cards because it's not 2006 anymore and benching a 8800GTS with IB isn't worth as much as benching it with Conroe.

 

I do agree that the older result should receive recognition (first in the list of 2nd place finishes), but I don't think that should translate to more points. Bottom line, the scores are the same.

 

People can say you should have beat the score and this and that, but in they end, they should know how it works. You push and push and sometimes you don't beat it. Tying someone in no way means you didn't try hard enough. They're not better because they got the score earlier... they just benched at a different time.

[Hyperhorn]

The current solution is the best way to encourage overclockers to stand clear of tactical backups and present the best achievable results right from the beginning. As Hwbot should be a place where folks can see full potential of hardware unleashed by overclockers and not only scores that are "good enough" regarding the competition, the status quo has its justification.

 

Also usually when you compete in a category at a subsequent date you have access to more data, newer/more software (BIOS, drivers, tools, benchmarks, OS improvements etc.), sometimes even small hardware advantages (new revisions/batches, improved silicon quality etc.) and know exactly what scores you have to beat. Most Hwbot overclockers I know don't try very hard to get an improvement of 0.001 s in a benchmark on principle, but to beat an established result.

 

When time goes on you get a natural advantage, but the current solution is a good way to encourage overclockers to submit their scores early (to get a possible advantage over the competition) and by a side-effect it balances the natural advantage that later overclockers have in very specific cases a little.

[Vinster]

T

When time goes on you get a natural advantage, but the current solution is a good way to encourage overclockers to submit their scores early (to get a possible advantage over the competition) and by a side-effect it balances the natural advantage that later overclockers have in very specific cases a little.

 

I agree with this... also the guy who benched an 8800 in 2009 probably also didn't have access to as many FW's that are available today for tweaking... also better drivers... if they ran a 15x.xx driver, there were a lot of improvements in just that alone...

 

OC'ing 3-6yrs ago was harder... Drivers change, FW's have changed better documentation on Volt Mods... back then you had to figure it out... now you just google it and you got it....

 

older HW has gotten easier...

 

I still prefer Older is first... , I understand people changing to Method 1.. it does make sense... but the guy who did it first shouldn't lose points just because other ppl caught up 5 years later so I stick with my method 2 vote if it were to change.

 

Vin

Edited March 8, 2013 by Vinster
more detail

[Massman]

Interesting discussion and feedback! Dennis is looking up information on how much scores are affected by this, which should also be interesting.

 

Personally, I voted for option #2. I think that ranking by date is only relevant in a system that has an end, for example in a competition. The nature of the HWBoint rankings is that it does not have an end, but just a database of benchmark results with a specific hardware configuration. Points are a value given to a specific result with specific hardware; ie: 10 seconds SuperPI with a Phenom II X4 940 is worth X amount of points. It does not really matter whether that result was obtained today or two years ago - the obtained performance result with specific hardware is the same.

 

I do understand the argument that it would be easier to obtain a certain score later in time as more knowledge surfaces, but at a certain point there is a knowledge saturation point. There is today not more information available about Athlon XP overclocking than there was three years ago. Additionally, someone that wants to score points with Athlon XP still has to do research; it's not something that you inherit from your parents or so.

 

In an ideal world, this would be an easy question: rank by frequency (efficiency). But since it's really easy to lie about the used frequency, that option is simply not practically doable. So, we have to look for the best alternative. A system where you valued on the same basis as one was rewarded five years ago, namely "X result with Y hardware" sounds quite nice to me. That system does not favor age or youth, doesn't punish when others tie your score and rewards for reaching a better-than-not-tie result.

 

77 votes is not that much, though. Let's try out the new HWBOT Poll feature to see if we can have more opinions on the matter!