HWBoints Adjustment #1: 3D WR Bonus

[Massman]

From the feedback we received after launching the actual HWBoints Rev3 product, we noticed two big problems. The first was the low reward for breaking the overall world records. Since pushing the limits at the absolute top of the benchmark rankings does require skill and effort, two of the main ideas behind rev3, not awarding big numbers indeed doesn't sound right. So, we have whipped up a fix that is supposed to, well, fix this.

 

What benchmark applications?

 

As the name already indicates, this is only applied to the 3D benchmark applications because that's where the main issue is located. Running 5 pots under LN2 (1CPU, 4GPU) is a hell of a work and should be rewarded accordingly if the score turns out to be great. Benchmarks applicable:

 

- Aquamark

- 3DMark01

- 3DMark03

- 3DMark05

- 3DMark06

- 3DMark Vantage

 

One might ask, why not do it for all benchmarks. Well, single-category benchmarks, such as SuperPI and Pifast already discriminate the world records from the non-world records as they are all in the same ranking. In other words, the top score will get the most points. For Wprime, this could've been a solution as well, although this benchmark doesn't scale with skill per definition. World records are set using a highly expensive server; rewarding financial input is not a key characteristic of the Overclockers League. PCMark05 might also apply for this rule, but as far as we can see, the top scores are already awarded the top points as well.

 

Points distribution?

 

As the name already suggests, this is a bonus to reward 3D WR attempts, not to reward buying expensive systems. So, these points are awarded based on the world record page, which contains the results of HWBoints-participants users, the non-participating users and the scores using unreleased hardware which receive no points. ALL of these scores have an affect on the 3D WR bonus; in other words, if someone breaks the world record using unreleased technology, then this will affect the 3D WR bonus points.

 

Why?

 

These points are WR points. World records are what they are: world records. They are not based on anything related to 'being fair' or anything related to this concept. The rankings that are fair fall under the HWBoints algorithm.

 

Now, points are awarded to the best 20 scores of each benchmark application, as an added bonus to the existing hwboints. The points are awarded as follows:

 

 

We've added a poll to see if everyone can live with this. If you have any comments, don't hesitate to throw a reply into the thread. But please, keep it civil and understandible. For complaints, go here.

[TerraRaptor]

Sounds good, imho. That makes sense to bench top CF and SLI for WRs. And as far as 01 & AM are mostly cpu dependant in case of hi-end gpus, that would suit as single GPU benchers, so multi GPU fans in case of 03/05/06 and Vantage. I.e. you can get your bonus using single GPU, so you can get your bonus in case of CFX.

[thebanik]

Even the No. 20th Position would be taken by one of the top most overclocker in the world, is it fair to award such low points to scores which are the best in the world???

[TerraRaptor]

That is bonus. It would be added to general hwpoints. It is just a reason to get higher.

[K404]

I voted against. The top score is always going to be a world record, theres already a points bonus by being No.1 over No.2 etc

 

The top rankings are also dominated by guys who arent buying their own gear, It... doesnt feel right to award them further for pushing things that most guys cant, because they've had to pick which new toy they can afford with their payslip.

 

Thats not meant as a disrespect to the guys involved... it just doesnt feel right to me.

 

 

A while ago I had an idea to reward those who take all the No.1 spots in a given category. That idea got shot down... I think this has less merit.

[speddy411]

If this adjustment (and the adjustment for old hw) goes online Rev3 will be perfect for me...

[SoF]

sounds interesting but it has some minor problems: does not fix the problem of low 2x / 3x hwboints (e.g. ranked top20 with 2x 260 GTX @LN2 in 01 SLI) as it will mix all numbers of GPUs as far as I understood. As it is a "one-time-Bonus" it will only help some guys at the top to extend their lead maybe but unlikely help to balance the single-gpu weight. And 10th get's 20 points while first gets 150 - no doesn't sound good for me to balance anything, before doing strange things like that better leave it out.

[Massman]

That's exactly what the bonus is for: absolute WR attempts. Not to fix low competition.

[r1ch]

While I am all for rewarding world records, I think the current Rev 3 already does this. This "extra" points total would take things too far.

 

For example, take the Aquamark #4 postion, 1x HD 5970, stock cooler, not overclocked.

 

http://hwbot.org/result.do?resultId=925857

 

Scores 29 points already, with this change it would score 110 points I think that's too much.

[Massman]

And 10th get's 20 points while first gets 150 - no doesn't sound good for me to balance anything, before doing strange things like that better leave it out.

 

A steep curve is needed to differentiate those who actually break the world record by going all LN2 from those who want to come close only using air vga's.

 

Absolute WR = absolute WR.

[SoF]

I got the point and I think it would unbalance what you have just tried to bring into the overclocker's league with rev3. The bonus system seems heavy and so you need the big cards again for the first 10-20 places.

My initial complain was about no even ranking for 1/2/3/4 GPUs, e.g. giving each category 1-200 places (or 1-500 for famous) for global and everybody get's (fixed) points according to his place or by percentual computation from the first rank (WR = 100%).

[SF3D]

Current point system is taking too much away from WR attempts. For example, when I did my 3DM01 runs last summer and got 135K, it was extremely hard work and took time and money from my pocket. In this current system you will get 14,7 points with that score and it is not fair at all.

 

Yuo can use ~5.5GHz E8600 and stock single GPU vga near stock speed to get same amount of global points as my 01 run: Ranking number 110. http://hwbot.org/result.do?resultId=847873

 

That is not fair at all, if you ask me.

 

This new suggestion Massman made here is to give some bonus for ultimate WR attempts. I think this needs to be done. This way game will be really equal to everyone joining the overclockers league.

[r1ch]

Current point system is taking too much away from WR attempts. For example, when I did my 3DM01 runs last summer and got 135K, it was extremely hard work and took time and money from my pocket. In this current system you will get 14,7 points with that score and it is not fair at all.

 

I take nothing away from your previous score, we've all spent time, money and effort getting scores that have now lost a lot of points, but that was always going to happen with the change.

 

You have got to give the current system time to settle down.

 

If you benched a GTX 280 or 285 in the same way you did before, you would score lots of points.

 

We should be looking into the future, seeing what is the best for future scores, not what makes past score 'fair'.

[K404]

Current point system is taking too much away from WR attempts. For example, when I did my 3DM01 runs last summer and got 135K, it was extremely hard work and took time and money from my pocket. In this current system you will get 14,7 points with that score and it is not fair at all.

 

Yuo can use ~5.5GHz E8600 and stock single GPU vga near stock speed to get same amount of global points as my 01 run: Ranking number 110. http://hwbot.org/result.do?resultId=847873

.

 

Points dont make or break achievements though. People still recognise the work you've done. Like you said... the work was done last year. Who remembers 16K in 3D03 or 40K in 01? Those achivements are worth nothing now, points-wise, but guys still pushed hella-hard to get there first.

[Guest cowgut]

something has to be done with this idk what thuo...you have to reward the top scores in each bench...its a wr for gosh sakes.

it has to be fair for the pro's too.

[SF3D]

r1ch - You missed my point a bit I guess. There will be always new WR's with new hardware. Only the highest score in some benchmark is world record. Not some single GPU run, no matter how good it is. (Unless real WR can be taken with single GPU)

 

These new WR's have to get some bonus. That is the "fair" part in there.

 

My example was too personal and that score is done last summer, so it wasn't good for it. I mean, that this change is needed for the future. That way game will be fair for all type of users.

 

 

 

K404- That is a good point. I think old records should be in some "museum of overclocking", but I think we don't have time to make such a thing. It would be cool to see, who was the first to break 1GHz barrier and under 1 minute in Superpi 1M.

 

I did not ment my post that way, cause I am looking in to the future.

[Massman]

I got the point and I think it would unbalance what you have just tried to bring into the overclocker's league with rev3. The bonus system seems heavy and so you need the big cards again for the first 10-20 places.

My initial complain was about no even ranking for 1/2/3/4 GPUs, e.g. giving each category 1-200 places (or 1-500 for famous) for global and everybody get's (fixed) points according to his place or by percentual computation from the first rank (WR = 100%).

 

That would mean that we litterally equalize "effort/skill" and "able to buy hardware" in terms of points and reward. I'm sure that's not how any ranking should be.

[TerraRaptor]

As far as I'm unable to spend much for WRs in nearest future, I won't envy they can - even money they put in hobby are worth of this bonus. I like others to take dizzy heights - that is cool!) Hope me to be one of them. I think guys from top-50/100 should speak on this adjustment.

Edited January 6, 2010 by TerraRaptor

[Warrior_oF_Byte]

I Vote yes, but the first position have to earn 100boints, because 150 is too much!

[K404]

The points difference between 1st, 2nd, 3rd etc is already big enough... why make it worse?

 

3D01:

1st place:154

2nd place: 112

3rd: 67

[r1ch]

r1ch - You missed my point a bit I guess. There will be always new WR's with new hardware. Only the highest score in some benchmark is world record. Not some single GPU run, no matter how good it is. (Unless real WR can be taken with single GPU)

 

These new WR's have to get some bonus. That is the "fair" part in there.

 

My example was too personal and that score is done last summer, so it wasn't good for it. I mean, that this change is needed for the future. That way game will be fair for all type of users.

 

I don't think this change is fair for all types of users - this is going back towards Rev2.

 

Why can't Rev 3 be given a chance to settle down? At the moment the single GPU categories are very hit and miss for the quality of results.

 

Once there are more results in the single GPU categories, it will take much more skill to rank highly in the single GPU ranking than to bench 5 pots.

 

Just because a user can manage 4 cards, and 5 pots, doesn't mean they're not being rewarded in hwpoints - they can prove their skill and earn their deserved points with one of the cards - this is the Rev 3 competition. The single card category is where overclockers skill can be fairly evaluated without money or sponsorship getting in the way. This is what hwboints are for and the reason Rev 3 was introduced, was it not?

 

This is the whole point of Rev 3 which has barely been given a week to settle in.

 

WR's are a significant achievement, yes I completely agree, but I don't think they should be given more points (in terms of the pure ranking for differentiating who's a better overclocker).

 

Maybe this is the perfect opportunity for Achievements to be promoted more. "Ruled the rankings: 3DMark01", "I was the king: 3DVantage" etc? Maybe there should be a "World Record" medal that appears next to someone's name when they've been skilled enough to take a world record? I think this is the road we should be going down.

 

If this must go ahead, I would like to see it delayed for a week, a month, something to let the Rev 3 settle in a get a better idea for how the points distribution works out and the skill required to get those points. I would also like to see the points greatly reduced, and the number of positions reduced to the top 3, or maybe top 5.

 

The points difference between 1st, 2nd, 3rd etc is already big enough... why make it worse?

 

3D01:

1st place:154

2nd place: 112

3rd: 67

 

Those points are only achievable to the single card categories though, I think?

 

I think hwbot is in a difficult place - you are trying to give people who don't have massive sponsorship and endless money the chance to compete, but trying to keep those people who do have that happy as well.

 

I don't envy you the decision!

Edited January 7, 2010 by r1ch

[stummerwinter]

For hwbot without awarding WRs open the gap between bot and WRs and make people to decide only one thing...

 

I think, WRs need to be awarded...

 

But anyway people like kingpin and AndreYang are in the lead, 3.0 didn't changend that much here...

[r1ch]

For hwbot without awarding WRs open the gap between bot and WRs and make people to decide only one thing...

 

I think, WRs need to be awarded...

 

But anyway people like kingpin and AndreYang are in the lead, 3.0 didn't changend that much here...

 

If points are the only reason to attempt a world record then I think this is very sad.

 

Compare this to footballers who get paid £100k a week to play football because they are the best in the world for their clubs but then refuse to play for their country at the world cup "because they don't get paid enough".

 

On the development server, kingpin dropped to 4th I think. He benched 1x5870 quickly and that's the only reason he's #1. It will take time for everyone to do this - not everyone can react so quickly.

 

You are riight at the end, the best overclockers - people like you, hipro, kingpin and andre are still there at the top of the leaderboard!

[gradus]

I Vote yes, but the first position have to earn 100boints, because 150 is too much!

 

+....but i think 100 is too much too.....50-75 bonus for 1st place is normal.

[stummerwinter]

For me it's less because of the points, it's more about getting hwbot as an WRs-ranking system, and one thing is from my point of view to award people (with points) to attract them submitting scores...