Jump to content
HWBOT Community Forums

R5: UGP/GTPP fix for exotic categories


Recommended Posts

As expected, the min_points=60 rule for UGP categories has made it possible to dominate rankings by simply running very obscure hardware.

 

http://hwbot.org/league/enthusiast

 

Dead Things is now #1 of the Enthusiast League mainly by hitting the #1 spots in multithreaded benchmarks with his stock clocked single, dual and quad 10-core cpus. That brings 60p x 3 (config) x 3 (benchmarks) for just having the hardware. In these categories, there's almost no competition.

 

I think we should have implemented the variable min_points from the start. This is suggested by a member and seems good.

 

- 10pts for a global with 10 or less submissions

- 20pts for 10-20 subs

- 30pts for 20-30 subs

- 40pts for 30-40

- 50pts for 40-50

- 60pts for 51+

 

The exact point scheme isn't fixed yet, but the scheme Borandi (again!) suggested seems pretty good. Currently the scheme is

 

1 - 60

2 - 45

3 - 33.6

4 - 26.2

5 - 22.4

6 - 21.7

7 - 21.1

8 - 20.5

9 - 19.3

 

If we follow the community suggestion from Borandi, if there are 9 participants in a ranking, the scheme would be:

 

1 - 10

2 - 7.5

3 - 5.6

4 - 4.4

5 - 3.7

6 - ...

...

 

Any comments?

Link to post
Share on other sites
  • Replies 144
  • Created
  • Last Reply

Top Posters In This Topic

Sounds......alright? What are the requirements for >60 points for #1? Most of the 2x/3x/4x 3D leaderboards have 60 points for gold and there are hundreds of submissions..... thousands? 60 points for gold for 50 submissions or 500, it's a big zone for 60 points to cover

Link to post
Share on other sites
Sounds......alright? What are the requirements for >60 points for #1? Most of the 2x/3x/4x 3D leaderboards have 60 points for gold and there are hundreds of submissions..... thousands? 60 points for gold for 50 submissions or 500, it's a big zone for 60 points to cover

 

This COULD be because an additional "feature" got implemented - the engine doesn't count subs that are more than one year old. That's why singlecore wprime rankings, with an insane number of subs, get alot less than you'd think.

 

Get rid of this, pleeeeeeeeeease:D If it's there, maybe I'm just dreaming:o it doesn't really do any good at all, as time has got nothing to do with the quality of a result. A sempron 140 result from two years ago is just as good as a sempron 140 result obtained today, for example.

Link to post
Share on other sites
Sounds ok to me...just because someone has access to exotics shouldn't be a criteria to get more points...afterall...there are no effort involved in those rankings...

no offense intended...

 

This is not entirely true, unless the ranking is REALLY exotic (as in top spot ran at stock). You see the same thing in the popular rankings, although not near the top spots. Buy random highend HW, run full pot, and *bam* you get a bunch of points.:P Not exactly a similarity between points and skill there either:D usually top spots with more than a handful subs are hard to get, as people will still fight for the gold cups. Probably really easy to get top 5, but grabbing gold cups isn't as easy as you think just because there are less submissions:)

Link to post
Share on other sites
^well..am not too well versed with sub zero...so couldn't comment there...but some of the submissions really sticks out like anything....

at times am just like..wtf? no tweaks..nothing..they just ran it and bam? 60 freaking points?

 

 

the problem is that i have access to those stock hardwares and u don't :P

Link to post
Share on other sites

Get rid of this, pleeeeeeeeeease:D If it's there, maybe I'm just dreaming:o it doesn't really do any good at all, as time has got nothing to do with the quality of a result. A sempron 140 result from two years ago is just as good as a sempron 140 result obtained today, for example.

 

I agree with this. Heavily.

Link to post
Share on other sites
This COULD be because an additional "feature" got implemented - the engine doesn't count subs that are more than one year old. That's why singlecore wprime rankings, with an insane number of subs, get alot less than you'd think.

 

Get rid of this, pleeeeeeeeeease:D If it's there, maybe I'm just dreaming:o it doesn't really do any good at all, as time has got nothing to do with the quality of a result. A sempron 140 result from two years ago is just as good as a sempron 140 result obtained today, for example.

 

I agree with this. Heavily.

 

QFT. I think if you're going to implement a feature like this, 12 months is too short - especially when the best hardware could still be the old stuff. Especially on single core wP et al.

 

We all know the basis behind this change was to drive people towards benching the stuff that interests the majority, but there's still just as much enthusiasm going for some of the old tough stuff as the newer stuff.

 

If there is a benchmark, we will run it. So to speak.

Link to post
Share on other sites

Interesting points - I'll bring them up at our next staff meeting (which we hold every day). I tend to agree that even though a time-based algorithm might be more elegant, it's not really bringing to the table. Just more frustration about dropping points.

Link to post
Share on other sites
Get rid of this, pleeeeeeeeeease:D If it's there, maybe I'm just dreaming:o it doesn't really do any good at all, as time has got nothing to do with the quality of a result. A sempron 140 result from two years ago is just as good as a sempron 140 result obtained today, for example.

 

seconded here. many i know bench 2 or 3 gens behind purely for financial reasons. i could never have afforded a Q9770 setup when they came out but now you can find them cheap and have a go at the older scores with everyone rewarded for their efforts not just the recent sub.

Link to post
Share on other sites
  • 2 weeks later...
  • 2 months later...

Small update:

 

The update was planned for this sprint, but will be postponed until the next one. Biggest problem at the moment is figuring out how to not overload the server when recalculating the submissions when adjustment goes live. It takes about 4 days to completely recalculate all submissions.

Link to post
Share on other sites
Small update:

 

The update was planned for this sprint, but will be postponed until the next one. Biggest problem at the moment is figuring out how to not overload the server when recalculating the submissions when adjustment goes live. It takes about 4 days to completely recalculate all submissions.

 

"The update" meaning the Rev 5 changes, or just this points adjustment in this thread?

Link to post
Share on other sites
  • 4 months later...

Alright, update. So, it took a while before we could apply this patch on the production server. The main reason was to determine when we would hurt our little engine by requesting a large recalculation. We've decided to do it this development sprint. Now, before we do so, here are the now and improved algoritms. Because of the complexity of the HWBoint algoritm, we had to adjust more than just the global points for exotic categories to ensure future-proof behavior.

 

We decided to go with a more simple logarithmic distribution. The maximum number of points that can be earned is determined by the number of participants. The idea is that 1st place always gets 100% of this, and lower ranks get a certain percentage, which is based on another logarithmic distribution. Points given are largely the same as with the old system, the major changes are:

  • No guaranteed 60 points for more exotic categories
  • A lot more points can be gained from medium-popular categories (100 to 500 participants)
  • Slightly more points for lower rankings
  • There's a little hiccup for last place though, you actually gain points when more participants join for exotic categories. That's something that can't really be fixed but the impact is relatively minor we think.

 

We ran the new algorithm on a test server with a database snapshot of 23/02/2013. You can check the differences of the top 20 in each league below. Main thing to notice is that rankings largely stay the same, but Dead Things is completely gone (dropped to position 40 with 383 points).

 

Points for first and last place (global): Old versus New algoritm

 

oldNewOneParticipants.jpg

 

XOC Ranking Difference (snapshot 23/02/2013)

 

oldRankingsXOC.jpgnewRankingsXOC.jpg

 

EL Ranking Difference (snapshot 23/02/2013)

 

oldRankingsEL.jpgnewRankingsEL.jpg

Link to post
Share on other sites

Is the database still in the progress of updating? I see some real weird things going on in the Xtreme OC League rankings.

Last week I was ranked at around position 150, over the weekend I jumped to position 96 and today I'm at 482??? Same thing is happening to my teammates Leeghoofd and FunSkilZZ.

Link to post
Share on other sites

Sad day for HWbot, too bad core misunderstanding of what drives HWBOT still alive and well. Can't wait until all points are disabled unless you are using latest CPU (4770k) and latest GPU (Titan). Very disappointing, I'm sure policies that treat certain segments of the site like second class members will continue to grow and expand the membership.

Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.


×
×
  • Create New...