Jump to content
HWBOT Community Forums

R4.2.4/R4.3: HWBoints finetuning


Massman
 Share

Recommended Posts

As mentioned earlier in this subforum, we have been looking at the feedback we received from the transition to HWBOT R4 and the changes it brought to the point algoritm. We're currently in the process of finetuning the boints and will introduce all changes in one big package which will be included in either R4.2.4 (due 19/11) or R4.3 (due 01/12).

 

FYI - next to the updates mentioned below, we will also implement a fix for the existing problems with the Global Team Powerpoints as described here. The fix is currently planned for R4.3, but the development is R4.2.4 is going so incredibly smooth that we are considering moving the fix to the current development sprint. If we can't include the GTP fix in R4.2.4, we will postpone the fixes described below and package them in R4.3. Just a matter of keeping things coherent.

 

Changes:

 

Currently, we have installed two updates to the member hwboints algoritm for global rankings:

 

1) participation weight based on 12-month period instead of 6 months

2) introduce min_participation constant

 

Background information:

 

The current hwboint algoritm used for calculating the points awarded to global member rankings is, just like the very first hwboint revision, mainly based around the amount of competition there is within the ranking. In Rev4, we introduced a time limitation to calculated the level of competition within one ranking. From Rev4, competitivity is determined by the amount of different users who have submitted to a certain global ranking over the past 6 months.

 

Based on feedback such as "I'm losing points in Wprime if I submit more results", we found out that a six month period is quite short. We also found that the drop in points is quite big if, for some reason, a benchmark is suddenly not so popular anymore. Extreme scenarios can be found in Wprime where the global points for #1 position dropped from 133p (without 6-month period) to 13,1p (with 6 month period).

 

To resolve this situation we've prolonged the period for calculating the level of competition in a specific ranking as well as introduced a minimum participation level for every global ranking.

 

Practical consequences:

 

Most importantly, practically this means that pretty much everyone is gaining points. Hooray!

 

On a more serious note, these two updates do have some interesting consequences. For one, in case a benchmark is no longer popular, the points will drop at a much lower speed than they did now. Secondly, this also means that every global ranking has a minimum amount of points. For every #1 global rank, you will now receive a minimum of 59,8 points. This is quite interesting for:

 

- people who frequently run 3/4-way setups, but not subzero

- for enthusiast league members who score top scores in overall benchmark rankings

- 1xCPU multi-threaded CPU applications (where competitivity drops due to lack of newly released single core cpus)

- unpopular but challenging benchmarks (eg: pcmark)

 

In general, you can just think about this update as a guarantee that a global #1 will always receive a minimum of 59.8p. I also want to stress again that these fixes only affect the points awarded in global member rankings and that the formula for calculating the league ranking is not affected. In other words: the game is completely the same, only a few more ways to be competitive.

 

Test environment:

 

This weekend, we'll updated our test server environment with a new dump from the result database and open it up for you to check it out and see your rank. For now, here are just some examples.

 

Some examples:

 

- Booj' 3xGPU Heaven DX11: 5.1p -> 59.8p

- Ananerbe's 1xCPU Wprime 1024M record: 11.0p -> 59.8p

- **DP**'s 3xGPU Vantage record: 8.1p -> 59.8p

- Nacho_arroyo's 6xCPU Wprime1024M record: 9.6p -> 59.8p

- Moonman's 3xGPU 3DMark01 record: 6.3p -> 59.8p

 

Example of the updated leagues. Note: this is using an outdated dump of the production database. It's very likely that your latest submissions are not taken into account yet.

 

pro1.jpg

 

ranking.jpg

 

enthusiast.jpg

Link to comment
Share on other sites

  • Replies 78
  • Created
  • Last Reply

Top Posters In This Topic

im confused x 10^2

 

What's there to be confused about? To sum up massman's lengthy explenation in one sentence: #1 in global points will always get you about at least +-60 points, no matter how unpopular the ranking may be.

 

Another change scheduled for the near future is a better way to handle team power points, but that has not been implemented yet.

Link to comment
Share on other sites

Well I guess if the unpopular crap gets awarded a lot more points, more people will be motivated to bench and fewer categories will be unpopular.

 

Good for keeping a wide selection of hardware under some form of spotlight, bad (open to interpretation) because again points are the motivator to bench. Points bring binning. Binning starts to undermine the "little guy"

 

I know not everyone feels the same way, that's cool :)

 

 

 

Weirdly, getting a good score with.... e.g.... an AMD tri-core is worth more points because it has a global ranking than it does for being any kind of popular chip

Edited by K404
Link to comment
Share on other sites

FYI, the point distribution comparison between "uncompetitive Wprime32M 3xCPU" (global points) and "competitive 8800GTX 3DM01 1xGPU" (hardware points).

 

Beating lots of people in a hardware class yields more points (in general) than beating few people in a global class.

 

gl-distribution.jpg

 

hw-distribution.jpg

Link to comment
Share on other sites

So, the competition points are still not implemented? The ideea is still on the "to do list"?

 

Ontopic: I don't think it's a good ideea to give so much global points to all the shitty overall's WR :D. Makes the WR's that really matters (single card 3D for example) less important...

Link to comment
Share on other sites

@ RB.... Either too many points for #1 or not enough for #10 :D

 

Massman and me spoke about this once.... what if the points increase was smoother, but with "bumps" at.... eg.... 100, 50, 20, 10 and 5

 

pureawesome.jpg

 

@PJ.... at the moment, there is a difference. If there is enough of a reason "for the masses" to bench the category then the points gap will close, you know that :)

Edited by K404
Link to comment
Share on other sites

My profile says I got 12 points from competitions but I cannot find any details about that.

I'm reffering to MOA, GOOC, Country Cup, etc. Where can I see what points I have and how can I add the competitions I've won? :)

 

FYI, the difference between "shitty overall wr" and a "single gpu wr" is over 100p: 59.8p (eg: 3x CPU wprime) versus 168p (eg: 1x GPU 3DM06).

 

Yes, but 2 x "shitty overall wr" = "single gpu wr" and that's not exactly fair in my book. Single GPU WR takes much more work to obtain that 2 x 1-core WPrime WR's for example :)

Edited by matose
Link to comment
Share on other sites

?

 

It's 'unreasonable' that the points gap closes when more people compete in a ranking?

 

I honestly don't know. :)

 

 

Basing points purely on popularity....... if there were 10,000 results each of 3DM11 single card and 3xCPU wP32M.... are they equally impressive achievements that deserve equal points?

 

 

Should the least popular global leaderboard be worth 1/3-1/2 of the points of the most popular?

 

 

Not commenting, just giving ya food for thought :)

 

 

EDIT: Matose and me are thinking along the same lines. Quick, everyone take cover! :D

Link to comment
Share on other sites

My profile says I got 12 points from competitions but I cannot find any details about that.

I'm reffering to MOA, GOOC, Country Cup, etc. Where can I see what points I have and how can I add the competitions I've won? :)

 

An overview is coming in R4.2.4 and will be visible under the points tab in your profile. Example:

 

cp.jpg

 

For MOA and GOOC we need additional code to support competitions that are not based on the hwbot database. The code is planned for R4.3 release.

Link to comment
Share on other sites

Why? If there's only a few results in the category.......

 

Because of kinda random categories like the (AMD) 3x CPUs, it's *almost* the same as the hardware ranking. That's not popular, so it only gets a few hardware points. Why should a category with 292 results be worth 60 global points? The PheII 555 only gets 5.8 hardware points

 

 

I have the Unigine DX9 world record and that category has 626 results.... zero ponts. It's a world record isn't it?

 

(I don't want points added for Unigine DX9, but I hope you see my point)

Link to comment
Share on other sites

The points don't define how good someone is 100% accurately; it's merely an estimation using some algoritm that weighs in the amount of competition and the position in a certain category. The idea of "good" is very subjective and everyone has a different opinion about it. For you, "good" may mean having a Single-GPU record, for others "good" might mean scoring top positions on various platforms and architectures.

 

Let's take some examples of those so-called "shitty" records:

 

- Single core Wprime 1024M record (sempron over 6.1GHz)

- Kingpin's 12xCPU Wprime32M record (2 cpus at 5.6G+)

- SF3D's 3xGPU 3DMark11 score (3 GPUs over 1350MHz core)

- Mtech's 4xCPU PCMark05 record (overall record)

 

Each of these scores have some level of difficulty here, but are awarded too little points because there's no competition in the ranking. If someone would try to beat any of these scores, it would take a lot of time and effort to get there. Now, if someone were to be the holder of these 4 records, it would give that person ~ 240p.

 

Do you honestly believe that, for instance, Single GPU 3DMark06 + Single GPU 3DMark05 (together ~320p) is actually underrated in comparison to those 4 other records?

 

How much importance you attributed to a certain ranking depends on how you personally feel about benchmarks, how the benchmarks behave (cpu-limited 3D benchmarks), etc. We all know how the Sandy Bridge has affected the benchmarking scene or how Gulftown affected the scene.

Link to comment
Share on other sites

You're missing the point Pieter :).

If I have 6 x "shitty WR" and you have 3 x "single-card WR" we will have the same points, this makes me as good as you?!

 

Even the "shitty" WRs are hard to beat (unless we're talking about 24x wprime and stuff like that). It's not fair that you can buy some random gulftown and a gtx580, ln2 both, and get plenty of globals in all 3dmarks with no effort, whereas you get only a handful of boints when submitting a score that's actually really good - for example a wprime 3x gold cup. Being top 10 is quite easy in some of those rankings, but making it to top 3 requires alot of skill and effort, so it's a good thing that such submissions will receive an amount of boints that shows the quality of the result.

 

Edit: this shouldn't apply to ALL rankings, only where you can actually overclock - we don't have to give 60 boints to a server ran at stock, that's just silly. But anything up to and including 12x CPU - yes :)

Edited by knopflerbruce
Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

 Share


×
×
  • Create New...