Adjustment for Global Points - Work in progress

[Massman]

Thanks for the feedback everyone. This week I will try to run another Adjustment test with:

• keep the less steep slope for globals
  
• keep reduced threshold for globals
  
• not increase the maximum points for globals*
  
• improve slope for hardware
  
• reduce threshold for hardware
  

(*): it is inevitable that the the global points contributing to your total will increase if we make the point slope less steep and reduce the threshold as both actions increase the points for positions below 10 by quite a lot.

[Massman]

Then Id also add the new way of calculating points to HW points (stay with 50 as maximum), for really.. HW points are where it is, Globals shows that you spend alot on binning the latest CPU.

Just look at the 900 series for example, 980ti gets all the globals but so few bench it compared to for example 970 that it hardly gives any HW points atm.

 

I had a quick peek at the usage figures for the GeForce 900 series graphics card and found something interesting. The GTX 980 has 21% more users than the GTX 970, 25% higher Max HWpts across all hardware rankings and 173% more total hardware points across all hardware rankings.

 

It may be worthwhile for hardware rankings to calculate the popularity (partially) based on the overall usage of the hardware model rather than specifically for a hardware ranking (defined by model and benchmark).

 

[TABLE=head]GPU Model|Users|Max HWpts|Total HWpts

GeForce GTX 950|13|2|168.1

GeForce GTX 950M|2|2|44

GeForce GTX 960|99|9.4|427.4

GeForce GTX 960M|10|2|111.6

GeForce GTX 965M|1|2|2

GeForce GTX 970|395|39.8|3847.6

GeForce GTX 970M|12|2|102.2

GeForce GTX 980|477|49.6|10537

GeForce GTX 980 (Notebook, MXM)|1|2|16

GeForce GTX 980 Ti|248|28|3337.7

GeForce GTX 980M|20|2|173.7

GeForce GTX Titan X|130|16.4|1485.8

[/TABLE]

 

//EDIT with some additional information. In the chart below you can find three data-sets:

• Participation: amount of users in a certain hardware ranking
  
• Max HWpts: maximum hardware points in a certain hardware ranking (typically for first place)
  
• Total HWpts: volume of all hardware points in a certain hardware ranking
  

We see that 64.50% of the hardware rankings have up to 100 users participating in it. With 100 partitipants, the maximum hardware points is 19.6pts, which is ~40% of the absolute maximum. The 64.50% accounts for 27.60% of all hardware points.

 

At 250 participants the Max HWpts nears the absolute maximum of 50pts. Up to 250 participants accounts for 79.25% of the results. It accounts for 51.15% of the total volume of hardware points.

 

Edited December 22, 2015 by Massman

[Schmuckley]

In b4 I drop 30 places.

[Rauf]

Thanks for the feedback everyone. This week I will try to run another Adjustment test with:

• keep the less steep slope for globals
  
• keep reduced threshold for globals
  
• not increase the maximum points for globals*
  
• improve slope for hardware
  
• reduce threshold for hardware
  

(*): it is inevitable that the the global points contributing to your total will increase if we make the point slope less steep and reduce the threshold as both actions increase the points for positions below 10 by quite a lot.

I like this change, seems good and would address many of the current drawbacks!

[lanbonden]

Thanks for the feedback everyone. This week I will try to run another Adjustment test with:

• keep the less steep slope for globals
  
• keep reduced threshold for globals
  
• not increase the maximum points for globals*
  
• improve slope for hardware
  
• reduce threshold for hardware
  

(*): it is inevitable that the the global points contributing to your total will increase if we make the point slope less steep and reduce the threshold as both actions increase the points for positions below 10 by quite a lot.

 

i think this looks beter but to counter your * that everyone gets more oc points you might want to look into is to run a test where you reduce the maxium global points to for example 100 points aswell.

[Taloken]

Thanks for the feedback everyone. This week I will try to run another Adjustment test with:

• keep the less steep slope for globals
  
• keep reduced threshold for globals
  
• not increase the maximum points for globals*
  
• improve slope for hardware
  
• reduce threshold for hardware
  

(*): it is inevitable that the the global points contributing to your total will increase if we make the point slope less steep and reduce the threshold as both actions increase the points for positions below 10 by quite a lot.

 

This sounds good indeed, what about apply this progressively on the time ? (Disclaimer : potential bullshirt idea)

 

Like a brand-new VGA will keep HWP as it is now (and why not less), but older card get a boost in his HWP each year (or 6 months, or month, but it's time consuming and a more frequent update need more precise calculs)

 

Let me take an exemple (with arbitrary numbers, just to show my mind) :

 

Year 3015

- GTX 800 and R9 800 are brand-new cards. HWP thresold is X, slope is Y.

- GTX 700 and R9 700 are last year cards. HWP thresold is lowered X-5%, slope is increased Y+5%.

- GTX 600 and R9 600 are 2-years old cards. HWP thresold is lowered X-7%, slope is increased Y+7%.

- GTX 200 and R9 200 are old-school cards. HWP thresold is lowered X-20%, slope is increased Y+20%.

 

Year 3016

- Release of GTX 900 and R9 900. Standard HWP thresold/slope.

- GTX 800 and R9 800 are now 1-year old. HWP thresold is lowered X-5%, slope is increased Y+5%.

- GTX 700 and R9 700 are now 2-years old. HWP thresold is lowered X-7%, slope is increased Y+7%.

- GTX 600 and R9 600 are now 3-years old. HWP thresold is lowered X-10%, slope is increased Y+10%.

- GTX 200 and R9 200 are more-old-school cards. HWP thresold is lowered X-25%, slope is increased Y+25%.

 

And so on, with of course a cap on the HWP awarded by the time cards get older.

 

Wanna GLP ? Buy new-gen.

Wanna HWP ? Dig old-gen.

 

Same can be applied to CPU if you want.

 

Ofc, this suppose we have the release date of each model in database, and if not, time to recruit an army.

Also (less impacting), theres the issue of rebranded AMD as 2 or 3 gen can share the same GPU ID.

[Doug2507]

No point increasing reduction in threshold as time goes by for hw. Longer hw is out then more subs, less need for increased reduction.

 

Agree with Rauf, looking forward to seeing how new test works out. Sounds promising PJ.

[Taloken]

Yep but there's a time where HW dies.

 

Almost nobody bench 8800 GT today (except when Country Cup).

 

With a valorisation, it will force people to dig for old cards every time.

Edited December 22, 2015 by Taloken

[Mr.Scott]

This sounds good indeed, what about apply this progressively on the time ? (Disclaimer : potential bullshirt idea)

 

Like a brand-new VGA will keep HWP as it is now (and why not less), but older card get a boost in his HWP each year (or 6 months, or month, but it's time consuming and a more frequent update need more precise calculs)

 

Let me take an exemple (with arbitrary numbers, just to show my mind) :

 

Year 3015

- GTX 800 and R9 800 are brand-new cards. HWP thresold is X, slope is Y.

- GTX 700 and R9 700 are last year cards. HWP thresold is lowered X-5%, slope is increased Y+5%.

- GTX 600 and R9 600 are 2-years old cards. HWP thresold is lowered X-7%, slope is increased Y+7%.

- GTX 200 and R9 200 are old-school cards. HWP thresold is lowered X-20%, slope is increased Y+20%.

 

Year 3016

- Release of GTX 900 and R9 900. Standard HWP thresold/slope.

- GTX 800 and R9 800 are now 1-year old. HWP thresold is lowered X-5%, slope is increased Y+5%.

- GTX 700 and R9 700 are now 2-years old. HWP thresold is lowered X-7%, slope is increased Y+7%.

- GTX 600 and R9 600 are now 3-years old. HWP thresold is lowered X-10%, slope is increased Y+10%.

- GTX 200 and R9 200 are more-old-school cards. HWP thresold is lowered X-25%, slope is increased Y+25%.

 

And so on, with of course a cap on the HWP awarded by the time cards get older.

 

Wanna GLP ? Buy new-gen.

Wanna HWP ? Dig old-gen.

 

Same can be applied to CPU if you want.

 

Ofc, this suppose we have the release date of each model in database, and if not, time to recruit an army.

Also (less impacting), theres the issue of rebranded AMD as 2 or 3 gen can share the same GPU ID.

 

That's bunnyextraction.

Let's everybody jump on the current gen or die bandwagon now.

Why penalize old gen hardware benchers that are a non factor to current gen benchers anyways?

Edited December 22, 2015 by Mr.Scott

[Taloken]

Where you see a penalization ?

 

New hardware : Normal GLP, normal HWP

Old hardware : Normal GLP, increased HWP

 

My idea is about make people go on oldies to get HWP (GLP will always be win with new gen).

 

 

Edit : Forget about X/Y and %, look it as a good wine : the older it get, the better it reward.

Edited December 22, 2015 by Taloken

[Vivi]

with regards to that xtu picture massman posted, i would not eff with the slope too much, top 3 or 5 should always get a lot more points than below.

[Mr.Scott]

Where you see a penalization ?

 

New hardware : Normal GLP, normal HWP

Old hardware : Normal GLP, increased HWP

 

My idea is about make people go on oldies to get HWP (GLP will always be win with new gen).

 

 

Edit : Forget about X/Y and %, look it as a good wine : the older it get, the better it reward.

I'm sorry. I misunderstood. Need to read a little slower.

[xxbassplayerxx]

 

 

 

This ^^^ , the points system is already heavily skewed to globals, if the solution is increasing the amount of globals given out, please leave it alone.

 

I still have hopes that HWBOT will one days recognize time, knowledge, and skill in all generations equally and steer away from rewarding the biggest wallets.

 

I think the idea to increase the points for people that didnt place first is good and I think the idea to decrease the cap for how many submissions before the global reach the maximum is good BUT Id go the other way and reduce the maximum globabal possible to 100 points instead of increasing it too 200 points.

 

Then Id also add the new way of calculating points to HW points (stay with 50 as maximum), for really.. HW points are where it is, Globals shows that you spend alot on binning the latest CPU.

Just look at the 900 series for example, 980ti gets all the globals but so few bench it compared to for example 970 that it hardly gives any HW points atm.

 

I agree with both here. Global points carry quite a bit of weight maxing out at 167 points at the moment. I would like to see these scaled back to 100 points max and I agree with lessening the slope on the point distribution but I feel there could be a happy middle between the current and proposed solutions. Not as steep as the original but more steep than the proposed.

[zeropluszero]

Yep but there's a time where HW dies.

Almost nobody bench 8800 GT today (except when Country Cup).

With a valorisation, it will force people to dig for old cards every time.

Speak for yourself.

[Rasparthe]

Speak for yourself.

 

Aye, +1 to this, if I could get away with only benching vintage hardware and still participate in competitions, I would.

 

I think the general conflict here comes from the fact that some guys are heavily into newest generation (maybe because HWBOT rewards that but probably because next generation is always cool) and some that are heavily into vintage or hardware benching, and then a great number that are in the middle.

 

Maybe HWBOT should allow each individual overclocker to make the choice for themselves.

 

Perhaps every overclocker is given a set number of submissions and you can spend them how ever you wish. If the max for Globals is 100 (as some have proposed) and the max for hardware is 50 (as currently allowed), So every global score will cost you 2 submissions and every hardware submission will cost you 1 submission. If the max submissions that count are (throwing out numbers) 40, you can split them how ever you wish. Put all 40 points in globals (20 submissions allowed), or all points in hardware (40 submissions allowed).

 

It would be equal playing field for both kinds of overclockers if you allowed the potential max points for each submission to be equal. That is why I proposed 2 hardware subs equals 1 global. If each overclocker is allowed to reach the same potential points but they can choose how its done, perhaps it wouldn't feel like an unlevel playing field for one type of overclocking or the other.

 

Just another idea to muddy the waters even further.

[Mr.Scott]

Aye, +1 to this, if I could get away with only benching vintage hardware and still participate in competitions, I would.

 

I think the general conflict here comes from the fact that some guys are heavily into newest generation (maybe because HWBOT rewards that but probably because next generation is always cool) and some that are heavily into vintage or hardware benching, and then a great number that are in the middle.

 

Maybe HWBOT should allow each individual overclocker to make the choice for themselves.

 

Perhaps every overclocker is given a set number of submissions and you can spend them how ever you wish. If the max for Globals is 100 (as some have proposed) and the max for hardware is 50 (as currently allowed), So every global score will cost you 2 submissions and every hardware submission will cost you 1 submission. If the max submissions that count are (throwing out numbers) 40, you can split them how ever you wish. Put all 40 points in globals (20 submissions allowed), or all points in hardware (40 submissions allowed).

 

It would be equal playing field for both kinds of overclockers if you allowed the potential max points for each submission to be equal. That is why I proposed 2 hardware subs equals 1 global. If each overclocker is allowed to reach the same potential points but they can choose how its done, perhaps it wouldn't feel like an unlevel playing field for one type of overclocking or the other.

 

Just another idea to muddy the waters even further.

 

Mmmm.....I don't know Tony. I hate the idea of a limited amount of subs. I have guys that will max that out in a day.

[Rasparthe]

Mmmm.....I don't know Tony. I hate the idea of a limited amount of subs. I have guys that will max that out in a day.

 

Me too, you know that I would prefer unlimited subs that count for league, but even now we are limited to 20 subs in hardware and 15 in global. I'm not talking about the Hardware Masters league as I hope they don't change anything there (finally closing in on 100!), but this seems like an easy fix that allows both camps to bench where their interests lay but still compete against each other in a level playing field.

 

The slope and amounts can be adjusted for sure, I was just using 40 as comparable number. If globals want to be worth more say 200 then adjust the amount of hardware subs that equal globals accordingly. Its all about keeping the POTENTIAL points the same.

[lanbonden]

Aye, +1 to this, if I could get away with only benching vintage hardware and still participate in competitions, I would.

 

looking forward to seeing you in next round of old school is best school that starts with the new year then

[Rasparthe]

looking forward to seeing you in next round of old school is best school that starts with the new year then

 

I plan to be there, although still trying to source SLOT A, one platform I don't even have a board and chip for.

[Rauf]

Thanks for the feedback everyone. This week I will try to run another Adjustment test with:

• keep the less steep slope for globals
  
• keep reduced threshold for globals
  
• not increase the maximum points for globals*
  
• improve slope for hardware
  
• reduce threshold for hardware
  

(*): it is inevitable that the the global points contributing to your total will increase if we make the point slope less steep and reduce the threshold as both actions increase the points for positions below 10 by quite a lot.

 

Been thinking more about possible changes. I think adjusting the slope for globals is very important and can also have a positive effect on 3D-popularity. As it is now once you reach into the mid thirties on your lowest globals, benching is not very rewarding. You have to have a top 5 cpu to really gain points (more than 5-10 points) in the 2D-department. If you look at 3D you have to have a top 3 position (exept last gen 3Dmark) to really be able to gain points. If you look at multi-GPU it's often only first or second places that have considerably more points than 35-40. When you consider a majority of "3D"-benches is actually a CPU-bench it gets even more discouraging. It's either find a top 5 CPU or quit.

 

If the slope is adjusted so that it's more rewarding to get a top 10 position this should help 3D especially because it's here the "competition" is at its lowest right now. More points to top 3-10 positions should help make a lot of the 3D-benches more popular.

 

However, placing 1-3 should of course still give significantly higher points than the rest, but not as much as now. Maybe first is X points, second is X*0.85, third is X*0.75...

[Massman]

Responding to @Rasparthe and @Mr\.Scott's conversation, I think we should definitely tip the scale back in the direction of skill and effort rather than the latest hardware. I am not particularly keen on eliminating the necessity to master the latest platform for being ranked high in the League. Historically speaking, competitive overclocking grew mainly because of the challenges that new hardware posed and the drive to get the highest benchmark score. That's what we saw with Bunny's SuperPI leaderboard and how the 3DMarks become so popular. The paradigm of the Overclockers League has always been to identify the most complete overclocker. That's why it takes into account the global rankings for the latest hardware, the hardware points for older platforms and the competitions for time-restraint overclocking.

 

The binning aspect of contemporary overclocking is something I am personally not very keen on and can be addressed partially by adjusting the slope for globals. In terms of finding the right balance between global and hardware points, we can adjust the amount of scores which contribute to the league total. In the past, we have made such arbitrary decisions to balance the two by capping hardware points to 500. It's justified to consider doing it again. I like the idea of matching the maximum potential points of both categories. For example, if the maximum global points is 150 and there are 15 global score contributing to the league total, we can allow for 45 hardware scores which max out at 50 points.

 

Important note on the hardware points: we have an incredible large spread between the least active and most active hardware rankings. It goes from 1 participant to 5650 participants. That makes it incredibly difficult to design a single algorithm which allows for healthy reward in the categories with up to 100 participants and the ones with thousands of participants. In addition, we have to keep in mind the the XTU categories have high participation so there will be people grinding the mobile rankings to get the maximum amount of points.

 

On the topic of testing out parameter Adjustments

 

Our UAT server is slightly under-powered to run this kind of massive calculation and it seems some ranking recalculations are getting out of sync. Regardless, I found that the XTU 4xCPU ranking was updated correctly and here are the comparisons.

 

Global ranking: less steep slope, reduced the maximum global points to 150. Left: current; right: new.

 

 

Hardware ranking: less steep slope, increase the maximum global points to 52. Left: current; right: new.

 

(Can anyone give me a couple of reference hardware rankings to tune the hardware points? What I would need are rankings which are fairly competitive but under-rewarded and rankings which are not competitive and over-rewarded)

 

[Doug2507]

 

I think the general conflict here comes from the fact that some guys are heavily into newest generation (maybe because HWBOT rewards that but probably because next generation is always cool) and some that are heavily into vintage or hardware benching, and then a great number that are in the middle.

 

Maybe HWBOT should allow each individual overclocker to make the choice for themselves.

 

Perhaps every overclocker is given a set number of submissions and you can spend them how ever you wish. If the max for Globals is 100 (as some have proposed) and the max for hardware is 50 (as currently allowed), So every global score will cost you 2 submissions and every hardware submission will cost you 1 submission. If the max submissions that count are (throwing out numbers) 40, you can split them how ever you wish. Put all 40 points in globals (20 submissions allowed), or all points in hardware (40 submissions allowed).

 

It would be equal playing field for both kinds of overclockers if you allowed the potential max points for each submission to be equal. That is why I proposed 2 hardware subs equals 1 global. If each overclocker is allowed to reach the same potential points but they can choose how its done, perhaps it wouldn't feel like an unlevel playing field for one type of overclocking or the other.

 

Just another idea to muddy the waters even further.

 

I think this is getting off the beaten track and away from original OP with regards to 2d/3d balance....

 

Don't increase global points, smooth out slope a bit (still giving top 10 (ish) the bigger % point reward as deserved), lower threshold. Job done. No point complicating matters further imo.

[Noxinite]

HW points slope looks much better. DDR2 mem frequency for under-rewarded?

[ObscureParadox]

Just curious if we can have a link to the test server so that we can first hand browse it and see the effects for ourselfes. Some people like myself just find it easier to go on and figure it out ourselfes.

[GENiEBEN]

Just curious if we can have a link to the test server so that we can first hand browse it and see the effects for ourselfes. Some people like myself just find it easier to go on and figure it out ourselfes.

 

http://uat.hwbot.org/