Adjustment for Global Points - Work in progress

[Massman]

The URL is http://uat.hwbot.org/. But as I said, the test server is severely under-powered for resource hungry processes like calculating points for 1.5M results. So certain rankings have not been correctly updated.

[Mr.Scott]

Interesting. I pick up roughly 50 global points but lose 200 places in standings. Team CP loses 3000 points over all.

[Massman]

Please don't look at the global rankings ...

 

I said, the test server is severely under-powered for resource hungry processes like calculating points for 1.5M results. So certain rankings have not been correctly updated.

[Mr.Scott]

Please don't look at the global rankings ...

I understand that it's not exactly accurate, and personally, I don't really care about the global's. I do however care about my team losing 3000 points. That amount doesn't seem to jive with just modification of the global slope, as CP is primarily a hardware points team. I'll keep watching/listening. It would seem I'm not totally understanding this yet.

Edited December 24, 2015 by Mr.Scott

[Massman]

I want to make this extra clear: the comparison of production and UAT overall leaderboards has absolutely no meaning because they are based on incomplete global and hardware recalculations. It is not possible to tell "how accurate it is".

[Mr.Scott]

You were clear. I'm not condemning anything yet.

[K404]

Hello

 

I really like the idea of smoothing out the point differences as scores creep towards #1. I've always felt that the points difference was FAR disproportionate to the typical score difference at that level....especially when it's usually a backup rivalry.

 

I know i've suggested this before...how about a score "bump" as someone enters top 50, top 25, top 10, top 5? Those are "golden rankings" that everyone understands, so pushing to enter the next "zone" gets a proper points motivation.

 

The rest of the ideas..... no comment for now

Edited December 24, 2015 by K404

[Rauf]

It's a step in the right direction. But too big a step. This would take too much of the competition out. Not rewarding enough to go for the top rankings.

 

If the current slope is a black skiing slope, the one tested here is a blue. Then we want to go green or maybe red I think.

 

Hw slope is fine. But threshold is still to high.

[Massman]

I am unfamiliar with skiing terminology.

 

Right now top 3 is defined as: 100%, 95%, 92.5%, and it used to be 100%, 75%, 56.25%. I guess we can try 100%, 90%, 80%.

 

As for Hardware Points: what would be a good target threshold for maxing out on points?

[Vivi]

I think top should be rewarded a lot more. This is like fighting for equality where it shouldn't be a thing.

 

Dancop gets 150 points for spending days, and enormous amounts of money on a top top top tier score, then someone who might have only done one decent session coming 10th, gets 110 points.

 

It will make coming 1st place boring, people won't chase #1. because top 10 is enough and then move on to other benches. If this sport wants more spectators we shouldn't remove the exciting aspect of it, seeing people fight for #1

 

my suggestion is this:

 

100%, 75%, 56.25% /50 /50 /50 /50 /50

 

give more 50 pointers but not more top 3 style points

Edited December 24, 2015 by Vivi

[Doug2507]

Sounds good for top spots. Obviously deserve a good proportion of overall % due to time/effort/skill etc etc needed for them. I'm kind of on side with Rauf on this and think drop away point could maybe be around 10th place.

 

Does HW threshold need changed? I thought original discussion/direction was to reduce 3d threshold for better balance?

[ObscureParadox]

Wait so have the point system changed now on the regular site? I'm seeing submissions with more than 50 points but didn't realise this was happening.

 

Edit : also how is the system changed exactly? I'm seeing that this submission http://hwbot.org/submission/3050898_vadimua_3dmark06_geforce_7600_gt_pcie_gddr3_52.1_hardware_points is worth more than this with 4 times the competition http://hwbot.org/submission/2279464_spl_3dmark06_geforce_8800_gt_512_mb_25088_marks

Edited December 26, 2015 by ObscureParadox

[newlife]

Wait so have the point system changed now on the regular site? I'm seeing submissions with more than 50 points but didn't realise this was happening.

 

Edit : also how is the system changed exactly? I'm seeing that this submission http://hwbot.org/submission/3050898_vadimua_3dmark06_geforce_7600_gt_pcie_gddr3_52.1_hardware_points is worth more than this with 4 times the competition http://hwbot.org/submission/2279464_spl_3dmark06_geforce_8800_gt_512_mb_25088_marks

 

That would be good to know and my hardware points for a 7600gt went up by 11 points same for all my XTU subs

[der8auer]

I think top should be rewarded a lot more. This is like fighting for equality where it shouldn't be a thing.

 

Dancop gets 150 points for spending days, and enormous amounts of money on a top top top tier score, then someone who might have only done one decent session coming 10th, gets 110 points.

 

It will make coming 1st place boring, people won't chase #1. because top 10 is enough and then move on to other benches. If this sport wants more spectators we shouldn't remove the exciting aspect of it, seeing people fight for #1

 

my suggestion is this:

 

100%, 75%, 56.25% /50 /50 /50 /50 /50

 

give more 50 pointers but not more top 3 style points

 

I have to agree with vivi. There has to be a clear reward for top1 and top3 compared to the rest. Maybe not that hard but 100, 80, 75, 70... could be good.

[Guest george.kokovinis]

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

 

My 0.02 cents.

 

In Formula 1 Car racing the Ferrari fans in Monza stand on their toes for

that split second, that will make the difference and the flag will go down for the FIRST Ferrari crossing the line.

 

Passion and enthusiasm.

That is what keeps the champions of every sport fighting till the last drop

of their sweat or LN2 or both.

 

Please keep that in mind.

If you take passion out of the equation and things start getting boring,

I really do not know what the consequences might be.

 

Math is good. I am a true believer of mathematics.

Passion is better. It drives your soul to heaven.

[Mr.Scott]

Did we decide on a formula?, because I see points being changed right now.

[Guest george.kokovinis]

Did we decide on a formula?, because I see points being changed right now.

 

 

Scotty,

 

Seems that decisions are taken at a much higher level.

I really do not know if our opinion is really important at all.

 

Reminds me of something I will never forget.

Some 15 years ago, I was appointed Managing Director of a pretty big

company, with 3000 workers.

Every six months we had a gathering and asked the opinion of the workers

for several aspects considering the company.

 

After each event, the Board, gave me a paper with ORDERS to follow.

One day, the Vice President of the Board told me :

" Let the mob believe that their opinion counts "

[Massman]

The dev accidentally pushed a part of the test algorithm to to production when deploying maintenance code.

 

Hoping for a rollback as soon as possible.

 

Sorry for this.

[Rauf]

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

 

My 0.02 cents.

 

In Formula 1 Car racing the Ferrari fans in Monza stand on their toes for

that split second, that will make the difference and the flag will go down for the FIRST Ferrari crossing the line.

 

Passion and enthusiasm.

That is what keeps the champions of every sport fighting till the last drop

of their sweat or LN2 or both.

 

Please keep that in mind.

If you take passion out of the equation and things start getting boring,

I really do not know what the consequences might be.

 

Math is good. I am a true believer of mathematics.

Passion is better. It drives your soul to heaven.

In F1 they have hardware restriction exactly for the reason to make it more exciting. They put the rules in to level the competion because some companies spent so much money to develop better cars they became too good and races were not exciting any more.

 

I'm not saying we should take the binning aspect out of OC, it is and always will be the most deciding aspect. But maybe it would be good to tone it down a notch.

 

Perhaps 100, 85, 75 and then a bit more slowly downwards.

[speed.fastest]

Scoring system looks good, but i agree 1st place maybe should be 100, 85, 75 and the slowly downward for the 1st place competition

[vadimua]

Good balance is when you put a lot of effort. Improves skills in the hardware modifications and tweaking. Long work, some years, to get a good result.

I can see here, more very easy points to attract more users. This monetization of the project? Cash back points, selfie points, follower points, boints shop. What's next?

Where to vote? I think that the overclocking community would not approve these changes. Especially the destruction of the system hardware points.

 

I don't care about the overall ranking, globals. But I care about the hardware points. HARDware points, it must be HARD, leave them alone.

[Massman]

The dev accidentally pushed a part of the test algorithm to to production when deploying maintenance code.

 

Hoping for a rollback as soon as possible.

 

Sorry for this.

 

Rollback done, points will return to normal very soon.

 

-----------------------------------------------------------------------------------

 

Important update on the Adjustment

 

We discussed the adjustment for the points internally and one of the key problems with the current calculation method is that it's based mainly on the position of a result within a ranking. The consequence of this method is that a new, high score in the ranking will affect many other results and trigger the chain of updater and notification algorithms. For example, if you submit a #5 result in a ranking with 2000 participants, our server needs to:

• update the rank position for 1995 results
  
• recalculate the points for 1995 results
  
• update the league totals for 1995 users
  
• update the team league totals for ? teams
  
• store historical points information for 1995 results and users
  
• create notifications for the result, user and team

In short: a ton of work.

 

In the long run, this is not sustainable. This year we breached the barrier of 1.5 million benchmark results as 25,000 overclockers submitted about 350K results. That's a ton of recalculations, hence we're having recalculation slowdowns once in a while. Since we're trying to improve the points anyway, why not give it a shot.

 

I spent this beautiful afternoon working on a new algorithm that takes into account two parameters: participants and a submission's score in relation to the top score. The algorithm I am playing with right now has the following characteristics:

• Same algorithm for Global and Hardware rankings
  
• GL Min Pts = 1
  
• HW Min Pts = 0.1
  
• Threshold for Global ranking is ~ 1000 participants
  
• Threshold for Hardware is ~ 55 participants
  
• GL reaches Min Pts at ~ 65-70% of top score, depending on popularity
  
• GL Points at 1000 participants for #1, #2, #3 are 150, 135 (90%), 127.5 (82.5%)
  
• HW Points at 55 participants for #1, #2, #3 are 50, 45 (90%), 42.5 (82.5%)
  
• Max points for #1 when 1, 10, 20, 30 participants is respectively: 3.9, 16.4, 26.7, 34.8 points.
  

This is a pretty radical paradigm shift for HWBOT points. I want to emphasize that this is something that will be mandatory in the near future. No other way. These are some of the effects:

• Improving your score will always yield more points (except if you're #1)
  
• The only way to reduce your opponent's points is by improving the #1 score
  
• Equal score = equal points (ie. XTU 2xCPU -> top50 has 100pts)
  
• Less steep slope, so more points
  
• Top-3 always has an extra bonus
  
• A lot more points in HW rankings (easy 20pts), even for uncompetitive rankings
  

As an example, I've taken three rankings from the database and plotted the points in Excel. The rankings are Fire Strike 1xGPU (Global), XTU 2xCPU (Global) and Fire Strike Extreme 1xGPU GTX 970 (Hardware). You can see the points plotted as position in the ranking for the three rankings below.

 

CAVEAT: this is the first time I'm trying out this new algorithm, so please understand that I am aware that the scaling is far from perfect yet. There are three things I want to look into. First, used fixed multipliers for top-10 instead of top-3 to give higher reward for pushing into the top-10. Second, make a steeper slope for the categories with low participation so that being 1st out of 30 gives maybe 20 pts rather than 35. Three, find a way to ensure poorly scaling benchmarks (like XTU) don't generate so much points for everyone. (*)

 

Anyway, that's it for this Sunday. See you all tomorrow! :celebration:

 

 

(*): and of course find a way to include the parameter 'skill' somehow.

Edited December 27, 2015 by Massman

[Vivi]

thanks for adjusting 1st 2nd and 3rd place, i like the look of this system, good work mass!

[Mr.Scott]

I think so too. In theory, it looks pretty good.

[Mr.Scott]

Scotty,

 

Seems that decisions are taken at a much higher level.

I really do not know if our opinion is really important at all.

 

Reminds me of something I will never forget.

Some 15 years ago, I was appointed Managing Director of a pretty big

company, with 3000 workers.

Every six months we had a gathering and asked the opinion of the workers

for several aspects considering the company.

 

After each event, the Board, gave me a paper with ORDERS to follow.

One day, the Vice President of the Board told me :

" Let the mob believe that their opinion counts "

 

We get the same thing at my smaller company.