Adjustment for Global Points - Work in progress

[Doug2507]

So this is the same way as challenger is currently scored? Sounds good. Already thought in challanger it's good how you can keep pushing your score for small point improvement even though the next guys score is too high to catch.

[Xtreme Addict]

Hmm Looking at R7 picture I am not sure that such version is good enough. Too low difference in points. I mean 1st place difference to 10th place which is relatively much easier is just 22,4 points. I am not sure if such difference is worth rerunning and pushing higher. I am more like a fan of leaving scaling system like it was in the past, just trying to apply 1.25 or 1.5 base for 3D.

 

The main test is the transformation from rank-based algorithm to percentage-based algorithm. Here's the explanation of why it's necessary. Then it includes the rough version of my latest post. I received a message from the developer we're moving to further performance testing, which is basically ensuring the scaling with all the rankings works fine. Once this is completely I can go back to the theory and verify if the algorithm works as intended.

 

Once that's verified I will update the thread here again with more detailed information and the community can check UAT and feedback on the points (max, min, scaling, etc). Once we have a general consensus on the forum and among staff, we'll do a full write-up of the change and post it in public. If the general consensus is positive too, we can set a date of deployment for Rev 7. Still a long way .

 

As a preview, here is the 3DMark Vantage 1xGPU ranking on our test server. The main difference is the scaling of the points as they are determined not by which rank you have but by how far your score is off the top score. The scaling is very flat, but this is something we can address by tuning the various algorithm parameters

 

 

 

This is written 3 posts above your reply: "Oh, we also added one more parameter in the algorithm design to determine the weight of a benchmark. In short, this parameter will allow us to arbitrarily limit points of certain benchmarks that are over valued. I'm looking at you, XTU.".

 

I mean, I call XTU literally by its name ...

[Rauf]

I know it's just a test but this is not how I recall the discussion went. Shouldn't there be a bonus for the top scores in addition to the %-score? I think top 10 should get a bonus for placing in the top ten. With some kind of slope like there is today. Otherwise it will just turn into scatterbenching and there is not enough incitament to push for the really high scores. Or push just ahead of one of your competitors... Ranking is still very important. Particularly in competitions. What other sport is based on how much you are beaten by the winner? It's all about position.

[Massman]

I know it's just a test but this is not how I recall the discussion went. Shouldn't there be a bonus for the top scores in addition to the %-score? I think top 10 should get a bonus for placing in the top ten. With some kind of slope like there is today. Otherwise it will just turn into scatterbenching and there is not enough incitament to push for the really high scores. Or push just ahead of one of your competitors... Ranking is still very important. Particularly in competitions. What other sport is based on how much you are beaten by the winner? It's all about position.

 

Yes, there will be a bonus as presented in this post.

 

I just want to keep you guys updated on the progress. Please give us some time to make sure everything is working as intended ...

[Alan_Alberino]

Spent some time looking UAT page, this is also part of the update? +80 HW Points

 

http://uat.hwbot.org/submission/3068409_dancop_3dmark06_geforce_gtx_580_57201_marks

 

http://uat.hwbot.org/submission/2816515_strat_3dmark03_geforce_8800_gtx_73611_marks

[speed.fastest]

I think R6 version is good enough, that will push user to get 1st place

[der8auer]

[Guest george.kokovinis]

 

Deep genuine humor, from a man with a very high IQ.

 

Love it.

[Massman]

Another update from our side.

 

As you may have noticed, the past couple of months were very busy for us as we had to prepare for the World Tour events in Brazil, South Africa and France. It's a bit quieter for me in the next month, so I will be able to finish the R7 proposal.

 

Very recently you may have also noticed some problems with the HWBOT submit function. Especially those present at our last WT event will know. From what we gather based on the server logs and reports, the issues are due to the way R6 is designed and our database structure. Once a heavy load action is ongoing on our database, the MySQL database locks all impacted rows until all queries are completely finished. The R6 calculation method requires all results in a certain global or hardware ranking to be updated. As I mentioned earlier in the thread, updating one result affects other update query triggers such as updating the User Total, the League Ranking, Notifications, Achievements, etc.

 

I hate to say this again, but with XTU throwing so much data at our servers it's not surprising the current calculation method is stretching our servers. To give you a figure, in January alone we had to process 80,000 data requests from XTU.

 

The calculation method for R7 will reduce the transaction load by a great margin, luckily.

[xxbassplayerxx]

As usual, XTU is the problem.

 

Possible to parse XTU results on a completely separate server *cough* Intel's Servers *cough* and pass changes over to HWBot's servers?

[Massman]

Sure, but that would not solve the problem. The XTU results are triggering ranking and point updates. Whether it's instantly or delayed (from another server), the update scripts will always trigger.

[xxbassplayerxx]

 

Any way to cut down the quantity of submissions?

[Massman]

Hah, I'm not sure if "less submissions" is the best approach here. The large quantity is a good thing.

 

What R7 is about is reducing the amount of updaters being triggered per submission so that a large quantity of submission doesn't end up locking the database for too long.

[Massman]

Making a bit more progress. I've asked the developer to move the R7 test to a higher capacity server to have a fully calculated test server.

 

Below a comparison between R6 and R7 for the global ranking (+ WR points). What is implemented is:

• Points are awarded based on relation of your score versus the top score
  
• Participation threshold for maxPoints was lowered
  
• For XTU we artificially reduced the points

You can see the effect of the XTU restriction by comparing it to the HWBOT Prime points. Note that the Fire Strike 4xGPU and XTU 8xCPU category also has WR points included.

 

 

Below are screenshot comparisons for the Hardware Points:

• SuperPI 32M 2600K: R6 vs R7
  
• Fire Strike GTX 970: R6 vs R7
  
• XTU 6700K: R6 vs R7
  

[Rauf]

Thanks for the update. Nice to see things are moving along.

I would appreciate if you could define what the goals are for this update. I know about the load on the servers, and XTU points. But what else?

 

Some thoughts on the figures above:

- WR points should be decreased since the global points are higher now.

- Why artificially limit XTU, why not a cap for maximum number of scores that affect all benchmarks the same?

- Points scaling seem better, but would need to see the whole deal to really know. Test server usually don't have all scores to give correct reading.

[Alan_Alberino]

I like the idea, but I went from 511 points on R6 to 1366 on R7, and have some scores with Dice with +100 GP (#680 on SuperPi 1M gave me 125 GP), I think its too much increase...

Edited April 17, 2016 by Alan_Alberino

[Guest Digg_de]

It is nice to see that things like Reference Clocks on Mainboards or Memory giving GP.

[TaPaKaH]

What about the current hardware 2-pointers?

 

Hwbot prime #20 gets too many points, IMO.

You probably assumed that the benchmark score scales near-linear with CPU overclock, which it doesn't.

[Gunslinger]

What about the current hardware 2-pointers?

 

Hwbot prime #20 gets too many points, IMO.

You probably assumed that the benchmark score scales near-linear with CPU overclock, which it doesn't.

 

lol, instead it scales with Java version used, which is not ideal.

[rtsurfer]

lol, instead it scales with Java version used, which is not ideal.

New java version is like a new OS.

 

No one said anything when legacies got faster with Win10.

[Gunslinger]

New java version is like a new OS.

 

No one said anything when legacies got faster with Win10.

 

How many times does Java get updated vs. a new OS being released?

[rtsurfer]

How many times does Java get updated vs. a new OS being released?

Valid point.

But we didn't get a significant speed up with releases until Java 9.

 

Hopefully it settles down after the Beta stage is over.

 

Sent from my Nexus 5X using Tapatalk

[Noxinite]

It seems to be going in the correct direction (which benchmarks get more global points, example = SuperPi 32M), but I question how high some of the global points are. When points are too easy to get they individually seem worthless and might also detract from benching. If anybody understands what I'm getting at?

Edited April 18, 2016 by Noxinite

[xxbassplayerxx]

Agreed! 100 = max

[Cavemanthe0ne]

The latest points update seems a bit skewed for Maxxmem... idk tho lol. Getting #465 score globally and getting nearly 72 points for it seems like a lot... Then again, maybe not. I was just a bit surprised to see 760 points on uat when I have 140 on the live server

attached is my global points as of this post

 

 

also, seems really cool with the updates you are working on for points. Hopefully you can end up on one that is fair and also not skewed to certain benches like xtu or others.