3D Benchmark proposal list for 2021

[Mythical tech]

On 12/20/2020 at 4:40 AM, Leeghoofd said:

Closing the thread in 3 days, last inputs welcome, otherwise this is it for 2021

Safe to assume this list is final or close to final for 2021?

[Rheinlaender]

When will the recalculation begin?

[Leeghoofd]

Whenever the new point system is in place. Realistically we aim for february/march. It would have been perfect timing if the new HWBot would be released this weekend, but that was mission impossible. FYI thx to  Corona paperwork was only completed in Fall iso early March... than slow access was granted , but its moving now.

Ths points for 2/3D will be introduced as soon as possible. Ranking points somewhere in Q1

[Leeghoofd]

Just finished a meeting with the programmer, new benchmark global point system will be setup this weekend, first on the test uat.hwbot.org, afterwards on the main site.

[Leeghoofd]

Looks really Fab, thx Tim. will be on normal server tomorrow

[rbuass]

I would like to do a resquest, to your analisys.

There are 2 special benchmarks that I think should be included for Global.

 

One is GPUpi 1B, because will show the highest frequency some graphic card can run a test.

GPUpi 32B is heavier, and so, the overclocker cannot run with too high clocks, so is very impressive to see running @ 3 Ghz or more.

 

The second is Catzilla, that's a very nice 3D bench allowed for Global points in the past (2 versions, 720P and 1440P).

It would be great to see Catzilla 1440P again on ranking for Global points, since requires very strong performance and tweaks.

Just for reference, Allion Labs, I think one of the most important certification labs in the world, uses Catzilla for tests, since will have very strong requests and peaks.

Who remember the crashs under Raymark, know what I'm talking about.

 

Best wishes for all overclocker.

[Rauf]

After seeing the new benchmarks in place for a while now. And benching 3D pretty heavily lately I have some feedback regarding globals. 

It's pretty clear the total number of relevant rankings/points is waaay higher for cpus. Just counting the most popular rankings for Geekbench 3 multi there are more than all 3d combined (counting for 3090/6900xt where sli is almost non existant and so rare on hwbot that it does not give points anyway). In this regard I think we need more 3d benchmarks with globals enabled. Lets bring back some of the removed benchmarks If catzilla is fixed so that it actually runs with mb/cpu drivers installed, why not.

Also, maybe consider setting fixed points for sli categories as they are very rare here on hwbot. But in 3dmark hof they are still filling up the top 100.

[Leeghoofd]

We already are thinking of awarding more (global) points for 3D with the new point system...

If you guys want the Puss back, give it some spins and report back 

[yosarianilives]

I think we shouldn't have a 3d bench where load time is part of the score (leave that for pcm) also the support for catzilla is even worse than for 3dm. Although the load time does give an idea, what if we made a new category called "system benchmarks" that's apart from cpu and 3d and put pcmark and catzilla in there as well as any similar bench that benches cpu, gpu, storage, etc we may add (if we can ever determine a non stupid passmark version for example, etc). 

[Papusan]

Yeah bring in Catzilla. With all the mess from 3Dmark suite I can't see that Catzilla is a muchworse benchmark regarding buggy software. Atleast bring in 1080p or 1440 for Globals. And the bench is fun too look at as well.

 

Edited February 6, 2021 by Papusan

[Rauf]

2 hours ago, yosarianilives said:

I think we shouldn't have a 3d bench where load time is part of the score (leave that for pcm) also the support for catzilla is even worse than for 3dm. Although the load time does give an idea, what if we made a new category called "system benchmarks" that's apart from cpu and 3d and put pcmark and catzilla in there as well as any similar bench that benches cpu, gpu, storage, etc we may add (if we can ever determine a non stupid passmark version for example, etc). 

Load time is not dependant on storage performance. It's all tweak and cpu perf. And it's only a small part of the score anyway. 

[phil]

7 hours ago, Rauf said:

After seeing the new benchmarks in place for a while now. And benching 3D pretty heavily lately I have some feedback regarding globals. 

It's pretty clear the total number of relevant rankings/points is waaay higher for cpus. Just counting the most popular rankings for Geekbench 3 multi there are more than all 3d combined (counting for 3090/6900xt where sli is almost non existant and so rare on hwbot that it does not give points anyway). In this regard I think we need more 3d benchmarks with globals enabled. Lets bring back some of the removed benchmarks If catzilla is fixed so that it actually runs with mb/cpu drivers installed, why not.

Also, maybe consider setting fixed points for sli categories as they are very rare here on hwbot. But in 3dmark hof they are still filling up the top 100.

So true!!!

[FireKillerGR]

To be a bit more precise on the issue, I collected all the points one could get (a month ago so it might be a bit different now) from 2D and 3D and its as follows.

2D ->  8532.1
3D ->  4081.5

(of course I am aware of the 30x subs GFP cap)

Only calculated the scores that get you above 80 Global Points + included per core category for each benchmark + multi-gpu configs for the 3D ones.

From 2D, the benchmarks that award the most points are GPUPi and 4K. (2023.8 and 2147.6).
From 3D its 03 (774.5) and 3D11 (595.5).

Last but not least, there are 67 individual scores (core categories * benchmarks) that give more than 80 GP on 2D and 31 on 3D benchmarks.

--------------------------

What could help balance things out:
1) Give points back on 3DFSU and 3DTS. 
2) Explore adding more benchmarks like Catzilla or find alternatives
3) fixed points as Tobias suggested. If one benchmark gets no points, it will be even less attractive for a less experienced (with hwbot) user. Also, its impossible to calculate how a benchmark will scale in 6 months as one might see way more growth than another.

Edited February 6, 2021 by FireKillerGR

[Leeghoofd]

And again... We are thinking of awarding more (global) points for 3D with the new point system...

And again... the point system will be totally overhauled end of Q1...

And again... people don't ever read what I type...

And again....many of the top 3D guys have great cpus too... You are not telling me that 5950X can't do 5800 CB?

Did I already mention we are in a transition phase?

[FireKillerGR]

I know its a bit hard, but maybe having a base formula/sheet that calculates what can be gained per benchmark/category & then adjusting it (before introducing changes) to see if it balances things out would be a cool idea.

I doubt anybody thought/calculated/expected the current system to be that broken.

At least that way, you will know where each category stands and can keep adjusting stuff till you find a formula that makes more sense.

PS. obviously not meant to disrespect the amount of work and time put by Leeg and others into adjusting and making hwbot better.

Edited February 6, 2021 by FireKillerGR

[phil]

3 minutes ago, Leeghoofd said:

And again... We already are thinking of awarding more (global) points for 3D with the new point system...

And many of the top 3D guys have great cpus too... You are not telling me that 5950X can't do 5800 CB?

I am just saying that getting a good 3d score requires more skill and hard work than a 2d so it should be encountered accordingly.

[Leeghoofd]

that's not true as you share braincels with Stavros, statement overruled

[phil]

Just now, Leeghoofd said:

that's not true as you share braincels with Stavros, statement overruled

True but there are other peolpe like Rauf that don't share braincels and do all the hard job alone.

 

[NATA 58]

catzilla. it is better than many bench 3d.

[MetalRacer]

Would like to see global and hardware points for the Puss.

[yosarianilives]

I think the visuals and the music is nice but it should not be added for globals. Bench is a pain to get working, then it breaks all the time, often have to retry running it many times until it finally completes a run. The support for catzilla is also nonexistent, I've never heard back from contacting catzilla support ever. When I contact FM I always hear back relatively quickly. Catzilla is a fun bench when it works, but it should not seriously be considered for globals. 

[Leeghoofd]

I have absolutely no problem to run this benchmark on the kids 8600K & 980Ti, the 9900K with a 1080 and the 5950X with RTX3070... all platforms on Win10 , just run the benchmark in administrator mode...

Is it an idea to do Catzilla in the upcoming challengers just to gather more feedback on the pros and cons ?

[keeph8n]

7 minutes ago, Leeghoofd said:

I have absolutely no problem to run this benchmark on the kids 8600K & 980Ti, the 9900K with a 1080 and the 5950X with RTX3070... all platforms on Win10 , just run the benchmark in administrator mode...

Is it an idea to do Catzilla in the upcoming challengers just to gather more feedback on the pros and cons ?

I would be up for doing this in the Challengers series yes

[Leeghoofd]

Yeah might be better than to have this here say, than after a few weeks we will launch a poll so users can share their experience with the 2.0 version from September 2020.

[chispy]

Hola everyone. First and foremost i hope everyone is doing ok with this hard times we are having with covid19 situation globally.

@Leeghoofd , @der8auer I think the priority #1 should be fixing the point algorithm. As it stands right now it does not help at all and is very disappointing this have not been taken care of sooner , very un-balanced and unfair point system.

 I have been on a benching spree and i have started loosing interest the more i bench , why ? Simple , the points algorithm is all screwed up and only the top 10 results gets HUGE ...  global points and if you fall below top 10 scores you will get so little points that it seems like a waste of time , yes it is that bad and very disheartening and sometimes i question myself if i should keep going or quit.

 That goes for both 3d and 2d globals. Also we need more 3d benchmarks with globals , at least add 3~5 more because as it stands there is more 2d bench than 3d bench. (  you can ad = heaven extreme , VR Mark , wild life , 3dmark06 , etc..  )

Catzilla has been beat to death with all kinds of cheats , it has been tore apart and mess with at the code level by some benchers and changed ( hackable )  + it is 100% un-reliable and cannot re-produce repetitive scores , please do not use it , it is  ?  . My 2 cents on why we cannot use Catzilla , see the reality of it , open your eyes and see where this bench stands in terms of reliability. it will be hwbot worst nightmare if you added and will make the mods work very , very hard with reports to the point where it will be take it down again , so why go thru this same shit again ??? , want to make hwbot implodes ? approve Catzilla for 3d points again and mark my words , you will see it this bench taken down again. Waste of time.

 This are obviously my opinions , but i believe they are very valid and reasonable , please take it in consideration when making a decision. Thank you for all the hard work you guys have been doing to update hwbot , appreciate it.

 

Kind regards: Angelo

Edited February 8, 2021 by chispy