3D Benchmark proposal list for 2021

[superpatodonaldo]

13 hours ago, Digg_de said:

Imagine there is something like Efficacy Points. ? Too complex to add i guess.

 

it's been a long time since I have this idea: give point (hw point? global point? ...to decide) balancing the result with the hw price and or oc %

overclock mean push further

[Strunkenbold]

Would be cool to have points only enabled for specific hardware generations. Running 3DM99 on TNT2 can be fun, why not give points for this? On the other hand, awarding hardware points running 3DM05 on a GF 3090 makes not much sense. 

IMO we give way too many points, I found it pretty overwhelming when I benched something 3D in the last days. So many benchmarks, so much to install and run. Every bench has it own clock limits. Looking for the best clocks for every benchmark can be very time consuming and frustrating. I mean, instead of soldering, flashing and clocking you do now run benchmarks for hours just too have them all. This feels very repetitive. There are now over 30 benchmarks to score hardware points. That is way too much. Sure, you will say no one forces you to run each bench, just run the ones you like. But ranking should be about who is the best tweaker not about the person who has unlimited time.  Maybe we should focus on just 10 benchmarks for every generation if something like that can be implemented.

Source of the problem is clear, new generation GPUs makes old benchmarks obsolete as they run pretty early in the CPU limit. Thats not the case for 2D benchs. Even with new gen CPUs, people hunt SuperPi 32m record. But with each new gen we enable another bench for points and suddenly you have simply too many.

[Leeghoofd]

The ideas are cool but it's impossible to define bro:

are you a good tweaker because you got every last mhz out of the card or did you just get lucky with a gem GPU.

Are you a good tweaker because you can only get 2400Mhz but you are very efficient or was that a once in a lifetime run. Or did the overclockere even downclock to make his run more efficient?

 

We want to simplify the point system, ditch the far too complex algorythms that messed up the rankings the last 3-4 years. The Bot currently does not recalculate all submissions, so it leaves people in the cold.

Noone forces you to bench all of them available benchmarks, hence why many just focus on the benchmarks that support globals. First we need to simplify things and highest on the list is to award benchers which are  lower down the ranking with more points to motivate them iso this curently installed cut off system.

From there on we can evaluate and adapt stuff... but one thing at a time 

 

 

[Leeghoofd]

By popular demand:

• Removed points from the Catzillas and the Heavens
• Talked to Matt about GPUPI 4.0, perf will be very similar to 3.3 (so they can be merged in the future). Thus we can swap from 3.2 to 3.3, so runs will be a bit less time/ln2 consuming
• Vantage Extreme will be released and will get hardware points too

 

Any further ideas/proposals

 

[Rheinlaender]

 

What a shame. Why are the 3Dmarks 03-06 with BM removed? I actually thought that these would be
prioritized in the future, I'm confused.
Firestrike Ultra is out * gulp * why I love the bench * sad *
Please leave the 2011 on Global. Above all, however, I would like the old 3Dmarks, including the 2000,
to be fully rated again. I still have a lot of old graphics cards and it is not worthwhile to benchmark 
them for the few points. I would be glad.
It was asked who benchmarks the VRMarks. So I think they are great and I bench them too. Global points would 
be great, but of course not a must. Catzilla is terrible. That could all be out for me. The installation has 
not worked for me for a long time and is also something for cat lovers ;-)
Errors in the translation are due to the translator ;-)

[Leeghoofd]

8 minutes ago, Rheinlaender said:

What a shame. Why are the 3Dmarks 03-06 with BM removed? I actually thought that these would be prioritized in the future, I'm confused.

Current BM 3Dmark scores will be merged with the normal ranking. if you want max scores Win7 is not always the way to go , especially not with old cards, some like XP , some benches prefer Vista. 

[Antinomy]

On 12/3/2020 at 6:23 AM, Strunkenbold said:

IMO we give way too many points, I found it pretty overwhelming when I benched something 3D in the last days. So many benchmarks, so much to install and run.

Totally agree. I've suggested this before, will write again:
Give 99 and 2000 points (HW, TPP) for hardware that can't run 3DMark 03. I have to remind that since Aquamark on legacy hardware is currently dead, some categories have only one benchmark awarded.

Edited December 12, 2020 by Antinomy

[Leeghoofd]

I can understand, but we have to keep an open mind:

I prefer giving for nearly each bench at least hardware points, so the user gets at least something in return for his efforts.

Globals will be a continuous evolution, but again taking into account not only the latest and greatest hardware deserves them...

Now we are currently brainstorming for the new points system:

• There will be points attributed also for the lower ranked (no more cut off system)
• Points will be linked to the rank and will be more of a fixed value: eg spot takes 150 points, doesn't matter if there's 100 or 100000 subs for it
• With the above in mind, the given points will gradually become lower if you go down the ranking, so no more weird jumps from 167.3 for nr 1 to 99.7 for nr2 and 2 points for nr 10

[Leeghoofd]

I tested the GFXbench yesterday on 3 setups, it is only working on one setup properly. Always get this annoying server error issue (even though it did downloaded and installed the required files on all 3 machines from the internet at first start) Next reboot the server error is there. Will cut it loose if there's no reply from their support...

 

 

[Rauf]

Looks good in general, but I think 3d11 and vantage should maybe be with the extreme preset instead? I haven't tried them, but I do know 3d11 perf is very much a cpu freq benchmark with latest gpus. Sure there is some scaling on gpu clocks, but not much. I tried running 5950X at 5.8ghz paired with a 2080ti at 2.7ghz. A 10900K with same gpu still beat this, if barely. But if you consider the phys score with 5950X was 44k, compared to a 36k with 10900k, you see the real diff is cpu freq in gpu tests. Game tests were 6k lower due to cpu freq. Combined was the same.

As for vantage it is kind of the same but point scaling is a little different. A high cpu score helps overall score more. Still I was not able to beat top 2080ti score with 5950X, even with 144k cpu score. Again, lower cpu clocks capped the game tests too much.

I think the extreme presets might make these benchmarks more balanced. 

[Leeghoofd]

I rahter keep these for the peeps with less powerfull setups, you guys can have fun with Timespy and co

[FireKillerGR]

was thinking about a potential swap between super 1080p and tsx (re globals).
Thats based on the fact that tsx is not maxed out (both gpu and cpu) yet so it would be more of a futureproof solution (?) to avoid having massive changes in an anual base.

[Rauf]

54 minutes ago, Leeghoofd said:

I rahter keep these for the peeps with less powerfull setups, you guys can have fun with Timespy and co

Won't the effect be the opposite, because ln2 cooled cpu scales on both game tests and cpu tests? I'm not sure but wouldn't an air/water cooled or older setup perform closer to top scores, relatively speaking, in extreme presets? 

[FireKillerGR]

30 minutes ago, Rauf said:

Won't the effect be the opposite, because ln2 cooled cpu scales on both game tests and cpu tests? I'm not sure but wouldn't an air/water cooled or older setup perform closer to top scores, relatively speaking, in extreme presets? 

yes but my take was more of aiming to have heavier tests (in general) whenever we add something new from now on.
Super 1080p does ok at the moment but afraid it might turn into 3D11, 3DV etc that are heavily bottlenecked by the cpu even on their respective graphic tests.

Even TS has started showing this with dual gpu setups which will potentially turn into a single gpu issue as well in a couple generations.

Edited December 12, 2020 by FireKillerGR

[Rauf]

36 minutes ago, FireKillerGR said:

yes but my take was more of aiming to have heavier tests (in general) whenever we add something new from now on.
Super 1080p does ok at the moment but afraid it might turn into 3D11, 3DV etc that are heavily bottlenecked by the cpu even on their respective graphic tests.

Even TS has started showing this with dual gpu setups which will potentially turn into a single gpu issue as well in a couple generations.

I agree, I was speaking of 3d11 and vantage, extreme vs perf presets. I'm all for TSX instead of super 1080, it's a bit of cpu benchmark already I guess. 

[Leeghoofd]

Ticked Vantage and Time Spy Extreme

 

 

[AKM]

I like the TSX change, but why Firestrike Extreme instead of Ultra? If we are thinking of shifting toward heavier benches because of new gens of cards, I would think Ultra would be the way to go here.

 

Is it more so to keep a middle ground for older hardware to score points as well in Firestrike?

[Antinomy]

15 hours ago, Leeghoofd said:

I prefer giving for nearly each bench at least hardware points, so the user gets at least something in return for his efforts.

I'd support Greg on this one - too many benches isn't good too. You don't buy an RTX 3090 to play DOS games so running '99 and 2000 on it is pretty lame, so I'd stand against giving those points on modern hardware, only old school as I said. And in general, making a ridiculous number of (point awarded) tests will make it less fun as I think.

 

15 hours ago, Leeghoofd said:

Now we are currently brainstorming for the new points system:

This will bring us right between Rev. 2 and 3. Removing cut-off should be good - motivates people to bench no matter how bad the situation in the rankings is (how tight the comp). Now I open say Ryzen 3500 and see 5 results, 4 of which are LN2 and just close it. The second will bring back trash-boints when instead of competing in categories with high number of subs you'll be focused on grinding less competitive ones which is easier and will bring same gain for less effort. I think №1 points should correlate with number of competitors. Grinding is good, just don't over-value it with points. The third is a bit of an issue, but on the other hand, back in Rev.3 we got a situation when say a popular category with 200 subs you can get 31 points for being 50th and 33 points for 20th and you thought - why try so hard for those extra 2 points, just run other tests and get moving instead if pushing to the limit. So the slope should be steep maybe a bit not that steep as now.

[Leeghoofd]

There will never be a perfect solution that fits all. Same as for your proposal on the 2D list its impossible to configure to specify which hardware gets points or not. Either I exclude that modern hardware in the rules per benchmark but it will be uncontrollable for proper modération. 

For the points, if I pull some hardware points or team power points from specific benchmarks there will be more of a riot from specific teams. I see it like this: The ones that put it in a lot effort will get rewarded. Do they need to have top scores? Do they need to bench them all? Nope, but for some users it might be quantity over quality of their scores. And therefore contributing to their hardware ranking and team ranking 

 

The main concept for 2021 is to minimize the nr rules and allow faster and easier calculation for the Bot. No more complex stuff. 

[Noxinite]

Wait 1st place always gets the same points? That would only work for global points imo, as there will always be enough total subs. For hwpts this just won't work I don't think. It needs to scale with number of subs.

[Leeghoofd]

It might look something like this:

Up to 50 subs: maximum of 50 points 

Up to 100 subs: maximum of 75 points

More than 100 subs 150 points. 

It will not be one fixed value. There will be a correlation with the amount of subs, but no more 167.3 and stuff

[Noxinite]

On 12/13/2020 at 9:41 AM, Leeghoofd said:

It might look something like this:

Up to 50 subs: maximum of 50 points 

Up to 100 subs: maximum of 75 points

More than 100 subs 150 points. 

It will not be one fixed value. There will be a correlation with the amount of subs, but no more 167.3 and stuff

Ahhhh, fair enough - I didn't get that from your previous messages.

Might be a struggle to balance all the point caps, but I guess that can be figured out after the transition when people start to farm stuff - flashbacks to XTU. ?

Edited December 14, 2020 by Noxinite

[Leeghoofd]

Closing the thread in 3 days, last inputs welcome, otherwise this is it for 2021

[IvanCupa]

Just try to write my complaint opinion here..

Overall I see it on the latest rev. is okay to me. You have 720p up to 8K in DX10 - 12. + DXR. And heavy CPU in TSE. *Oh, We missing 1080p 3DMark here...

GPUPi Move to 32B, I assume because is getting to short bench for modern GPUs? Battle of miliseconds? ?

I just asking VRMark GL points. What's the consideration? I even didn't notice it's get GL point since long ago hahah.
Is there a lot subs? Or is reflecting somewhat real world result?

I'll vote for 3DMark FS still, for DX 11 1080p. Why? It's "free", you can use it for free, as long you watch the demo before the benchmark run & run it on a full test basis. ORB Validation still work even you didn't have valid key.
We Indonesian, still use this in 3 series of local AMD OC comps (AMD Rock Tour) because of this. We must design the comps is accessible for everyone / make the entry point as low as possible even free. This strategy encouraging people to try & enter the comps, thus we have good number on subs & participants. (Over 300 people on last edition).
I know it kinda CPU benchmark right now, with relatively new HW. IDK how Superposition on 1080p behave though, didn't try yet. If behave relatively the same maybe it's good to be back to FS? ?

CATZILLA, no comment here, just try it again last week & it's just somewhat work, can install it fine, up & running. But still, some glitch on the benchmark launcher & System detection. Throw 1-2 random error tho.. So let's just say money gone because bought this software.
Even though I kinda like it, just so refreshing as alternative. It has cats with iconic music. We need moar cats videos benchmark! LoL ?

Nightraid is more like Ice Storm, Cloudgate / Sky Diver but in DX12. So maybe treat it the same.
Wild Life is Vulkan API, cross platform. So your system can be compared to Mac, iOS & Android. Still occasionally shows glitch / bug, like incorrectly rendered texture / corruption. Especially in mobile & gadgets. Maybe because Vulkan API implementation on each GPU & driver. Didn't really have suggestion here. Nice concept, though. We'll see in the future.

I'd like to if 3DM06 still get GL points. As last native DX9 benchmark.

Thank you.. ?

[unityofsaints]

05 removed, then added again, then removed again - what is this, Schrödinger's rankings update? 

Edited December 26, 2020 by unityofsaints