Giant 3D elephant in the room no one wants to talk about...(why is it always me bringing this stuff to light?)

[Splave]

47 minutes ago, Noxinite said:

I don't understand what all the fuss is.

All people have to do it select one setting when they start benchmarking, reboot, bench, change setting back if finished benching, reboot...

There are plenty of other benches where you need to jump through more hoops to get a good score (or even a score).

So?

100% agree, would be really great if we could convince GPU-Z to add ECC check box near where the CUDA etc check boxes are. 

[zippytek]

Question then before I run some more 3D:

 

Front page says:

 

Quote

Due to the inflated and bugged scores we have seen lately pop up at the UL Hall of Fame and at HWBOT it would be recommended to enable the ECC Memory setting in the nVIDIA Control Panel. This is a request for testing to see if this is a solid fix by UL.

However just enforing this in a screenshot would not be sufficient. Therefore UL Benchmarks has implementend ECC detection in their latest Systeminfo release 5.55

 

 - Therefore only use systeminfo 5.55 or newer to be used from the 23 November 2022.

Am I reading this correctly? Any subs after 11/23 with older systeminfo *or* ECC showing "disabled" in 3DMark verification links will be invalid for hwbot? 

Please confirm or correct me. Thanks!

[Leeghoofd]

Correct , we need to analyze things

just wait for benching on LN2 till this is sorted

[zippytek]

2 hours ago, Leeghoofd said:

Correct , we need to analyze things

just wait for benching on LN2 till this is sorted

sounds good, thanks. fwiw, it sounds like there is an issue disabling ECC on some cards too which makes this all even more fun. 

 

[Splave]

3 hours ago, Leeghoofd said:

Correct , we need to analyze things

just wait for benching on LN2 till this is sorted

So all current scores that don't have ecc on but "feel" correct can enjoy being permanently 2-3% better performance? 

If we require it for some we must require it for all. Time to press delete on all 4090 subs without ecc if this is the path we are taking.

[saltycroissant]

There's a far more simple solution.... just remove the 4090 from hwbot ?

[Rauf]

1 minute ago, Splave said:

So all current scores that don't have ecc on but "feel" correct can enjoy being permanently 2-3% better performance? 

If we require it for some we must require it for all. Time to press delete on all 4090 subs without ecc if this is the path we are taking.

Same as for the scores with superposition release candidate version

there are some unbeatable hw points because they were done when rules did not explicitly say beta or rc versions were not allowed.

Fyi, I don't agree with this line of thinking, but removing all old scores is also a bit drastic. And checks can only be done for 3dmark anyway, not superposition so not sure how much we gain by this. For sure there will be huge moderation work still because I guess most users won't read the rules that carefully.

[Leeghoofd]

1 hour ago, Splave said:

So all current scores that don't have ecc on but "feel" correct can enjoy being permanently 2-3% better performance? 

If we require it for some we must require it for all. Time to press delete on all 4090 subs without ecc if this is the path we are taking.

I'm not even sure forcing ECC will solve it:

 - UL so far still validates with ECC on or off.

 - Some cards can't enable ECC 
 

It's not entirely up to HWBOT to take a stand here. If the scores are out of bounce with or without ECC, UL should get something working that invalidates the score right after the benchmark... being it via an algorithm or whatever.

We can still do manual moderation but it's a bit weird that the scores get validated on the Benchmark's site, but gets removed here...

[Splave]

7 hours ago, Rauf said:

Same as for the scores with superposition release candidate version

there are some unbeatable hw points because they were done when rules did not explicitly say beta or rc versions were not allowed.

Fyi, I don't agree with this line of thinking, but removing all old scores is also a bit drastic. And checks can only be done for 3dmark anyway, not superposition so not sure how much we gain by this. For sure there will be huge moderation work still because I guess most users won't read the rules that carefully.

Always welcome to flag scores that are out of line. In fact we need more intelligent people like you that know what to look for to do so. Perhaps if reporting scores was anonymous less people would be afraid of repercussions by making reports.

Superposition has the fps graph that is easy to see what's wrong or off about a run. UL should do the same or force ecc for everyone. Instead of requiring new scores to run the race with one leg and compete with other people's two leg scores.

Requiring higher standards than UL shouldn't be an issue. I've never even looked at the hall of fame there. Hwbot is supposed to be the best most secure WR database. They have an entire staff. One guy fixed 2D benching security in his spare time and still takes shit for it on a daily basis. No wonder 3D will never be fixed. Have fun I don't care anymore. Sub whatever you want.

 

 

[_mat_]

This shows how volatile competitive 3d benchmarking is. There is no common standard for results, so a common issue with a new GPU generation, a vulnerability, tweak or cheat brings unavoidable chaos.

And for each common issue there will always be the same questions: Will all benchmark dev companies fix? How? Are overclockers even part of their target audience/business strategy? Making benchmarks XOC-capable is a lot of work and each company will decide if it's worth the trouble.

It's flawed by design and HWBOT is left to pick up the pieces to somehow make it work. I am not talking about a specific company/benchmark, just the general concept of competitive 3d benchmarking. In the end the quality of results goes down, people get unhappy because they no longer believe in each others work being done fair and square.

PS: Is Superposition even guarded against bus clock timer skew?

[Mr.Scott]

20 hours ago, Splave said:

Always welcome to flag scores that are out of line. In fact we need more intelligent people like you that know what to look for to do so. Perhaps if reporting scores was anonymous less people would be afraid of repercussions by making reports.

Superposition has the fps graph that is easy to see what's wrong or off about a run. UL should do the same or force ecc for everyone. Instead of requiring new scores to run the race with one leg and compete with other people's two leg scores.

Requiring higher standards than UL shouldn't be an issue. I've never even looked at the hall of fame there. Hwbot is supposed to be the best most secure WR database. They have an entire staff. One guy fixed 2D benching security in his spare time and still takes shit for it on a daily basis. No wonder 3D will never be fixed. Have fun I don't care anymore. Sub whatever you want.

 

 

You care.

You're just angry at the ignorance, and rightly so.

You simply cannot trust everybody to do the right thing.

It's not HWB's fault, it's the community's fault.

[spit051261]

Does seem weird that you can go out and buy a very expensive card and without any overclocking knowledge can get 100 points or more by just plugging it in and some basic settings.

I have run a 4090 , I know how easy it is.

Sort of takes away the time and effort we put into this hobby.

Saying someone who is having more fun running a 295x2 on legacy.

That is real fun ?

Edited November 30, 2022 by spit051261

[Fasttrack]

Is there any update on this very serious issue ?

A good RTX 4090 costs approximately 2500 Euros in Europe.

A very significant amount of money to invest in a hobby, and then find yourself in front of nasty surprises.

Thanks.

[Leeghoofd]

We are still looking for a suitable feasible solution for all new 40 series cards

till now this is the current opinion by UL:

"I'd still suggest for now that we just start requiring ECC for competitive OC and not care that without it you would get higher FPS"

 

[Fasttrack]

Well then, I will skip entirely this generation.

Melting connectors, weird behavior when overclocking.

Let's see what AMD has to offer.

[Johnksss]

On 11/20/2022 at 4:54 AM, Nizzen said:

Did a scaling test in PR:
Same ~3045 core on every run
27584p +0 mem
27989p +500mem
28353p +1000mem
28639p +1500mem
28790p +1700mem

Might want to try the with ECC enabled because those scores are no where even remotely close. Unless us peasants do not have the secret hand shake back room benching tips needed to make those scores happen?

 

Edit: Nevermind. Stupid user error! I forgot one is using the Resizeable-Bar

Edited December 17, 2022 by Johnksss

[Johnksss]

On 11/23/2022 at 1:27 PM, Mr. Fox said:

It seems so.

I am not sure what that means.

Futuremark's products were better before UL got involved and messed things up. But, we see that scenario play out with lots of things. Big company buys little company, then turn their gold into garbage... then eventually get rid of it because it wasn't their baby and became too much of a burden to support it. 

He means Catzilla I suspect....

[Mr. Fox]

2 minutes ago, Johnksss said:

He means Catzilla I suspect....

Yeah, probably right. I hadn't thought of that, but now it makes sense. I love Catzilla, so that would be fine with me.

[Johnksss]

Retraction due to user error...?‍♂️

Edited December 17, 2022 by Johnksss

[FireKillerGR]

On 11/20/2022 at 3:21 AM, Splave said:

I propose that:

1. If you post a known bugged run you get a warning, then a vacation.

2. If you are "unsure" then ask a mod or even me if it seems out of line. It is quite easy to read. 

3. We require monitoring already in 3dMark benches, I propose we show the FPS and FPS average graph section in the result screenshot, if you cant fit it add a second screenshot with it. 

4. Always welcome to video / disclose tweaks to mods for doubters. 

 

With the blessing of Roman/Albrecht, people have a week to remove results that "oh maybe it did have a lot of artifacts, or maybe it does seem too high" before we dive in and hand our warnings, or vacations for multiple infractions. 

 

Im sorry if you feel like I am always the one exposing this stuff, but why do I always have to be the one to do it?  We all work hard on this stuff and an even playing field is the of the utmost importance. Without it this places means nothing. Let's do better, and be better. .

This is getting confusing now

Is yours with ECC on? Cause this is how it looks at the moment.

Scene 16 average FPS: 282 FPS

vs ours:

Scene 16 avg FPS: 208 FPS
 

and Rauf's:

Scene 16 avg FPS: 209.78 FPS

Also this seems bugged and ecc is incorrectly stated as on

Edited February 6, 2023 by FireKillerGR

[nv1diafan]

2 hours ago, FireKillerGR said:

This is getting confusing now

Is yours with ECC on? Cause this is how it looks at the moment.

Hello guys, just a tip about easy ecc detection: the supo benchmark runs with ecc off state vram 24564MB in the verification link, while with ecc on 23028MB. In 3dmark with ecc off: 24576MB, with ecc on: 23552MB.

When ecc is enabled less amount of vram is reported because it reserves some amount.

[FireKillerGR]

19 minutes ago, nv1diafan said:

Hello guys, just a tip about easy ecc detection: the supo benchmark runs with ecc off state vram 24564MB in the verification link, while with ecc on 23028MB. In 3dmark with ecc off: 24576MB, with ecc on: 23552MB.

When ecc is enabled less amount of vram is reported because it reserves some amount.

Exactly; same on GPUZ window as well.

[Splave]

How would you know what a result with ECC looks like, your scores apparently dont require it right  

 

what's really confusing is forcing ECC on people now and leaving old ECC results. How am I supposed to compete? Seems fair right? Only 1.3k less from only enabling ECC. Also safedisks bugged runs are also left so whats up with that? Only my results get removed it's hilarious to me at this point. 

 

Edited February 6, 2023 by Splave

[FireKillerGR]

30 minutes ago, Splave said:

How would you know what a result with ECC looks like, your scores apparently dont require it right  

 

what's really confusing is forcing ECC on people now and leaving old ECC results. How am I supposed to compete? Seems fair right? Only 1.3k less from only enabling ECC. Also safedisks bugged runs are also left so whats up with that? Only my results get removed it's hilarious to me at this point. 

 

Actually spent enough time comparing ECC on vs ECC off on multiple benchmarks and already provided input I believe to both the mod team and FM last year (25th of November to be precise).

So there is an understanding of the actual impact of the perf loss from our end.

To my understanding, old scores that seem to be in-line have remained till a decision is made.
In between, the shift on ECC has been made to avoid additional bugged runs + have it as a testing to better identify what the best outcome would be. But hey, thats up to the mods to elaborate on.

From our end, we have already communicated that if ECC On solves it (even with perf penalty as long as it applies to all and solves the issue) we are totally fine on having our scores removed as long as we are given a notice in advance allowing us to rebench with ECC On.
That has been our only request so we don't wake up one day with the need to rebench everything overnight.

Edited February 6, 2023 by FireKillerGR

[Splave]

15 minutes ago, FireKillerGR said:

Actually spent enough time comparing ECC on vs ECC off on multiple benchmarks and already provided input I believe to both the mod team and FM last year (25th of November to be precise).

So there is an understanding of the actual impact of the perf loss from our end.

To my understanding, old scores that seem to be in-line have remained till a decision is made.
In between, the shift on ECC has been made to avoid additional bugged runs + have it as a testing to better identify what the best outcome would be. But hey, thats up to the mods to elaborate on.

From our end, we have already communicated that if ECC On solves it (even with perf penalty as long as it applies to all and solves the issue) we are totally fine on having our scores removed as long as we are given a notice in advance allowing us to rebench with ECC On.
That has been our only request so we don't wake up one day with the need to rebench everything overnight.

So you should get the courtesy of notice when ecc results would go live. While I sit here with a $2600 paperweight currently. 

The way things move around here you won't have to rebench you will just sub 4090ti. Sounds great for everyone ?

Edited February 6, 2023 by Splave