Giant 3D elephant in the room no one wants to talk about...(why is it always me bringing this stuff to light?)

[Splave]

3D benching as we know it is broken. Plain and simple. What are we going to do about it? RTX-4090's are great cards but they also bug very easily. We need to be better users. If you see your score jump 1k point after raising your mem clock 15mhz that is not normal and not a "lucky" run. When your entire screen is tan from an artifact spanning your entire monitor and your score increases 10% it is not a "lucky run." In my opinion after a first warning we should issue a vacation for the user. 

After 2 minutes of research it is quite easy to see a bugged run. 

Which 2 look normal, which 2 look equally strange? Going from the pictures alone is easy but when I add the clock frequencies maybe it can help you even more. 

 

I propose that:

1. If you post a known bugged run you get a warning, then a vacation.

2. If you are "unsure" then ask a mod or even me if it seems out of line. It is quite easy to read. 

3. We require monitoring already in 3dMark benches, I propose we show the FPS and FPS average graph section in the result screenshot, if you cant fit it add a second screenshot with it. 

4. Always welcome to video / disclose tweaks to mods for doubters. 

 

With the blessing of Roman/Albrecht, people have a week to remove results that "oh maybe it did have a lot of artifacts, or maybe it does seem too high" before we dive in and hand our warnings, or vacations for multiple infractions. 

 

Im sorry if you feel like I am always the one exposing this stuff, but why do I always have to be the one to do it?  We all work hard on this stuff and an even playing field is the of the utmost importance. Without it this places means nothing. Let's do better, and be better. 

 

 

 

Edited November 20, 2022 by Splave

[rbuass]

35 minutes ago, Splave said:

3D benching as we know it is broken. Plain and simple. What are we going to do about it? RTX-4090's are great cards but they also bug very easily. We need to be better users. If you see your score jump 1k point after raising your mem clock 15mhz that is not normal and not a "lucky" run. When your entire screen is tan from an artifact spanning your entire monitor and your score increases 10% it is not a "lucky run." In my opinion after a first warning we should issue a vacation for the user. 

After 2 minutes of research it is quite easy to see a bugged run. 

Which 2 look normal, which 2 look equally strange? Going from the pictures alone is easy but when I add the clock frequencies maybe it can help you even more. 

 

I propose that:

1. If you post a known bugged run you get a warning, then a vacation.

2. If you are "unsure" then ask a mod or even me if it seems out of line. It is quite easy to read. 

3. We require monitoring already in 3dMark benches, I propose we show the FPS and FPS average graph section in the result screenshot, if you cant fit it add a second screenshot with it. 

4. Always welcome to video / disclose tweaks to mods for doubters. 

 

With the blessing of Roman/Albrecht, people have a week to remove results that "oh maybe it did have a lot of artifacts, or maybe it does seem too high" before we dive in and hand our warnings, or vacations for multiple infractions. 

 

Im sorry if you feel like I am always the one exposing this stuff, but why do I always have to be the one to do it?  We all work hard on this stuff and an even playing field is the of the utmost importance. Without it this places means nothing. Let's do better, and be better. 

 

 

 

That was all that was missing.
It's been a long time that I don't post results in the league, but also it's been a long time the 3D benchmark has more influence on a good assembly (mounting), not breaking the thermal paste (which needs to be the best and from the best batch), and of course, having a chip gold awarded.
But result bug for records is the last.

Thanks for sharing. 

[Rauf]

Well, already talked to Pasi about it  a bunch because 3dmark hall of fame is full of bugged runs. But it's a nightmare to moderate. Especially on cold as there is some weird cold efficiency bug that hurt efficiency when you run colder also.

Especially superposition seems sensitive to cold efficiency...

But anyone benching on Hwbot should be able to tell if their score is bugged or not. As you say, you don't get a huge bump in score from doing nothing or just slightly higher clocks. But how do you reach everyone and get everyone to follow the rules?

[TASOS]

I dont know if your posts are about something or someone in particular.

But i will speak in general.

Since you know/find out some very specific issues about this RTX 4xxx series , why not make an article about it ?

This stuff is new and many people would like to overclock it up to the edge of stability.

They probably wont have a clue if their big score is bugged (except if the result/run is too obvious , like in 2k3).

Are we having something similar like the 3DMark Vantage era ?

 

Give them a heads up

Dont assume that everyone is acting with bad intention 

[Xtreme Addict]

looks like we will have to find the way to impose ECC on rule  

Edited November 20, 2022 by Xtreme Addict

[Leeghoofd]

I talked to Pasi already about this Allen, Ronaldo, Micha, it should be possible to integrate ecc detection in the benchmark. So we are awaiting UL for a fix... 

[Nizzen]

Did a scaling test in PR:
Same ~3045 core on every run
27584p +0 mem
27989p +500mem
28353p +1000mem
28639p +1500mem
28790p +1700mem

[Leeghoofd]

Let us also take into account a "renown" Youtuber posteds several "bugged result" , so people start thinking this is normal...

For the moment it is manual labour for HWBOT and UL, but the Hall of Fame should really get cleaned up ASAP.

 

However what I find more worrying is that some of our "experienced" clockers posted their bugged results at the Hall of Fame.... they should lead by example, not like this... E-peen over common sense

[FireKillerGR]

43 minutes ago, Leeghoofd said:

Let us also take into account a "renown" Youtuber posteds several "bugged result" , so people start thinking this is normal...

For the moment it is manual labour for HWBOT and UL, but the Hall of Fame should really get cleaned up ASAP.

 

However what I find more worrying is that some of our "experienced" clockers posted their bugged results at the Hall of Fame.... they should lead by example, not like this... E-peen over common sense

Lets not open this can of worms cause:

a) this wasn't accidental
b) went on with subbing it
c) it wasn't the first time  

Edited November 20, 2022 by FireKillerGR

[Splave]

To those responding we are talking to FM already, why not do it in public so I can be aware of the issue too and everyone else? Thats why I made the thread. 

On 11/20/2022 at 4:58 AM, TASOS said:

I dont know if your posts are about something or someone in particular.

But i will speak in general.

Since you know/find out some very specific issues about this RTX 4xxx series , why not make an article about it ?

This stuff is new and many people would like to overclock it up to the edge of stability.

They probably wont have a clue if their big score is bugged (except if the result/run is too obvious , like in 2k3).

Are we having something similar like the 3DMark Vantage era ?

 

Give them a heads up

Dont assume that everyone is acting with bad intention 

I guess this is an article here? I also didnt name any names which I could (and will if they dont pull them themesleves. one person already did)  and oh yes the ones I see should know way better. These are not noobs. The information is here, they can choose to apply it to themselves or not. 

[Mr.Scott]

20 minutes ago, Splave said:

I also didnt name any names which I could (and will if they dont pull them themesleves. one person already did)  and oh yes the ones I see should know way better. These are not noobs. The information is here, they can choose to apply it to themselves or not. 

Public humiliation is a viable deterrent.  Just sayin ........... 

Edited November 21, 2022 by Mr.Scott

[Leeghoofd]

12 hours ago, Splave said:

To those responding we are talking to FM already, why not do it in public so I can be aware of the issue too and everyone else? Thats why I made the thread. 

I guess this is an article here? I also didnt name any names which I could (and will if they dont pull them themesleves. one person already did)  and oh yes the ones I see should know way better. These are not noobs. The information is here, they can choose to apply it to themselves or not. 

Talk to Kayla as we discussd it already on discord Allen

[Fasttrack]

On 11/20/2022 at 11:58 AM, TASOS said:

I dont know if your posts are about something or someone in particular.

But i will speak in general.

Since you know/find out some very specific issues about this RTX 4xxx series , why not make an article about it ?

This stuff is new and many people would like to overclock it up to the edge of stability.

They probably wont have a clue if their big score is bugged (except if the result/run is too obvious , like in 2k3).

Are we having something similar like the 3DMark Vantage era ?

 

Give them a heads up

Dont assume that everyone is acting with bad intention 

For me, being a simple member, that has never benched any vga under cold, the post of Tasos is the most reasonable in my eyes.

From what I read ( assuming I understood correctly ), HWBOT ( and maybe UL ? ) will require even more verifications to judge if a run is legit or not.

OK, whatever you say.

Just jeep in mind that 99% of members come here to spend some hours and have some fun.

  

[Splave]

3 hours ago, Leeghoofd said:

Talk to Kayla as we discussd it already on discord Allen

What's a discord

[Leeghoofd]

Just got this in from UL: 

 

We have released SystemInfo 5.55 that has detection for ECC status for NVIDIA GPUs. 
 

When ECC is supported by the card (for GeForce this is 4090 and 4080 and maybe 3090 Ti?) the Graphics card data segment should include info on whether ECC was enabled or not when using this SI, so technically you could now start requiring it and checking the status from our result page. There is a performance penalty of about 4-5% for turning it on, though.

 

Going to ask community to try to break it before we require it for anything, but now the information is there so we can start investigating whether this is a fix for the issue.

 

So we are going to require ECC set to enabled from now on

[saltycroissant]

On 11/21/2022 at 12:20 PM, Splave said:

To those responding we are talking to FM already, why not do it in public so I can be aware of the issue too and everyone else? Thats why I made the thread. 

I guess this is an article here? I also didnt name any names which I could (and will if they dont pull them themesleves. one person already did)  and oh yes the ones I see should know way better. These are not noobs. The information is here, they can choose to apply it to themselves or not. 

Id say the most annoying thing was asking UL to delete the 10+ backup bugged score that ppl would have, because after subbing 1 bugged score and having it removed they'll still post their second bugged score. 

Im on the side of Leeg this issue got worst and more widespread mostly because of some yOutUbEr posting bugged result on stream.

Hopefully this get sorted out quickly by UL, will save me a lot of email.

 

Edited November 23, 2022 by saltycroissant

[Leeghoofd]

Systeminfo 5.55 only from now and ECC enabled

 

 

[Mr. Fox]

The more complicated things get, the less enjoyable it is to participate. I sure am glad I haven't already wasted a butt-load of money an overpriced 40-series GPU, and seeing that doing so is going to create extra rigmarole and nonsense to have to be concerned about with respect to EEC on versus off, chances are a lot greater that I am not going to bother. Lack of Windows 7 driver support was a good enough reason not to, but this new reason is an extra excuse to vote "no" by not opening my wallet.

[Papusan]

6 hours ago, saltycroissant said:

Id say the most annoying thing was asking UL to delete the 10+ backup bugged score that ppl would have, because after subbing 1 bugged score and having it removed they'll still post their second bugged score. 

Im on the side of Leeg this issue got worst and more widespread mostly because of some yOutUbEr posting bugged result on stream.

Hopefully this get sorted out quickly by UL, will save me a lot of email.

 

7 hours ago, Leeghoofd said:

Just got this in from UL: 

 

We have released SystemInfo 5.55 that has detection for ECC status for NVIDIA GPUs. 
 

When ECC is supported by the card (for GeForce this is 4090 and 4080 and maybe 3090 Ti?) the Graphics card data segment should include info on whether ECC was enabled or not when using this SI, so technically you could now start requiring it and checking the status from our result page. There is a performance penalty of about 4-5% for turning it on, though.

 

Going to ask community to try to break it before we require it for anything, but now the information is there so we can start investigating whether this is a fix for the issue.

 

So we are going to require ECC set to enabled from now on

Does this means all posted scores on the leaderboard for 3090 Ti and newer cards will be removed? 4-6% performance penalty from the new rules will reduce your chance to beat older scores with same or sligtly higher clock speed. Or will only the scores from 4080 and 4090 be deleted/removed?

Edited November 23, 2022 by Papusan

[Fasttrack]

Wrong decision. Dead wrong.

I was thinking to buy an RTX 4090.

Nope.

ECC enabled ?

This means shared memory.

This is useful when the card operates on workstations for computing.

Not 3d benching.

Shame, shame and shame again.

Big elephants indeed exist in HWBOT members.

Why do we all have to suffer in order to kill the big elephants ?

[Leeghoofd]

@Papusan Only 40 series are involved in the ruling, the inflated scores will be pulled by us and UL and will need to be rebenched correctly...

 

[Mr. Fox]

How are they inflated? i would have ECC turned off for normal everyday use as well. i've never wanted or needed ECC for anything. This means anyone that buys the most expensive GPU has to deliberately cripple it. Makes no sense you are required to gimp performance in order to be allowed to submit benchmarks. Is it because the RTX 4090 is flawed and engineered incorrectly and produces erratic results unless it is gimped?

Edited November 23, 2022 by Mr. Fox

[Leeghoofd]

Seems you missed all the fun... users on stock cooling beating LN2 clocked scores, 15mhz more on the ram and here comes the gold...

By the way this solution is a community fix, not only our call... but noone will ever believe that I guess...

On request of UL : Going to ask community to try to break it before we require it for anything, but now the information is there so we can start investigating whether this is a fix for the issue.

[Papusan]

26 minutes ago, Leeghoofd said:

@Papusan Only 40 series are involved in the ruling, the inflated scores will be pulled by us and UL and will need to be rebenched correctly...

 

Thanks. And you are sure that the 3090 Ti isn't affected by this? And what with UL. Will none of the posted scores forwards (from anyone) be validated and checked "Valid" on the leaderboard on Futuremark if you don't have ECC checked in NCP? I dont t'alk about the scores that will be put on Hwbot. But on UL(Futuremark) leaderboard.

Edited November 23, 2022 by Papusan

[Mr. Fox]

16 minutes ago, Leeghoofd said:

Seems you missed all the fun... users on stock cooling beating LN2 clocked scores, 15mhz more on the ram and here comes the gold...

By the way this solution is a community fix, not only our call... but noone will ever believe that I guess...

Correct, I did not catch all of that. Maybe the LN2 clocked scores were run with the ECC handicap enabled by mistake?

The part that confuses me is singling out one GPU and enforcing a performance-crippling feature on it just because it has the exclusive option. Nothing else ever had it before, and it is not clear why anyone buying a 4090 would want it enabled unless they were using it for business-related reasons. To make that logical would require the exclusion of all submissions  by GPUs that do not have the ECC feature as well. Unless what I mentioned before is accurate and the 4090 is a flawed product that produces consistently erroneous results without ECC enabled, in which case there should not be an option available in NVCP to disable it. It shoulld be enabled with no option available to disable it if that is the case. Or, not allow scores for 4090 GPUs because they are an inherently bugged product?

Edited November 23, 2022 by Mr. Fox