Deliberately disabling GPU Cores for 3D Benchmarks

[FireKillerGR]

Hi everyone,

after the recent discussion about the loopholes of old benchmarks (especially 01) and how they can be tricked by running them with less GPUs enabled in SLI/Crossfire while subbing with full Crossfire/SLI enabled, we did some research today to see whats up.

We recorded a video of 21 mins non-stop testing showing enabled 4x GPU Crossfire vs Single gpu and their differences & also sent a comparison screenshot to the moderating team (full video available to them at any point of time)

As the performance gap between single and multi-GPUs on 3DMark2001 is quite close, some seem to have taken advantage of it and run the benchmark with one GPU as they couldn't figure out how to get the performance to work on multi-GPU configurations.

Thinking that everyone is cheating and instead of raising the question towards the mod team or reaching out to other users in a discrete way to check what's up, chose to take advantage of the small performance differences in order to run the benchmark with a single GPU before enabling the Crossfire for screenshot purposes.

Eventually this is not just about figuring a potential concern but also about deliberately running a false configuration on purpose for personal gains.

The performance difference being small or the lack of system info should not be the reason to call it anything differently than what it is.
To our eyes this is NO different than disabling cores on a 12900K/12900KS and ruling the 1c/2c/4c/6c cpu categories other than this being visible to the eyes of the community and the moderating team.

Same way one could do the exact opposite and run 2x GPU in SLI/Crossfire and disable it to sub it under the single gpu category of any 3D benchmark.
How visible it is makes no difference to the action itself.
NOTE: It is SO visible as a specific test of 3DMark2001 performs worse with a high clocked LN2 CPU (running single gpu) vs a run at 5.6 GHz with Crossfire enabled.

With new competitions coming in and HWBot slowly growing again, it is also on the users to maintain a certain level of transparency and avoid clear wrong-doing and not just on the moderating team.

We are coming out publicly on this as killing a benchmark cause some(one) CHOSE to cheat while others invested time & money to figure it out shouldn't be the way.
Additionally, thinking that everyone is cheating and submitting scores with wrong configurations on purpose for personal gain should not be left as is either.

We mentioned it before and we are mentioning it again as we stand by it; we are always open to make video-proof a necessity for top-10/20 scores & comps to maintain fairness towards all users.

Regards,
Stavros & Phil

Edited June 29, 2022 by FireKillerGR

[Splave]

So we cant just require videos for everyone. I for one flat out refuse to do so. 

for other benches that we couldn't police effectively we removed. 

is there any other ideas besides requiring videos?

Can UL or whoever throw us a bone and make use a systeminfo or something

Edited June 28, 2022 by Splave

[AKM]

Throwing in my two cents here..

I personally don't think disabling/re-enabling CF/SLI for multi-card benches should be a thing. Like Stavros mentioned, disabling cores on a 8 core cpu to compete in the 2 core rankings is definitely not a thing. Seems the logic should be the same all around?

Then again, 3D01 is so unique because what other bench can you run in any order you please and still get a valid score? What other bench can you legit change settings between subtests and still continue on? Imagine if we could run Geek 3 test by test, etc. So in that regard, its all very grey because the bench and scores themselves date back so far before all the various rule changes and what not.

Idk what the line between "cheat" and "tweak" is, or what it should be, but I think that in this context this should not be considered a tweak at all.

1 hour ago, FireKillerGR said:

We mentioned it before and we are mentioning it again as we stand by it; we are always open to make video-proof a necessity for top-10/20 scores & comps to maintain fairness towards all users.
 

Requiring proof for competitions I can understand as there are legitimate prizes on the line, but for every top score just seems like a huge pain. I'm sure we all have regular lives as well and time is already limited for a lot of us, and this just seems super extra for regular subs. But honestly, I myself don't have any valid ideas to police it. In my opinion we just nuke the globals for 01. It sucks because legacy 3D is my favorite but at this point don't see any other way to avoid situations like this. Maybe let it ride out until the end of the year/season and then change for next year? Hardware points should be good still because of how big the sub database is for 01, but we would hopefully not run into things like this again.

[der8auer]

1 hour ago, FireKillerGR said:

Same way one could do the exact opposite and run 2x GPU in SLI/Crossfire and disable it to sub it under the single gpu category of any 3D benchmark.
How visible it is makes no difference to the action itself.
NOTE: It is SO visible as a specific test of 3DMark2001 performs worse with a high clocked LN2 CPU (running single gpu) vs a run at 5.6 GHz with Crossfire enabled.

Two things here: Before I opened my thread 2 days ago I looked into this and from my perspective it was not clearly visible in all cases if single or multi GPU was used. 01 and 03 were the benches I probably used the most in my active carreer. And especially with the different tweaks (and run oder for 01) there are sometimes huge fluctuations in subtests. And especially for blocking people for "cheating" we have to be 100% sure that it was actually a cheat. And if there is a tiny uncertainty left, we have to give benefit of the doubt.

The other thing is that new benches don't really allow this way of cheating because we always ask for the futuremark verification link. Just disabling SLI after running Time Spy for example won't work. That was my intention why I asked to just disable globals for 01/03 for the multi GPUs. 

Looking at this from a neutral point of view and especially business point of view (the last one I don't really care about tho) I would just disable them right away and move on. We have so many other topics to care about that spending time on this doesn't really make sense. But then again from a bencher point of view I know how much time and effort (and money) it takes to create these scores and I respect that. That's why I was looking for your feedback on this matter. From my side there are just two options:

- It's detectable and people report the scores -> moderation investigates and decides. This can cause drama for both sides. Moderation will have to spend time on it and potentially can take a wrong decision here if the detection is not clear. This would be at your risk

- We disable points and move on

[FireKillerGR]

33 minutes ago, AKM said:

Requiring proof for competitions I can understand as there are legitimate prizes on the line, but for every top score just seems like a huge pain. I'm sure we all have regular lives as well and time is already limited for a lot of us, and this just seems super extra for regular subs. But honestly, I myself don't have any valid ideas to police it. In my opinion we just nuke the globals for 01. It sucks because legacy 3D is my favorite but at this point don't see any other way to avoid situations like this. Maybe let it ride out until the end of the year/season and then change for next year? Hardware points should be good still because of how big the sub database is for 01, but we would hopefully not run into things like this again.

Yes tbf maybe its coming from us as its easy to hook up the capture card, hit record and bench normally.
It just sucks that some manage to ruin it for everyone else.

24 minutes ago, der8auer said:

Two things here: Before I opened my thread 2 days ago I looked into this and from my perspective it was not clearly visible in all cases if single or multi GPU was used. 01 and 03 were the benches I probably used the most in my active carreer. And especially with the different tweaks (and run oder for 01) there are sometimes huge fluctuations in subtests. And especially for blocking people for "cheating" we have to be 100% sure that it was actually a cheat. And if there is a tiny uncertainty left, we have to give benefit of the doubt.

The other thing is that new benches don't really allow this way of cheating because we always ask for the futuremark verification link. Just disabling SLI after running Time Spy for example won't work. That was my intention why I asked to just disable globals for 01/03 for the multi GPUs. 

Looking at this from a neutral point of view and especially business point of view (the last one I don't really care about tho) I would just disable them right away and move on. We have so many other topics to care about that spending time on this doesn't really make sense. But then again from a bencher point of view I know how much time and effort (and money) it takes to create these scores and I respect that. That's why I was looking for your feedback on this matter. From my side there are just two options:

- It's detectable and people report the scores -> moderation investigates and decides. This can cause drama for both sides. Moderation will have to spend time on it and potentially can take a wrong decision here if the detection is not clear. This would be at your risk

- We disable points and move on

For 03 its quite clear as the scaling is evident. 
Can do some further tests to confirm but its basically visible here:

Recent 1x GPU score with alder cpu:
https://hwbot.org/submission/4858339_chispy_3dmark03_radeon_r9_290x_273295_marks

Our 3x:
https://hwbot.org/submission/5020268_ogs_3dmark03_3x_radeon_r9_290x_420566_marks

vs our 4x:
https://hwbot.org/submission/5020272_ogs_3dmark03_2x_radeon_r9_295x2_446238_marks

For 01 its a bit more tricky but its also evident by judging based on Nature's FPS. 
2+k FPS on 21.5.2 drivers with 290x4 should be easily done with cpu at 5G or so (2.3k fps at 5.6G cpu).
Cant comment about older cards/drivers.

Below you can see 4x xfire vs single gpu with cpu clocked at 5.6GHz in both cases.
I believe single gpu scales better on most tests when pushing the cpu (or requires different LOD) but either way the gap in Nature is massive.

 

Edited June 29, 2022 by FireKillerGR

[saltycroissant]

Just now, der8auer said:

Two things here: Before I opened my thread 2 days ago I looked into this and from my perspective it was not clearly visible in all cases if single or multi GPU was used. 01 and 03 were the benches I probably used the most in my active carreer. And especially with the different tweaks (and run oder for 01) there are sometimes huge fluctuations in subtests. And especially for blocking people for "cheating" we have to be 100% sure that it was actually a cheat. And if there is a tiny uncertainty left, we have to give benefit of the doubt.

 

I think most of the issue comes from AMD gpu driver needing some tweaks to perform correctly and if you don't know exactly how the other score will look like they are all cheating. Im going to be very clear. This thread was probably made because of a single person who ran 3D01se on a single GPU and subbed in x3/x4 categorie thinking other ppl cheated. Like team OGS pointed out this is very ez to spot if the entire run was run on a single gpu due to the Nature score being so low. 

Personnally, i reported the x3 score the day it was posted because the DragoL was outside the possible effi range for the clocks and amount of gpu supposedly used, in fact its so high that it also beat all the x2 sub on win7. 

Id be really sad to lose 3D01se because its a great benchmark in my opinion and that it require quite a lot of work. Pls don't remove this because of 1 person. This thread is a little late sadly because all of this could have been avoided by asking said person to run 3D03 x3 or x4 and it would have been obvious to everyone, but now is too late.

[Splave]

3 hours ago, der8auer said:

 

- We disable points and move on

Since I don't currently have any of these golds I agree to this.

Can we send ogs some thermal paste and a hat or something as congratulations at least. 

[yosarianilives]

Here's what I have to say about sli for this bench. For better or worse you can run single card without disabling sli, try different sli modes. As far as I know this is within the rules and let's you have your cake and eat it too. What the difference between this and disabling sli? Not for me to say

[yosarianilives]

Anyways I second the opinion that we shouldn't pull pts, if we pulled every bench that took some tweaks to os and drivers to get a decent score we'd have almost nothing left. Asking to pull 01 globals is like asking to pull 32m globals imho

[unityofsaints]

With a small browse of the standings I could find the score that is being talked about just by looking at screenshots so in my opinion it doesn't make sense to throw the baby out with the bathwater by changing the points system for one isolated incident. Even with points hardly anyone benches these categories, without they would die for sure.

[Sweet]

Interesting, reminds me of excessive hacking in PcMark05 in old years..But this is not the case

Imo: sustainability in the future of HWBot lies in scores and honest competition
I don't know if 1 gpu or 2 gpus is better but I do know that there are many new guys who want to reach the top positions, as I once thought myself (and I didn't achieve it)
So for me, many more benchmarks should have global ones and give new one the chance to see if they can get closer to the best.
For the sake of sustainability, for this year, the global ones should be left to 3D01 and 3D03
Next year will see...

My 2cents: Sustainability is the word
Chers

Edited June 29, 2022 by Sweet

[TaPaKaH]

Wasn't disabling SLI the oldest multi-card trick in the book for 01?

Quite a few 01 world records in the G80/G92/GT200 era, including my own, were set this way and no one ever complained.

[yosarianilives]

32 minutes ago, TaPaKaH said:

Wasn't disabling SLI the oldest multi-card trick in the book for 01?

Quite a few 01 world records in the G80/G92/GT200 era, including my own, were set this way and no one ever complained.

Then why not do 16 way 3dm01 score. You gotta at a minimum be able to enable it and show enabled in gpuz. As for the sli mode I mainly know about it because that's what superposition defaults and you have to manually force a real sli mode to get scaling

[saltycroissant]

Just now, TaPaKaH said:

Wasn't disabling SLI the oldest multi-card trick in the book for 01?

Quite a few 01 world records in the G80/G92/GT200 era, including my own, were set this way and no one ever complained.

Imo disabling gpu is cheating even if other ppl did it.  When hawaii became the go to for x3/x4 everyone had similar scaling for cpu clock/gpu clock , there's even some score out there of ppl who didn't use the right driver setting, the score difference is massive. But the solution is not to think that everyone cheated so you should do it too. In fact props to that guy for subbing much lower score being legit and not cheating.

[Guest hammertone]

2 hours ago, saltycroissant said:

Imo disabling gpu is cheating even if other ppl did it. 

Yup. Another one of many loop hole bandits here.

My thoughts on this. Why waste electricity installing the second gpu lol

[der8auer]

7 hours ago, saltycroissant said:

Id be really sad to lose 3D01se because its a great benchmark in my opinion and that it require quite a lot of work. Pls don't remove this because of 1 person. This thread is a little late sadly because all of this could have been avoided by asking said person to run 3D03 x3 or x4 and it would have been obvious to everyone, but now is too late.

Single Card would not be affected by this. Maybe also not dual. But will see what this thread brings and then discuss with Albrecht

[phil]

6 hours ago, Splave said:

Since I don't currently have any of these golds I agree to this.

Can we send ogs some thermal paste and a hat or something as congratulations at least. 

Lately I don't like you very much ?

[Splave]

3 hours ago, phil said:

Lately I don't like you very much ?

I still love you though. 

 

The problem as well what if the cheater is actually smart. Benching 4x and his say car low is way too high since he's using 1x gpu. Why not down clock the cpu a bit or gpu clock below stock to get the performance drop you want. ? If you are using a bench that doesn't scale with card count this will always be the problem. 

 

@Leeghoofd not sure what you mean by 01 not being cpu bench? That's all it is. 

Edited June 29, 2022 by Splave

[Guest hammertone]

If you knowingly cheat then you defeat yourself. What is point? Is best effort not good enough?

__________

My humour about all this sad business. ?

1. You all get bans.

2. Loose all your points.

3. Confiscate all your best parts

4. Winning 

__________

My serious side.

1. Leave 3D01se alone. Most of us old guys still enjoy it ?

[phil]

21 minutes ago, Splave said:

I still love you though. 

?

 

25 minutes ago, Splave said:

The problem as well what if the cheater is actually smart.

 ????

[FireKillerGR]

2 minutes ago, hammertone said:

If you knowingly cheat then you defeat yourself. What is point? Is best effort not good enough?

__________

My humour about all this sad business. ?

1. You all get bans.

2. Loose all your points.

3. Confiscate all your best parts

4. Winning 

__________

My serious side.

1. Leave 3D01se alone. Most of us old guys still enjoy it ?

Jokes aside there needs to be a distinctive line to whats punishable and what isnt and to what extent.

Recall @Raufgetting banned cause he altered a screenshot of a legit score as he added a memory tab that was missing.

[Leeghoofd]

32 minutes ago, Splave said:

 

@Leeghoofd not sure what you mean by 01 not being cpu bench? That's all it is. 

Yes for you it is, not everybody benches 12th gen and/or on ln2. 

Easiest solution drop globals for submissions with more than one card. 

[saltycroissant]

Just now, Leeghoofd said:

Yes for you it is, not everybody benches 12th gen and/or on ln2. 

Easiest solution drop globals for submissions with more than one card. 

Might be a ez solution but it would also punish everyone that behaved well because of 1 person

[Leeghoofd]

Well it's nearly impossible to moderate, so this indeed the easy route.  But a single card can still mean there are 2 gpus onboard. I need to check with the programmer if it's doable to perform this.

In the end I would focus your frustrations on the ones who pulled this one. HWBot is again with the back agains the wall as we are entitled to take a decision.

As usual some that did it will still call it a tweak, others a cheat. An endless discussion.... but to bench it in the same way as the other one got away with it, that act can't compute in my tiny brain. Plus it could lead to full removal of this fun benchmark.

Think of possible solutions. Video proofing maybe, but doubtfull as the benchmark just takes too much time to film everything... 

[saltycroissant]

Just now, Leeghoofd said:

Well it's nearly impossible to moderate, so this indeed the easy route.  But a single card can still mean there are 2 gpus onboard. I need to check with the programmer if it's doable to perform this.

In the end I would focus your frustrations on the ones who pulled this one. HWBot is again with the back agains the wall as we are entitled to take a decision.

As usual some that did it will still call it a tweak, others a cheat. An endless discussion.... but to bench it in the same way as the other one got away with it, that act can't compute in my tiny brain. Plus it could lead to full removal of this fun benchmark.

Think of possible solutions. Video proofing maybe, but doubtfull as the benchmark just takes too much time to film everything... 

I understand that its a nightmare to moderate. And no video proofind isn't the solution, i do agree with Splave on that.

I was really thinking that this wouldn't create such drama because if you have benched those card in question its pretty clear if someone cheated or not ( because lets be honest disabling GPU is cheating its not a tweak im sure 99% of ppl would agree). 

I also understand that its a difficult decision and not a funny 1 to take for you and Der8auer. For 01se if you want to have it 100% safe and clean you can only keep the x1 categorie. With dual gpu you could still disable 1 gpu on it and sub as x2.