If you could change 1 thing on hwbot what would it be?

[_mat_]

I think the new points system is not a good solution because it doesn't motivate to compete in all benches equally. In fact the whole idea of "the more subs the more points" is shifting the interest towards certain benches where it is next to impossible to compete against the well known overclockers while others are not benched at all for obvious reasons. A fair points distribution system could make it easier to compete in the long run which could attract new benchers.

There is also the big topic of trust issues. Too many benchmarks can be easily tricked because the screenshot is the only source to judge by. These benchmarks need to be updated or removed. Although this is a very painful process, it is necessary for extreme overclocking to be taken seriously. But that's not all, the following things need change in my opinion:

• Timing and result handling have to be unified! Benchmarks are currently horrible at this, some are better some are so easily cheatable it's a joke.
• The submission system should have a SDK, so it can be easily integrated into benchmarks and updated in a unified way when changes are needed. HWiNFO, a unified timer and result handling have to be a part of this SDK. The benchmark just feeds the right values to it and fires the submission process.
• The logic/validation of submissions have to be server-side. The benchmark can do it on its own to improve user experience but the server decides if this is a valid score. This would allow old benchmark versions to be valid in certain situations and would have circumvented the timer skew bugs in Windows 8 and higher. The next bug will come along sometime and the already difficult OS/hardware/benchmark combination will end up as a clusterf***.
• Benchmarks should not bother with vendor and device names of hardware, because there is always a problem with matching the local and server names. The benchmark will therefor only send the vendor and device ids, the server translates them back into readable names.
• Benchmarks should be thorougly inspected by someone with knowledge of reverse engineering and locating serious security issues. Together with the SDK submission integration this could end up in a certificate of some kind.

I know that this is a lot of input on benchmarks, but don't forget that benchmarks are the main focus of overclocking. Without them there is nothing to measure so they are key to the overclocking business. To understand how bad the situation currently is I decided to finally release the following video to the public. This is a tool I wrote in not even a day that completely screws with CPU-Z and XTU in an undetectable manner. I won't go into technical details for now for the sake of extreme overclocking in general. With GPUPI 4 I already found a way to fix these security issues (and lots of others).

 

[Dancop]

Holy shit!!! That's just unbelievable!

[Casanova]

1 hour ago, _mat_ said:

To understand how bad the situation currently is I decided to finally release the following video to the public. This is a tool I wrote in not even a day that completely screws with CPU-Z and XTU in an undetectable manner. I won't go into technical details for now for the sake of extreme overclocking in general. With GPUPI 4 I already found a way to fix these security issues (and lots of others).

 

That's it, i give up. Stand up paddle will be my new hobby.

[Achill3uS]

jeeez, i wont run a single XTU anymore...

[subaruwrc]

just ban that shit finally ... 

[_mat_]

Casanova, this is in no way intended and should not be the message taken away from my post.

The truth is that I have been researching on timers and benchmark security in general for quite some time now and found multiple issues. The issue shown in the video is just one of those, although it's a pretty tough one. It's important to know that not a single bit of executables, drivers or OS code was edited to achieve these results. What has changed is how the system is perceived through the eyes of the benchmark. It's just not reliable the way it's done currently.

What really should be taken away from this is the fact that this needs attention, needs to be changed and fixed. And it's not only one thing.

Edit: Just to clarify, HWBOT knows about these problems since last year.

Edited July 13, 2018 by _mat_

[_mat_]

Don't focus on XTU too much. Although I have been in contact with Intel and they changed nothing (as always), it's not only their problem. ALL benchmarks are affected by one or more issues.

[Casanova]

18 minutes ago, _mat_ said:

Casanova, this is in no way intended and should not be the message taken away from my post.

The truth is that I have been researching on timers and benchmark security in general for quite some time now and found multiple issues. The issue shown in the video is just one of those, although it's a pretty tough one. It's important to know that not a single bit of executables, drivers or OS code was edited to achieve these results. What has changed is how the system is perceived through the eyes of the benchmark. It's just not reliable the way it's done currently.

What really should be taken away from this is the fact that this needs attention, needs to be changed and fixed. And it's not only one thing.

Edit: Just to clarify, HWBOT knows about these problems since last year.

I totally understand man. Congrats for all your effort.

Have you tried to reproduce this with pifast and superpi? Do u think they can be cheated too?

Edited July 13, 2018 by Casanova

[_mat_]

Yes.

[bigblock990]

mat is the pilot, hwbot is the target  RIP

Edited July 13, 2018 by bigblock990

[Casanova]

Rbuass told me once to bench because i like it and for the fun, not for the points... That totally makes sense now.

[der8auer]

In the end every software can be fooled. I'm pretty sure mat is the only guy in here who can do it  So personally I wouldn't worry too much about it. 

[Leeghoofd]

So this all boils back to the statement I have made in several threads in this forum over the years:

 

the real problem is not solely the bugged over-algorythmened HWBOT and its choice of benchmarks,  it its the OCer trying to "tweak" his ranking.

 

If we have to remove all trickable benchmarks we will have not much left ?  Now if you guys want to run 6 benchmarks, fine by me lol, I even bet some are trying to reproduce Matt's testing right now.

 

I'm all in for a simplified ranking based on a simple to understand algorythm.

Maybe an idea to Focus on that guys, how do you see the new algorythm, what should Frederik base himself upon. This insane complex algorythm has been developed to plz top competitors and especially 3D scores, too boost competition participation and to avoid complete recalculation of too many rankings each time a new submission was done ( server load reducement)

Edited July 13, 2018 by Leeghoofd

[Splave]

you dont even need matt's hax to do the simplest BS possible. Saving the screen shot then apply as background, then put in any processor you or GPU you want open the cpu tab and take another screenshot on top of it. This ruins all no validated benching and I honestly think its more common than people think in HW benching. IE run a vantage with a 5870 and then save only the score window. Set that picture as the background reboot with your ln2 4870 and you have the Worldrecord without even benching congratulations. Atleast with a wrapper or systeminfo there is an extra layer of security even if its not offically supported it will say the card name in the link that was used. 

 

Its also not uncommon for ln2 benchers to save the screenshot of just the score immediately incase CPUz causes a system crash. Then you can just apply as background and open your cpuz with slow mode enabled flip off quickly save and good to go. Atleast in this case the bench is actually run at the reported frequency. 

 

(Dont be distracted by the cute baby he has since grown into that massive head if you were concerned  )

 

Matt's fixes are a great starting point to the future of secure submissions, hopefully he finds a place that takes him seriously. 

TLDR: Screenshots are not secure and we need better validation than that

 

Edited July 13, 2018 by Splave

[Achill3uS]

Absolutely with you Alby, you can die tryin to avoid these tricks with a more and more complex algo what eats up all the resources, but if someone is happy with fooling the system shame on him. Actually im benching long time ago only what i enjoy to run not what worth the most by points. Just never lose our motivation to bench and compete each other bcoz of a furstrating system what suppose to give us space to live our passion. +1 to make it simple.

[mickulty]

One single thing?  Give ycruncher points, it's an interesting benchmark that can run on an extremely wide range of hardware with a validation system and a developer who cares about security.

In a broader sense, the problem with insecure benches like aquamark or 3dmark2001se is not that they necessarily are cheated, but that when someone puts a lot of effort into finding legit tweaks and gets monster scores other people will accuse them of bugging the bench or cheating - often without any sense of irony from either group.  It doesn't matter if anyone is actually cheating - if everyone knows you *could* cheat that alone is a salt mine.

On the other hand if those benches lost points it would REALLY fucking suck because there's not much else you can even run on the kind of ultra cheap old-ass agp cards that are good for getting into competitive OC.

[speed.fastest]

Cheater is always cheater, find way to cheat everywhere, the problem here isnt cheater, but HWBot Database is broken, and money oriented overclocking industry is what make it broken.

[marco.is.not.80]

13 hours ago, mickulty said:

One single thing?  Give ycruncher points, it's an interesting benchmark that can run on an extremely wide range of hardware with a validation system and a developer who cares about security.

Actually, that is one question I've had since joining the bot. Why are some benchmarks allocated points and others aren't? What is the criteria? Is it because of the algorithm? Too many points from too many sources equals overload?

Edited July 15, 2018 by marco.is.not.80

[Noxinite]

I was thinking kinda similarly, if benchmarks are not buggy and cheatable (on the whole) then surely they could all have HWpts enabled? And vice versa.

Not sure about global pts as I'm not sure what the aim of them is. The number of benches with them was reduced with Rev7, but I can't remember why.

[speed.fastest]

I think global point is related to money oriented industry, other than that global doesnt matter a lot except from vendor that use overclocker to show score from new hardware. So what about remove global point and only hardware & wr point? It should be enough i think. Requirement from new hardware is what make overclocker goes i think, this because global points.

[mickulty]

9 hours ago, marco.is.not.80 said:

Actually, that is one question I've had since joining the bot. Why are some benchmarks allocated points and others aren't? What is the criteria? Is it because of the algorithm? Too many points from too many sources equals overload?

For new benches to have points added they usually need to be secure, I think the exception is hwbot prime which served the purpose of being cross-platform with mobile.

For old benches to have points removed, that's a bit more difficult.  Putting aside my usual extreme cynicism and all the obvious cynical reasons for established people to object, a lot of people might feel that it would devalue a lot of overclocking history to make those benchmarks "irrelevant".  PCMark05 (which was described as "unmoderatable") faced a lot of resistance when removing points was proposed.

[RULE]

I waited a bit before replay, i usually read everything also if i don't take part in the discussions i'm watching you

In 3 pages no one have mention how much is our fault this situation? yes i'm talking also of myself as we should be shame of ourself 'cos as a community we sucks hard fellas.  If you like something why you wanna try to kill it? I really can't understand where this auto-destructive attitude come from. Is really come the time where we need to put our personal interest aside and start think as a community, otherwhise can close everything and go golf like old cucks.

What i love of hwbot is the database where i can compare my score with other people from all around the world. Don't care really much about boints no more, basically boints for me are cancer, i prefer work hard in just some scores and make it properly.

So now i should say what i would change but....

@richba5tard  i really wanna know what's your plane for the future. I mean you build this and i assume u care to keep it alive. Sadly i believe the only way to keep overclock alive is to burn hwbot from the ground and start over, probably move database in another more active community like overlcok.net or something like that. Seems people have no problem to pay an annual amount of money for have something smooth to play with, so i'd start from there. I know this might be painfull for you but really now are we talking about life or death for hwbot.

This are my 22cents, I know we can sort things out and make something better than before, just please stop to suggest a point system at your advantages, we are lucky that a lot of this kind of people left so we can make what will come even better than before.

Edited July 15, 2018 by RULE

[speed.fastest]

Database without point will be great, who care about points anyway? Nice write @RULE.

[mickulty]

10 minutes ago, speed.fastest said:

Database without point will be great, who care about points anyway? Nice write @RULE.

The thing with points is they provide a way for legacy stuff to have some kind of meaning, providing instant context.  Without them only the latest top-end stuff will matter because that's all people care about, and that makes XOC a lot more exclusive and hurts the community.

4 hours ago, RULE said:

@richba5tard  i really wanna know what's your plane for the future. I mean you build this and i assume u care to keep it alive. Sadly i believe the only way to keep overclock alive is to burn hwbot from the ground and start over, probably move database in another more active community like overlcok.net or something like that. Seems people have no problem to pay an annual amount of money for have something smooth to play with, so i'd start from there. I know this might be painfull for you but really now are we talking about life or death for hwbot.

HWbot is alive.  This would be death.

[superpatodonaldo]

On ‎7‎/‎12‎/‎2018 at 4:24 PM, Splave said:

See title. 

 

Only get to choose one thing. Also choose 1 thing you like. And go

1 thing I like: vmod database

1 thing I'd like to change: points assignment; new point = actual point divided by (average?) hardware price

Edited July 16, 2018 by superpatodonaldo