Jump to content
HWBOT Community Forums
_mat_

Intel's XTU analyzed (and it's not looking good)

Recommended Posts

Posted (edited)

Now that XTU will have its global points removed soon, I am officially publishing my findings on this benchmark. I tried to give full insight on how to disect and uncover the security issues of XTU but also some tweaks and the possiblity to run the inner benchmark executable on its own for quick performance testing and points calculation.

xtu-dll-injection-example-to-redirect-ac

https://www.overclockers.at/articles/intels-xtu-analyzed

This is not some kind of personal vendetta against Intel; far from it. The article's purpose is purely educational to raise awareness for benchmark security and timer reliability. This is not only about cheating, it's about the credibility of benchmarks and result databases like the bot as well. Security vulnerabilities are not taken seriously enough by benchmark developers and HWBOT in my opinion. Yes, I am going the hard way with XTU in my article of course and that's not for everyone. But there are already tools available for download that will get you ahead without any effort.

So I'd like to start a discussion here on how we can improve the situation permanently. It goes without saying that any serious initiative would require a cooperation from all sides involved.

Edited by _mat_
by staffs request
  • Like 6
  • Thanks 8

Share this post


Link to post
Share on other sites

Rip xtu, you will be missed - the king is dead, hail to the new king hwbotprime 1.00 :)

  • Haha 1

Share this post


Link to post
Share on other sites

XTU is one thing... but with enough effort I'm sure many benchmarks can be hacked like this. And I seriously doubt those vulnerabilities will be fixed. :|

  • Like 1
  • Thanks 5
  • Sad 1

Share this post


Link to post
Share on other sites
Posted (edited)

... and the last one can turn off the server and switch the lights off. :P

//edit... but similar analysis of x265 would be interesting of course. On the other hand It holds a risk of finding some kind of a security hole which may not be possible to fix at all... or would take too much of time to be worth it. The fact the benchmark developer is still here doesn't guarantee unlimited maintenance and support. ;)

Edited by havli
  • Like 1
  • Thanks 1

Share this post


Link to post
Share on other sites

Yes, every benchmark is breakable and nothing is 100% bulletproof. But that's not necessary, we only need to increase the cost for cheating to a level where it's easier/cheaper to just overclock and make the damn score. Overclocking has no 100,000 USD competitions, so it's not an impossible task.

Regarding your benchmark and its current state of security:

1626703302_hwbotx265-99_999.thumb.png.d963654b858eab52d9f52ff95aba95d8.png

It took ~2 hours to break HWBOT Prime 1.0 and your bench and I've never written a single Java desktop application. it. No debugging or code changing involved, only static analysis of your executable. If you need details on how to reproduce this, just let me know and we'll chat.

  • Haha 1
  • Confused 1

Share this post


Link to post
Share on other sites
Posted (edited)

XTU actually is fun to bench, only if it support AMD and not only 2 seconds count for scoring. There is still no successor for SuperPi 32m for new platform, proper multithreaded & memory scales a lot! I only think is 3DMark11 Physics is only option for now to replace SuperPi 32m.

And please dont do this unless you submit this to their developer only, so xoc is not dead. We dont know this if someone not publicing this. I respect your good job still!

Edited by speed.fastest
  • Like 2

Share this post


Link to post
Share on other sites
1 hour ago, speed.fastest said:

XTU actually is fun to bench, only if it support AMD and not only 2 seconds count for scoring. There is still no successor for SuperPi 32m for new platform, proper multithreaded & memory scales a lot! I only think is 3DMark11 Physics is only option for now to replace SuperPi 32m.

And please dont do this unless you submit this to their developer only, so xoc is not dead. We dont know this if someone not publicing this. I respect your good job still!

What about GPUPI (even though its a royal pain to get everything working)? Hopefuly v4.0 will be out soon and and the woes of installing OpenCL support will be at an end.

  • Like 1

Share this post


Link to post
Share on other sites
8 minutes ago, cbjaust said:

What about GPUPI (even though its a royal pain to get everything working)? Hopefuly v4.0 will be out soon and and the woes of installing OpenCL support will be at an end.

GPUPI is good, but its pure cpu, it doesnt even care if single channel memory only.

  • Like 1

Share this post


Link to post
Share on other sites
9 hours ago, speed.fastest said:

XTU actually is fun to bench, only if it support AMD and not only 2 seconds count for scoring. There is still no successor for SuperPi 32m for new platform, proper multithreaded & memory scales a lot! I only think is 3DMark11 Physics is only option for now to replace SuperPi 32m.

I already have a Prime95 benchmark ready that scales pretty much perfectly and is very hungry for memory bandwidth. It is using the latest Prime95 version (29.4) and includes a few fixes like NUMA awareness, improved thread synronization and error checking. It's also faster than the XTU version. :D

It seamlessly integrates into XTU so you can start XTU with an AMD CPU and just run the benchmark. It won't be far off to create a WIN32 application that runs the bench and uploads the score to HWBOT. Is this something the community would be interested in?

  • Thanks 4

Share this post


Link to post
Share on other sites
Posted (edited)

we need wrappers, everywhere just to keep OC alive, you can make those Matt... However due to our own self destructive community attitude it might not be enough... sigh

Honestly beats me why people always have to go the extra mile for some bragging rights on the internet...

Edited by Leeghoofd
  • Like 1
  • Thanks 3

Share this post


Link to post
Share on other sites

Newsflash, pretty much every benchmark can be hacked given enough time and skill. Question is is it worth spending dev time actively combatting hackers when the only gain is temporary internet fame?

Benchmark makers should prevent easy cheating (like adjusting the windows clock), but injecting dll's/modifying code?

  • Like 3
  • Thanks 1
  • Confused 1

Share this post


Link to post
Share on other sites
6 minutes ago, richba5tard said:

 

Benchmark makers should prevent easy cheating..like adjusting the windows clock

no offense by why did you not then with hwbotprime? 

  • Confused 1

Share this post


Link to post
Share on other sites
2 minutes ago, Splave said:

no offense by why did you not then with hwbotprime? 

No offense taken. My focus was not hwbotprime the past years, and I hate shifting focus between projects. Should have done the clock cheat fix earlier, I know.

Byte code alteration or adjusting memory is a whole other story though. As long as you allow your app to work offline, it can be hacked.

  • Like 2
  • Thanks 1

Share this post


Link to post
Share on other sites

modern OC is a moderators nitemare, even the latest timespy and co can still be tricked and submitted valid sigh...

 

Why can't peeps just press run benchmark

  • Like 9
  • Thanks 2
  • Sad 1

Share this post


Link to post
Share on other sites
On 1/3/2019 at 10:16 PM, speed.fastest said:

GPUPI is good, but its pure cpu, it doesnt even care if single channel memory only.

Not now that it uses avx on intel...

 

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×