Jump to content
HWBOT Community Forums
Leeghoofd

The official BenchMate support thread

Recommended Posts

Good idea.

It would also be great to let people know that BenchMate 0.9.3 should only be used on a separate OS, so it is not mixed with BenchMate 0.10. That causes a lot of frustration on all sides.

  • Thanks 1
  • Confused 1

Share this post


Link to post
Share on other sites
On 7/7/2020 at 12:22 AM, Alex@ro said:

This software  is junk with capital letters.

Secondly, this made me consider retiring from benchmarking for the first time in 12 years, unbeliavable. Not even ryzen on cold, not even B-die benching, not even cracks when full-pot, nothing did the job except the amazing bench-banana-mate.

It makes absolutely no joy to bench anymore with this application.

Yeah, everything is nice and dandy, unicorns are pink and gooses are flying until this software will simply not work. 

In my experience over time i found that eventually you can find answers to most common issues with benchmarks. Google it, this forum, ask a fellow overclocker, eventually you know what to do. But with benchmate, you are left blind and dumb, both at the same time.

Thanks for speaking out. There are others around here who share your opinions but don't talk about it because they're afraid it would come across in the wrong way or because they've given up trying.

It's interesting that your post has drawn so much criticism for being too ranting, if anything it it wasn't ranting enough and left out some things. While it touches on the many bugs and frustrations with Benchmate, it leaves out the two other valid criticisms: single point of failure for support, and benchmarks scoring higher with Benchmate than without. Really, any of those three issues on their own are enough to disqualify Benchmate from serious use. Maybe the Google model of "perpetual Beta" applies here.

Please don't retire! You're a great bencher and many of us share your frustrations, even if we aren't necessarily posting about it. And as they say, don't feed the trolls 😉

  • Like 1
  • Thanks 1

Share this post


Link to post
Share on other sites
Posted (edited)

Please specify the bugs that you are referring to. It would really help to come forward and offer feedback and bug reports.

There are currently a few limitations mainly because BenchMate currently works best with modern platforms. These are:

  • Old or cheap CPUs won't work because they don't have the AES-NI instruction set. The benchmark will just close.
  • Don't use a 12 year old NVIDIA card because the driver wants to inject stuff into the benchmark, which is a big no-no and BenchMate is right to disallow this.
  • Windows 7 without SHA2 patch. BenchMate won't start. Installt the patch first.
  • Don't mix BenchMate 0.9.3 and 0.10. If you want to bench Geekbench, you are on your own. Thank the Geekbench guy for that!

All of the above was named many times in this thread and in the Discord channels. And every single point is already fixed and will be released with the upcoming version.

I really don't know how to be more understanding, open and thankful for feedback, reported bugs and critique. I answer fast and always, nearly around the clock. If there is a problem, just let me know! I'm sure it can be fixed.

The only thing I currently need is your patience. We are working very late hours on BenchMate 1.0 and the new online validation platform to make it happen asap. We gunned for July but it seems like August will be more realistic.

In the meantime just bench without BM if you have a problem. The way I see it, HWBOT will always stay open for manual submission as long as the results can still be moderated. Especially old hardware will always need a fallback.

As for the improved speed with BenchMate: I looked into every single benchmark, saw the time meaurement mistakes and fixed them with an appropriate, more reliable method. That's what you get for doing stuff the right way. I'm not even sorry. :)

 

Edited by _mat_
  • Like 1

Share this post


Link to post
Share on other sites
2 hours ago, _mat_ said:

As for the improved speed with BenchMate: I looked into every single benchmark, saw the time meaurement mistakes and fixed them with an appropriate, more reliable method. That's what you get for doing stuff the right way. I'm not even sorry. :)

 

what the actual fuck.

  • Like 1

Share this post


Link to post
Share on other sites
3 hours ago, _mat_ said:

 

There are currently a few limitations mainly because BenchMate currently works best with modern platforms. These are:

  • Old or cheap CPUs won't work because they don't have the AES-NI instruction set. The benchmark will just close.
  • Don't use a 12 year old NVIDIA card because the driver wants to inject stuff into the benchmark, which is a big no-no and BenchMate is right to disallow this.

 

 

matt does this mean that it will affect the older hardwares used for eg the  hardware selected for this years teams cup ?

Share this post


Link to post
Share on other sites

I don't know what stages and hardware will be used with BenchMate. Can you list them?

Share this post


Link to post
Share on other sites
10 hours ago, _mat_ said:

I don't know what stages and hardware will be used with BenchMate. Can you list them?

it would be probably easier and quicker if you looked at the legacy stages in ddr, ddr2, ddr3 matt 🙂

Share this post


Link to post
Share on other sites

I only see a single stage for BM and that's wPrime 1024M with AM4. All good there.

Share this post


Link to post
Share on other sites
56 minutes ago, _mat_ said:

I only see a single stage for BM and that's wPrime 1024M with AM4. All good there.

ok matt, thankyou mate 👍

Share this post


Link to post
Share on other sites
On 7/21/2020 at 8:17 AM, _mat_ said:

As for the improved speed with BenchMate: I looked into every single benchmark, saw the time meaurement mistakes and fixed them with an appropriate, more reliable method. That's what you get for doing stuff the right way. I'm not even sorry. :)

There's the problem right there. We're not scientists, no one is _actually_ interested in how quickly the scene is rendered in Cinebench 15. We are interested in scores that are comparable to each other in a competitive ranking.

Your efforts are like someone analysing all 3-point shots in the NBA since its introduction in 1979 and concluding that an outside shot is actually 55% harder than an inside shot instead of 50% harder and should therefore be awarded 3.1 instead of 3 points. Just imagine players in 2020 could get 0.1 more points as long as they wear a tiny bracelet around their wrist when they shoot. Some would forget. Some would refuse. But anyone making as many shots as Kobe during his famous 81-point game would end up beating his record. I don't think the fans or players would want that!

Change the rules of the game or make a new game but don't change how an existing game is scored, it messes too much with the integrity of the rankings.

  • Thanks 1

Share this post


Link to post
Share on other sites

It's far more complicated than your analogy assumes. Using CINEBENCH on Windows 7 has a completely different timer implementation than running it on Windows 10. By changing OS settings you can additionally change the hardware being used for the meaurement. Actually you don't even need to change anything, Windows might select different time keeping hardware on its own depending on the available platform. And that's alright because benchmarks use WIN32 API functions to measure time and these functions exist to separate application code from platform-dependent code.

All of the above truly leads to comparing apples and oranges in line with your analogy. The worst part is that we have no data showing time measurement information to understand what is happening behind the scenes.

Now with BenchMate in control of the benchmark code we can do two things to fix this:

1) The same time measurement code, logic and hardware is used for all benchmarks.

2) Add timer statistics to benchmark runs for better analysis

BenchMate obviously does much more to ensure reliable results. Like using multiple timers to at least triple check the measured time period.

Of course it is true that there might be differences between BM runs and unprotected runs. The same goes for runs between different systems and OS, so there wasn't any integrity in the first place. But it's never too late to do things the right way.

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.


×
×
  • Create New...