GPUPI - SuperPI on the GPU

[Leeghoofd]

Matt will be back end of next week of a biss trip, meanwhile I want to make a select group of people of e.g. teamcaptains (as we have noticed making a poll never worked in the future) to talk about points and benchmark your users want. Only topic which is not debatable is that 3.3 scores will be mixed with prevvious scores. We also have two cinebenches, why not two GPUPIs?

[websmile]

First of all there is no doubt 3.3 is a new benchmark because the speed of calculation changed drastically with the new coding. This means also that 3.3 normally can get no points for a while because it needs a beta test phase if submission is allowed to find vulnerabilties and problems. Maybe hwbot finally learns from big problems with x265 2.0, realbench, several xtu versions and so on, but I have doubts. It is simply  madness to completely rewrite the code and make it submittable as same benchmark. I really estimate the work of Mat and idea of GPUPI 4 for exampple as new benchmark standalone appeals to me because you can never have enough good and safe benchmarks, but this needs to be tested longer before you use it for comps or points

[GtiJason]

Wonder why this hasn't been done with XTU then, per core (w/ HT) has artificially gone from 330 points 

4770k 330+/-

To 

8700K 470+/-

Is it fair to say there's been a real world 42% increase per core (at same clock, 5ghz) from Haswell to Coffee ?

Just Sayin'

[websmile]

Yes, this was watched. If you do not know the reason for this, maybe simply ask and someone explains you that this is caused by new instructions implemented from skylake on  and then also on Coffeelake, knowledge  is always better than conspiiracy theory

[bolc]

13 hours ago, Leeghoofd said:

I already talked to Matt today and this version will probably be called GPUPi4 and we will have to decide on other benchmarks for it. 

 

Let it be clear that I will not let this benchmark version interfere with scores of previous versions, not on my watch. Whatever happened in the past does not interest me at al.

After some testing and debating I decided that these boosted algorythm scores will not be included in any current GPUPI ranking. Hate me for it, I don't care. The created stir just proves again that people just prefer easy points and not a challenge.

 

@bigblock990You can bench whatever version you like, but till now only 3.2 or below will give you globals and hardware points. 3.3 will only give you pleasure and nothing in return at the Bot, untill a new benchmark is introduced.

 

 

 

+1

glad to see the idea I had was not that bad and will be enforced, hehe

 

Edited April 8, 2018 by bolc

[yosarianilives]

3 hours ago, websmile said:

Yes, this was watched. If you do not know the reason for this, maybe simply ask and someone explains you that this is caused by new instructions implemented from skylake on  and then also on Coffeelake, knowledge  is always better than conspiiracy theory

I think this makes the big difference. When I first heard about the speed up in his initial announcement it sounded like it was because he implented avx and some other things. Which would be the same as new xtu version. However this is just fancy coding and not implementing a new instruction so it can't be the same bench.  Although it could also just be that matty doesn't have piles of money to throw at the bot...

[websmile]

The main difference is that with old or new xtu a 2600k scores the samebecause these instructions are not in the cpu, at haswell should be same, marginal difference of course can happen.

As said I appreciate the work Matthias does and also the dedication to perfection, but we have 33k subs for gpupi that are worth zero in every aspect, not only points but also comparability over cpu or vga generations, if 3.3 is introduced and subs are integratted in the same ranking like 3.2 and older. If we (hwbot, which I am no part of staff anymore but still look around sometimes) do a introduce 3.3 as seperate ranking we will have eight benchmarks that measure more or less the same despite the improvement that comes with 3.3. Freezing points for 3.2 and older and introducing 3.3 is also no option as I explained, you would have to abolish globals and wr points immediately plus you cannot keep frozen points alive if you see that newcomers or guys that start to bench again have no option to get points for 3.2 as sub is closed. This would give a big advantage to guys who benched this is the past and kill fair play at ranking. So what is left to do?

P..S. Moneywise i think this was never planned to benefit either hwbot nor Mat, it was a tribute to a true legend and an interesting project

[GtiJason]

4 hours ago, websmile said:

As said I appreciate the work Matthias does and also the dedication to perfection, but we have 33k subs for gpupi that are worth zero in every aspect, not only points but also comparability over cpu or vga generations, if 3.3 is introduced and subs are integratted in the same ranking like 3.2 and older. If we (hwbot, which I am no part of staff anymore but still look around sometimes) do a introduce 3.3 as seperate ranking we will have eight benchmarks that measure more or less the same despite the improvement that comes with 3.3. Freezing points for 3.2 and older and introducing 3.3 is also no option as I explained, you would have to abolish globals and wr points immediately plus you cannot keep frozen points alive if you see that newcomers or guys that start to bench again have no option to get points for 3.2 as sub is closed. This would give a big advantage to guys who benched this is the past and kill fair play at ranking. So what is left to do?

 

I'm just playing devils advocate, points wise for me keeping original gpupi is obviously best since I can't bring my dead gems back to life or buy back the ones I've sold.

It's just guys like Matthias should be rewarded for their hard work and since GPUPI is a tribute to the Hardware Terminator and true legend Turrican it should be treated the way Mat believes he'd want.

[Mysticial]

I just want to highlight this statement by Mat in his justification of reworking the implementation:

Quote

I always want to release the best version of GPUPI that my current abilities allow me to. It's kind of my personal way of overclocking (with code).

As a hobbyist software developer myself, I can completely relate to that. When you put that much time and effort into making something, there is a strong desire to make it the best it can be. And it becomes a personal challenge to make it better and better. And quite often, "better" means faster and more efficient.

However, from the perspective of competitive benchmarking, you want everything to stay the same. Once the benchmark is released, it never changes. No speedups, no utilization of new processor features, it needs to be frozen in time - forever.

In other words, there is a conflict of interest between competitive benchmarking and benchmark writers doing personal projects. So these sorts of breaking changes are bound to happen eventually. It's a normal part of software development. The only thing you cannot change is change itself.

So I think it would probably be more fruitful to start coming of up with better ways to cope with speed changes in benchmarks.

• Do you periodically introduce new benchmarks? If so then maybe consider phasing out old ones - even if they are still popular.
• How about making the speed changes part of the game? If the benchmarker wants to stay on top he/she must stay up-to-date with the latest versions of the program and the improvements that they bring.

 

For what it's worth, the y-cruncher benchmark which I maintain has never kept speed consistency. It gets faster with almost every single release. But nobody on HWBOT notices since there aren't any points for it. Furthermore, the improvements are much more incremental and are spread out over many releases so you don't see the massive 50% speedups that we're seeing with GPUPI 3.3.

In reality, y-cruncher is a scientific application first and a benchmark second. One* of the goals of the program is to compute Pi as fast as possible by any means necessary - on any hardware and with any software changes. This is why it can utilize stuff like AVX/AVX512, unlimited memory, etc... But fundamentally, this is incompatible with competitive benchmarking as it is today so I've never really been bothered that it never "caught on".

*The "real" goal is of course to set size records. But that's outside the scope of HWBOT since the hardware needed to do this is typically on the order of 5-6 figures USD and requires months of computation.

 

 

Edited April 10, 2018 by Mysticial

[websmile]

The problem is comparability of results long term in a database that hwbot once was designed to be.

Do you know what would be the best solution for this GPUPI problem without phasing out old results or having two benchmarks that compute the same problem? If Mat would add new features to 3.3 like he plans anyway so that users have a new benchmark and he can go on on his path to perfection. Iirc it was said that plan was to add maybe avx anyway = do this and you do not have to put 33000 submissions to garbage people invested time in and power plus you have a new, better coded benchmark that also covers new instructions and fulfills the need of people (not only hwbot, but also testers, reviewers and so on) for a new perfect measuring program

 

[1whiteshark1]

I believe i have found a Bug. Im running GPUPI 3.3 Legacy on 2 Gtx 260s with CUDA 6.0.5. On the second card the Memory is not released after a run leading to out of Memory errors after a few runs. Closing GPUPI makes it release it.

[Mr.Scott]

You should be rebooting in between runs anyway.

[1whiteshark1]

Im just binning for now

[bolc]

7 hours ago, Mr.Scott said:

You should be rebooting in between runs anyway.

rebooting what for...? :o :D

[Splave]

How is does this work? Lol seems fair /10 2.3.4 subs blocked... 

[yosarianilives]

7 hours ago, Splave said:

How is does this work? Lol seems fair /10 2.3.4 subs blocked... 

Did you disable hwinfo on the second? 

[Splave]

1 hour ago, yosarianilives said:

Did you disable hwinfo on the second? 

Do u think it will make it 3.5s faster? 

[havli]

Yeah, 3.2 is slower on Fermi... in fact it is even worse for me. For example I can't get anywhere near this score anymore http://hwbot.org/submission/3301090_havli_gpupi___1b_geforce_gtx_465_1min_5sec_319ms/

[_mat_]

The problem is that the GPUPI 3.2 Legacy version is compiled with CUDA 6.5 to be able to be compatible with as many NVIDIA cards as possible (all that support double precision calculations in fact). But CUDA 6.5 will never be as fast as GPUPI 2.3.x with CUDA 8 support. Same goes for GPUPI 3.x with CUDA 9 btw.

Well, it seems like this will be a huge problem with continued support for new graphics card architectures and especially new CUDA versions. But I may have already found a way to tackle this with runtime compilation or by running PTX assembly directly. I will have a look at this in the next few weeks, it will need a lot of testing.

As for submission with GPUPI 2.x please read my answer here:

 

[Strunkenbold]

Is there anything need to be done on hwbot side that gpupi 3.3 results just get uploaded for the new gpupi 3.3 category, or is this working now?

Do you plan to continue the support for the older gpupi or will it stay unmaintained?

Just asking because hwbot don't need another unmaintained benchmark. And I'm so happy that it's in this good shape for some time now. 

Would it be possible to create one unified app, one with the old calculation style and one with your new one?

[_mat_]

GPUPI 3.3 will be discontinued, so no, there won't be a direct upload to the new benchmark category.

BUT there will be good news soon. I am currently working on GPUPI 4 which will introduce a whole new foundation for timing and handling of benchmark results. I won't get into it right now, but I can guarantee some surprises.

I also have the intention to maintain GPUPI 3.2 by releasing a minor version until GPUPI 4 gets out of the beta phase.

[unityofsaints]

On 7/10/2018 at 11:15 AM, _mat_ said:

GPUPI 3.3 will be discontinued, so no, there won't be a direct upload to the new benchmark category.

BUT there will be good news soon. I am currently working on GPUPI 4 which will introduce a whole new foundation for timing and handling of benchmark results. I won't get into it right now, but I can guarantee some surprises.

I also have the intention to maintain GPUPI 3.2 by releasing a minor version until GPUPI 4 gets out of the beta phase.

Can the current GPUpi 3.3 category be rolled into version 4? There are hundreds of submissions in this category already, including competion ones. It would be a shame to lose those.

[_mat_]

The performance of the new versions in the GPU categories will be the same. The CPU categories are a different chapter as there will be a native AVX path that should be faster than OpenCL on modern CPUs.

So yes, my recommendation will be to rename the 3.3 categories to GPUPI 4. Just Like Geekbench.

[cbjaust]

Trying to run GPIPI 3.2 Legacy on s478 (P4 3.2GHz Prescott) and it crashes out with the faulting module hwinfo32.dll. The current beta of hwinfo 587_3495 runs without issue.

[_mat_]

I have reuploaded GPUPI 3.2 with the latest HWiNFO libraries. It is still called 3.2, just be sure that the HWiNFO*.dll is dated 13th of August 2018.

GPUPI 3.3.2 was updated as well.

Download: https://www.overclockers.at/news/gpupi-international-support-thread