Math turns benchmark: y-cruncher meets HWBOT

[Mysticial]

I'm not sure how many of you are familiar with the y-cruncher Multi-Threaded Pi Program.

 

 

It's been around for quite a while now (since 2009). In short, it's a program that computes Pi and other constants to billions/trillions of digits. It currently holds the world record for the most digits of Pi ever computed (13.3 trillion digits) as well as a bunch of other less popular constants.

 

y-cruncher is also the first Pi computing program that can:

• Use multiple threads for a worthwhile (sometimes linear) speedup.
• Use (and stress) an unlimited amount of memory.
• Utilize ISA extensions (SSE, AVX, etc...) for nearly all modern processors.

There has been some hope that it could be a SuperPi/PiFast alternative. But that never really happened.

 

Over the years, I've been asked numerous times why this program never became part of HWBOT. In fact, I've had many chats with Massman. But none of them really got anywhere until recently. Also, the fact that the program lacks a GUI didn't really help either. And to be fair, y-cruncher was designed as a math program with one purpose in mind - to break size records. Competitive benchmarking and user-friendliness was always secondary.

 

Official XtremeSystems thread here. (Though it's been a while since I've updated it.)

 

-----

 

Anyways. It took about a week of work, but I've thrown together an app complete with a GUI that can run and submit y-cruncher benchmarks to HWBOT.

 

This app is called the "y-cruncher HWBOT Submitter". It's written in Java and requires the Java 8 runtime to run. (y-cruncher itself has no requirements other than Windows Vista or later.)

 

 

For now, I've only enabled 3 benchmarks for HWBOT:

• Pi - 25 million digits
• Pi - 1 billion digits
• Pi - 10 billion digits

 

The 25m benchmark will go under a few seconds for modern hardware. That's too fast, so we'll probably drop that at some point. The only reason it exists in the first place is because it's fast and easy to test.

 

The 1b benchmark is the standard size. It requires about 5 GB of ram to run and will take a few minutes to run for most high-end systems.

 

The 10b benchmark will require 48 GB of ram. That basically implies a minimum of Skylake, Haswell-E, or some server. If you don't have enough memory, it's possible to run it using swap mode. But that's more complicated to setup and will be slower than doing it all in memory.

 

 

How does it work?

 

Anyone who's familiar with y-cruncher will know that it outputs a validation file at the end of every computation. The submitter app is a runnable .jar file that you can put in the y-cruncher folder.

 

When you run it, it automatically searches out all the validation files and verifies the checksums in them. The ones that are valid and match a supported HWBOT size are available for submission to HWBOT.

 

The submitter app is a wrapper on top of y-cruncher itself. No changes to y-cruncher were needed for this to happen. And quite frankly, I designed it this way so that I could keep all the Java networking/GUI separate from the 300,000 lines of ugly C++ that is y-cruncher.

 

 

Download:

 

Current y-cruncher version: 0.7.7.9495

Current HWBOT submitter version: 1.0.1.133

 

• y-cruncher v0.7.7 with HWBOT Submitter

 

Version Support:

 

The submitter app:

• Supports all validation files generated by y-cruncher v0.6.1 - v0.7.7. So you can retroactively submit old benchmarks if you still have the validation files for them.
• Supports benchmark integration with y-cruncher v0.6.6 - v0.7.7.

Despite being written in Java, the submitter app does not run in Linux (at least I couldn't get it to run). But at the very least, validation files generated in Linux can still be submitted to HWBOT if you transfer it to Windows and run the submitter app there.

 

I have yet to figure out why it's broken in Linux, but it seems to involve the JavaFX library. In any case, even if someone does manage get it to run, the benchmark integration will still be broken since it uses Windows-specific command-line parameters to launch y-cruncher.

 

 

Version History:

 

Main Page: http://www.numberworld.org/y-cruncher/version_history_ui.html

Edited January 24, 2019 by Mysticial

[Massman]

Thanks for this - I really like Y-cruncher, especially for testing big memory configs. It seems that 25m might be a bit too easy for the current systems as we're already below 2 seconds. Maybe swap this one out for a more heavy test than 1b?

 

I wouldn't mind seeing a test which requires 32GB memory or more ... just to see how the high density guys clock their gear!

[Massman]

Just tried the submitter - I'm very impressed with the way you handled the flow from bench to submit! :celebration:

 

One issue though, I can't seem to submit my Pi-25m result. I'm using the "y-cruncher v0.6.9 with HWBOT Submitter"

 

Edited March 27, 2016 by Massman

[newlife]

That's some awesome news

[Mysticial]

Just tried the submitter - I'm very impressed with the way you handled the flow from bench to submit! :celebration:

 

One issue though, I can't seem to submit my Pi-25m result. I'm using the "y-cruncher v0.6.9 with HWBOT Submitter"

 

Ugh... I tested this thing on at least 4 different computers before I felt confident enough it would work.

 

I'm unable to reproduce it locally, but HWBOT.org is intermittently timing out. (not sure if that's in any way relevant)

 

How consistently are you getting that error? Is 100%, what about on other machines? As of right now, there's no way for me to debug it unless I have it output both the datafile and the server-sent error message.

 

(Btw, I'm willing to share the source code of the submitter app with HWBOT if that makes anything easier.)

[GENiEBEN]

Java.

[skulstation]

same here.

evri time i try to submid i have this error.

[sdougal]

http://forum.hwbot.org/showthread.php?p=438309#post438309

[Mysticial]

same here.

evri time i try to submid i have this error.

 

Thanks. Since this is happening with more than one person and HWBOT.org is stable atm, then it's a problem with the app itself.

 

I'll push out an update (hopefully tonight) that will log the datafile and error message that HWBOT sends back.

[skulstation]

Thanks. Since this is happening with more than one person and HWBOT.org is stable atm, then it's a problem with the app itself.

 

I'll push out an update (hopefully tonight) that will log the datafile and error message that HWBOT sends back.

 

ok thx.have the same error on my wifes Laptop whit win 8

[flanker]

good work guys!

[flanker]

awesome, this benchmarkm is also very stressfull. But I like it

[Mysticial]

One issue though, I can't seem to submit my Pi-25m result. I'm using the "y-cruncher v0.6.9 with HWBOT Submitter"

 

ok thx.have the same error on my wifes Laptop whit win 8

 

I've just uploaded a new version (v0.9.1.61) that will output the datafile and the full error message when a submission fails.

 

If it fails again, please send me the following files: (zip them up and attach here)

• error-response.txt
  
• error-datafile.hwbot
  
• Validation - Pi - 25,000,000.txt (or whatever you ran)
  

 

Hopefully that will be enough for me to figure out what the hell went wrong. Unfortunately, that's the best I can do since I've been unable to repro the error on any of my own machines.

 

(Btw, the datafile is encrypted. But it's being sent unencrypted to HWBOT since I can't get that to work.)

[skulstation]

stil not working on wyfes loptop.

when i am back home after work i gone try it on the other config and send all the files.

what is your e-mail to send it.pm don't like one hwbot file

[JeRiKo1]

Here are some error files

Edited March 29, 2016 by JeRiKo1

[Mysticial]

stil not working on wyfes loptop.

when i am back home after work i gone try it on the other config and send all the files.

what is your e-mail to send it.pm don't like one hwbot file

 

Right, I haven't actually fixed anything yet with v0.9.1.61. All I did was add the logging.

 

Here are some error files

 

Thank you! That did it:

org.xml.sax.SAXParseException; lineNumber: 8; columnNumber: 27; cvc-datatype-valid.1.2.1:

 

'9,518' is not a valid value for 'decimal'.

A localization bug. Decimals were being converted with commas instead of a decimal point. And because of that the HWBOT server couldn't parse them.

 

I just pushed out a fix for that. See if v0.9.1.62 works for you guys.

 

I can't guarantee that's the only localization bug. But at least we're one step closer. And that would explain why it worked on all my machines.

Edited March 29, 2016 by Mysticial

[skulstation]

Right, I haven't actually fixed anything yet with v0.9.1.61. All I did was add the logging.

 

 

 

Thank you! That did it:

 

A localization bug. Decimals were being converted with commas instead of a decimal point. And because of that the HWBOT server couldn't parse them.

 

I just pushed out a fix for that. See if v0.9.1.62 works for you guys.

 

I can't guarantee that's the only localization bug. But at least we're one step closer. And that would explain why it worked on all my machines.

 

it works on the tyan test setup whil also test it on wifes Laptop

 

edit : it also works on the wifes laptop

http://hwbot.org/submission/3173421_skulstation_y_cruncher_pi_25m_2x_xeon_5148_15sec_993ms?recalculate=true

Edited March 29, 2016 by skulstation

[Massman]

Works here too: http://hwbot.org/submission/3173424_massman_y_cruncher_pi_25m_core_i7_4500u_5sec_787ms?recalculate=true

 

Will try to fire up a more capable system later today

[ZFeSS]

Good work!

Mysticial, can I ask you some question?

You used Java only for GUI? So is it possible to run benchmark, save validation file and submit through java app on other machine? I think most benchers don't like 2D benchmarks that require additional software to run.

As far as I can understand benchmark run linux, problem is in java submitter?

[Mysticial]

Good work!

Mysticial, can I ask you some question?

You used Java only for GUI? So is it possible to run benchmark, save validation file and submit through java app on other machine? I think most benchers don't like 2D benchmarks that require additional software to run.

As far as I can understand benchmark run linux, problem is in java submitter?

 

Correct. Only the submitter app is in Java. The main app has always been in C and C++. I chose Java for the submitter app because GUI and networking is a pain-in-the-ass in C++. (I also expected it to work out-of-the-box in Linux as well.)

 

Yes, the validation file is the only thing that matters. So you can run the benchmark one machine, and transfer it to another machine to submit it.

 

For that matter, any Pi computation to one of the supported sizes can be submitted to HWBOT. It doesn't have to be one of the benchmark options. You can use the custom compute menu to fine-tune the parameters. (Hint: The default parameters are often sub-optimal. So you'll probably need to do this anyway to get the best possible time.)

 

In the future, I'm probably going to add one limitation in that the computation has to be contiguous. (So that you can't start a computation on a fast computer, pause it, and finish it on a slower one to make the slow one appear faster than it really is.)

[Mysticial]

Update: I finally got the datafile encryption to work. So that has been enabled along with a new version of the submitter. (v0.9.2.64) This also means that older versions of the submitter will no longer work. So you'll need to download the new version.

 

With this, I also switched the benchmarks to public beta.

[Guest nexus35]

Hello

I nevertheless uses the latest version.

I do not understand?

Edited April 2, 2016 by nexus35

[Mysticial]

Hello

I nevertheless uses the latest version.

I do not understand?

 

That's strange... It works for me.

 

Since the version restrictions clearly aren't working consistently, I went ahead and disabled them. I added those yesterday because I wanted to blacklist v0.7.1 until it is released. (There are a few of copies floating around out there. And they are faster than v0.6.9, so it wouldn't be a fair to those with only v0.6.9.)

 

I guess that probably explains why there haven't been any submissions lately... Oops...

[Massman]

Works here - http://hwbot.org/submission/3177103_massman_y_cruncher_pi_25m_core_i7_4500u_5sec_536ms

 

I'll set up a small competition to test the benchmark

[Guest nexus35]

It's good now