Restrictions for Y-Cruncher 0.8.6

[Leeghoofd]

A decision has been made for the new Y-cruncher 0.8.6 and future versions:

We will only allow the 0.8.6 to be used for 5B, 10B and 25B competitive rankings.

 

The main goal is that we don't want to affect current ranking on benchmarks that have over 20K submissions. Secondly most of the 5B and higher are mainly ambient cooled submissions, with far lesser popularity thus current efforts get rewarded with a low amount of HW points for the top spots (6-15 pts maximum). This versus the 1B ranking where several CPUs get 50 HW points for top spot.

Via the numerous PMs it is clear that not all users want to rebench "once" more. This stand versus the ones chasing the globals ofc which obviously don't mind the improved performance as they can to grab 'easy' points.

We have debated it internally and took the path that HWBot wants to keep the 1B and 2.5B ranking as is and lock it at maximum Y-Cruncher 0.8.5 version. This taking into account the warning of the developers of both Y-Cruncher and BenchMate about the loss of 'optimal' support with newer hardware or Operating Systems.

This decision will have pros and ofc haters, but internal testing showed that the performance gain with the newer version would once more obliterate the current rankings with not only the latest and greatest hardware, but will also have an impact on older CPU gens. 

 

So what will happen with future releases of Y-cruncher?

Since the 5B and higher presets require a massive amount of memory, we will keep that ranking open for new unlocked performance releases, so rebenching to keep eg top spot and idem ditto points is a must with each new software release. 

Globals will be added to the 5B preset so people will be motivated to bench it. 



So this decision is a bit of both worlds, the most fair choice to make from our point of view. Secure the current and open up for a new ranking.

[Mysticial]

y-cruncher dev here: I want to go on the record and say that both mat (Benchmate dev) and myself strongly disagree with this decision. It is by far the worst of both worlds (static vs. latest) and only serves to add confusion with rankings. (This is putting it lightly as much harsher words were used in those "internal" discussions.)

The proper thing to do here is to implement a proper versioning system within HWBOT so that all versions can be submitted. Then rankings and points can be computed separately and adjusted as the site evolves. I won't try to speculate on why HWBOT cannot do this. Multi-threaded benchmarks are already divided up by core-count and nobody has any problems with that.

--------

Quote

This taking into account the warning of the developers of both Y-Cruncher and BenchMate about the loss of 'optimal' support with newer hardware or Operating Systems.

To expand on this, we mean all support. No new instruction sets for new chips is just the tip of the iceberg. If the program breaks on a new platform or OS (won't run at all), only the newest version will be fixed. Older versions will not (and often cannot) be fixed and will stay broken.

 

Edited January 24 by Mysticial

[aperacer]

Personally I liked the idea of a living benchmark, it rewarded the active benchers.

But I understand why others did not like it.

For the future I would propose that we clearly label the "living" nature of y-Cruncher 5B, 10B and 25B

In the rules and maybe also on the submission page it should be clearly stated that if a new version gets integrated in Benchmate, the performance per clock could increase significantly, thus making it necessary to rebench if you would like to keep your place in the ranking.

What is the plan for y-Cruncher 1B? Will it keep Globals?

y-Cruncher 5B is also a good compromise for the global rankings, still runable on old 1C, 3C and 5C CPU´s, as you can use 4x8GB DDR3 for max performance and with DDR4 and DDR5 platforms the necessary RAM Capacity is not an issue.

 

[Leeghoofd]

1B will keep globals, 5B will get globals, we will see what happens first

Seems there are many variants of active benchers, you can pick whatever one suits you the best...

If the software developer adds free performance with each release I don't see why its HWBot's task to mention that in the rules... However we mentioned the 0.8.5 as maximum version for 1B and 2.5B.

Neither is it the standard that we should always blindly follow the path depicted by the software developer, as in this case as points of view can be pretty different.

 

 

[SparkysAdventure]

Is there any chance there would be a userbase poll/vote on the matter? I actually really enjoyed the iterative nature of the benchmark, and the changing metas.

[Leeghoofd]

6 minutes ago, SparkysAdventure said:

Is there any chance there would be a userbase poll/vote on the matter? I actually really enjoyed the iterative nature of the benchmark, and the changing metas.

Nope

 

and I'm happy for you