The official HWBOT Team Cup 2012 - SC2:

[I.M.O.G.]

Being familiar with the benches and the accepted practices/rules at hwbot has always been part of the competition to me... Unfortunately rule documentation has always been a weak part of HWBot. There is a lot that isn't made unmistakeably clear.

 

Our team has been applying the UCBench tweaks for months (year?), and the submissions have never been contested. If you look around at submissions, many of them are using thread/instruction set tweaks that are just now coming under debate here.

 

In the rules, the "use default options" to me meant under advanced for "bench mode" you leave default selected - not time based, not password based.

 

I think the rules should just be clarified to reflect what has been enforced historically. I don't see any inconsistency in enforcement.

[nedernakker]

so what you say is that i'm a bad sport?

 

working the bench for this competition i allready found this out. I chose not to submit scores using these settings due to the strange outcome.

 

the discussion during CC 2011 was about wether to allow the adjustment of which test to run, not how it is run. this second thing is pure manipulation of the test. it's not that hard.

 

imho you are looking at things in a wrong perspective.

 

lets just let the mods decide..

[mr.paco]

Default Settings; interesting.

 

All the bench marks here at Bot are to be run at default settings according to the rules BUT you are allowed to run em and are legit if in:

3D Mark 01 selecting a few tests and change the order they are ran etc

3D Mark 03 UN check CPU Test

3D Mark 05 UN check CPU Test

3D mark Vantage UN check FEATURE TESTs

PC Mark 04 Too many to name

PC Mark 05 WAY TOO many to name

Wprime 32/1024 change thread count

and so on.

All NON default settings. These are just a few examples and UC Bench is no different, No special 3rd party application is being used. I have been running UC Bench since it was first introduced when there were no points given for it and in that time I have learned a few things about the bench,

yes high cpu speed is essential BUT UC Bench is NOT 110% solely dependent on just CPU clock speed alone. So a 500mhz cpu difference is nothing when it comes to the end results. Its not what you run but how you run it.

There are those that have ran UC Bench with cpus that have up wards of 16,32 even 64 threads at so called default settings and found that they had to change the line command. So whats the difference.

I am not trying to defend uc bench or even my runs, but All of my runs on this or any other benchmark are done legit and if I feel its questionable believe me I do contact a mod before submitting it and get an OK or NO GOOD.

With these runs mentioned (thank you btw, feels good to see that my efforts are noticed ) there is no bug, no illegal tweak or anything, its part of the bench mark. I dont see any one debating the non default settings in any of the other bench marks.

But I guess sooner or later some one had to say $#!?... Why I cant score like that

[I.M.O.G.]

so what you say is that i'm a bad sport?

 

working the bench for this competition i allready found this out. I chose not to submit scores using these settings due to the strange outcome.

 

the discussion during CC 2011 was about wether to allow the adjustment of which test to run, not how it is run. this second thing is pure manipulation of the test. it's not that hard.

 

imho you are looking at things in a wrong perspective.

 

lets just let the mods decide..

 

I don't know you and I didn't read your posts specifically. If you took my statements personally, please don't. It's just the way I feel based on my experience benching at hwbot.

[nedernakker]

I just don't understand why anyone thinks this has to do with jealousy or envy.. I don't give a shit about who has the highest scores.

 

 

i just want clear rules. people have been comparing and discussing the wrong things today. Like Mr. Paco is doing in his post.

 

even if i uncheck the cpu test in 3dmark tests it does not affect the GRAPHIC settings of the benchmark. which are the one of influence of the score.

 

if you allow the affinity 'tweak' we should also allow 3dmark 11 entry submissions as performance ones. Cause in fact that is what happens when fiddling around with the ucbench settings.

 

but lets just stop this discussion, it is leading to nowhere. just want to know if it is allowed so i can 'tweak' my runs in the same way or the tweaked submissions need to be removed...

 

@I.M.O.G.

 

that was not in response to you.. you ninja posted while I was responding to Rasparthe

Edited July 3, 2012 by nedernakker

[mr.paco]

I just don't understand why anyone thinks this has to do with jealousy or envy.. I don't give a shit about who has the highest scores.

i just want clear rules. people have been comparing and discussing the wrong things today. Like Mr. Paco is doing in his post.

Because based on your comment "how come I scored higher than you if I was clocked 500mhz lower than you". Thats why

 

even if i uncheck the cpu test in 3dmark tests it does not affect the GRAPHIC settings of the benchmark. which are the one of influence of the score.

The point is, its NOT truly default if you check or un-check ANYTHING except for RUN BENCH

 

if you allow the affinity 'tweak' we should also allow 3dmark 11 entry submissions as performance ones. Cause in fact that is what happens when fiddling around with the ucbench settings.

I'm not much of 3d bencher dont know where that is going. but as far as the affinity tweak as you call it. Its not a tweak if all I did was to allow the bench to run longer with more thread count as in wprime

 

but lets just stop this discussion, it is leading to nowhere. just want to know if it is allowed so i can 'tweak' my runs in the same way or the tweaked submissions need to be removed...

If that was true, your question should have simply been: "Is it allowed?..." Period.

As all the submissions till now have been allowed. I am no mod so I do not speak for them I can only say what has been told to me by them: It is allowed

[knopflerbruce]

Default settings usually means that whatever settings that changes the amount of data computed should be left unchanged. Removing unneeded tests has nothing to do with default settings for the part of the benchmark that produces the score - which is why it's perfectly fine to skip CPU tests in 03 and 05. The question here is if ucbench calculates different things when forcing 64 threads rather than the default number of threads. wprime is very easy to understand that way, there the work is split into x number of threads and all threads run simultaneously. That COULD be the case with UCBench, but it's not that clear - it just seems odd to me that you gain 20% this way, when common sense says the performance should drop.

[knopflerbruce]

Because based on your comment "how come I scored higher than you if I was clocked 500mhz lower than you". Thats why

 

 

The point is, its NOT truly default if you check or un-check ANYTHING except for RUN BENCH

 

 

I'm not much of 3d bencher dont know where that is going. but as far as the affinity tweak as you call it. Its not a tweak if all I did was to allow the bench to run longer with more thread count as in wprime

 

 

If that was true, your question should have simply been: "Is it allowed?..." Period.

As all the submissions till now have been allowed. I am no mod so I do not speak for them I can only say what has been told to me by them: It is allowed

 

It's allowed if it's not "bugged", which is what you've been told.

[Antinomy]

This is a very old case. Carl used it a year ago: http://hwbot.org/submission/2197835_turrican_ucbench_2011_athlon_xp_1800_thoroughbred_60.9_mpt_score after he saw GENiEBEN using it. If you check the rankings: http://hwbot.org/benchmark/ucbench_2011/rankings?hardwareTypeId=processor_221#start=0#interval=20 you can see that it gives up to 30% performance. Same way as wPrime sometimes.

 

What I usually don't get is why people start asking this long after the case appeared. Either everybody is slow to ask the crew before or they just know how slow the crew is so why bother?

I remember the codec rule update for PCMark being added only after 1-2 years after I've asked.

 

Pretty suspicious but such discussions mostly appear during Competitions and are rare for usual submissions.

[Turrican]

hey bro it's Karl.

[GENiEBEN]

Our team has been applying the UCBench tweaks for months (year?), and the submissions have never been contested. If you look around at submissions, many of them are using thread/instruction set tweaks that are just now coming under debate here..

 

The "Thread" tweak has been used since I've shown it after CC. The first "SSE3" only tweak was used during CC12 and I had to explain that one too. F.T.S.

 

This is a very old case. Carl used it a year ago: http://hwbot.org/submission/2197835_turrican_ucbench_2011_athlon_xp_1800_thoroughbred_60.9_mpt_score after he saw GENiEBEN using it.

 

Yep Ney and Karl noticed quite quick But Karl EDITED the score so it's not APRIL 2011 but MAY.2012.

Whatever you guys decide, legal or not (PowerToy anyone?), I still have the 3rd tweak I haven't used yet, so I for one couldn't care anyway.

 

 

PS: Ohh and no one noticed the Unigine one so far What takes you guys so long? I can take LN2 records on air with this one, not so good imho.

[Mr.Scott]

The question here is if ucbench calculates different things when forcing 64 threads rather than the default number of threads. wprime is very easy to understand that way, there the work is split into x number of threads and all threads run simultaneously. That COULD be the case with UCBench, but it's not that clear - it just seems odd to me that you gain 20% this way

Slippery slope you're approaching here. The same thing could be said for most of the PCM05 tweaks. You can't have it both ways, so choose your words and battles wisely. Honestly, there's quite a few threads where people have asked if it was 'legal' already. The answer has always been yes, including from the staff. Why all of a sudden is this such a problem?

Anybody that looks at a tweaked submission could easily see what was tweaked and just mimic it.

Edited July 3, 2012 by Mr.Scott

[knopflerbruce]

Slippery slope you're approaching here. The same thing could be said for most of the PCM05 tweaks. You can't have it both ways, so choose your words and battles wisely. Honestly, there's quite a few threads where people have asked if it was 'legal' already. The answer has always been yes, including from the staff. Why all of a sudden is this such a problem?

Anybody that looks at a tweaked submission could easily see what was tweaked and just mimic it.

 

I'm not writing the rules right now, I'm just participating here:)

 

Most of the gains from the pcmark tweaks can be explained once you know how they work, right? That's not the case here - I've never seen 32 threads per core be a very efficient way of running a benchmark. What is it with this benchmark that makes it scale that way?

[knopflerbruce]

I don't know if i have the power to decide anything, but I'm thinking we allow this at least for this competition unless anything else has been decided within 45h30min from now (midnight july 5th my time:p). Simply to not ruin the competition and give people a fair chance to rebench if they want to.

[Mr.Scott]

I'm not writing the rules right now, I'm just participating here:)

 

Most of the gains from the pcmark tweaks can be explained once you know how they work, right? That's not the case here - I've never seen 32 threads per core be a very efficient way of running a benchmark. What is it with this benchmark that makes it scale that way?

I wasn't trying to antagonize you.

Here's what I've found after tweaking UCB for quite a while now.

There is no set pattern for number of optimal threads. It varies across platforms. The 'sweet' combination for threads for say 939, is not the same combination that you would want for socket A or AM3 for example. Number of physical cores seems to make no difference either. I've been testing thread combinations for UCB before Frank even knew there was a tweak. I turned him on to that. The combinations he uses on his Intel submissions aren't even close to being good on an AMD setup, so there's a difference in the way the bench is calculated right off the bat, based on CPU manufacturer.

Edited July 4, 2012 by Mr.Scott

[nedernakker]

but the remaining fact is that running the bench on just the threads it autmatically sets and adjusting the threasd results in more computing time..

 

where a normal submission would take 20 seconds. Mr. Paco's submission of his E4300 shows 20 threads all running ~8.5 seconds a piece.. (and that's what's only visible in the screen, there are 5 more not visible)that is 12~13 times as long as normal, that could easily explain the gain in numbers even though the threads are set higher...

 

and please Mr. Scott. I don't know you and you don't know me. there is absolutely no use in talking to me like you do here:

 

Shall I get you a tissue?

 

You mean like all the PCM05 so called 'tweaks' that are legal?

There is no difference, and the staff has already said it's ok in many different threads, so stop your crying.

 

I don't disrespect you so i'm expecting the same back. you have no right to talk to me like that.

 

This is just how the system works, mods can't look at every submission theirselves, that's why the report button is there just like the forum to discuss it. there is nothing wrong with that or me asking what is allowed or not. I'm just writing down what i see is happening and looking for an explanation for those numbers.

 

English is not my native language, so i'm guessing people are misunderstanding what I really mean since again there is assumed I can't stand losing which really isn't the case.

I'm just writing down what in my opinion looks like causing the strange behaviour of the benchmark. I don't know what kind of discussions have been held about this benchmark in the past, I haven't been using UCbench for years, nor have te time to dig through the forum to find posts from the past. I noticed it in this competition and found this the place to ask. I find it kind of harsh that some people react to me in such a disrespectfull manner and totally lose the point of this discussion.

 

So now I'm just gonna wait for an awnser.. this discussion allready took up too much of my time..

[Massman]

Just tested this little trick - doesn't do much aside from boosting the score by 20pts here.

 

- default thread count: 1224.6

- all threads selected: 1231.7

- only 64 threads: 1245.5

 

Link to screen: http://hwbot.org/blog/wp-content//Capture1.png

 

It's a legit tweak for the benchmark.

[GENiEBEN]

Just tested this little trick - doesn't do much aside from boosting the score by 20pts here.

 

- default thread count: 1224.6

- all threads selected: 1231.7

- only 64 threads: 1245.5

 

Link to screen: http://hwbot.org/blog/wp-content//Capture1.png

 

It's a legit tweak for the benchmark.

 

You're doing it wrong, but yeah legit.

 

The tweak consists in finding the threads that boost the score, not just selecting one like you did. I.e. I found best avg score is given by running in steps of 8 (1-2-3-4-8-16-24 etc), but like Mr.Scott said, it depends a lot on each cpu/platform.

[Massman]

Oh, pretty interesting. Did a run with all threads selected (link) and then one with 1/2/4/8/16/32/64 (link). Upto 1270 now. I disabled HT for the second run, though!

 

Cool stuff, lots of fiddling needed to get the highest performance

[knopflerbruce]

Did any of you guys actually rerun the benchmark? I tried it using myq7700, and I didn't get the boost in the same spot both times... rofl.

[nedernakker]

just tried it on my 2160 and it gains 30 points at 150 MHz less cpu speed. all runs after were giving lower scores.

 

it's very inconsistent

[I.M.O.G.]

Oh, pretty interesting. Did a run with all threads selected (link) and then one with 1/2/4/8/16/32/64 (link). Upto 1270 now. I disabled HT for the second run, though!

 

Cool stuff, lots of fiddling needed to get the highest performance

 

That has been a red herring in my experience for LN2 benching. Different rules may apply on air/other cooling when you aren't stressing the limits of stability.

 

On the second run, if your clocks were maxed, you should not be stable enough to run 8-16-32-64 threads sequentially... If your clocks were maxed you could complete 1 out of those 4 tests if selected alone, but one after another would cause instability... Similar to the ability to complete wprime32 vs wprime1024. The shorter tests run at higher clocks - independently 8, 16, 32, or 64 may complete when ran alone, but will fail when ran in succession.

 

I've never been able to get higher scores at lower clocks on ucbench by tweaking... It's best to tweak, then do your run at the optimal settings you have found with your max clocks.

 

If I'm wrong and running more thread tests increases the score more than maximizing clocks, someone should be able to beat me in this challenge: http://hwbot.org/challenge/i.m.o.g.s_ucbench_2011_global_challenge___jul_4_2012_until_aug_3_2012/

 

As is usually true with hwbot, lower in the rankings there are often many exceptions with scores versus clocks and how things scale... But at the top of the rankings where the competitors typically all have learned how to optimize well, there is more consistency with clocks vs results.

 

EDIT: There's no guarantee I'm right. But the only 2 scores beating mine in 4x CPU class currently are CPUs that can run higher clocks: http://hwbot.org/submission/2291870

 

If UCBench were worth globals, there would be more people pushing it to its limits. Unfortunately, there are few people really trying hard at the top of the UCBench rankings.

Edited July 4, 2012 by I.M.O.G.

[GENiEBEN]

Did any of you guys actually rerun the benchmark? I tried it using myq7700, and I didn't get the boost in the same spot both times... rofl.

 

Always restart PC after a run.

[Robi]

I wonder what you are all talking about. The rules for UCBench2011 say

 

Any online UCBench2011 submission must comply to the following rules:

 

- default UCBench2011 settings

- have a valid screenshot (see example below): clearly show ucbench dos box (with result), ucbench launch application, processor and memory settings in CPU-Z

 

using the advanced button to set the number of threads or editing the batch file etc. is of course not default UCBench2011 settings.

 

Some of the moderators should delete all the non default runs soon!

[knopflerbruce]

Maybe we need to define "default settings" - what we actually mean (I think) is that settings that change the amount of calculations are not allowed. Splitting the work into 64 equal amounts instead of 2 does not change anything. Removing unneeded subtests: the same. The score is not a combined score. it picks the best result out of whatever tests you select, and then it does make very little difference if you remove the ones you have no faith in.