PCMark05 , keep or remove from hwbot.

[Doc.Brown]

I posted a suggestion somewhere: Post the real result, type the real XP startup speed in a box while submitting, let the bot engine calculate the "new" score assuming 220 XP startup if the score is higher than 220mb/s.

 

Got the impression it was a good idea, but no decision was taken:p The score formula is pretty straight forward, so this should be easy to code.

 

I think it's a good idea too

 

@El Gappo:I respect your opinion but IMO tweaking is part of fun,trying to be under the 220Mb/s is a big waste of time because,when we are on Ln² or Dice,time = money.Ln² or dice cost one arm here ^^

[knopflerbruce]

I think it's a good idea for 3 reasons:

 

1: the cap is still there, so existing score are not worthless

 

2: you don't have to limit the performance of your storage setup, which is completely against what we should aim for

 

3: it can be coded:)

[Doc.Brown]

I hope the hwb staff to hear your idea ...

[Christian Ney]

He is in the staff

[Doc.Brown]

Oups ,good to know

[I.M.O.G.]

I think it's a good idea for 3 reasons:

 

1: the cap is still there, so existing score are not worthless

 

2: you don't have to limit the performance of your storage setup, which is completely against what we should aim for

 

3: it can be coded:)

 

+1!

[Mikecdm]

I'm just starting to get into this bench and i don't want to see it go. The xp startup cap is part of the challenge to get a good score and it's really not all that difficult to slow it down enough to be under 220. It may defeat the purpose of the bench, but having it there add a little bit of fine tuning. It also doesn't make older scores completely obsolete.

 

Now what christian has done, by making a thread and exposing some of the tweaks to improve scores is a great idea. If everyone is aware of most of the tweaks, then the bench in theory should once again be based on storage setup and clocks. There are those who still want to tinker and find new tweaks and there will be those who try to get away with cheating, but cheaters will always be cheaters.

[KONAKONA]

I posted a suggestion somewhere: Post the real result, type the real XP startup speed in a box while submitting, let the bot engine calculate the "new" score assuming 220 XP startup if the score is higher than 220mb/s.

 

Got the impression it was a good idea, but no decision was taken:p The score formula is pretty straight forward, so this should be easy to code.

 

This is a good idea.

 

Someone go code it for massman.

[ZMBKLN]

I myself think the XP Startup needs to be left alone.

 

It's one of the few things left that requires some skill and research to keep it in check.

[Massman]

It's a really, really dirty workaround for a very specific issue. In my opinion, if you need that kind of dirty workarounds, it's basically because the bench cannot be used anymore, at which point we should just drop it and move to a newer version.

 

I'd prefer a different solution, to be honest ...

[El Gappo]

Please don't mess with it again...

[dinos22]

why does chispy create a thread every 6 months asking if we should keep or remove PCMARK05. Should we have a poll if chispy should be making this poll?

[Massman]

Good question, let's vote about it.

[dinos22]

sorry i should have referred to chispy as the Godfather of PCMARK05 naturally

[Massman]

Could we perhaps agree on:

 

- No ramdisk, ramcache or MFT software allowed

- 220MB/s XP startup cap stays

- Benchmark must run on default web browser (Internet Explorer, any version) and default audio/video codec

- Browser enhancement tweaks such as disabling/removing features, plugins, make-up etc are allowed

- Audio and video codec tweaks are allowed. Powertoy is, as GUI for the registry, allowed

 

+ make system pictures mandatory for everyone. Top scores should show storage subsystem, but we will not block scores -just- because the storage subsystem is not in it. Blocking scores because a lack of picture verification is only necessary when the score is suspicious.

[El Gappo]

Sounds good. See if we can make it a few months before Christian ruins it this time

[knopflerbruce]

It's a really, really dirty workaround for a very specific issue. In my opinion, if you need that kind of dirty workarounds, it's basically because the bench cannot be used anymore, at which point we should just drop it and move to a newer version.

 

I'd prefer a different solution, to be honest ...

 

The same thign could be said when the 220 cap was invented - using this method is not worse in any way than having a cap present without haveing a correction formula. Plus, it's one of MANY subtests, it's not like 3Dmark05 where you have just 3...

[Christian Ney]

Why remove the cap:

- cause there is 3 HDD tests

- Now SSD are really fast so it's easy to have more than 220 mb/s

- ramdisk is easy to spot

- when under LN2 you waste time and money to be under the 220 mb/s

- rules up to date ?

 

Why keep the cap:

- cause it has been here from the beggining

- FM also have this cap

[SteveRo]

Could we perhaps agree on:

 

 

 

+ make system pictures mandatory for everyone. Top scores should show storage subsystem, but we will not block scores -just- because the storage subsystem is not in it. Blocking scores because a lack of picture verification is only necessary when the score is suspicious.

 

I think this would be a good addition to the rules - it should help.

[Massman]

The same thign could be said when the 220 cap was invented - using this method is not worse in any way than having a cap present without haveing a correction formula. Plus, it's one of MANY subtests, it's not like 3Dmark05 where you have just 3...

 

Yes, it is worse, because it requires us to maintain code that is only present to fix one subtest on one ancient benchmark. It sounds really simple, but coding-wise there's a lot of work involved. It's not just three lines of code.

 

The 220MB cap was a good solution, because it only required to write a check if startup was over 220. If yes => no good, if no => good. That's quite different from having to check the value, recalculate the score and override the result database.

 

If it would be applied for all benchmarks, it would be worth spending time on. But it's just for one 7-year old benchmark. And it's just one of the solutions that isn't perfect either. People learned to live with the cap, so why change it again.

[Doc.Brown]

Ok the cap has been here from the beggining but storage technology changed since this time,and if you just say"ok pcm05 is to old,not representative anything now so the only way is to remove it" I think you also should open a thread and poll(or simply do action to kill it) for the old,but one of Massman favorite's toy,3DM01 because old vga did equal or better score than newer vga(so finally what it is representative),like PCM05 it is very tweak sensitive and these tweaks aren't known by all OC'ers(i don't see any thread to reveal tweaks here)but oddly there's any controversy,and what think about the validation of the ultra bugged,binding Unigine(DX11 screen resolution,language,integrated audio activation etc) ...

 

To be clear i don't want to create another controversy but just show off that the subject can be enlarge to other software and at the end we're staying with 3 benchmarks for playing,is it enough or it will be better to keep rules up to date with technology and so keep a large choice of toys to play and have fun.

To stay under the cap limit I use ATTO during the start up test but I never obtain the same results one time it's good,one time not so that's why I think knopflerbruce idea's could be a good alternative.

[Massman]

I don't want to remove it because it's old. I want to keep it because it's (to some) a lot of fun.

 

I'm just trying to say that Knopflerbruce's suggestion is, for me, not a good solution because it's a lot of work and the code only applies to one specific benchmark. If we'd have to do recalculations for a lot of benchmarks, it might be worth investing time in the development. But since it's just for this one subtest of a 7-year old benchmark, the benefit from Knopflerbruce's idea does not outweigh the work needed to make it work.

 

I don't see why "keeping the cap" isn't just as viable an option as recalculating it to 220MB/s. Sure, it's a bit of extra work, but if you're prepared well enough, you won't have any issues under LN2.

[Doc.Brown]

Well,I understand the "investing time development" argument.

I developped all my arguments in this thread and really don't want to enter in long days controversy,so finally do what you think is the best for the future of Pcmark,I will doing with

[ZMBKLN]

Could we perhaps agree on:

 

 

 

+ make system pictures mandatory for everyone. Top scores should show storage subsystem, but we will not block scores -just- because the storage subsystem is not in it. Blocking scores because a lack of picture verification is only necessary when the score is suspicious.

 

Perfect!... But put in on the main rules page for PCM05.

 

Ok the cap has been here from the beggining but storage technology changed since this time,and if you just say"ok pcm05 is to old,not representative anything now so the only way is to remove it" I think you also should open a thread and poll(or simply do action to kill it) for the old,but one of Massman favorite's toy,3DM01 because old vga did equal or better score than newer vga(so finally what it is representative),like PCM05 it is very tweak sensitive and these tweaks aren't known by all OC'ers(i don't see any thread to reveal tweaks here)but oddly there's any controversy,and what think about the validation of the ultra bugged,binding Unigine(DX11 screen resolution,language,integrated audio activation etc) ...

 

To be clear i don't want to create another controversy but just show off that the subject can be enlarge to other software and at the end we're staying with 3 benchmarks for playing,is it enough or it will be better to keep rules up to date with technology and so keep a large choice of toys to play and have fun.

To stay under the cap limit I use ATTO during the start up test but I never obtain the same results one time it's good,one time not so that's why I think knopflerbruce idea's could be a good alternative.

 

Actually, there is now a tweak thread open and discussing all the little tweaks to get the best scores out of PCM05.

[knopflerbruce]

I don't want to remove it because it's old. I want to keep it because it's (to some) a lot of fun.

 

I'm just trying to say that Knopflerbruce's suggestion is, for me, not a good solution because it's a lot of work and the code only applies to one specific benchmark. If we'd have to do recalculations for a lot of benchmarks, it might be worth investing time in the development. But since it's just for this one subtest of a 7-year old benchmark, the benefit from Knopflerbruce's idea does not outweigh the work needed to make it work.

 

I don't see why "keeping the cap" isn't just as viable an option as recalculating it to 220MB/s. Sure, it's a bit of extra work, but if you're prepared well enough, you won't have any issues under LN2.

 

THe problem is that you run an SSD without much tweaks, and you can get a very nice score removed because "oh, Xp startup is a tad too fast", which itself is silly. It might have been OK in ancient times, but today it's a joke. There's no OC related skill in crippling performance.