PCMark05 , keep or remove from hwbot.

[knopflerbruce]

It doesn't need new rules, it needs the current rules to be enforced. I've reported numerous submissions from the same few people for cheating this bench, the users are still active and still doing it openly. No fear of "being removed instantly".

 

You need to punish the people cheating rather than the people playing fairly or this is going to be a reoccurring problem, not just pcmark.

 

The reason why we don't moderate this benchmark too well is because it's nearly impossible to tell what's a legit score and what's not. We can't ban people unless we're almost 100% sure that a cheat was used, obviously... and if we want to pull a score it has to be quite clear that it's bugged. If we can't tell, we can't block. Too many tweaks, and too much difference between a "stock" run and a tweaked one to be sure...

[Predator]

i even go beyond and suggest to remove the limit cap to 220 startup XP, why i say this, this is a limit FM set back then cause at that time it was no way a hard disk drive could score past those Mb/s , and it was a way to spot someone using ramdisk software, but it's not the case anymore, 2x SSD Sata2 in Raid0 can bypass the limit, and it's REAL performance, then why to cap it down to 220?

 

i guess it's in order to spot those who use ramdisk soft and i really don't know the benchmark too much but people here has said it should be easy to spot based in the virus scan test result (please correct me if i'm wrong)

 

would be viable to arrange some funds like in aquamark3 to make the PCMark05 "cheat" proof? , i'm saying something that just can spot if someone is using ramdisk should be enough

 

i see all other tweaks legit, as long as they reflect real performance gain like Gautam stated, i see nothing wrong to use them

 

my 2 cents

 

oh you guys should know as well, UD9 approves this benchmark

[knopflerbruce]

IF we say that all these tweaks that are classified as cheats now are allowed, then why not lift the startup as well? PCMark05 is retired from the ORB, so validation links aren't needed. XP startup is also real world performance, isn't it?

[pro]

we could certainly remove it and it certainly is real world performance (similar to virusscan performance)

 

the only reason i suggested to keep it as it adds another layer of complexity to the benchmark which is kind of neat

 

validations are still good on 3dmark.com for pcmark so we should require them still i suggest

[Guest subzero]

Transparent windows (maybe)

 

trasparent windows dont have cheats

or programm to manipulate that

send me a pm and i will explain you what happens

[Barton]

Exactly what are the alleged cheats and exactly how would they be detected and moderated if they are prohibited?

 

That's the problem.

 

We can prohibit certain things, but if there is no way to detect them and enforce the rules where does that leave us?

 

=========================================================================================

 

On a separate note, in another forum linked here, it is alleged that unnamed cheats are being used to lower Pi Fast scores. Is that benchmark also to be eliminated?

 

http://www.hwbot.org/forum/showthread.php?t=28784

 

http://www.benchtec.co.uk/forums/threads/8526-PCMark05-RIP?goto=newpost

[Guest subzero]

when you move your mouse on trasparent windows test you have a higher score this is is not a cheat.(higher aero refresh rate)

change ie 9 settings and force alternative software rendering(use cpu instead of gpu for acceleration)is not a cheat

check out all stripe sizes ,disc sizes,formmating system and allocation size+intel rst driver is not a cheat

play with ati ccc settings and drivers is not cheating

ok powertoys i dont think is good to use it

[knopflerbruce]

Exactly what are the alleged cheats and exactly how would they be detected and moderated if they are prohibited?

 

That's the problem.

 

We can prohibit certain things, but if there is no way to detect them and enforce the rules where does that leave us?

 

=========================================================================================

 

On a separate note, in another forum linked here, it is alleged that unnamed cheats are being used to lower Pi Fast scores. Is that benchmark also to be eliminated?

 

http://www.hwbot.org/forum/showthread.php?t=28784

 

http://www.benchtec.co.uk/forums/threads/8526-PCMark05-RIP?goto=newpost

 

Well, I don't know anything about these "cheats", this is the first time I've heard of it. Care to explain what this is about? I think Pifast can be cheated by manipulating the windows clock, but it's easy to spot that because the subtests won't have the same relationship as real scores.

[Gautam]

when you move your mouse on trasparent windows test you have a higher score this is is not a cheat.(higher aero refresh rate)

change ie 9 settings and force alternative software rendering(use cpu instead of gpu for acceleration)is not a cheat

check out all stripe sizes ,disc sizes,formmating system and allocation size+intel rst driver is not a cheat

play with ati ccc settings and drivers is not cheating

ok powertoys i dont think is good to use it

 

Powertoy is just a GUI for changing settings in the registry. I don't know why everyone has an issue with this either.

[SteveRo]

We built a wrapper around heaven right? Couldn't we build a wrapper around pcm05 and for example - have it report memory mapping to make sure ramdisk is not used?

[chispy]

i even go beyond and suggest to remove the limit cap to 220 startup XP, why i say this, this is a limit FM set back then cause at that time it was no way a hard disk drive could score past those Mb/s , and it was a way to spot someone using ramdisk software, but it's not the case anymore, 2x SSD Sata2 in Raid0 can bypass the limit, and it's REAL performance, then why to cap it down to 220?

 

 

^^ This , i agree 100% , Predator we tried this long time ago i think year 2008~2009 , furturemark was going to raise the 220 cap and in place it was going to be raised to 300 , but it never happenned as always we get very little support from FM , still 2011 and i still waiting from FM or hwbot to raise the XP Start Up to 300 . , hwbot can let loose if they let loose everything else allowed , why cap the benchmark , it would not be fair for some people the all goes but 220 xp because , thats where they can catch up and have advantages in the Hard drive test , then why penalized some and reward others ???

Edited July 25, 2011 by chispy

[SteveRo]

Mr Chispy, I'm not sure but looks to me like maybe FM is not enforcing the 220 limit anymore anyway - see this one -

 

http://3dmark.com/pcm05/3077474

 

probably ramdisk but a good example of over 220 any way.

[chispy]

Mr Chispy, I'm not sure but looks to me like maybe FM is not enforcing the 220 limit anymore anyway - see this one -

 

http://3dmark.com/pcm05/3077474

 

probably ramdisk but a good example of over 220 any way.

 

Hola Mr.SteveRo , thats a software Ram-Disk for sure , hmm , food for thought ,@ Massman can we raise the XP Start Up Cap to lets say 400 ? i still think 220 its too low cap for people with good hardware in the HDD department.

[Gautam]

I would vote for no cap. As you guys can see, it's not hard to spot ram drive results. We can kill those on sight.

[El Gappo]

The 220 limit makes it like a game and adds another layer of complexity as pro said. Why change something that is going to even further invalidate old scores that people have sweated over for hours? We don't need that...

 

Not to mention it makes it even more inaccessible to people that don't fancy dropping a bomb on raid and a-cards.

[chispy]

I would vote for no cap. As you guys can see, it's not hard to spot ram drive results. We can kill those on sight.

 

+100% agree with you on this one Gautam , why a cap if everything else goes , if everything else goes and the 220 cap its not lifted it will be very unfair , as you see how easy its to spot someone using a software Ram Disk. @ Massman if you are going to let everything goes , please think about the possibility of removing the 220 cap please .

[Gautam]

The 220 limit makes it like a game and adds another layer of complexity as pro said. Why change something that is going to even further invalidate old scores that people have sweated over for hours? We don't need that...

 

Not to mention it makes it even more inaccessible to people that don't fancy dropping a bomb on raid and a-cards.

I don't like it because its arbitrary and artificial in principle. FM put that there because many years ago (2006 or 2007 probably) it was practically impossible to cross 220mb/sec without the use of software. Things are very different nowadays. It was just a quick and dirty fix to eliminate ram disks, but it makes little sense today. The goal is to prevent the usage of ram disks, not to arbitrarily limit the bench. The 220 cap was just a poor and sloppy way of doing so that's stuck just because it's the way it is. Yes removing it will make current scores relatively weak, but since we're trying to iron out a new set of rules, that's pretty much inevitable.

[mr.paco]

TWEAK or CHEAT, Everyone has their own understanding of just what that is and what should be allowed and not. And as stated earlier here when someone cant figure out how someone else got such a high score in an area they automatically want an investigation done because they cant get such good scores and don't know why. So now because of soo many "I cant get that score - he/she is doing something illegal" crys, you wanna now consider removing a bench mark. There are many TWEAKS known and still more unknown, but for me my understanding is with a bench mark it is to get your system/component (CPU or GPU or RAM or HD) to perform faster and above manufacturer specification and not how it will run under NORMAL everyday configuration. So wouldn't it be a form of MANIPULATING a benchmark to perform faster the moment you start to OC your 2ghz cpu to 3ghz? Oh wait, that's an ALLOWED TWEAK. Sounds simple minded I know and I may have it wrong but my point is the DEFINITION of ILLEGAL TWEAK is what BOT Staff say it is. I understand not everyone whats to drop $hundreds on a hd setup just to get better score with XP start up or HD Usage, but you'll spend $800 on a GPU so you can get higher 3DMark 06/05 runs, or who knows how much on SS/Cascade/LN2 so you can get a high CPUZ. Not every bench is for Everyone, you work the bench you feel you're best at. EVERY benchmark can be manipulated/tweaked one way or another if its by a small degree or by a large one. If that is the case than what is the point of OCin/Benchmarking if every time some unknown tweak or system adjustment is used or made by some one its considered bad because they got a higher score. I dont know anything about this Powertoys & MIPS & LOD and a bunch of other stuff that has been mentioned but, OCin is OCin = make it go faster/higher... Period

[Gomeler]

I'd like to see all the caps removed from the benchmark. I have mixed feelings on the HDD tests being run in RAID card cache but they are at least partially limited by the PCIe bus and all the latencies involved with that. All the tweaks I've found/learned from others/heard about are software tweaks that provide greater performance to the subtest. AFAIK nobody is outright circumventing a test. They're just providing more efficient or faster ways to perform something.

 

Also, why haven't we brought PCM7 into the fold yet? It is a fun benchmark and we need more system benchmarks that don't completely depend on a cherry CPU/GPU.

[chispy]

PCM7 would be a nice alternative to PCMark05 , i agree with you Chris.

Edited July 26, 2011 by chispy

[ShrimpBrime]

Why would PCmarko5 be any different then any other Benchmark??

 

May as well remove PCMark04 while your at it. Plenty of ways to cheat that one too right?

 

Didn't vote, but it should stay. Funny how people know what the cheats are and what to look for. Only cheaters really know about/how to cheat IMO. My scores are so low with it, I hardly run it. Wish I knew more cheats, I mean tweaks....

[Mr.Scott]

Apparently my choice to remove it makes me an un-popular guy on my team now.

I can't put it any more plainly, when the moderators tell you they can't moderate it accurately, due to not knowing the so called tweaks themselves or what ever the reason, then who do you expect to do it? Or does it just become a free for all for whatever points you can steal any way you can.

It needs to go. It will be an endless headache of accusations and reports to look at. Is the staff up for that?, 'cause that's the way it will be.

[mr.paco]

Apparently my choice to remove it makes me an un-popular guy on my team now.

Not true, Everyone is entitled to have their own feelings n opinions. Youre still my hero .

But honestly; what bench mark can be 100% moderated & kept in check cleanly, honestly & correctly? None. So to remove PcMark05 because it can not be moderated you might aswel make a list of the other benchmarks that can not be moderated 100%

There will ALWAYS be a way to TWEAK ANY Benchmark. May it be with hardware or software. Like putting 2 SSD in raid "0" and using it as your page file instead of the NORMAL c-drive or using memset to tighten ram timings, or flashing your GPU bios to another for possibly better shadder performance or using the various OS for different bench runs with or with out Service Packs. Its ALL TWEAKS. The manipulation of ANY bench mark happens the moment you ADJUST ANYTHING in your system including OCing (that is a TWEAK too) but the definition of LEGAL TWEAK n ILLEGAL CHEAT is in the eye of the person that is able to and not able to do it. And to some degree willing or not willing to spend the money on high priced components needed.

I am in no way justifying cheating, but what is REALLY cheating? Photo shopping your results is definitely cheating, using IE9 or Google Chrome instead of 8 is not.

NO one here can say what the TRUE number on what your setup will do in a benchmark because of ALL the many many various settings one has on their individual rigs outside of leaving it at STOCK speed n settings. Everyone's combination of hardware is different and the results will be too. Some by a little some by a lot. But who is to say what the RIGHT result should be?

Look at it, this is a world wide community of computer hardware & software enthusiasts, and everyone has different components. some have WAY more knowledge than others some have WAY more hardware than others but are all still under the same umbrella trying to learn/figure out how to get it to run cooler, faster, higher and so-on with hardware (LN2, Volt modding, etc)and/or software (memset, msi afterburner, etc)...Thats the bottom line.

I vote keep it.

[travy25]

It's a shame that the few have ruined this for the many as is often the case. I didn't even know software cheats existed. I've always treated it like gpu benching where faster cpu's yield better scores along with optimizing your OS. With PCMark05 it's about faster HDD's and OS optimization. Try to play with all the tools the OS provides and see what yields the best scores. Before we scrap this alltogether I think we should continue to search for a viable solution.

 

On HDD limits:

 

By the reasoning I'm seeing here are we going to start limiting cpu scores in 3DMarkxx, because all of those benches are dated and everyone keeps rebenching old gpu's everytime they do a cpu upgrade? Why isn't that capped? Either let that go or be consistent with that methodology. There a lot of gpu points here that are cpu dependent. That will hurt a lot of teams, but it's the same thing. Capping a variable that very significantly affects the end result.

 

@Scotty: No hard feelings here, everyone is entitled to an opinion and you've argued your point with sound reasoning.

[chispy]

Ok lets hear what massman and richbastard have to say about PCMark05 , can you guys please stop by and update us on whats going to happen and what actions are going to be taken on this matter ?

Edited July 26, 2011 by chispy