PCMark05 , keep or remove from hwbot.

[Mr.Scott]

Sorry, but when a bench can't be accurately moderated, it's time to turn it loose. You have mods here that already said they can't tell what's real and what isn't because they don't know. It's not beating a dead horse, it's the same un-resolved issues, and it will never end. How many patches does the leaky boat get before you finally let it sink.

[Massman]

Why does no one seem to listen when I suggest that photo verification should be used? I mean, you can still give PC05 the smack down with storage speed if you are willing to spend the money on some serious raid cards and SSDs or ramdrives or whatever. Everyone in this thread seems to be butt blistered because people are supposedly using software ram drives and such, if you have to show a picture of it then it should be very obvious if your score is in the ballpark or not.

 

I like the idea of photo verification, but do you require photo verification for all submissions? Even those that are clearly not very competitive at all (let's say: 10k PCM05 with i7 990X).

 

Not everyone has a camera around when he's benching.

 

I kind of like the idea of making pictures mandatory for all submissions; not just the PCMark05. We discussed this option before and then the conclusion was that it would hinder people too much. I guess pretty much everyone has a digital camera nowadays, so it should be that difficult.

 

One thing though: we can't automatically identify a picture and judge whether it's a system picture or not. Also, people can just download pictures from the web, I guess.

 

Here are the rest of my opinions on this mess:

 

*) I'd still like someone to explain to me what the XP startup limit is doing anymore. I mean, PC05 is still very much a storage benchmark. It limits people that have 20 SSDs in raid just as much as it does someone with a software ramdrive, and again picture verification would be able to solve the whole software issue. Unless it's there to try and "even out" the bench, so to speak. I don't have enough experience with PC05 to know if it is or not.

 

The XP startup was a difficult call: "Short comment on the cap: it was 50/50 on keeping it or not. Either way would've made some people unhappy. So, as the cap has pretty much been with PCMark05 since forever, I decided to keep it. It's a part of what defines PCMark05 as benchmark." (link)

 

Originally, the cap was set by Futuremark on their ORB to keep software ramdisks out.

[xXSebaSXx]

Please don't take this as an attack on you Scott... I don't know you so I would never even think of attacking you, but to me, all that sounds like "All my scores got made irrelevant by people that found new tweaks and used them so I want their scores gone; even if it means mine are gone too"...

Can't tell what's real? Fine; picture requirement. Problem solved. If someone is going to claim that they ran four MAX IOPS in RAID0 to get those HDD scores; let them show proof. Just like MTech and SteveRo do; you can see their storage setups on every submission.

 

I know this benchmark has a bunch of holes in it, but so do a lot of others... I don't think there is a benchmark on the Bot that can't be tweaked to achieve higher scores. So where do we stop? Do we remove 01 because you can trick better scores out of it with "modded" BIOSes? Do we remove 06 because you can get better scores out of it with LOD? Do we remove Spi? They are all tweakable, they are all open to "illegal" tweaks that would be super hard to spot too.

 

Or do we go to HWBot created wrappers for every benchmark where the thing runs on "pre-determined" settings chosen by the HWBot staff and no one is allowed to run them in any other way?

 

I just don't see it happening that way man. As much as some people may not want to admit it; the Bot is a site where we compete against each other; and where there is competition there will always be people looking for a way to be better than the next guy. I say; make it clear what's legal and what isn't legal and then let us play.

 

 

 

I like the idea of photo verification, but do you require photo verification for all submissions? Even those that are clearly not very competitive at all (let's say: 10k PCM05 with i7 990X).

 

Not everyone has a camera around when he's benching.

 

I kind of like the idea of making pictures mandatory for all submissions; not just the PCMark05. We discussed this option before and then the conclusion was that it would hinder people too much. I guess pretty much everyone has a digital camera nowadays, so it should be that difficult.

 

One thing though: we can't automatically identify a picture and judge whether it's a system picture or not. Also, people can just download pictures from the web, I guess.

 

 

Picture of the HW and there should be a piece of paper with used ID and date visible on the picture. Like most people use when they are selling HW at any of the boards. If you have a camera that can take pictures so you can sell your chips, you can use that same camera to take pictures when you bench. I don't know about anyone else, but my camera is always on my desk, next to my PC when I bench.

Edited January 8, 2012 by xXSebaSXx

[Mr.Scott]

Please don't take this as an attack on you Scott... I don't know you so I would never even think of attacking you, but to me, all that sounds like "All my scores got made irrelevant by people that found new tweaks and used them so I want their scores gone; even if it means mine are gone too"...

No offense taken at all. I don't bench PCM05 enough to fit your thought though. It would be no loss for me. A couple of my team mates excel in it though, and it would most definitely hurt them and my team. I'm only giving my opinion just like everybody else. Whether anybody likes it or not.

[mr.paco]

Every BenchMark here has their own set of H/W requirements n needs to be competitive and they "ALL" have their vulnerabilities to be exploited to gain a higher score in one form or another, by a little or by a lot "UNDETECTABLE".

Some folks invest their time n money on pricey GPUs for 3D benchs others on CPUs for 2D bench and in this case Storage Systems for PCMark (A-card, revo drives, I-ram, etc as well as a good gpu). And with each case you choose how much brain power you will put in to better your score/run. Some put in a lot and others just run it n forget it.

 

You choose the arena you want to play in with what you have and what you know and you do the best you can but don't start calling FOUL if you cant score as high as the next person not knowing how to adjust/tweak a particular setting.

If YOU dont like running a particular bench for what ever reason, dont use it.

 

Sure it would be great to be ranked amongst the likes of Steve, Mtech, Pro, SAV and some others and I do have some of the HW to do it but my scores fall short, call it a lack of skill, knowledge or what ever but the point is I dont cry out ""PCM05 sucks cause "I" cant get those high scores with my 12 x I-Rams storage array; get rid of it""

 

As with anything in life, as time goes by you learn how to do/be better. When PcM05 first came out all of these tweaks were always there just not openly known, but as time went by folks tryed to better their scores and thus discovered what can be done. The very same thing can be said for the the other benchmarks.

 

I dont know; to me its simply, dont get into a motorcycle race with a peddle-bike and cry "why cant I win :mad:"...

 

Yes I'm sure there those that don't play fair with illegal actions, but to remove Pcm05 because of inability to moderate 100% accurately, then by all means line up all 28 benchmarks here and start shooting

 

PCM05 should stay; you dont like it, dont run it. Let us that do like it play

 

Just my 2cents...

 

xXSebaSXx[/url]"]Picture of the HW and there should be a piece of paper with used ID and date visible on the picture. Like most people use when they are selling HW at any of the boards. If you have a camera that can take pictures so you can sell your chips, you can use that same camera to take pictures when you bench. I don't know about anyone else, but my camera is always on my desk, next to my PC when I bench.

+1

As do I, mandatory since I'm in the Enthusiasts League

Edited January 8, 2012 by mr.paco
added quote

[Mr.Scott]

Then you might as well remove the cap and restrictions and make it a free for all, because it's already un-moderateable with any kind of accuracy when the staff openly admits they can't do it because they don't know all the tweaks. Besides the fact that you have staff skirting the rules now too. That's all I'm saying.

Edited January 8, 2012 by Mr.Scott

[KONAKONA]

I like the idea of photo verification, but do you require photo verification for all submissions? Even those that are clearly not very competitive at all (let's say: 10k PCM05 with i7 990X).

 

Not everyone has a camera around when he's benching.

 

I kind of like the idea of making pictures mandatory for all submissions; not just the PCMark05. We discussed this option before and then the conclusion was that it would hinder people too much. I guess pretty much everyone has a digital camera nowadays, so it should be that difficult.

 

One thing though: we can't automatically identify a picture and judge whether it's a system picture or not. Also, people can just download pictures from the web, I guess.

 

You could pick a certain score level in some of the tests, wherever the threshold between someone benching 05 with not much in the way of storage hardware like just a physical disk or single low end SSD and someone benching with raid or fancy SSDs. Again, I don't have enough experience with the bench to know where that is, but I don't think it'd be that hard to find.

 

If a sub gets reported, it doesn't take very long at all for a mod to do an image search on the verification picture and see if it brings something up. Google has a "reverse image search" feature now, so it'd be rather difficult to find a picture of a system running the same MB/possibly ram with a storage setup that is going to be in the ballpark of their score without it being easily findable. Honestly though, I don't think anyone would go to that much trouble to forge a score though.

 

Well, maybe a staff member...

 

The XP startup was a difficult call: "Short comment on the cap: it was 50/50 on keeping it or not. Either way would've made some people unhappy. So, as the cap has pretty much been with PCMark05 since forever, I decided to keep it. It's a part of what defines PCMark05 as benchmark." (link)

 

Originally, the cap was set by Futuremark on their ORB to keep software ramdisks out.

 

Fair enough.

 

Then you might as well remove the cap and restrictions and make it a free for all, because it's already un-moderateable with any kind of accuracy when the staff openly admits they can't do it because they don't know all the tweaks. Besides the fact that you have staff skirting the rules now too. That's all I'm saying.

 

I haven't seen how 05 has become an "unmoderateable" benchmark yet. All I've seen is a staff member go test the limits of where the rules stand, and the entire time it was very obvious that he had a score that used some form of software ramdisk, regardless of it's name or technicalities. The only issue that has arisen is pertaining to the current exact context of the rules, not difficulties pertaining to the known rules being enforced.

 

The only time scores aren't moderateable is when people don't report suspicious scores. I keep coming back to this, but I really don't see how PC05 would be hard to moderate if you just had to include a photo if your storage based scores are over a certain point. I mean, if you have the money to get a few hard drives in raid0 or a high end SSD, or even a more expensive setup, you should be able to hunt down a picture to take of your rig, right? Even if you found a picture of a rig that looked exactly like yours with a big fancy storage setup, you would still have to limit your scores to something realistic on that setup if you were using a SW ramdrive. Even now, the only way you could go about using a SW ramdrive with any hope of going undetected is to limit it to within the range of a hardware based storage setup. And you could still get reported and have your post looked over.

 

It seems simple to me.

[xXSebaSXx]

Then you might as well remove the cap and restrictions and make it a free for all, because it's already un-moderateable with any kind of accuracy when the staff openly admits they can't do it because they don't know all the tweaks. Besides the fact that you have staff skirting the rules now too. That's all I'm saying.

 

I don't see this as an issue of tweaks... It's an issue of whether one uses something that has been forbidden in the rules (SW RAM Disks or not). It's already been shown that when one puts enough time/effort/money into the storage part of the bench great results can be had. That's not a tweak; that's someone going out and buying the HW needed to compete...

 

I go back to the 3D benches... Are we going to outlaw voltmods/franken cards/TiNed cards/Shammied cards simply because not everyone "knows" how to get such things done? Are we going to throw out Vantage because people don't have access to Untochables addon cards? Or because they don't have the experience and knowledge that the top dogs have? I don't think that ignorance (read as "not knowing about a tweak") is not enough of a reason to get all up in arms about it.

 

And believe me when I tell you; if I ever find a "tweak" that gets me 100+ webpages/s in PCM05; I am NOT going to come out and blabber it to everyone. Or are we going to ask everyone that spends time looking for tweaks that give them an edge to reveal their hard work for everyone? I'm sorry, but the fact that someone doesn't know a tweak doesn't make IT illegal... It just means that those that don't know it have not spent enough time with the bench to find it.

 

This is not elementary school where every kid gets a trophy whether they win or not. We're all adults here and those that put more time and effort into it are the ones that get the rewards.

Edited January 9, 2012 by xXSebaSXx

[Mr.Scott]

I don't see this as an issue of tweaks... It's an issue of whether one uses something that has been forbidden in the rules (SW RAM Disks or not). It's already been shown that when one puts enough time/effort/money into the storage part of the bench great results can be had. That's not a tweak; that's someone going out and buying the HW needed to compete...

 

I go back to the 3D benches... Are we going to outlaw voltmods/franken cards/TiNed cards/Shammied cards simply because not everyone "knows" how to get such things done? Are we going to throw out Vantage because people don't have access to Untochables addon cards? Or because they don't have the experience and knowledge that the top dogs have? I don't think that ignorance (read as "not knowing about a tweak") is not enough of a reason to get all up in arms about it.

 

And believe me when I tell you; if I ever find a "tweak" that gets me 100+ webpages/s in PCM05; I am NOT going to come out and blabber it to everyone. Or are we going to ask everyone that spends time looking for tweaks that give them an edge to reveal their hard work for everyone? I'm sorry, but the fact that someone doesn't know a tweak doesn't make IT illegal... It just means that those that don't know it have not spent enough time with the bench to find it.

 

This is not elementary school where every kid gets a trophy whether they win or not. We're all adults here and those that put more time and effort into it are the ones that get the rewards.

I don't know why you continue to try to make me the bad guy here. I don't care if everybody sits on all their tweaks or not. My point is that if the people who moderate the submissions don't know or understand said tweaks, how can they possibly look at a submission and judge whether it's valid or not if it's reported, when they don't know what to look for. Does that mean if you're the only one who knows said tweak, that they and everybody else have to take your word for it? That's what makes this bench impossible to moderate, and why it should be ditched.

Mind you, this is my personal opinion, and I'm entitled to that, like it or not.

BTW, loopholes in the rules do not constitute a 'tweak'.

 

Whatever. Do what you want.

I don't run it enough to care anymore.

EDIT- because I don't care, does that make me more 'ignorant' than just not knowing a tweak?

Edited January 9, 2012 by Mr.Scott

[Sweet]

Guys please, we (the staff) know all teewks right now, known questioned software is old and was used to obtain good results in some important on-line competition.

But when done right combination can not tell if software or hardware, which has been used.

 

I do not mean that all scores have been done with software, but it is impossible to know whether it was used to obtain scores, since those who have done know good how to do.

 

 

Finally a member of staff has used this software to prove that with this only, no need to spend many U$S. in storage, results have been achieved bulky, as you buy an raid 0 ssd ultimate or revodrive, and Without Accard controller, only you must have a lot of ram in your system and ready, software is much cheaper.

 

With this has shown that it has hurt those who have invested a lot of U $ S in its storage system. nothing more.

 

Picturer, maybe, or maybe not, because..........cheat can be done with them, no photoshop, just need to put two or three ssd and take the picture when they are connected to the rig, then remove and throw run the PCMark with this software and many tweaks, the ssd for photography, and then return intact to its rightful owner, like computation store or friend

 

@ xXSebasXx: you know me ....you know that it is perfectly detectable a tweak invalid in any 3DMark , I'm talking about software and SO optimized to give the same results in the PCMark as if he had used three revodrive

Edited January 9, 2012 by Sweet

[Mr.Scott]

we (the staff) know all teewks right now

Don't tell stories Alex. If the staff knew all the tweaks, don't you think they'd be using them for themselves?

 

Also, for the record, I mean no disrespect for those who excel at this bench honestly. But I know for a fact there are a few that aren't so honest. They'll never be caught, because everything is a 'tweak'.

Edited January 9, 2012 by Mr.Scott

[Turrican]

Don't tell stories Alex. If the staff knew all the tweaks, don't you think they'd be using them for themselves?

no.

[Sweet]

Don't tell stories Alex. If the staff knew all the tweaks, don't you think they'd be using them for themselves?

 

Yes, we can, and no is storie, I dont use this software never in my PcMark, only one guy (of the staff) has used, like i write before

 

I use this software for rending videos of many pictures (more than 1000pictures) of my family travel, only, see my Pcmark score, please.and you can see this in the profile of each member of staff

 

P.S: sorry by delay, the coffee is in my t-shirt:confused:

Edited January 9, 2012 by Sweet

[Mr.Scott]

I know you're an honest guy Alex. You don't have to prove anything.

And apparently, I must be outta line for speaking my mind, so I'll just bow out semi gracefully and leave PCM05 to those who bench it.

Edited January 9, 2012 by Mr.Scott

[Sweet]

A special mention for overclockers of my country, who have investigated this in the PCMark, long almost 4 months.

 

Some have begun to test this software I used for my videos., but then, they went much further and have discovery (if this is the right word) a tweak that greatly improves the PCMark software and this to increase the score, but using something new and also can regulate each subtest.

 

I'll try this soon (perhaps), and this a tweak is fully legal, not is software and no is a script, only tweak

 

For those who have helped me discover what many guys was being done and in turn, find something that is perfectly legal, is my appreciation for his research and many hours of testing

 

This is in trial and error stage right now, by now is somewhat complicated to do -at least for me-

 

THANK YOU !!!

Edited January 9, 2012 by Sweet

[xXSebaSXx]

I don't know why you continue to try to make me the bad guy here. I don't care if everybody sits on all their tweaks or not. My point is that if the people who moderate the submissions don't know or understand said tweaks, how can they possibly look at a submission and judge whether it's valid or not if it's reported, when they don't know what to look for. Does that mean if you're the only one who knows said tweak, that they and everybody else have to take your word for it? That's what makes this bench impossible to moderate, and why it should be ditched.

Mind you, this is my personal opinion, and I'm entitled to that, like it or not.

BTW, loopholes in the rules do not constitute a 'tweak'.

 

Whatever. Do what you want.

I don't run it enough to care anymore.

EDIT- because I don't care, does that make me more 'ignorant' than just not knowing a tweak?

 

Scott... That post was not aimed at you man. I have nothing against you. I agree with you on what you say about loopholes not being tweaks. I agree with you on most of your argument actually...

 

The one thing I don't agree with you on is the "let's ditch it" part; I'll explain my train of thought. If we say that PCM05 should be ditched because it's too hard to moderate and because it is too hard to separate tweaks from cheats; what happens next? Do we ban GPU BIOS edits for better 3D scores because they are impossible to spot? Do we ban the special "REX BIOS" that makes for much more efficient SPi runs because the moderators can't tell who's using it and who's not? Do we ban LOD for 06? My argument here is that if we start down this path, the time may come when "NO TWEAK" is allowed... And what fun would that be?

 

You are entitled to your opinion and I respect it, but I am entitled to mine as well.

 

I recognize that I, sometimes, come across a bit antagonistic, but trust me... My posts are in no way attacks on anyone in particular.

[Sweet]

Income for the last time in this post, just to clarify something that is confusing

 

 

Forbidden Hardware:

none

 

Forbidden Tweaks:

mipmap tweaking

Don’t disable post-processing

any software or human interaction altering the perceived speed of the benchmark program, tricking it to believe it ran faster

wireframe hacks: any software or driver causing a benchmark program not (properly) render the textures

rendered image differs too much from the original due to other software tweaking

 

LOD is a tweaks "allowed in 3D'06" since long time ago, only disable post processing and mipmap is not allowed.

 

Rex bios is not a tweak (cheat) is a bios optimization, why should not be allowed ?

 

regards

Edited January 9, 2012 by Sweet

[xXSebaSXx]

I understand that Sweet... My point is that if we start to go down the path of ditching benchmarks because the tweaks start to become too difficult to manage/identify... Who's to say that other benchmarks will not go out the same door in the future?

My examples of LOD and BIOSes may have not been the best, but the idea is there... Just because a benchmark can be tweaked in ways that are hard to manage/spot doesn't mean that it should be thrown out. In my opinion; it means that people should work harder at finding those "hard to find" tricks to get the most out of the bench.

[Christian Ney]

Finally a member of staff has used this software to prove that with this only, no need to spend many U$S. in storage, results have been achieved bulky, as you buy an raid 0 ssd ultimate or revodrive, and Without Accard controller, only you must have a lot of ram in your system and ready, software is much cheaper.

 

With this has shown that it has hurt those who have invested a lot of U $ S in its storage system. nothing more.

 

So just to be clear:

 

1. I dont care a fuck about the score I made.

 

2. I made this score to show the community the impact of a tweak/software/optimisation/whatever used and allowed for PCMark7 in PCMark05 (way more used than pcmark7).

 

3. Why not discuss in the forum ?

because only few people will see it.

having the score on the frontpage of hwbot was the best way lot of people see it and discuss about it and have more feedbacks.

 

4. Now imo there is a big difference between ramdisk and ramcaching.

just create a ramdisk doenst improve performance nor boost you system. Now maybe there is a way by creating a ramdisk and change settings/folders/registry to use the ramdisk to boost the system.

Select the ramdisk to bench in pcmark05, I see it as a cheat or at least I find it not fair/nice/allowed.

 

5. Community dont want it allowed, fine.

 

6. It shows also the use of ramcaching to people who arent familiar with, soon 16,32,64GB of memory will be common in your system, make the best use of it

 

Don't tell stories Alex. If the staff knew all the tweaks, don't you think they'd be using them for themselves?

 

No, too lazy

Edited January 9, 2012 by Christian Ney

[Doc.Brown]

I vote for the third option,i don't really want to completely remove the Xp Start-up limitation but i wish it will be adapt with the speed of the current storage solutions(hdd,ssd) of the moment(sata3).

Or another solution exist,keep the limit to 220 but build a wrapper or a little software wich could limit the start-up test at the same value for all guys.

 

Sorry for my bad english.

[Mr.Scott]

Scott... That post was not aimed at you man. I have nothing against you. I agree with you on what you say about loopholes not being tweaks. I agree with you on most of your argument actually...

 

The one thing I don't agree with you on is the "let's ditch it" part; I'll explain my train of thought. If we say that PCM05 should be ditched because it's too hard to moderate and because it is too hard to separate tweaks from cheats; what happens next? Do we ban GPU BIOS edits for better 3D scores because they are impossible to spot? Do we ban the special "REX BIOS" that makes for much more efficient SPi runs because the moderators can't tell who's using it and who's not? Do we ban LOD for 06? My argument here is that if we start down this path, the time may come when "NO TWEAK" is allowed... And what fun would that be?

 

You are entitled to your opinion and I respect it, but I am entitled to mine as well.

 

I recognize that I, sometimes, come across a bit antagonistic, but trust me... My posts are in no way attacks on anyone in particular.

Understood. We are quite similiar in personality. I do respect your opinion. No harm done.

 

If the staff knew all the tweaks, don't you think they'd be using them for themselves?

 

 

No, too lazy

@ Chris- LOL

[rbuass]

Oh guys....

I can not understand how somebody can like a bench that you don not need to overclock hard to brake the scores...

To me.... PCmark is useless...but is only a personal opinion...

Once lots of guys like... keep the bench

[knopflerbruce]

I posted a suggestion somewhere: Post the real result, type the real XP startup speed in a box while submitting, let the bot engine calculate the "new" score assuming 220 XP startup if the score is higher than 220mb/s.

 

Got the impression it was a good idea, but no decision was taken:p The score formula is pretty straight forward, so this should be easy to code.

[hokiealumnus]

Now that would be a wonderful thing!

[El Gappo]

Trying to hit 219.9 is part of the fun