PCMark05 HB Rules,FM Rules and correct submision for ranking.

[chispy]

This tread will be used to raise our opinions as benchers on this community

regarding PCMark05 HB Rules , FM Rules and the correct submision for ranking on PCMark05. Please disscuss the subject in here.

 

1. Can hwbot confirm exactly what FM have done as far as the 220 cap xp start up HD test , is it 220 ,300 ,400 ?

 

2. It would be nice to see in the screenshoot a pic of device manager with the alleged HD used (Acard raid0 , SSds Raid0 , i-Rams etc...).

 

Thank you.

 

 

chispy.

[RAMDAC]

Hi chispy,

thank you for this thread.

It is my first and at the same time my last posting here, as I've decided to leave this unfair community.

 

A few weeks ago I had 4x SSD which I optimized with MFT to obtain some weird 33k score. That result was deleted and after reading the hwbot rules, I understood it and accepted it of course.

So once I tasted the blood I wanted more, and I bought 4 additional SSDs with the hope for a top score without MFT or any other cheatings.

Well, even these 8 SSDs didn't brought me further than to 22k in PCM 05.

I decided then to go extreme and to install totally 16 SSDs.

 

 

The financial investment was gigantic, as I had not to buy only the disks but also a proper RAID controller for the dozens of disks.

 

The disks were easy available, so I had just to wait for the controller which came luckily right before weekend.

At the beginning the results were very poor and I was very disappointed for spending much money for almost no effects. The mistake was that I combined all disks to one array, awaiting an incredible data rate but it was a bit contraproductive, especially also because I wasn't familiar with all those SuperTrak settings.

With 2 OCZ APEX SSD I had an average rate of ca. 350MB/s, which was good but with 4 OCZ in RAID 0, I had just about 460MB/s, then with 6 APEX SSDs- it was a bit more than 550MB/s. (Comparing it to SuperTalent MasterDrive OX with 2 SSDs an average rate of 266MB/s, with 4 SSDs a rate of 512MB/s in Vista and over 700MB/s in Win XP)

 

 

I remember how I was dreaming about an incredible PCM 05 score while I installed the hardware but the first run with all SSDs on one array brought me solely some poor 98MB/s in PCM 05's XP startup.

After a few sleepless nights, I finally found the right combination for the arrays, for the stripe sizes, the block sizes etc. I found out which ones were better for OS installation, the OCZ or the SuperTalent, further I found out the optimum amount of RAID arrays to create a dynamic volume and I found the optimum amount of volumes for the OS array and much more... (like e.g. the best performance settings on the SuperTrak controller itself)

 

 

Further I played with variations of different RAID controllers, e.g. took 2 SSD for the onboard JMicron controller, 4 SSD for Intel's ICH10R and the other 10 were plugged into the Promise SuperTrak controller. With all three RAID controllers enabled I had some compatibility issues that forced me to disable one of them.

JMicron worked fine with SuperTrak but too slow in performance, while Intel's ICH10R was damn fast but with all 6 slots populated, it showed me "Reset Port Error!!!" on all disks and it didn't allow me to configure SuperTrak's RAID while booting. (Unfortunately or fortunately this error happens only with the OCZ disks, when using the SuperTalent with all slots populated, there are no errors?!)

Anyway, accepting the error messages through Boot and using SuperTrak's WebPAM console in Windows, allowed me to create finally my desired RAID arrays.

 

The first run with 2 logical arrays (12x SSD + 4x SSD), plus 2x SSD RAID 0 as primary and active on JMicron for the installed OS, brought me impressive 29k, which I submitted immediately to Futuremark and afterwards to hwbot.

 

 

http://service.futuremark.com/compare?pcm05=1887600

 

 

Later I found out that my score was deleted and I went to the PCM 05 rules page, seeing that PhysX wasn't permitted.

Alright, I disabled it, changed at the same time the configuration to ICH10R as the controller for the active, primary RAID disk for the OS and I made 3 logical RAID arrays through SuperTrak's console.

After creating the 3 logical arrays (2x 64GB SSD + 4x 32GB SSD + 4x 32GB SSD - Block size: 4KB, Stripe size: 128kb), I went to Windows' administrative tools where I made one dynamic stripeset volume out of the 3 logical RAID arrays (Block size: 8192 Bytes - but also 16kb and 64kb gave me the same best results and a data rate of over 220MB/s in PCM 05's XP startup)

 

 

The quintessence of the story is that I've spent a lot of time and a looooot of money, just to be stopped by 4 finnish letters and being treated almost as a cheater now.

 

That SF3D kid reported me twice, once my XP startup of 235MB/s was too high for him to climb and the other time, my Virus scan was too fast for him to follow.

It shows me only that this boy never ever had his hands on a dozen SSDs at once.

The funny thing was that the email came exactly when I was experimenting how to achieve even higher rates than 235MB/s at XP Startup.

The sad thing was that I turned off that pc from then on, seeing no reason to continue anymore. - I could perhaps find some tweaks to lower the scores, or I could degrade again to only 8 SSDs but I won't do this paradox favor to anybody!

 

Well, the only consolation is that I'm retiring at least as a clean world champion, without steroids, HGH, STH, IGF-1, EPO etc.

The knowing that I've reached a totally legal world record, is such a great feeling that I don't need any hwbots and boints to confirm it or even award me in form of a JPEG, PNG or GIF trophy.

I award myself by seeing live what this machine is able to reach.

 

 

And the acceptance in form of a valid link at FM is prize enough

http://service.futuremark.com/compare?pcm05=1888775

 

I feel only sorry for my team but as long as my scores aren't accepted among this illustrious society here, I'm useless anyway.

I will submit my future scores to FM and compete against myself only.

 

Chispy, you were the one who took all my GTX285 Gold awards but I like you as we have similar hardware and a similar view, and I wish you luck among those stone age competitors which are still comfortable with their antique methods.

 

You and the honorable eva2000 are right: "The Future Is Now!"

 

All the best to you!

 

(I'm for sure outta here

 

RAMDAC, former hwbot fan

[chispy]

Sad to hear that bro , i feel your pain and frustration. Dont give up and keep benching , i like a little competition on the GTX 285s category , and im sorry for taking all your trophies on that category . I think the rules of PCMark05 should be more clear as to what are the bounderies of HD xp start up , HD general usage and HD virus scan in order for things like this not happenning no more. The HWBOT Rules need to be rewritten for this benchmark as they are very unclear as of now and very confusing

I wish you the best RAMDAC and thank you for stand up and bring this to other members attention. Good Luck.

 

chispy.

[SF3D]

That SF3D kid reported me twice, once my XP startup of 235MB/s was too high for him to climb and the other time, my Virus scan was too fast for him to follow.

It shows me only that this boy never ever had his hands on a dozen SSDs at once.

The funny thing was that the email came exactly when I was experimenting how to achieve even higher rates than 235MB/s at XP Startup.

 

I didn't report your score. I blocked it.

 

If we have a 220MB/s limit in XP startup, what do you think 235MB/s is? We have to follow this rule strictly. No exceptions, no matter who have done the benchmark or how it was done. This rule will be overwritten when the time is ready.

 

And yes, that 9000MB/s virus scan was a bit too much for me.

 

So, if you can't follow the simple rules, you start to call me a kid? Well, if we will meet someday, come to say it on my face

 

I'm sick and tired of this PCMark05 and the "HDD" issue we have. No one is using HDD with it anymore, so we should just dump the whole benchmark.

[Massman]

I'll skip the emo-comments and go right to the issue .

 

Basicly, we are left in the unknown by FM. They've promised to raise the cap (new technology, of course), but sadly enough they still haven't raised OR informed us about the new limit. Yes, vague comments in mail, but other than that we have no precise information.

 

As far as I know, the limit would be raised to 300, but don't pin me down on that one. If turns out that your score falls within the new restrictions, your score will be restored, of course.

[TheKarmakazi]

I'm sick and tired of this PCMark05 and the "HDD" issue we have. No one is using HDD with it anymore, so we should just dump the whole benchmark.

 

Thats a bit rash dont you think? Unless you substituted points for PC Mark Vantage... Personally I enjoy benching pcm05, lots of things to tweak and squeeze some extra points out of. I dont think we should remove points from it just because new tech is stretching the limits. Give FM some time to determine a finalized policy and see what happens.

 

I really have no opinion on the startup limit, I will follow whatever rules are in place from FM and HWB.

[Gautam]

With 2 OCZ APEX SSD I had an average rate of ca. 350MB/s, which was good but with 4 OCZ in RAID 0, I had just about 460MB/s, then with 6 APEX SSDs- it was a bit more than 550MB/s.

 

And yet your HDD Virus Scan was 8979 MB/s. :

 

Yeah it's looking like time to can this benchmark...it's no fault of the community, it's just that the bench itself was poorly designed from the getgo and we've just all been working around it's shortcomings. =

[jabski]

the limit is 220mb.

[r1ch]

I'm sick and tired of this PCMark05 and the "HDD" issue we have. No one is using HDD with it anymore, so we should just dump the whole benchmark.

 

Is your only definition of a hard drive a rotational magnetic drive then? Or do you really want to start debating the semantics of what constitutes a "HDD"? I really really don't.

 

I don't understand the hate for PCMark05. Futuremark are responsible for the all of the confusion, by (rightly) imposing a 220 limit to start with, (wrongly) removing it and then (rightly) putting a limit back but a higher one than before....without public statements to this effect.

 

I feel sorry for hwbot having to moderate scores based on changing rules.

 

However, dumping the benchmark is not right. The only argument that there is against it is that current physical 'drives' that you can install an OS to are "too fast" and not what the benchmark was intended for. By that reasoning, if you dump PCM05, you should dump 3DMark01, probably Aquamark, and with the Windows 7 and ATI/AMD bugged runs with 3DMark03, probably that as well. No? I didn't think so.

[SF3D]

Is your only definition of a hard drive a rotational magnetic drive then?

 

For me HDD is: http://en.wikipedia.org/wiki/Hard_drive

 

This is not relevant at all. So please, no comments like that anymore.

 

We should take point's away from this benchmark. Just like we did with PCMark04. PCMark04 wasn't able to use dual cores in grammar check test and we dropped it. Now PCMark05 can't give any reliable results with those monster SSD setups. 9000MB/s virus scan is not the value FM had in mind, when they designed this benchmark. It is their error in design, not ours.

 

I understand why some of you like this benchmark so much. For me this is just like cheating with PhysX, cause benchmark is not designed to run some tests with some different type of hardware. Of course future will bring more powerful parts, but it doesn't mean that we can use them without any discussion.

 

I'm thinking what is best for HWbot. There is 16000+ users and only very few are defending PCMark05. The end have to be somewhere.. this benchmark can't be here forever.

[Massman]

The new results are reliable, the only thing we need is a cap limit set to new heights.

[Massman]

For me HDD is: http://en.wikipedia.org/wiki/Hard_drive

 

This is not relevant at all. So please, no comments like that anymore.

 

It's very relevant as the only issue with the PCMark05 benchmark is IN FACT related to the definition of harddisk drive.

 

We should take point's away from this benchmark. Just like we did with PCMark04. PCMark04 wasn't able to use dual cores in grammar check test and we dropped it. Now PCMark05 can't give any reliable results with those monster SSD setups. 9000MB/s virus scan is not the value FM had in mind, when they designed this benchmark. It is their error in design, not ours.

 

I believe 388k was not the value the AM3 team had in mind when designing the benchmark. I believe Madonion was not at all thinking about 130k scores in 3DM01, let alone 170k in 3dm03.

 

Skip them as well, then?

 

PCMark04 was dropped from the hwboints because one had to manipulate the program using external programs to make it work correctly. As far as I know, Pcmark05 does not need that.

 

I understand why some of you like this benchmark so much. For me this is just like cheating with PhysX, cause benchmark is not designed to run some tests with some different type of hardware. Of course future will bring more powerful parts, but it doesn't mean that we can use them without any discussion.

 

In your analogy, you should use software ramdisk instead of 'type of hardware'. PhysX is a part of the software that can be enabled/disabled; a harddisk isn't.

 

I'm thinking what is best for HWbot. There is 16000+ users and only very few are defending PCMark05. The end have to be somewhere.. this benchmark can't be here forever.

 

Even less people are supporting ditching PCMark05 from the hwboints ranking. Most people don't care what decision we take, they'll just follow our decision no matter what.

[SF3D]

Yes, but again we can say it was ment to be used with traditional HDD. Not SSD or even with I-ram or Acard or anything else.

 

If SSD's can get you a huge boost in real life usage, we have to have a reliable benchmark for it. Not 4 years old tests suite, which wasn't designed to hold this kind of storage power.

 

It is totally same situation if graphics processing unit can do physics calculation and it will do it 20x faster. It is clearly a better method for it, but some tests are not designed for it and we don't allow it to be used. This is same situation.. some test were designed to be run from HDD. We have better hardware nowadays, but it doesn't change the fact that this benchmark wasn't designed for new hardware.

 

I have to say again, that it is their error in design. It is not our fault.

[knopflerbruce]

For me HDD is: http://en.wikipedia.org/wiki/Hard_drive

 

This is not relevant at all. So please, no comments like that anymore.

 

We should take point's away from this benchmark. Just like we did with PCMark04. PCMark04 wasn't able to use dual cores in grammar check test and we dropped it. Now PCMark05 can't give any reliable results with those monster SSD setups. 9000MB/s virus scan is not the value FM had in mind, when they designed this benchmark. It is their error in design, not ours.

 

I understand why some of you like this benchmark so much. For me this is just like cheating with PhysX, cause benchmark is not designed to run some tests with some different type of hardware. Of course future will bring more powerful parts, but it doesn't mean that we can use them without any discussion.

 

I'm thinking what is best for HWbot. There is 16000+ users and only very few are defending PCMark05. The end have to be somewhere.. this benchmark can't be here forever.

 

Not sure it's possible to have it for hardware points only, but that would perhaps be the best..? It would really be a shame if thousands of normal results will be "worthless" because only a TINY minority of the results cause trouble.

[jabski]

but the whole point of the limit is what makes it fun. If your 8x SSD's in Raid are over 220 limit then obviously you would need to change your setup.

I vote keep pcmark

[TheKarmakazi]

 

I'm thinking what is best for HWbot. There is 16000+ users and only very few are defending PCMark05. The end have to be somewhere.. this benchmark can't be here forever.

 

And by your own logic very few of those 16,000 are attacking it or asking for it to be dropped...only you and Gautam?

 

Yes, but again we can say it was ment to be used with traditional HDD. Not SSD or even with I-ram or Acard or anything else.

 

But you never raised a problem or question prior to this when I-Rams were all over the place. Only after MFT scores came out did you raise opposition (and I agree with you on that issue).

 

So do you propose that pcmark vantage gets hwboints then in pcmark05's place? Vantage has no problems with large arrays of SSD's and still gives very viable and realistic numbers.

Edited March 10, 2009 by TheKarmakazi

[chispy]

but the whole point of the limit is what makes it fun. If your 8x SSD's in Raid are over 220 limit then obviously you would need to change your setup.

I vote keep pcmark

 

I totally agree with you jabski , Now that i have seen and been thru that myself (go past 220 start up with my 8 SSds set up and a Adaptec 5805) , it was fun finding that sweet spot below 220 , for my it was 165 xp start up wich brought me a score over 23k). The problem its FM have NOT set the new cap in placed and the hwbot rules NEED to be rewritten as to what are the bounderies on HD xp start up , HD general usage and HD virus scan. It would be very nice to add PCMark Vantage and award points for it , far more reliable than PCMark05 and it was meant for new technology in mind. I hope for a good decision of hwbot on this matter. Good day guys.

 

 

chispy.

Edited March 10, 2009 by chispy

[Monstru]

I really don't care about PCMark 2005, but I really do not like some attitude I see here. If you wanna drop a 4 year old benchmark because when it was made they wouldn't even think that HDD's could get this fast, then what about 2001, they sure as hell didn't think 8 years ago that GPU's and CPU's would get this fast. And it can only use max 2 GPU's, and it does not even scale with the latest video technology and so on....

 

That is not a right way to look at it. I can imagine that for some guys getting near the top must be a little frustrating to lose some points because they don't have what it takes to dominate all the benchmarks, but it is normal to be like that. Sure, everybody in the top has tremendous skills, but there are also specific skills and likes and dislikes. Real old school tweakers play 2001 and really show what tweaking is all about, with not so expensive machines. Guys with "big guns" really show their skills putting quad-GPU machines to the works and rule 2k3-2k6, guys with good Nehalem skills really kill everybody in Vantage, guys that test tens of CPU's and really have the patience to find the "one" pwn everyone in Pi 1M or CPU-Z, software maniacs really tweak that Spi 32M, guys with Skulltrails laugh in anyone's face with their WPrime scores and so on. There are enough games to play for everybody, everyone can show his skills in something, and mainstream overclockers and benchers have most of those applications to play with. What is wrong with having those CRAZY guys that use 20 SSD and thousand dollar RAID controller having their own game where they can shine? If PCM05 is their game, let them play their game, this guys that started this thread really looks that he knows what he is doing and really put some effort into this. Let them be....

 

And about bounderies and stuff....Hey people, newsflash...nobody uses LN2 on CPU and VGA's to play, so it really does not matter if it'a real life situation or not, it's all about if you are crazy enough to get the most points in one benchmark or not. And last time I checked that was what benching ia all about.

Edited March 10, 2009 by Monstru

[SF3D]

Monstru- You are not right on this one.

 

It is all about weighting a benchmark. PCMark05 was designed to run with normal HDD's. The HDD tests are giving too much gain to the final score and that is making final scores meaningless. Simple as that.

 

It is same with 3Dmark Vantage --> If you use PhysX, final score doesn't mean anything, cause "new technology" was used and score got a lot better.

 

I haven't seen any top 10 guys saying any bad words about this situation. (Gautam said something, but he doesn't care about those things you are suggesting) I'm not even top 20, so this is not issue for me.

 

I don' have time to answer most of the point you are making, cause they are not worth it now. We all know that you are good bencher. We all know you can tweak systems well, but there is no need to always bring these same issues up. Someone is always better than you, no matter what is the matter. THIS IS NOT the case here. It is all about the PCMark05 benchmark itself.

[r1ch]

SF3D, you should listen to what your fellow hwbot mods and admins are saying instead of repeating your flawed and blinkered view on the situation. Every argument you put forward has been proved groundless. Massman has this absolutely spot on - read his post again please.

 

You don't seem to understand that the single justification for your argument against PCM05 would have at least 3DMark01 and Aquamark points removed as well. Is this really what you want?

Edited March 10, 2009 by r1ch

[SF3D]

SF3D, you should listen to what your fellow hwbot mods and admins are saying instead of repeating your flawed and blinkered view on the situation. Every argument you put forward has been proved groundless. Massman has this absolutely spot on - read his post again please.

 

You don't seem to understand that the single justification for your argument against PCM05 would have at least 3DMark01 and Aquamark points removed as well. Is this really what you want?

 

We have a nice discussion going on in crew section about these issues. I know what these two guys are saying and it doesn't change anything for me. I still haven't got any valid argument towards my own.

 

My argument about PCMark05 being weighted wrongly in the first place haven't been proved groundless. Or if you can do it, please do so. I have to say it again, that it was a design error in first place. They developed this benchmark 2004-2005 and I don't know why they didn't thought about future. If there would have been different weighting method, none of these problems would be here.

 

And please, explain how 3DMark01 and Aquamark is connected to this issue, what PCMark05 have? I can't see any similarities.

 

And in general:

It is my job here to evaluate the benchmarks we have in our database. If there is something wrong or if there is even some questions, we have to check them all. It is so funny, when people always think, that I have some personal issues with this. If other moderators who have posted here don't see the problem, it doesn't mean there isn't one. If some benchmark loose points, it is the same situation for everyone. There is no winners or losers in these situations. Aquamark have been under discussion many times, but still there isn't any similar problems present, so there is no need to start taking it down.

[scooter.jay]

I was not going to say anything but pcmark05 is my favorite bench. The problem is that a few think the benchmark gives unfair high scores when a good ssd raid setup or ram drive is used but in think there is nothing wrong with it. Each bench has its own need imo i.e 3d03 does not need cpu speed its all about gpu. So why cant pcmark05 not be about hard drive/drive speed. As long as a piece of hardware is used acard or i-ram or drive setup then whats the problem thats all i think.

[TheKarmakazi]

It is all about weighting a benchmark. PCMark05 was designed to run with normal HDD's. The HDD tests are giving too much gain to the final score and that is making final scores meaningless. Simple as that.

 

It is same with 3Dmark Vantage --> If you use PhysX, final score doesn't mean anything, cause "new technology" was used and score got a lot better.

 

 

It is NOT the same...

 

PhysX (and MFT or RAMdisk) is a SOFTWARE based bump in points that skews the results of hardware into unrealistic scores. SSD's Irams and Acards are all HARDWARE which gives better performance because it is simply BETTER and more powerful! Your argument is like saying we shouldnt allow dual socket systems to play wprime because it so heavily skews results? Or that dual gpu single card setup is illegal to play 2003...

 

These are hardware based setups that make the scores so high, not software based like physx/ramdisk. And pcmark rightfully weighted hdd speed heavily because it is the slowest subsystem of 90% of computers! So it makes sense for it to increase scores so much!

 

If some benchmark loose points, it is the same situation for everyone. There is no winners or losers in these situations.

 

That is not entirely true! I dont have mega $$ for vantage setups with tri sli and binning 100 cpus. So I bought a few irams to have some fun with pcmark05. I have many results and points invested in pcmark05, since its one of my favorite benches along with 2001. Since you dont seem to even run pcm05 of course you dont care if points are dropped... In fact I would say all of i4memory team is invested in pcmark05 and would be a substantial "loser" if this happens... I know lots of OCF guys and Benchtek UK and Dimastech are hardcore pcmark05 benchers too! So you cant really say no one will be a loser....

[Buckeye]

WoW what another interesting read. It seems the PCMark05 problem just keeps going and going.

 

I have to add in here also how I feel about this bench. I was an early adopter of SSD's and ended up with my 9x 32gb MTRON PRO SSD - ARC-1231ML some time ago. I love this setup even tho newer SSD's are killing me, haha tech changes.

 

I played around with using 2x SSD's with onboard controllers with older systems and ran the benches for many, many setups. It's a blast seeing what a simple change in hardware can do for just about any older setup can do for performance and bring those old rigs back to life.

 

As far as PCMark05 goes I say no limit on XP startup or anything, use what ever you can bring to the table for this bench. This bench is about what the whole computer can do, every piece, no matter if its software or hardware. Be as creative as you can be to make the fastest running system you can.

 

SSD's, GPU's, PhysicX or MFT or what ever, it make no differance to me.

 

I get into this same discusion about using 3DMark01, Aquamark etc. with current gen hardware. Or for that matter mixing OS's up in Forum Warz.

 

Because any bench that is currently used right now can be torn apart and say that this was never designed for this or that so you cannot use this. Yet everyday people bench with hardware that ws never desgned for a given bench.

 

Those of us that like PCMark05 need a place to race our hardware and compare what each other can do. So what if PCMark05 was never designed to use SSD's, its what we have to use. Leaving a cap on HDD startup or what ever is like going to a F1 race where they put restrictor plates on the carbs to slow them down, I think you might get what I mean there.

 

If you can say that PCMark Vantage was designed to use SSD's then great I will use that. I am unclear if there is any bandwidth caps on that bench.

 

I say remove all caps and restrictions and let the races begin.

[knopflerbruce]

It's possible to have limits for the subtests that "kill" the benchmark. It's better than no benchmark, at least.