Crew pro Posted November 5, 2012 Crew Share Posted November 5, 2012 There will be an official statement addressing a few issues shortly, sorry for the delays Link to comment Share on other sites More sharing options...
Moose83 Posted November 6, 2012 Share Posted November 6, 2012 Sounds good, hopefully it will published today:) Link to comment Share on other sites More sharing options...
Moose83 Posted November 11, 2012 Share Posted November 11, 2012 Mh...still no decission about RST and Sector Size change? A lot of time went away:) Link to comment Share on other sites More sharing options...
Yamunsa Posted November 12, 2012 Share Posted November 12, 2012 (edited) i think the stumbling point of the discussion is about how to enforce the rule and deal with previous submissions, that have already been made using these programs, than whether the software should be permitted. knowing it would be much easier to allow these programs, then their use does not need to be policed, still doesn't make them right... if the use of software caching is permitted then using tools from dataplex, fancycache/supercache (pointing to ssds not system ram), velobit, enhanceIO or even using mklink to soft link directories on your SSD to directories in program files as a cache medium would have to be permitted because it would be impossible to draw a firm line in such a grey area. from my own research i've seen it to be quite easy to spot especially when the majority of submissions with HD GU over 300mbs and VS over 600mbs more often than not have listed their storage and or have posted a pic of their setup. ease of spotting is for me is further emphasised using my own or steve ro's submissions to see what large acard, iram or ssd arrays can do. permission of rst also limits the advantages to participants running Z68, Z77, H77 & Q77 as, as far as i'm aware it only works on those platforms. so if folks want to have a bash at older HW records they will have to have a large array only to watch it get smashed by inferior setups if they indeed move it to current gen platforms. looking at what pro has posted previously i think the writing is on the wall for raid expert & rst and any 3rd party caching program for use with pc05. it's just too large a can of worms to be allowed and the impact on other PCM benchmarks is obvious. highlighting notable points from the first page: why keep pcm05? for those that are saying "get rid of pcm05", we are basically at that point and i am sure if this doesnt work then that is what will happen, but there are lots of reasons to keep this benchmark, for example, we have guys like Steve and Aristides that are always strong in this bench and are getting beaten by tweaks that they are not comfortable using and rightly so, this benchmark has a place and is a source of enjoyment for lots of people, so i believe we should have a proper effort to try and fix it up before abandoning it i' pretty sure steve ro has not made a submission with either raid expert or rst. please be patient now guys i see lots of people saying, whys this not banned, why is mine banned and not his, etc, please be patient, this will takes weeks, months, and will be ongoing, but it is no different than any other benchmark, other than the fact it has gone unchecked for so long.. if this happened in 3dmark11 it would be immediately checked and in the future pcmark05 will be the same what is allowed and whats not? -snip- 1. "Using software, performing hardware modifications or by human interaction altering the perceived speed of the benchmark program, tricking it to believe it ran faster and thus producing a better result. (adjusted 31st of August 2010)" 2. not modifying the benchmark itself Yes d3d, video encoding via registry or powertoy and LOD go against these rules, BUT they have been used and accepted as officially OK, unless staff have specifically said something is allowed that goes against HWBOTs general rules then it is NOT allowed if what you are doing in pcmark05 goes against one of those 2 rules, remove it now. staff will be contacting users with out of line submissions asking for proof on how to replicate them within the rules, if you are worried you cant provide that proof, remove your score now or it will be removed for you use your brain guys, if you know something is not right, 2400MB/s virusscan on a weak AMD platform, then its probably not right, this is the ethical side of hwbot we can't enforce but hope is in place Key areas we are targeting to start with Some of the key areas we will be targeting, but not limited to: Video encoding replacement, audio encoding Irregularly large trans windows Irregularly large memory latency Massive browser scores Anything out of line with file decoding Crazy virusscans or hdd general What is OK I am not going to go through every tweak and say its ok or its not ok, thats what the general rules are for and they cover alot of the "super tweaks", but not all, FD2 as a program is totally allowed, but using it to manipulate the window size or minimize windows is not allowed, similarly using a script to do this is not allowed as it breaks the "benchmark percieved speed rule" -snip- Raid XPERT Still up in the air is raid xpert, we are discussing the outcome of this one right now, there and positives and negatives to both sides of the fence and we need to make sure everything is worked out properly Massman said it was allowed, but now that we are aware that their is some bugging going on, we are reviewing it. We don't want peoples expensive raid setups obsoleted by a bugged software, Fairness This is not a witch hunt, everyone will be given the oppurtunity to prove their tweak is fair in a confidential environment. If you tweak is legitimate, it won't be shared, it will be approved and you can continue using it privately if you so wish. If you choose to not share the tweak with staff and your score looks out of line then it will be removed. Lets hope we can clean the benchmark up once and for all, remember guys, please use your brain, we dont want to remove scores, so please do it yourself if you know they go against rules Edited November 12, 2012 by Yamunsa grammer & spelling Link to comment Share on other sites More sharing options...
Moose83 Posted November 12, 2012 Share Posted November 12, 2012 I only see Problem with sector size change;) Nearly 2G VS can done with areca and also with RST. i will not use RST or Raid Expert the next time, only Areca:) Link to comment Share on other sites More sharing options...
Crew pro Posted November 12, 2012 Crew Share Posted November 12, 2012 as ive previously said Moose, making proper decisions can take time, and even if it takes 6 months, we are going to take the time rather than make the wrong decision, i can give some insight into the current thinking of staff, but this is not finalized and could change as per our final announcement, but the current thinking is, rst/raid xpert allowed, sector size changes not allowed, if you have overly high looking hdd scores then you may asked to demonstrate how its possible and it will be your responsibility to prove its legal EDIT: Didn't realize I was logged into our team.au account, this is pro Link to comment Share on other sites More sharing options...
Yamunsa Posted November 12, 2012 Share Posted November 12, 2012 the current thinking is, rst/raid xpert allowed taking this example - link : StrategosSan comments that he only used 2 x Crucial M4 SSDs and his virus scan result exceeds the phisical limit of the drives let alone the translation into pc05. software cacheing looking to be deemed AOK.. this is a real surprise and a dissapointing ruling. Link to comment Share on other sites More sharing options...
gazza30 Posted November 12, 2012 Share Posted November 12, 2012 If rst uses another disk to cache to achieve its results why is fancycache frowned upon when you can assign one ssd to cache another seems like the same thing if you ask me. Rst / raid expert just seems like a moderators nightmare if you ask me going to be very hard to 100% stop bad results slipping through and ending up right back where we are now. Link to comment Share on other sites More sharing options...
Crew pro Posted November 12, 2012 Crew Share Posted November 12, 2012 Everyone is never going to be happy, facts of life Intel RST has long been allowed and AMD raid xpert is its equivalent from another vendor Other software caching programs have long been banned You think it's a nightmare to moderate rst? Try going back through the entire pcmark results and removing all the submissions using RST? Plus all the people using it for over a year and now their results being removed Similar to Lod this might not be the ideal situation but we think it's the best taking in all the factors We want a level playing field in the future without impacting existing scores in our ranking, no one said moderation would be easy, it won't, but people know now if there result is out of line they will be asked to explain it, so try it on and see what happens Blatantly breaking hwbot rules won't just result in submission removal Link to comment Share on other sites More sharing options...
Mr.Scott Posted November 12, 2012 Share Posted November 12, 2012 Everyone is never going to be happy, facts of life Intel RST has long been allowed and AMD raid xpert is its equivalent from another vendor Other software caching programs have long been banned You think it's a nightmare to moderate rst? Try going back through the entire pcmark results and removing all the submissions using RST? Plus all the people using it for over a year and now their results being removed Similar to Lod this might not be the ideal situation but we think it's the best taking in all the factors We want a level playing field in the future without impacting existing scores in our ranking, no one said moderation would be easy, it won't, but people know now if there result is out of line they will be asked to explain it, so try it on and see what happens Blatantly breaking hwbot rules won't just result in submission removal I'm liking your stance on this. Thank you for the effort. Link to comment Share on other sites More sharing options...
gazza30 Posted November 13, 2012 Share Posted November 13, 2012 Everyone is never going to be happy, facts of life Intel RST has long been allowed and AMD raid xpert is its equivalent from another vendor Other software caching programs have long been banned You think it's a nightmare to moderate rst? Try going back through the entire pcmark results and removing all the submissions using RST? Plus all the people using it for over a year and now their results being removed Similar to Lod this might not be the ideal situation but we think it's the best taking in all the factors We want a level playing field in the future without impacting existing scores in our ranking, no one said moderation would be easy, it won't, but people know now if there result is out of line they will be asked to explain it, so try it on and see what happens Blatantly breaking hwbot rules won't just result in submission removal Thanks for reply Pro, just my take on thing mate is all i know its not an easy job for yourself and the other Mods to sort out. Easier to moderate last couple of years since rst has been availiable then to have to deal with moderating and questioning results from here on in and years to come often the easy path may not be the best path. As for not affecting existing results i thought heaps of results had already been deleted either due to F2D or RST or other cheat/hacks etc why not continue the purge. It seemed like PC05 was getting back to more of a hardware bench not all that long ago with certain sowtware tweaks be disallowed i just don't think its gone far enough. But if Allowing rst/amd software caching is the road were taking then fine suits me hell i can sell off half my hardware for pc05 just don't think its the right path to follow. Link to comment Share on other sites More sharing options...
Crew pro Posted November 13, 2012 Crew Share Posted November 13, 2012 As for not affecting existing results i thought heaps of results had already been deleted either due to F2D or RST or other cheat/hacks etc why not continue the purge. your dead right on this one, but i think the key difference is those things are breaking fundamental hwbot rules, where as RST has been accepted in the past, i think everyone needs to understand there is no rule change here, its simply rule enforcement and in some tricky cases where things might fall between rules, clarification of rules take this sector size for example, by changing the sector size you are tricking the benchmark into giving a score thats not real, changing the percieved speed of the benchmark Link to comment Share on other sites More sharing options...
Yamunsa Posted November 13, 2012 Share Posted November 13, 2012 Everyone is never going to be happy, facts of life very true. Intel RST has long been allowed and AMD raid xpert is its equivalent from another vendor rst and xpert raid has been around, (used in conjunction with pc05), for less than 2 years. Other software caching programs have long been banned specific note to fancycache and perhaps supercache in regards to using system ram not other ssds? You think it's a nightmare to moderate rst? Try going back through the entire pcmark results and removing all the submissions using RST? Plus all the people using it for over a year and now their results being removed Similar to Lod this might not be the ideal situation but we think it's the best taking in all the factors i disagree. i think handling rst / xpert raid or indeed moderating old subs for rst could be quite easy. four different methods: (1) ask the community to report submissions and over time the problem will solve itself rather quickly as demonstrated recently with TE & TW. (2) if hwbot has a working relationship with futuremark ask them for access or a list of results in their pc05 db with scores exceeding a hdd general usage level of 300mbs and compare the md5 hash with hwbot submissions of known users of sw caching. with db level access at hwbot this would take less than an hour. (3) do it the hard way and scrutinise individual scores. this sounds worse than i think it would actually be - there are not many hwbot members benching pcm. even fewer who are actually competitive. realistically you are looking at the top 30 global subs for 1, 2 & 4 cores to identify those who use rst / xpert and those who have an array. moving forward the individuals best results would need to be looked at to confirm the use of sw caching and a circular pm could be sent to the member requesting self moderation. i believe this technique could be divided amongst a handful of admins and could be sorted over a weekend. (4) draw a line under the current subs and move forward with a final ruling. this bridge has been crossed before dating back to AM3 and the consideration of existing scores. We want a level playing field in the future without impacting existing scores in our ranking, no one said moderation would be easy, it won't, but people know now if there result is out of line they will be asked to explain it, so try it on and see what happens i agree, however the recent purge of scores highlights both the communities and individuals' backlash to the use of tools excessively altering the realistic score of the benchmark. the momentum to finally clean up pc05 is there. it is up to the community as a whole to finish the job. Blatantly breaking hwbot rules won't just result in submission removal i sincerely hope this is an indication to enforce the rules in a fair, blanket and firm fashion. to comment overall i think the following is worth pondering: The world of overclocking and the community it’s founded on is in essence not so much different from any other field of interest or social group. We have our own language, our own set of principles, our own set of shared motives and goals and (virtual) places we visit. Just like any other social group, once in a while we are forced to face an existential crisis; a problem that pushes us to think about what overclocking and benchmarking is, or what it is supposed to be. Over the years, we’ve had lengthy (and interesting) discussions regarding the usage of extreme cooling, a debate on whether or not graphical artifacts should be tolerated, if adjusting LOD values should be considered legit and many many more. link Link to comment Share on other sites More sharing options...
Yamunsa Posted November 13, 2012 Share Posted November 13, 2012 (edited) i know i've had a lend of you before but over here i have the utmost respect for you, (even more so when concerning pcm). i think everyone needs to understand there is no rule change here, its simply rule enforcement and in some tricky cases where things might fall between rules, clarification of rules in responce; the most widely used rules page for pc05 is dated May 15, 2008 well before the notion of this sort of software was about, (noting ramdisk was banned outright). link. using rst & xpert raid to trick the benchmark into giving a score that's not real, changing the perceived speed of the benchmark, is exactly what is happening. as linked above the theoretical and advertised limit of drives are being exceeded in the subtests. even these speeds do not take into account the translation of synthetic performance into pc05. 2 x retail sata lll ssd in r0 cannot physically hit 1200mbs let alone the convertion of speeds in to pc05's virus scan. the president of permitting / accepting even condoning the use of tools that previously has been shown to effect the benchmark into giving a score that's not real is no reason to retain the acceptance of them. F2D is an excellent example. we are still in the process of moving on from that era. Edited November 13, 2012 by Yamunsa Link to comment Share on other sites More sharing options...
Moose83 Posted November 13, 2012 Share Posted November 13, 2012 Dont talk about Fancy Cache! Over 2g gen usage and 7G VS on all Plattforms... Link to comment Share on other sites More sharing options...
Yamunsa Posted November 13, 2012 Share Posted November 13, 2012 (edited) when used to cache on ramdrives not ssds but i agree that sw caching can be pushed to the extreme. it's a route i'd much rather see halted before it becomes abused. here is a perfect example of a score that could fly under the radar if the user didn't list what he'd done - link. i know i have posted hard today but imo the sw caching discussion should be had. being used to seeing things open or shut the door very much looks ajar. Edited November 13, 2012 by Yamunsa added more content Link to comment Share on other sites More sharing options...
Moose83 Posted November 13, 2012 Share Posted November 13, 2012 No ramdisk 2 ssd + fancy Cache gives over 2g gen usage and 7g VS, no joke;) Link to comment Share on other sites More sharing options...
IanCutress Posted November 13, 2012 Share Posted November 13, 2012 taking this example - link : StrategosSan comments that he only used 2 x Crucial M4 SSDs and his virus scan result exceeds the phisical limit of the drives let alone the translation into pc05. software cacheing looking to be deemed AOK.. this is a real surprise and a dissapointing ruling. I agree. Strat gets: HDD - Virus Scan 1310.47 MB/s for using only two SSDs on ICH. I couldn't even get that using two ACARDs (http://hwbot.org/submission/2327525_borandi_pcmark_2005_core_i7_3960x_43966_marks), so something isn't right here. Link to comment Share on other sites More sharing options...
Moose83 Posted November 13, 2012 Share Posted November 13, 2012 1,3 with 2 ssd is no problem with rst, and 4 ssd can score 2g sometimes. Link to comment Share on other sites More sharing options...
knopflerbruce Posted November 13, 2012 Share Posted November 13, 2012 your dead right on this one, but i think the key difference is those things are breaking fundamental hwbot rules, where as RST has been accepted in the past, i think everyone needs to understand there is no rule change here, its simply rule enforcement and in some tricky cases where things might fall between rules, clarification of rules take this sector size for example, by changing the sector size you are tricking the benchmark into giving a score thats not real, changing the percieved speed of the benchmark This is true, but perhaps it's a good idea to make a newspost about these tweaks that we already know will be forbidden - instead of waiting for ALL the clarifications? It puts the benchers in a weird situation where nobody knows what we accept or not. Link to comment Share on other sites More sharing options...
SteveRo Posted November 13, 2012 Share Posted November 13, 2012 I think a progress newspost might be good. Link to comment Share on other sites More sharing options...
Moose83 Posted November 13, 2012 Share Posted November 13, 2012 I think a progress newspost might be good. Agree Steve;) By now: RST/Raid Expert allowed but unter discuss. Sector Size Change not allowed by now also unter discuss. Many new submits still do not show all subtests. . Link to comment Share on other sites More sharing options...
SteveRo Posted November 14, 2012 Share Posted November 14, 2012 (edited) ... breaking fundamental hwbot rules ... take this sector size for example, by changing the sector size you are tricking the benchmark into giving a score thats not real, changing the percieved speed of the benchmark Mr. Pro, Can we at least get an official read on removing scores that are "tricking the benchmark into giving a score thats not real". Edited November 14, 2012 by SteveRo Link to comment Share on other sites More sharing options...
IanCutress Posted November 14, 2012 Share Posted November 14, 2012 Half the argument was about tweaks that could easily beat $5000 storage setups. So why allow a software solution like RST with features that can easily beat $5000 storage setups? I say disallow *until* we are told otherwise. (Personally I'm fed up with 2 m4 SSDs attached to the ICH beating 2 acards when clearly the Acards should be ahead) Link to comment Share on other sites More sharing options...
Strat Posted November 14, 2012 Share Posted November 14, 2012 I agree. Strat gets: HDD - Virus Scan 1310.47 MB/s for using only two SSDs on ICH. I couldn't even get that using two ACARDs (http://hwbot.org/submission/2327525_borandi_pcmark_2005_core_i7_3960x_43966_marks), so something isn't right here. Hi all Just to clarify the situation, I only used Write back cache on Intel RST + very clean SSDs that's all folks! Link to comment Share on other sites More sharing options...
Recommended Posts