Jump to content
HWBOT Community Forums

pro

Crew
 View Profile  See their activity

  • Posts

    1099

  • Joined

  • Last visited

  • Days Won

    5

 Content Type 

Posts posted by pro

    Clarification in PCMark'05 about Tweaks // Tricks allowed

    in Benchmark software

    Posted

    Thanks for your input! I agree with most of what you say, but there are conflicting statements that I would like to clear:

     

    Much like PowerToy, all F2D does is change the registry. Albeit on the fly like other additional tools you mention.

     

    But *anything that changes the size of the windows is illegal*. EVEN IF it can be done by the registry.

     

     

    registry tweaks are always allowed by default, thats the general unwritten rule, but some have been deemed to be disallowed over the years because they decieve the benchmark too much, such as mipmap

     

    i think we need to be understanding here, there can be no clear cut, one size fits all rule for benchmarking, there will always be anomolies,

     

    lets take LOD for example, registry based or file based in the latest nvidia drivers, it DOES impact the visuals of the benchmark and does impact the end score, but it has been universally acccepted as OK,

     

    this transparent window test, the whole point of the test is to see how fast those windows move across the screen, so by resizing or minimizing the windows you are decieving the benchmark and it does have a significant impact on the score, hence this is disallowed

     

    i think we need to keep in perspective here, like any "sport", rules arnt just there to set a solid line that can NEVER be changed, rules are also there to be changed over the lifetime of the sport to keep things fair for all involved,

     

    mate you have proven my point, by making that list of what is allowed/disallowed and answering nearly everything correctly,

     

    most benchers are smart, they can work out if something is allowed in the benchmark based on the two key rules, if not, we will be here to answer grey areas, but please don't expect instant answers, we/i want to do this right and at times that will mean lengthy discussions between hwbot staff

    Clarification in PCMark'05 about Tweaks // Tricks allowed

    in Benchmark software

    Posted

    Steve what you were doing was the same as me and many others but theres no doubt that it's secirving the benchmark into giving a higher score by minimizing the windows

     

    I know for a fact that there are much nastier TW window tweaks but it is outside hwbot general rules do it must be removed

     

    Just do you guys know we will target the big obvious things first and progressively make our way down, but if you feel your own score is out of line save us the trouble and remove yourself

    Clarification in PCMark'05 about Tweaks // Tricks allowed

    in Benchmark software

    Posted

    If you know AMD raid expert is wrong, why do you continue to submit scores with it? Just because massman once said it was OK? Currently, everything is under speculation and you've been saying it since the team cup that you don't agree with it. How about having some principles and run the bench according to what you feel is right, instead of following the heard.

     

    yeah i think ethics goes a long way to resolving this

    Clarification in PCMark'05 about Tweaks // Tricks allowed

    in Benchmark software

    Posted

    okay guys let me weigh in on this,

     

    my position

     

    firstly im waiting for all my scores to be removed and i will not submit pcmark scores to hwbot anymore, most of my scores feature totally legitimate tweaks, some scores have things like transparent windows with resized windows that we have worked out is not allowed. the main reason i am removing my scores is so i can stay objective in decision making and no one thinks that i am using their tweaks against them... and just to clarify im not using any tweaks that are replacing benchmark files, so if your doing that, your doing it wrong ;)

     

    ill always bench pcm05 privately though as i love the benchmark

     

     

    why keep pcm05?

     

    for those that are saying "get rid of pcm05", we are basically at that point and i am sure if this doesnt work then that is what will happen, but there are lots of reasons to keep this benchmark, for example,

     

    we have guys like Steve and Aristides that are always strong in this bench and are getting beaten by tweaks that they are not comfortable using and rightly so, this benchmark has a place and is a source of enjoyment for lots of people, so i believe we should have a proper effort to try and fix it up before abandoning it

     

     

    please be patient

     

    now guys i see lots of people saying, whys this not banned, why is mine banned and not his, etc, please be patient, this will takes weeks, months, and will be ongoing, but it is no different than any other benchmark, other than the fact it has gone unchecked for so long.. if this happened in 3dmark11 it would be immediately checked and in the future pcmark05 will be the same

     

     

    what is allowed and whats not?

     

    lets get something cleared up, a cheat is not a super-tweak, i WOULD call bobs TW a supertweak, i would call replacing codecs in a benchmark cheating, if it pleases you, then call it a super-cheat, whatever, but if its outside the rules its not a tweak, and i am just as much to blame for things like TW window resizing as anyone else benching PCM,

     

    now lets get onto the juicy stuff, whats allowed and whats not, this is extremely simple, lets go back to two very basic hwbot rules,

     

    1. "Using software, performing hardware modifications or by human interaction altering the perceived speed of the benchmark program, tricking it to believe it ran faster and thus producing a better result. (adjusted 31st of August 2010)"

     

    2. not modifying the benchmark itself

     

    Yes d3d, video encoding via registry or powertoy and LOD go against these rules, BUT they have been used and accepted as officially OK, unless staff have specifically said something is allowed that goes against HWBOTs general rules then it is NOT allowed

     

    if what you are doing in pcmark05 goes against one of those 2 rules, remove it now. staff will be contacting users with out of line submissions asking for proof on how to replicate them within the rules, if you are worried you cant provide that proof, remove your score now or it will be removed for you

     

    use your brain guys, if you know something is not right, 2400MB/s virusscan on a weak AMD platform, then its probably not right, this is the ethical side of hwbot we can't enforce but hope is in place

     

    Key areas we are targeting to start with

     

    Some of the key areas we will be targeting, but not limited to:

     

    Video encoding replacement, audio encoding

    Irregularly large trans windows

    Irregularly large memory latency

    Massive browser scores

    Anything out of line with file decoding

    Crazy virusscans or hdd general

     

     

    What is OK

     

    I am not going to go through every tweak and say its ok or its not ok, thats what the general rules are for and they cover alot of the "super tweaks", but not all,

     

    FD2 as a program is totally allowed, but using it to manipulate the window size or minimize windows is not allowed, similarly using a script to do this is not allowed as it breaks the "benchmark percieved speed rule"

     

    BUT as with any benchmark, if you can do it via control panel such as personalization or registry then it SHOULD be allowed, but there are always exceptions to the rule such as MIPMAP and in the future there might be other exceptions, but this is the current position, there is no one rule fits all model

     

    so take TW for example, if you can make the windows really small and change the text type in control panel then that is legal,

     

    SCRIPTS ARE LEGAL, i use scripts in nearly every benchmark, but to do things like control cpu clock, change gpu clocks on the fly, and to change things like the drives for different harddrive during PCMARK, that is all OK

     

    Using a script to do something that goes against the general rules is NOT ok. For example replacing a file that the benchmark uses to make TE go faster, NOT OK.

     

    Things like codecs can be tricky, as microsoft may update them or another program installed for watching videos may influence the scores, this is not the case in PCM05,

     

    if you are putting codecs specially in the pcm05 directory or replacing codecs that pcm05 installs, this is definatly NOT OK, i think that is common sense

     

    also forcing pcm05 to look in a folder it doesnt normally so it uses your codec, not OK, again common sense,

     

    guys you get the drift right? if it breaks hwbot general rules then it is NOT OK

     

     

    Raid XPERT

     

    Still up in the air is raid xpert, we are discussing the outcome of this one right now, there and positives and negatives to both sides of the fence and we need to make sure everything is worked out properly

     

    Massman said it was allowed, but now that we are aware that their is some bugging going on, we are reviewing it.

     

    We don't want peoples expensive raid setups obsoleted by a bugged software,

     

     

     

    Fairness

     

    This is not a witch hunt, everyone will be given the oppurtunity to prove their tweak is fair in a confidential environment. If you tweak is legitimate, it won't be shared, it will be approved and you can continue using it privately if you so wish.

     

    If you choose to not share the tweak with staff and your score looks out of line then it will be removed.

     

    Lets hope we can clean the benchmark up once and for all, remember guys, please use your brain, we dont want to remove scores, so please do it yourself if you know they go against rules

×
×
  • Create New...