PCMark05 Tweaking Thread

[knopflerbruce]

explaining the tweak to a disinterested staff member that does not participate for points would be ideal as they are not "competition."

 

oddly enough I can see where Moose is coming from though. He has explained a tweak in private to a staff member, and that member used it for their own personal gain. The trust relationship between HWbot and the end user has been broken by the staff. If this had happened to me, I would be very reluctant to continue to explain a tweak.

 

 

 

The point is that we ALL understand that the staff has to moderate the scores somehow and that sometimes it involves revealing a tweak for judgement. That is fine, unavoidable, and expected in ANY type of competition. what is not right is that the staff participates for points, leaving no real 3rd party to truly moderate the scores while protecting the intellectual property of the end user.

 

give me a million, i will pull all 30 points i have and moderate all this biach...

 

 

I dont think we need a dedicated PCM05 moderator that doesn't participate in points. We need a dedicated moderator for ALL benchmarks that does not participate for points. By the current policy, the critical flaw still exists where any new legitimate tweak is not protected as it still has to revealed to staff members who participate in points and can use it for their own benefit... which has happened... and has not been addressed (as far as we know.)

 

It COULD be that CN got to know the tweaks by himself. However, as I've already said - it's probably easier to demand that the tweak is explained here, in a public section. Then no moderator has an edge compared to the other benchers (who can, if they wish, check this forum on a regular basis).

 

If you want a dedicated moderator that does not participate I think you have to dig deep to find enough. I'd consider it if I had alot less points and would get paid for the work I did, much like massman is doing now - getting an OK paycheck compared to the standards in the country he's living in (Belgian standard at least before he moved).

[zeneffect]

Funny thing is for the 2nd "way" Windows gives you the tool. No matter how much I work on the wrapper I still find ways to beat myself :\

 

 

 

Besides me and CN I dont recall any HWB staff using any of these tweaks. CN got it from me (after gluv already leaked it from me...) and I made it myself (good thing I keep Sent PMs). Care to explain?

 

this has already been touched upon in this thread. Moose shares tweak... moderator says its "ok" then uses the tweak themselves. Regardless, if CN got it from you after Gluv leaked it to you, he is then in violation of hwbot's ethical policy as he has now used intellectual property that was discussed in private for a ruling on legality. THIS is the problem, we can not have active benchers also play judge.

 

Knopflerbruce makes a valid point... it will be next to impossible to find a "voulenteer" that is trustworthy to not disseminate trusted information.

Edited October 18, 2012 by zeneffect

[Moose83]

GenieBen gives TE and WebPR to Christian, i only told him, how to do Audio Comp. I told him there are more ways to go, simply methode is replacing codec at the right time...but this dont work in every run with orb. And i didnt get any of these tweaks by GenieBen, i did it by my own. Maybee you didnt want, that staff use these tweaks, but they did;)

 

And again, Christian gives me ok for TE and WebPR:)

[GENiEBEN]

after Gluv leaked it to you

Went the other way around.

 

Thanks moose for clearing up!

[zeneffect]

Went the other way around.

 

Thanks moose for clearing up!

 

ah... thanks for the clarification. the wording was a little confusing... still does not deter from the fact that this example actually occurred (as stated by Moose.) Which subtest it does not matter, the simple fact that confidential information between the end user and hwbot was exploited (once again as stated from Moose. I dont even bench pcm05)

[der8auer]

Its hard work, to get such tweaks, it tooks many time. And then i should give it the staff, that staff can use it by own???

 

Feel free to explain all PCMark2005 tweaks to me. The problem is that I propably won't understand them as I never do any PC05

 

I dont think we need a dedicated PCM05 moderator that doesn't participate in points. We need a dedicated moderator for ALL benchmarks that does not participate for points. By the current policy, the critical flaw still exists where any new legitimate tweak is not protected as it still has to revealed to staff members who participate in points and can use it for their own benefit... which has happened... and has not been addressed (as far as we know.)

 

No

 

One reason that we are part of the staff is because we are experienced in benchmarking. Obviously we like benching as much as everybody else here and I won't give up points and everything just to be moderator

Don't forget that we're volunteers and don't get payed for it. And it's good that we don't get payed because I can decide the time to moderate. If I'm not in the mood - I just don't do anything.

[GENiEBEN]

Bad news for those skipping ORB error 10...

 

I've played more with fooling ORB into accepting modified dll's, it's all a matter of running the script at the proper time. Good news is that my wrapper is able to detect those files, so goodbye any method of fooling around with PCMark's files.

Now it's down to Windows files and registry, grab your tools, err, apps!

 

Custom zlib: http://www.3dmark.com/pcm05/3174110

Default zlib: http://www.3dmark.com/pcm05/3174112

[Moose83]

Then you must work again

I dont skipp this error, i havent this error and didnt need an script;)

[zeneffect]

Feel free to explain all PCMark2005 tweaks to me. The problem is that I propably won't understand them as I never do any PC05

 

 

 

No

 

One reason that we are part of the staff is because we are experienced in benchmarking. Obviously we like benching as much as everybody else here and I won't give up points and everything just to be moderator

Don't forget that we're volunteers and don't get payed for it. And it's good that we don't get payed because I can decide the time to moderate. If I'm not in the mood - I just don't do anything.

 

being a volunteer does not make one impartial though. you yourself said you actively bench. without an impartial judge or referee, how can there be any trust relationship between an end user and hwbot? sure, one could make it public so that staff would not have a clear advantage, but how fair is that? why should staff have a distinct and clear advantage if they are supposed to be competing at the same level with the same rules? I understand that staff has to be notified of tweaks that seem suspicious, but the current state of how it is moderated leaves no possibility for the protection of one's intellectual property other than relying on a person's integrity.

 

mabye full, open ended disclosure would also act as a viable solution? it would certainly level the playing field, but would make benching more dependent on pure hardware rather than a skilled bencher who knows how to push the hardware as well as tweak the system for maximum efficiency.

Edited October 18, 2012 by zeneffect

[GENiEBEN]

Then you must work again

I dont skipp this error, i havent this error and didnt need an script;)

 

At some point the original dll has to be replaced, no matter what your options to do it were.

We will see down the road, when I have a compiled downloadable wrapper you're getting the first run

 

being a volunteer does not make one impartial though.

 

The way I see it anything outside an error margin (lets say 5%) of what the wrapper finds is deemed questionable immediately. That brings us to the part where the wrapper knows what is legal and what is not.

 

Option 1. Score is removed, tweak not shared, no points and user monitored for repeated offences.

Option 2. Score is allowed, tweak is verified for legitimacy, points awarded, wrapper updated to allow it or not.

 

No matter who you choose as 3rd party, it is still down to persons integrity, so it may aswell be HWBOT's staff. Any other takes on this?

/rant

 

I think I should dedicate more spare time to writing our own System Benchmark than trying to make a wrapper for an obviously poorly written benchmark (yes FM, you suck..or at least did in 2004).

Edited October 18, 2012 by GENiEBEN

[zeneffect]

At some point the original dll has to be replaced, no matter what your options to do it were.

We will see down the road, when I have a compiled downloadable wrapper you're getting the first run

 

 

 

The way I see it anything outside an error margin (lets say 5%) of what the wrapper finds is deemed questionable immediately. That brings us to the part where the wrapper knows what is legal and what is not.

 

Option 1. Score is removed, tweak not shared, no points and user monitored for repeated offences.

Option 2. Score is allowed, tweak is verified for legitimacy, points awarded, wrapper updated to allow it or not.

 

No matter who you choose as 3rd party, it is still down to persons integrity, so it may aswell be HWBOT's staff. Any other takes on this?

/rant

 

I think I should dedicate more spare time to writing our own System Benchmark than trying to make a wrapper for an obviously poorly written benchmark (yes FM, you suck..or at least did in 2004).

 

when the 3rd party has nothing to gain from the disclosed tweak, then the level of trust goes up. Reliance on a wrapper and the determination of if the tweak is legal or not by active benchers still does not resolve the doubt that confidentiality between hwbot and the end user is maintained as there IS an incentive to use XXX tweak after it has been disclosed privately to the staff. there is no way anybody can claim that they are a disinterested party if they are actively benching for points in the same arena as those who are being questioned.

 

its easier for most all other benchmarks with the exception of PCM05 as the trend is to develop a tweak and use it to one's own advantage. once disclosed to staff, the advantage is lost as they also bench (based upon the assumption that a new legal tweak is found.)

 

/end run-on sentences

Edited October 18, 2012 by zeneffect

[Sweet]

I think I should dedicate more spare time to writing our own System Benchmark

 

Yes, fully agree, but it takes time and money, something complicated

 

when the 3rd party has nothing to gain from the disclosed tweak, then the level of trust goes up. Reliance on a wrapper and the determination of if the tweak is legal or not by active benchers still does not resolve the doubt that confidentiality between hwbot and the end user is maintained as there IS an incentive to use XXX tweak after it has been disclosed privately to the staff. there is no way anybody can claim that they are a disinterested party if they are actively benching for points in the same arena as those who are being questioned.

 

its easier for most all other benchmarks with the exception of PCM05 as the trend is to develop a tweak and use it to one's own advantage. once disclosed to staff, the advantage is lost as they also bench (based upon the assumption that a new legal tweak is found.)

 

/end run-on sentences

 

Pro, also known yougpro, will be a moderator direct (or better judge) for PCMark'05, is one of the most knowledgeable people of this benchmark, off course the rest of the moderators we will help

[der8auer]

being a volunteer does not make one impartial though. you yourself said you actively bench. without an impartial judge or referee, how can there be any trust relationship between an end user and hwbot? sure, one could make it public so that staff would not have a clear advantage, but how fair is that? why should staff have a distinct and clear advantage if they are supposed to be competing at the same level with the same rules? I understand that staff has to be notified of tweaks that seem suspicious, but the current state of how it is moderated leaves no possibility for the protection of one's intellectual property other than relying on a person's integrity.

 

mabye full, open ended disclosure would also act as a viable solution? it would certainly level the playing field, but would make benching more dependent on pure hardware rather than a skilled bencher who knows how to push the hardware as well as tweak the system for maximum efficiency.

 

I understand your point but actually we never had issues with impartiality in the last years. PCMark2005 is different to every other benchmark. Normally a result gets reportet and in 99% it's clear whether to block or check the result.

Now PCM05 comes up with unique tweaks we have to ckeck and test. That's actually the problem. If you tweak any other benchmark you might get 2-5% higher score. If we see a good run we say "hey - nice performance" and don't have to know each tweak.

You guys are tweaking PCMark more than 100%. Obviously this is completely different and needs a different approach.

[zeneffect]

I understand your point but actually we never had issues with impartiality in the last years. PCMark2005 is different to every other benchmark. Normally a result gets reportet and in 99% it's clear whether to block or check the result.

Now PCM05 comes up with unique tweaks we have to ckeck and test. That's actually the problem. If you tweak any other benchmark you might get 2-5% higher score. If we see a good run we say "hey - nice performance" and don't have to know each tweak.

You guys are tweaking PCMark more than 100%. Obviously this is completely different and needs a different approach.

 

i think this stems to how pcmark is played. everybody keeps the tweaks close to home and does not really want to share unless they gain something in return. this (i think) adds another dimension to benching which is pretty cool. i never had any problems with impartiality personally as every tweak i use has been well documented and is used by countless others. pcmark... COMPLETELY different animal.

 

Pro as the moderator for pcm05 is an excellent choice as he is retired from PCM05 and thus an impartial judge, as well as having direct pcm05 experience with the integrity and respect behind him to back up his ruling/decisions.

 

now that THAT has been clarified.... how are you going to moderate previous scores when it is well known that "there is more than one way to skin a cat" (not sure why anybody skins cats...) a perfect example is TW. Moose knows what im talking about.... flash windows isnt the only way to do it hell you dont even need to resize the anything to get the same boosted score. oddly enough, Pro already knows the other tweak im referring to as well as I told him before I told my own team... months and months ago. (yes ive figured out the TE and WPR replacement method a long time ago but passed on it as it is an invalid exploit in my opinion. Id rather not distribute information like this as misuse could do the team more harm than good.)

 

rabblerabblerabble

Edited October 18, 2012 by zeneffect

[Moose83]

i think this stems to how pcmark is played. everybody keeps the tweaks close to home and does not really want to share unless they gain something in return. this (i think) adds another dimension to benching which is pretty cool. i never had any problems with impartiality personally as every tweak i use has been well documented and is used by countless others. pcmark... COMPLETELY different animal.

 

Pro as the moderator for pcm05 is an excellent choice as he is retired from PCM05 and thus an impartial judge, as well as having direct pcm05 experience with the integrity and respect behind him to back up his ruling/decisions.

 

now that THAT has been clarified.... how are you going to moderate previous scores when it is well known that "there is more than one way to skin a cat" (not sure why anybody skins cats...) a perfect example is TW. Moose knows what im talking about.... flash windows isnt the only way to do it hell you dont even need to resize the anything to get the same boosted score. oddly enough, Pro already knows the other tweak im referring to as well as I told him before I told my own team... months and months ago. (yes ive figured out the TE and WPR replacement method a long time ago but passed on it as it is an invalid exploit in my opinion. Id rather not distribute information like this as misuse could do the team more harm than good.)

 

rabblerabblerabble

 

Hey, mh, thats actually funny Trading tweaks is an new area in this Bench:)

I did so also, me like

Edited October 18, 2012 by Moose83

[Moose83]

I understand your point but actually we never had issues with impartiality in the last years. PCMark2005 is different to every other benchmark. Normally a result gets reportet and in 99% it's clear whether to block or check the result.

Now PCM05 comes up with unique tweaks we have to ckeck and test. That's actually the problem. If you tweak any other benchmark you might get 2-5% higher score. If we see a good run we say "hey - nice performance" and don't have to know each tweak.

You guys are tweaking PCMark more than 100%. Obviously this is completely different and needs a different approach.

 

Not exactly right Roman, we tweaked it more than 300%

[zeneffect]

Hey, mh, thats actually funny Trading tweaks is an new area in this Bench:)

I did so also, me like

 

trading tweaks is half the fun of this bench, either to do or to watch the progression of scores and trying to figure out who has what tweak.

[Sweet]

Hey, mh, thats actually funny Trading tweaks is an new area in this Bench

I did so also, me like

 

You Moose better not change (or trade) anything with anyone

 

 

I read the news today oh, boy

About a lucky man who made the grade

Edited October 18, 2012 by Sweet

[AUST DeVinE]

nice knowing you PCmark05

 

may you RIP

[sheffline_85]

Why are these tears and saliva? remove and ban cheaters results, first for 2 years. Why when a person kills someone, it does not try to justify the mass? Impunity creates lawlessness. If someone disagrees fundamentally is a cheater, you are welcome to my house, the descent will not)))

[knopflerbruce]

There is a difference in type of cheating. Should we ban people when they use Physx as well? Hey, I'd be happy to ban you for 2 years if you have the wrong tab open in CPUZ let me know if you think it's OK and I'll check some of your results for these things.

 

This was done in good faith, although common sense should've kicked in and made people realize these tweaks really couldn't be legit because of the crazy boosts.

[Mr.Scott]

So.....when exactly are these BS scores to be blocked/removed? I see more piling up every day.

[knopflerbruce]

Whenever a newspost pops up. I'm not the guy writing it, but I guess massman and/or pro should have an idea.

 

I'll start pulling scores on thursday afternoon my time if nothing has happened until then, got some assignments at uni next week so that seems like a very nice time to do some work.

[Sweet]

I see more piling up every day.

 

 

Imo., Meanwhile notification arrives. members who have used the benchmark cheats like modify files either as replacement or modification of files (temporary or not), themselves could go erasing these submissions that are cheats.

 

By the way, Those have many submissiones with cheats, at time of communication, I think proposing punishments, because as Mr.Scott written and others, there are benchers who are already aware that they are cheats and continue to use to this day. 21-10-2012

 

So...They are NO benchers, are Cheats. Sorry but I think that

Edited October 22, 2012 by Sweet

[I.M.O.G.]

nice knowing you PCmark05

 

may you RIP

 

I've heard this a lot the past 12 months, as interest grew, "tweaks" exploded, then things got kind of off the chain and only a handful of people bothered making competitive submissions, while the rest turned away.

 

Doubt its dead. If policed in a way where people understand the rules, and believe they are enforced/enforceable, people will keep running it.

 

Edit: I can't really tell what has been ruled as invalid lately. Interested in hearing what has changed.

Edited October 22, 2012 by I.M.O.G.