Petition for a verification of all scores. Why are scores that easy to submit?

[Max Silencio]

I cannot understand why there are scores here without any verification.

So scores can easily be sophisticated and it makes no sense for the honest to post them.

I've seen submissions here which have the same hardware and clocks like me but with extremely higher scores. And there's no 3dMark url or cpu-z screen which shows there real clocks. So the ranking is in that respect pointless.

Or did I misread this ranking system?

 

How do you think about this?

[thebanik]

I cannot understand why there are scores here without any verification.

So scores can easily be sophisticated and it makes no sense for the honest to post them.

I've seen submissions here which have the same hardware and clocks like me but with extremely higher scores. And there's no 3dMark url or cpu-z screen which shows there real clocks. So the ranking is in that respect pointless.

Or did I misread this ranking system?

 

How do you think about this?

 

I doubt that its true. If you do see any submission which is missing cpu-z or gpu-z (for 3d), do report them and the moderators would surely take care.....

[Max Silencio]

But for 3dMark11 there are submissions without an url. So they are not proved.

One example:

http://hwbot.org/submission/2268692_pixy_3dmark11___performance_geforce_gtx_680_13351_marks

 

Here's no 3dMark url. Isn't this mandatory?

 

http://hwbot.org/benchmark/3dmark11_-_performance/rankings?hardwareTypeId=videocard_1913#start=0#interval=20

 

Just view rank 2th to 7th in the GTX680 3dMarks11 ranking (29 submissions): There is no verification url.

As well as 12th, 13th, 15th, 17th, 18th, 19th, 22, 23, 24, 25, 26, 27

Edited April 13, 2012 by Max Silencio

[K404]

The current rules say that only top 20 GLOBAL need verification. What is so special about these scores that they need extra verification? Good for the hardware, yes..... but not world-changing.

 

 

 

BUT..... maybe it would be interesting to require validation for the top 20 scores of each top-end card, competitive or not

[thebanik]

But for 3dMark11 there are submissions without an url. So they are not proved.

One example:

http://hwbot.org/submission/2268692_pixy_3dmark11___performance_geforce_gtx_680_13351_marks

 

Here's no 3dMark url. Isn't this mandatory?

 

http://hwbot.org/benchmark/3dmark11_-_performance/rankings?hardwareTypeId=videocard_1913#start=0#interval=20

 

Just view rank 2th to 7th in the GTX680 3dMarks11 ranking (29 submissions): There is no verification url.

As well as 12th, 13th, 15th, 17th, 18th, 19th, 22, 23, 24, 25, 26, 27

 

 

You mentioned that you see scores with missing url as well as cpu-z whereas all the scores have GPUz as well as CPUz screen as mentioned in the rules. And the scores are well in line to what is expected. If you doubt any score, again report and the mods with their experience will be able to judge whether anything fishy is going or not and will take appropriate action.

 

I've seen submissions here which have the same hardware and clocks like me but with extremely higher scores. And there's no 3dMark url or cpu-z screen which shows there real clocks

[Max Silencio]

Sorry, but I think this goes into a wrong direction as my intention is not to doubt the scores generally. I just would like to propose a mandatory 3dMark11-url-rule. It's that stupid easy to arrange cpu-z, gpu-z and the small 3dMark11 result window screenshots side by side with no reliability that these are related to the submitted scores.

But if I'd misunderstood the rules please clear me up.

And the gpu clocks are normally already back to stock when snapshotted. Some cpu-z and gpu-z screens are in a very low resolution which makes it sometimes hard to read the numbers at all.

 

 

@forum mods

So I may extend my question to the forum mods here.

What are your reliability notions and criterias for the submissions?

Edited April 13, 2012 by Max Silencio

[K404]

It's that stupid easy to arrange cpu-z, gpu-z and the small 3dMark11 result window screenshots side by side with no reliability

 

If someone was determined to cheat then yes, what you say is true, but those software windows are "all we have" to decide if a score is legit or not.

 

Futuremark .3dr files can be hacked as well apparently.

[knopflerbruce]

I don't think asking for ORB link to all top 20 scores in each HW category is something we should do. These licenses are not free, and adding this rule to the categories with 50 points for 1st place also means we have to do the same for the ones where the top score has 2 points, which is way overkill. Most of the sers post here because it's free. If you need a license in most cases, then it's not free anymore.

[FM_Jarnis]

If someone was determined to cheat then yes, what you say is true, but those software windows are "all we have" to decide if a score is legit or not.

 

Futuremark .3dr files can be hacked as well apparently.

 

Confirming this, but 3DMark.com will barf if you try to submit them. So all you really can do is to have a 3dr file that can be opened in a benchmark but you still can't submit it.

 

We have fairly recently hardened SystemInfo against result file tampering but like before, all the validation is server side. There is no way to do client side validation that isn't hackable, so we really aren't even trying to handle that. In other words, a result file that opens and shows you a number is NOT proof that the run was valid.

 

Result file that opens and submits to 3dmark.com without the site throwing a fit about the file, however, is considered proof of a valid run. Any cases where you think this is not so, we are very very interested of hearing about (including example files etc.). Feel free to send any to info [at] futuremark.com with details about the problem. And yes, people have found loopholes in the past and we're committed into plugging them as they come up. Granted, we have no illusion about bulletproof security - it really isn't possible without something like "bundle Punkbuster with benchmarks" and that would really be overkill. But we have worked towards blocking any casual result file editing and the system should be able to detect tampering like that - but only on 3dmark.com submit because it is all server side.

Edited April 13, 2012 by FM_Jarnis

[FM_Jarnis]

I don't think asking for ORB link to all top 20 scores in each HW category is something we should do. These licenses are not free, and adding this rule to the categories with 50 points for 1st place also means we have to do the same for the ones where the top score has 2 points, which is way overkill. Most of the sers post here because it's free. If you need a license in most cases, then it's not free anymore.

 

You can have a (single) result in 3dmark.com at no cost with basic edition.

 

You also can have any number of results from any benchmarks in 3dmark.com as long as you have one valid license for any benchmark on your account. In other words, if you have, say, advanced 3DMark 11, you can submit as many results as you want with basic edition of Vantage or 06 - just a single paid license turns your 3dmark.com account into fully featured one. Considering for example how many Vantage and 11 keys have been bundled with video cards, this shouldn't be a huge issue... I'm sure anyone serious about latest hardware probably has one or two of those keys just lying around

 

The only use case that really "fails" on the submit front is usage of pirated/keygenned 3dmark advanced/pro keys to get pretty screenshots of the score - you can't submit those to 3dmark.com since the run was not done with a valid key.

 

Of course in theory this should be a non issue - you already have to show the result in a screenshot for hwbot, so you need a non-basic version for that on all benchmarks - basic edition shows the score only on the website. If you have that screenshot showing the result in 3DMark UI but "cannot submit to 3dmark.com because licenses are not free", then you are basically stating that you are using a pirated/keygenned 3dmark. It is up to the community to decide how "okay" that is - we know the realities of pirate use (heck, we have plenty of stats showing how many people attempt to submit with pirated copies every day) and have no illusions about everyone suddenly rushing to buy the application(s). We just won't provide the online service for illegitimate keys.

Edited April 13, 2012 by FM_Jarnis

[Max Silencio]

You can have a (single) result in 3dmark.com at no cost with basic edition.

 

You also can have any number of results from any benchmarks in 3dmark.com as long as you have one valid license for any benchmark on your account. In other words, if you have, say, advanced 3DMark 11, you can submit as many results as you want with basic edition of Vantage or 06 - just a single paid license turns your 3dmark.com account into fully featured one. Considering for example how many Vantage and 11 keys have been bundled with video cards, this shouldn't be a huge issue... I'm sure anyone serious about latest hardware probably has one or two of those keys just lying around

 

The only use case that really "fails" on the submit front is usage of pirated/keygenned 3dmark advanced/pro keys to get pretty screenshots of the score - you can't submit those to 3dmark.com since the run was not done with a valid key.

 

Of course in theory this should be a non issue - you already have to show the result in a screenshot for hwbot, so you need a non-basic version for that on all benchmarks - basic edition shows the score only on the website. If you have that screenshot showing the result in 3DMark UI but "cannot submit to 3dmark.com because licenses are not free", then you are basically stating that you are using a pirated/keygenned 3dmark. It is up to the community to decide how "okay" that is - we know the realities of pirate use (heck, we have plenty of stats showing how many people attempt to submit with pirated copies every day) and have no illusions about everyone suddenly rushing to buy the application(s). We just won't provide the online service for illegitimate keys.

 

+1

That's a powerful argument. So there's no substantial reason not to post a result link for validation. And it should not be that exhausting to copy and past one link which 3dMark11 automatically offers after benchmarking.

[I.M.O.G.]

Thats pretty much the problem I believe. A lot of people on hwbot use illegitimate keys.

 

Without using a key, it is slower to find out your result - often times the benching machine does not have networking installed, so you transfer the result file off, and upload from another machine in order to see your result. If you have a key, it is much faster and easier to tweak and get runs in. For PCM05, it is critical to have a key to change the destination drive for storage tests in order to get top scores.

 

I think its hard for hwbot, because they don't want the burden of enforcing the use of legitimate keys, and they don't want to make benchmarking less accessible.

 

Futuremark makes a strong point though. Only one key need be purchased really, then anyone can submit as many things as possible, and everyone is already posting screenshots of their scores meaning they are using a key already.

 

Dark side of overclocking and benchmarking - pirated keys and RMA abuse. :-/ Despite it, the industry has come around from disliking and discouraging overclocking, to promoting their products through it. Interesting to me the way things have changed over time.

[knopflerbruce]

+1

That's a powerful argument. So there's no substantial reason not to post a result link for validation. And it should not be that exhausting to copy and past one link which 3dMark11 automatically offers after benchmarking.

 

It's better than I thought, but not ideal anyway. What's sad is that so many free keys are thrown away. maybe we should collect them here and give them away for free. Then you get both ORB funcionality with legit keys, and the cost is zero:) THEN it would be easier to do this.

 

I also suggest that FM benchmarks that are retired are available as pro versions for free. At the moment you get no ORB funcionality, but you still pay $15 for a 3dmark03 key... that makes no sense.

[FM_Jarnis]

3DMark 03 key should give you ORB functionality. 03, 05, 06, Vantage and 11 are currently supported. Toss me a PM with details if you are having issues. 03 was gone temporarily when old ORB was replaced by 3dmark.com but it was re-implemented based on community feedback. Only 3DMark 2001 (SE) was retired for good - and even that was mostly because we had no other choice, the codebase for that was just simply too old.

 

Granted, we are currently contemplating on potential retirement of 03 at some point but if we do so, sales of new keys will obviously cease before that and we'll give plenty of warning. It is almost ten years old and all that...

[FM_Jarnis]

[double post]

Edited April 16, 2012 by FM_Jarnis
"oops"

[knopflerbruce]

OK:p my bad

[Vinster]

It's better than I thought, but not ideal anyway. What's sad is that so many free keys are thrown away. maybe we should collect them here and give them away for free. Then you get both ORB funcionality with legit keys, and the cost is zero:) THEN it would be easier to do this.

 

I also suggest that FM benchmarks that are retired are available as pro versions for free. At the moment you get no ORB funcionality, but you still pay $15 for a 3dmark03 key... that makes no sense.

 

I have one that relates to this, what about PCMark04? it's still a valid bench for older CPU's (which is all I have/get) but the ORB no longer accepts an online submission. How do I get that bench to display my score so I can get atleast screenshot?