3DMark Steel Nomad

[Seby]

I made a tiny program to do this for you, just put the saved IFR on top of it. I'm a noob at programming so hopefully it doesn't break too easily.

ECCfixer.exe

[Mr. Fox]

On 5/28/2024 at 5:38 PM, Seby said:

nvflash64.exe --save oldifr.ifr

open file in hex editor, set everything past offset 0x5568 to just 00s. 

save as whatever.ifr

nvflash64.exe --flashinforom whatever.ifr

Thank you. 👍 Really appreciate it.

  

On 5/28/2024 at 7:50 PM, Seby said:

I made a tiny program to do this for you, just put the saved IFR on top of it. I'm a noob at programming so hopefully it doesn't break too easily.

ECCfixer.exe 96 kB · 5 downloads

That seemed to work fine.

 

Edited May 30 by Mr. Fox

[FM_Jarnis]

You are right in that the API used should be more visible on the result screen. We'll improve that in an update. Trivial issue, did not consider the importance for this use case.

As for the runs with corrupted rendering. This is partially "thanks nvidia"-level issue with 40-series, but we are indeed working on a solution. First part of this is now ready internally (in our admin interface) and we can trivially easy see these now manually. If you think a result is bogus, send a link to our support address and we can check and invalidate if needed.

We cleaned up stuff from the top of the hall of fame this morning (invalidating results that are obviously not rendered correctly) - if we missed something, let us know and we'll check.

We are also working on a fully automated solution for future submissions based on the same mechanic we currently have. This will take some time - implement, then test it in shadow mode to make sure we do not suddenly start rejecting legit results - but we will definitely do this.

I do agree that in the push to get this test out, result validation side did not get the attention it deserves. We are right now working on fixing that. And once fixed for Steel Nomad, we'll go back to implement the same thing for older tests. 

 

[FM_Jarnis]

On 5/25/2024 at 4:48 PM, Papusan said:

Why not put in a CPU test same as for 3DM Time Spy and older benchmarks?

This is bit of a design question.

Old tests tried to give you a "overall system performance" score. Which involved the CPU - even if the weight in overall score was small. 

But that was a major problem when people kept using overall score as a way to compare their GPUs, which of course caused issues. "Why is my 3090 so much lower score than your 3090" when the guys had wholly different CPUs. Some understood why, some declared the benchmark unreliable. People just didn't understand to use the graphics score which was the correct way to do that comparison. There were also complaints from OC people who didn't like that they had to OC the CPU for competing on a GPU overclocking HoF since overall score was used.

Current design is that the tests are isolated. GPU test will measure your GPU perf. If the test ever becomes meaningfully affected by your CPU (ignoring completely lopsided stuff like "hey lets put 4090 to this 10 year old dual core celeron system, yeah that makes sense") that is a sign that the test is getting too light and needs replacing soon.

And CPU test (CPU profile) will test your CPU and only your CPU. Zero effect from the GPU. 

We've internally floated and idea of offering some kind of "batch run" feature where you could choose to add a CPU test to a GPU test or even to multiple GPU tests to get some kind of combined result as a separate run mode, but this is at this point just an idea. We have not yet done any work on this and have made no decisions. Not promising anything and do not expect anything anytime soon, we have a long list of things on our plate. May never happen and not even sure how this would somehow interact with result search or Hall of Fame either (which is frankly suffering from "list bloat" - too many different tests, too many lists).

But we are open to feedback, so if someone has grand ideas how to do such a thing (as an addition to the current isolated tests), we are listening. Just have to consider that we are doing these benchmarks to a number of different audiences - normal "gamers", casual tweakers, extreme overclockers, business users, GPU manufacturers... - and sometimes they have conflicting needs and sometimes a thing that makes sense for one user just won't work at all for some other user. 

[FM_Jarnis]

Saw the rules for Steel Nomad on HWbot and I would strongly recommend requiring SystemInfo 5.73 or later.

Valid Steel Nomad submits will require this on 3dmark.com sometime next week.

[Mr. Fox]

On 5/30/2024 at 4:27 AM, FM_Jarnis said:

But we are open to feedback, so if someone has grand ideas how to do such a thing (as an addition to the current isolated tests), we are listening.

I'd really like to see something in Systeminfo that identifies GPU benchmark runs as "Stock" and "Overclocked" based on NVIDIA and AMD reference boost clock, with a corresponding filter available for search results. The actual boost clock will vary with temperatures, as will the boost clock of "OC" versions of non-reference designs, but the base boost clock will remain constant regardless of the actual boost clock on reference cards. The moment you overclock the GPU even a little bit, the base clock value will change. You can see this in GPU-Z. Once that is in place you can create a filter based on stock versus overclock and exclude anything from HOF ranking that isn't an overclocked submission. Conversely there would be times that it would be very useful knowledge to know what stock performance should look like based on averages. You could also more easily quantify how things like CPU clock, memory clock and BCLK, and OS tweaks, affect benchmark scores with stock GPU runs.

It would also be nice to see a filter added to search results based on GPU brands and models as identified by the firmware. This data is already captured, so why not include those as options in the filtering menus for selection?

Because so many of the 3DMark submissions are overclocked systems, which all of us here absolutely love doing, the UL database has little or no value in terms of identifying, with any level of accuracy, what appropriate stock performance should look like and thus it becomes more difficult to evaluate (from that data UL is compiling).

(Note: I am speaking of UL/3DMark search results in my comments above and not HWBOT.)

Edited June 1 by Mr. Fox

[FM_Jarnis]

The issue for filtering for stock results is that it has to work across all cards from all vendors and pretty deep into history. I'm not sure what the status with that is if you go just by GPU-Z data. I know this was the issue last time we looked into this (which, I admit was some years ago) and rejected doing it due to lack of reliable automated way.

We do already offer searching by GPU core clock range, is there some issue there that makes that unworkable? I mean, I understand it is an extra hassle to figure out values to enter there. And yes, that is the actual recorded core clock, not what it says on the spec sheet. Wonderful world of "it depends" clock speeds of modern hardware

The main issue is, we are bit afraid of taking a stance ourselves placing a label "stock" on results because that is potentially a can of worms - if the label cannot be applied reliably, it could create a huge mess.

GPU brand based search is on our list, so that may happen at some point. Can't promise it, but that is definitely doable.

And on the database quality - average score (visible at the top of the page on search) is still quite representative of the real peformance of non-overclocked cards. It might slightly overstate it for super popular "competitive" cards like RTX 4090 right now that has more than usual volume of seriously overclocked results, but the vast majority of all submits are still from systems that have not been modified in any way.

Checking average score for 4090 from Speed Way with a 12900K, average is 10052. Then did a bit of a search online for what a review of a Founders Edition 4090 card reported for Speed Way and a review using 12900K said 9916 (ServeTheHome review, March 12 2023). I'd venture a guess that this 136 points is a fairly good value for the potential bias from overclocking and factory OC cards together. Roughly 1.3%. This is within the margin of error of 3DMark runs (officially 3%, even if in practice it is closer to 1%). And this is with a CPU model that was popular when overclocking 4090s early on, and without caring one bit about potential gains from driver improvements as the score from the review is from over an year ago. 

The nice bell curve also shows we have plenty of systems that are overheating, have faulty cooling (either due to things like badly applied paste or due to bad system design, hello throttling prebuilts!) which do average out the mega-overclocked ones. 

Only case where average score is probably not a good indicator of stock is if you find a very rare hardware combination with very few submits (say, less than a few hundred) and there happens to be unusually high percentage of results from an overclocker. And even there you can probably sidestep the issue by just using a similar but more common CPU model for the search to get a larger pool of results.

Edited June 2 by FM_Jarnis

[Leeghoofd]

We adapted the Vulkan, DX12 ruling for the Steel Nomad verification screenshot. Removing the User Setting from the verification screenshot specifics
 

Currently it is a mess, as users don't read up, or don't have the resolution to do so and not sure how fast UL will do anything about their GUI layout...


Spread the word