A new way to address CPUs (Cores/Threads) on HWBot

[Splave]

If you call it a 8 core its is fighting against 5800x and 11900k IMO that doesnt seem fair at all.

If you were to pickup the phone and say "hello MR.Intel, how many cores does a 12900k have?" they would say it has 16.  
 

In my opinion its a 16 core processor, this is due to the fact that it has 16 cores.  We need to not dilute the rankings by catering to hopeful companies to claim PR victories.

[GtiJason]

Just to play devils advocate, was the FX 9590 put into 8 core because that's what AMD called it ? What if they had called it a 4 core with SMT, It competed pretty well against the 4770k ? Benchmarks that AMD is usually good at like Cinebench put these 8 cores on par with 4770k when maxxed out on Lin. So were any of these cpu's capable of getting Global First Place back when they were new. I'd imagine CPU Frequency would of been the only one with GpuPi the only other possibility.
I'm not saying to have it one way or not, just want to make sure we are considering everything before making a decision we may regret in the future

[Splave]

47 minutes ago, GtiJason said:

Just to play devils advocate, was the FX 9590 put into 8 core because that's what AMD called it ? What if they had called it a 4 core with SMT, It competed pretty well against the 4770k ? Benchmarks that AMD is usually good at like Cinebench put these 8 cores on par with 4770k when maxxed out on Lin. So were any of these cpu's capable of getting Global First Place back when they were new. I'd imagine CPU Frequency would of been the only one with GpuPi the only other possibility.
I'm not saying to have it one way or not, just want to make sure we are considering everything before making a decision we may regret in the future

Im not following, 4770k has 4 cores and ht, 9590 has 8 cores. Therefore 4770k is in the 4 core category and 9590 is in the 8 core category proving my argument that 12900k should be in 16  or am i missing something. Its not about what they call, its about what it is. The PR alone I cant wait to hear what they say "worlds fastest big little"/ "first 16 core over 7ghz" etc going to be a real laugh. 

[Leeghoofd]

Thats why I'm in favor for Option 4:

We list the 12900K twice. This solution would be a mix of #2 and #3. We list the 12900K as:

- i9 12900K (8P + 8E) [this would be in the 16C ranking]
- i9 12900K (8P + 0E) [this would be in the 8C ranking, the user would have to manually disable the E-Cores in BIOS to participate]

Only advantage I see in that option is that you will get twice points for the same processor...
 

We have no control over marketing or creative PR people you know that Al. Perfect example the WR on the Crucial kits, HWBOT apparently certified the BabyJ score in their markting slides, while we in fact pulled it and it never came back ?

[julienricqrt]

In my opinion physical cores are what categories should be based on for CPU, it's easier to compare.

Right now old CPUs without HT are in the same category in HWBot as CPU with HT, for instance i5 2500k & i7 2600k are both in the 4 cores category. Hyper-threading is a feature, like AVX 512 for instance, so I don't think it should matter to sort categories. If a 16 cores CPU with big/little cores is slower than a "real" 16 cores CPU in the rankings, so be it.

So my choice is option #2, because it's consistent with the way results are sorted in the database nowadays.

[Splave]

5 hours ago, Leeghoofd said:

Thats why I'm in favor for Option 4:

We list the 12900K twice. This solution would be a mix of #2 and #3. We list the 12900K as:

- i9 12900K (8P + 8E) [this would be in the 16C ranking]
- i9 12900K (8P + 0E) [this would be in the 8C ranking, the user would have to manually disable the E-Cores in BIOS to participate]

Only advantage I see in that option is that you will get twice points for the same processor...
 

We have no control over marketing or creative PR people you know that Al. Perfect example the WR on the Crucial kits, HWBOT apparently certified the BabyJ score in their markting slides, while we in fact pulled it and it never came back ?

I just feel like we are calling it something it isn't for the sake of it getting global points. Id be salty if I had 11900k scores and a 16 core took my points that's all. 

Not an easy choice and I will participate however you guys decide. ?‍❤️‍?‍?

[Leeghoofd]

6 hours ago, Splave said:

I just feel like we are calling it something it isn't for the sake of it getting global points. Id be salty if I had 11900k scores and a 16 core took my points that's all. 

Not an easy choice and I will participate however you guys decide. ?‍❤️‍?‍?

That's just the thing with Option 4 that we want to avoid.
 

2 CPU entries will be created:  i9 12900K  &  i9 12900K(8P + 0E)
 

1)  i9 12900K - this would compete automatically in the 16C ranking

2) i9 12900K(8P + 0E)  - this would compete in the 8C ranking (the user would have to manually disable the E-Cores in BIOS to participate)

 

So you might get double the fun for the same prize ? It is really no matter how we approach it, comparing apples with oranges

 

 

[GtiJason]

13 hours ago, Splave said:

Im not following, 4770k has 4 cores and ht, 9590 has 8 cores. Therefore 4770k is in the 4 core category and 9590 is in the 8 core category proving my argument that 12900k should be in 16  or am i missing something. Its not about what they call, its about what it is. The PR alone I cant wait to hear what they say "worlds fastest big little"/ "first 16 core over 7ghz" etc going to be a real laugh. 

Exactly, They are in the correct categories. Just saying the argument people used to make about  9590 (and other cpu's like it) that it was made to compete against the 4 core Intel's is not based on logic. I just failed bad at getting my point across ? because I wasn't making an argument for or against anything. I'm not worried about it, I have trust that the correct decision will be made

[d0minat0r]

From my side 12900k is 16 Core cpu. Yes we see Intel banner on HWbot and probably Intel do not like to see 5950 beat 12900k but that is not our problem, when I say our I think we who buy Intel CPUs.

[Gulftown]

My opinion, 12900K - 16 core CPU, simple. 

[_mat_]

That way the benchmark integration can stay as it is and the category and therefor the amount of cores is decided by the cpu model detection. It's practical and works out of the box even for old benchmarks without BM integration.

[yosarianilives]

On 8/16/2021 at 3:21 PM, Leeghoofd said:

Thats why I'm in favor for Option 4:

We list the 12900K twice. This solution would be a mix of #2 and #3. We list the 12900K as:

- i9 12900K (8P + 8E) [this would be in the 16C ranking]
- i9 12900K (8P + 0E) [this would be in the 8C ranking, the user would have to manually disable the E-Cores in BIOS to participate]

Only advantage I see in that option is that you will get twice points for the same processor...
 

We have no control over marketing or creative PR people you know that Al. Perfect example the WR on the Crucial kits, HWBOT apparently certified the BabyJ score in their markting slides, while we in fact pulled it and it never came back ?

Getting people to follow rules is hard enough, now add them to have to disable cores for a ranking? Not possible, will just make a big mess

[denvys5]

On 8/11/2021 at 10:02 PM, denvys5 said:

Question: do we have verification from Intel that P and E cores can do same workload in parallel? Or do they move workload, like on ARM bigLITTLE?

 

#4 sounds reasonable. I would also add 0P+8E core configuration to that list. Why? Coz we might see 8P+64E cpus in near future, if this architecture succeeds.  And that means little cores become competitive in MT benchmarks on their own.

But, this all ranking split is possible only if core configuration can be verified for each individual submission. So 2D only benchmate subs, as far as I understand. 

Looking at Intel information on new thread scheduling rules (Intel Thread Director), it makes most sense to do one single ranking for each cpu, by adding up cores. My previous point was based on expectation that cores have different instruction sets. With confirmation that 8E cores are running same instruction set (AVX2 enabled), just as other guys above suggested, 8P+8E = 16C makes the most sense.

[der8auer]

While discussing things internally we also had the idea at some point to add "mixed" categories. So instead of adding this kind of CPU to the normal 16C ranking we could also create "16C Mixed". 

However we would still see different types of CPUs in the mixed rankings. For example "12C Mixed" could contain 8+4 desktop CPUs and at the same time 4+8 mobile CPUs. To me it felt like this cause more problems than it would solve. Opinions on this?

[TerraRaptor]

55 minutes ago, der8auer said:

Opinions on this?

Count any cores as classic cores. not caring if they are P or E. Any other approach will make it unfair for users of previous generations.

Besides, if we allow disbaling E-cores to allow P-cores to participate in lower core count ranking shouldn't then we allow disabling cores on any other cpu to stop L3014/phenomx2 domination in x1/x3/x5 rankings?)

[der8auer]

Unless there will be a new single core CPU this won't change but I don't see how this is part of this discussion. Same as we don't see quad GPU stuff anymore these days. Tech changes and now it's changing to something new and we have to think about how to work with it. You might only be worried about how it is unfair for previous generations but we also have to make sure that HWBot stays interesting in the future. 

Intel showed during the architecture day that Ultra Mobile will have 2P Cores and 8E Cores. This would be in the 10 Core ranking against 10900K. 

 

[Noxinite]

It still feels like option #4 is the best put forward. Having mixed would be strange is you got 2P+8E & 6P+4E in the same global ranking.

[mickulty]

If globals are to be divided by core count then they should be divided by core count, that's that.  Benchmarks should not be bent to how fast we feel a CPU should be.  HWBOT has had big.little for a long time and always handled it this way, the difference is that now it affects a big boy.

A mobile CPU is unlikely to be competitive in any global ranking anyway, and if it was that would be a bad thing for the site and community since they can't really be OC'd.

Switching to thread count would be pretty reasonable from the perspective of "How do we compare this hardware?", but if that's done I want to know who's going to pick up the task of updating the database.  To my understanding it's managed by volunteers at the moment.  I don't think the site can reasonably embark on such a thing at the moment.  Maybe in the future if the site has had the rewrite, had the bugs fixed that are waiting for the rewrite, and has paid stuff who can do it.

Adding a separate hardware listing for disabled cores is not without precedent - after all, that's roughly how unlocking works with AM3.  However, we need to think about when hardware qualifies for extra listings.  I would propose;

"When the average amount of execution resources per core is higher with some cores disabled, and it is possible to disable cores down to this configuration, an additional hardware listing should be added for the highest core count configuration that has the maximum amount of execution resources per core."

Why this slightly awkward wording, you might ask?  Well, all FX-based CPUs except the FX-4200 have execution resources shared between cores.  And yet, unfairly, these CPUs have always had to compete with other chips that have the same core count.  An FX-8350 has to compete with an i7-5960X!

This proposal would allow an FX-8350 in one core per CU mode to compete fairly with an i7-4770K.  It would still get roflstomped, but I think it's important on principle.  This way there's a neutral wording, applied to everyone, rather than reference to a specific technology.  It feels fairer.  The effect is that a CPU with n strong cores gets to compete with other CPUs that also have n strong cores without the weak ones being relevant.

The bit about it being possible matters because otherwise you'd get a load of random phone processors involved.

Edited August 20, 2021 by mickulty

[Leeghoofd]

Roman also had the idea of threads not core count...  Might shake up the ranking though

[yosarianilives]

Ht would become a huge handicap until you get above the non-ht thread counts. So currently 8 thread. Throw away 7700k, throw away 5300g,7350k is useless. It's time for 7640x, 9700k, and all that other shit nobody has spent as much time benching hard. 

Not sure this is good or bad, but that's mainly how it would shake up rankings. Oh and g470 would become even more irrelevant lmao. At least 1, 3, 5 cores wouldn't really change in the slightest 

[Splave]

I understand what roman is saying, looking to the future is important because if hwbot is gone the choice doesn't matter. 

[TAGG]

9 hours ago, yosarianilives said:

Ht would become a huge handicap until you get above the non-ht thread counts. So currently 8 thread. Throw away 7700k, throw away 5300g,7350k is useless. It's time for 7640x, 9700k, and all that other shit nobody has spent as much time benching hard. 

Not sure this is good or bad, but that's mainly how it would shake up rankings. Oh and g470 would become even more irrelevant lmao. At least 1, 3, 5 cores wouldn't really change in the slightest 

I think that would mean new tops for 2/4/6/8 threads would be G4520/7640X/9600K/9700K(all but gpupi) honestly that sounds hillarious cause 90% of all the serious scores of past gens become worthless ?

As much as i would love to see that for the pure chaos i'm honestly for the "a core is a core" approach, so 12900K is 16C and done

[der8auer]

On 8/21/2021 at 6:12 PM, yosarianilives said:

Ht would become a huge handicap until you get above the non-ht thread counts. So currently 8 thread. Throw away 7700k, throw away 5300g,7350k is useless. It's time for 7640x, 9700k, and all that other shit nobody has spent as much time benching hard. 

Not sure this is good or bad, but that's mainly how it would shake up rankings. Oh and g470 would become even more irrelevant lmao. At least 1, 3, 5 cores wouldn't really change in the slightest 

Yea I would like to avoid drastic changes like that. So right now I think we will go for the dual listing. If we figure out after one or two generations that it was a bad idea we can still remove one of the two listings. That will be easier than changing the entire bot.

[mickulty]

1 hour ago, der8auer said:

Yea I would like to avoid drastic changes like that. So right now I think we will go for the dual listing. If we figure out after one or two generations that it was a bad idea we can still remove one of the two listings. That will be easier than changing the entire bot.

This makes sense but there has to be a consistent standard applied to everyone, not just the new Intel chips.  FX chips getting to play in 1 core per CU configurations would show it's not just a way to keep intel competitive for globals.

[Leeghoofd]

Again, AMD is heading into the same direction with future processors, I fail to see the relation with FX and what's upcoming... we can test with Alder Lake, if it works out fine ( there will be an occasional user error ) than we are ready for Team Red too...