Jump to content
HWBOT Community Forums

mickulty

Members
  • Posts

    544
  • Joined

  • Last visited

  • Days Won

    4

Everything posted by mickulty

  1. Now that Country Cup 2023 is over and there are no longer stakes in terms of how the competition goes, I’m hoping we can have a sensible and constructive discussion about how it was run. Let me say up front, I know some people may find all this as a bit much. They might wonder why I care so much. The truth is, I care so much because it would be so easy to do the right thing - and when the right thing is easy, not doing the right thing is terrible. What Happened: During the competition, there ended up being several changes to stages. I can’t actually say for sure how many, since there’s no transparency about when stage rules are changed. However, there are two that I know for sure, and think are worth discussing. First, the superpi 32M bclk stage. This was changed to exclude socket 775 due to benchmate issues, and as a result the multi cap was raised from 8x to 9x to include socket 1156. I bring this up as an example of a stage where the changes were well justified. I want to show that I’m not just blindly opposed to any action, any change – even though that change wasn’t good for me personally. Second, the Cloud gate iGPU stage. At some point in “early november” – therefore at least 2 weeks into the competition – “Xeon CPUs allowed” was silently added to the rules of this stage. This had the effect of changing the optimal hardware to exclude one iGPU from a fully unlocked CPU, in favour of a BCLK OC’d Haswell Xeon (costing $100+ and with little reason to own for OC) that benefits from a separate hardware listing. The justifications given when this was talked about on Discord were, in chronological order: 1) “so people can complain and keep the discord thread alive” This was seriously the only justification given at first. 2) “This wasn't added recently fyi…” This was said on the 1st of December – since the change was silent a lot of people took a while to notice. That doesn’t make it better! 3) “Xeon iGPUs don't offer any significant advantage over their non server counterparts, in fact they will be less performant in this benchmark” This simply wasn’t true, as allowing xeons meant you could have HD 4600+P4600 rather than 4600+4000. CPU clock matters, but not that much. So ok, it was an error. But also – in that case what about the double-digit number of other stages where xeons would have genuinely offered no advantage? So, unlike the superpi stage, there was no real justification given for the change. Eventually, after having to drag Roman into it, I got one more drop of reasoning: 4) leeghoofd didn’t think there were xeons with iGPUs when the change was made. Had he been right about this, adding “Xeon CPUs allowed” would have had zero effect on what hardware could actually be used for the stage. OK. So it wasn’t supposed to have an effect. But then why make the change at all? Why play with fire? When I talked to Roman on the 4th of December he said he’d “try to follow up with Albrecht and that he gets in contact with you again”. A patient month later, that hasn’t happened. I am still none the wiser as to what led to the bright idea to make this arbitrary change in the first place. Why This Matters: This has happened before, and it earned hwbot a write up in igorslab, as well as completely turning off at least one participant from future involvement in hwbot: https://www.igorslab.de/en/the-dirty-business-behind-hwbot-competitions-a-field-report-as-editorial/ Those events, within the last year, led to a statement from Roman, including 2 key points: “I totally agree that it’s unacceptable to edit rules of the competition after it started.” “It is also not in [HWBOT mods] interest to cause any unnecessary conflicts or drama – nor do they personally benefit from it.” So… why did the “unacceptable” happen AGAIN, causing nothing but unnecessary conflicts and drama? Why was arbitrary hardware silently added to the cloud gate stage weeks into the competition? I was very enthusiastic about country cup when it started, participating very heavily and encouraging others to do the same. The fact that this change happened completely ruined the competition for me. Afterwards, I made one other submission as a favour to teammates with conflicting hardware. I honestly regret participating at all. The truth is, I’m not sure if I want to get involved in another hwbot competition now – and I know I’m not the only person to feel that way. What Next? Personally, I believe very strongly in seeing the best in people. I want to understand why this happened so that I can forgive and forget. But I can’t do that if the chain of thought that led to this change is kept a secret. I am begging leeghoofd to please meet me halfway here by explaining the thought process that inspired this change specifically being made. So far all I’ve heard is retroactive justifications. I’m sorry, but how can I trust that a mistake like this won’t be repeated if I’m not even allowed to know where the mistake came from? I would also love to hear from other users on what they think could be done differently so that problems like this don’t happen again. Not just “don’t do it again” – we had that back in March and it didn’t stick – but structural/procedural changes. There have been multiple competitions in 2023 tainted by unjustified after-the-whistle changes to rules. How can hwbot do better in the new year, and begin to mend what has been broken?
  2. Hah. Yeah, it's not great... though I have a 3930K that weirdly does 2400 1T on the same mem with it.
  3. Yeah it's fun! And I was serious about having a faster score incoming anyway, because this will get removed from the comp at some point even if no-one reports it.
  4. Don't worry, I already have a walletectomy in progress to satisy the pointless arbitrary rule that I could swear wasn't there when the competition started. Faster, compliant score is coming.
  5. Please add Plextor S3C series https://www.goplextor.com/Product/Detail/S3C#/Features
  6. First impression: isn't country cup normally shorter than team cup? I appreciate the configured stages are kinda random but here are my thoughts on them as they are: ycruncher 5b isn't a good idea, a nonstandard benchmark always causes problems. Could be removed to shorten competition. Intel HEDT 32M looks fun. Cinebench R24 per core count incl. AM3 is a good idea to get some subs going. Is it supported by benchmate yet? Thuban GB6 single is redundant because the CB24 stage includes thuban. Could be removed to shorten competition. Time Spy Extreme 4-core CPU you'll get a lot of people complaining about having to do the full run. There's argument for excluding LGA1700 because of AVX512 lottery. HWBOT x265 Core i5 kinda random but sure. 7zip 2-core could probably benefit from being per socket, otherwise it's all lga1700 bclk. Does 7zip benefit from AVX512? If so I'd prefer to see LGA1700 excluded otherwise the optimal hardware is gonna be rare. GPUPI 3.3 for CPU 6-core DDR4 looks fun. GPU Stages no strong opinions other than cloud gate probably needs a yos pass to figure out how to define hardware without loopholes/issues Memory Stages look great Misc Stages see attachment (but seriously... aquamark dogpile would just be swapping GPUs on a cold CPU and the comp is super long as it is)
  7. I was 4 hours in, the IMC hated me because of thermals, and after being on for a couple of minutes the board would require a hard power cycle to restart which would also cause it to lose all settings.
  8. The "4P" option represents one core per compute unit mode - certainly available on gigabyte boards, supposedly a certain bios on CHV, not sure about asrock and msi. It was added because it's vaguely analagous to p-core modes on ADL (rankings like 12900K (8P)). The difference compared to just disabling cores is that each core gets a full 3-wide decoder 100% of the time, a full 256-bit FPU 100% of the time, and an exlusive 2MB of L2 cache. You can look at ths as the "P-core" and the extra integer units that normally share these resources as the "E-core". Another way to think of it is disabling CMT. I found a screenshot of the settings on a 990FX-UD7 but even cheapo gigabyte AM3+ boards like the 78LMT-USB3 have the setting. Also attached are block diagrams (By Shigeru23 - Made by uploader, CC BY 3.0, https://commons.wikimedia.org/w/index.php?curid=17130259) annotated to show the difference. Realistically a 78LMT-USB3 even being a fairly bunnyextraction board but with true 1 core per cu mode will still beat a crosshair with disabled cores. Imagine 12900K with 4P and 4E-cores vs 8P...
  9. read the rules before posting score is bus clock, not data rate + you don't get to round up your clock, it's exactly as cpu-z reports, not what's running in your head...
  10. If it was likely the winners would be at stock, I'd be against it. However, with overclockable threadripper pro, xeon wx etc and limits on how many threads benchmarks scale up to, there's a possibility we could see some monster OCs on monster chips. That'd be awesome! I'd suggest ycruncher 10b, but then, of course I would...
  11. This is also true of eg the Phenom II X4 965 BE https://www.cpu-world.com/CPUs/K10/AMD-Phenom II X4 965 Black Edition - HDZ965FBK4DGI (HDZ965FBGIBOX).html https://hwbot.org/submission/2503960_maedhros_superpi___1m_phenom_ii_x4_965_be_10sec_578ms
  12. afaik steam deck is LPDDR5 not DDR5 so would be excluded by default (same as how GDDR5 is excluded)
  13. Brace for a long one. Comments about the AMD AGP stage, which needs it own section: The AMD AGP stage is currently set up to exclude AGP 8x. I'm not sure if this is intentional, but it's interesting. Top AGP 8x cards are obviously a bit rare and expensive, but I'm concerned the top AMD AGP 4x cards - eg Radeon 9100 and 8500 - aren't any easier to get hold of. Call me sentimental but they're also collectors items that it might be a shame to mod to death, whereas for top AGP 8x people already have super modded cards. AMD AGP Stage has a lot of other 'loophole' boards/platforms not currently banned. There are a couple of ASRock 775Dual series boards, but also AM2/AM2+ AGP options that even enable Deneb+AGP (this is afaik 'optimal' for 3DMark2001SE AGP). As it stands only the relatively common stuff is banned, which would just hand an even greater advantage to people like me who have non-listed 'loophole' hardware. The mobo restrictions should either be made much more comprehensive, or relaxed entirely. If AGP 8x is really not allowed, then IMO the restrictions might as well be relaxed entirely as CPU won't make much difference. If AGP 8x is allowed, it's probably a good idea to either restrict by CPU core and accept people running 4CoreDual+P4, or to restrict by CPU socket eg to 462/478/754. Restricting to older sockets and allowing AGP 8x would push the bottleneck to the CPU, which would at least nerf the unobtainium cards. An alternative might be to specify certain relatively widely available cards, eg Radeon 9200 IMO the best thing to do is to allow AGP 8x and restrict CPU socket rather than specific boards. It would also be viable to just make the stage 'full out', if that's done I think we've had 01se relatively recently so maybe 05 or 06 would be better? But full out 01se on AMD AGP is fine. Comments about benchmark/hardware selection: Requiring 4 scores seems really excessive for the Intel LGA1156 stage, it's a relatively niche platform, I'd suggest 2 in line with eg AMD DDR2 GPU. Is Intel 3DMark Vantage CPU score DDR4 intended to be one score per gen, or just i5s only? One score per gen would be good. Is DDR5 iGPU really intended to be x1 score? 3 scores are needed in other DDR5 categories. I suppose many people will have -KF or non-iGPU boards but you'd expect at least 2? Is GPUPI 1B on AMD CPUs for the DDR4 stage even memory bound? Conventionally GPUPI isn't. If a benchmate-based memory sensitive benchmark is desired there's always good old ycruncher. This could be swapped with the Intel CPU DDR5 stage. I'm slightly disappointed there's no max mem clock stage on any platform but the current stages look fun so I wouldn't want to suggest scrapping any of them. GPUPI 3.3 32B on geforce 900 series is a 15+ minute benchmark. I'm not complaining, but just as long as you're aware. I'm a big fan of the use of unique video card cores, rather than unique specific models, for the iGPU section. Had to get at least one positive comment in there! AMD AM3 HWBOT x265 seems like a missed opportunity to give people a reason to bench 1 core per CU FX by allowing all 4-core AMD DDR3 ? Comments about loopholes: Noticed the Intel DDR5 CPU stage specifies "Use 1 processor(s)" - isn't this redundant as server stuff is conventionally banned by default? If it needs to be specified many other stages might be lacking it. DDR1 SuperPi 16M is going to get 4CoreDual'd unless core 2 arch is explicitly banned. DDR3 x265 per socket is at real risk from chinese DDR3 "X99" boards, Z170, and CFL modded Z170. Probably laptop stuff too? It might be best to just have a socket whitelist. Is it intended to allow TR4 Threadripper for the DDR4 AMD GPUPI 1B stage? What about TRX40 threadripper, and WRX80 threadripper pro? Might make sense to restrict to dual channel. Geforce 900 series (Nvidia GPUPI 3.3 32B) as configured on HWBOT includes the Titan X. Is this intended? I don't think it's too bad by modern GPU standards... Currently the DDR4 AMD GPUPI 1B stage would include mobile-only "Zen 3+" as optimal hardware. If that's banned, it'll require Bristol Ridge which afaik is locked to 2400 max mem mult and a bit of a compatibility minefield. Might be a good idea to ban mobile and reduce to 4 subs (zen 1/+/2/3). Although I supposed it would be funny to see people benching mem on bristol ridge, so I won't complain if it's kept at 5 ? Things I suspect are mistakes: Many stages don't explicitly list what you'd assume are standard restrictions (eg use only 1 GPU core), but presumably this is to be added? AMD GPU stages all say "unique processors" rather than "unique videocards", except GDDR6 Nvidia 3DMark03 GT215 says "unique processors" DDR2 iGPU says "unique processors" Some of the memory stages say "unique processors", some say "unique hardware", despite having the same restrictions written I haven't looked too carefully for loopholes based on mobile, socket adapters and 'consumer' multi-CPU, so there might be some that I haven't flagged up.
  14. If anything this thread demonstrates that the community can be relied upon to spot when something is wrong with scores that get big points, and raise a stink. Obviously only scores getting big points get community attention, but that would be the case however moderatable the benchmark is. The rules page used to have a "disallowed tweaks/cheats" section. Bring that back and put the onus on the benchers to report any of their own scores where they might have got a bit overzealous, and to stop taking the wee in the future. Then, if people are still cheating, leave it to community members with a personal interest to figure out what they think is wrong and explain it. Leave it to the person accused to show how they got the score and prove it is legitimate. Leave it to the community to accept the proof or point to a problem with it. Mods shouldn't have to run around collecting evidence, it shouldn't be much more difficult than checking a verification link. It's easy to fall into the trap of holding others to the same standards to which we hold ourselves, maybe I'm biased because I was willing to send a board and CPU across europe just to remove doubt when @unityofsaints on my then-team had a score in question. But if you're claiming to be fastest in the world, backing that up with a video doesn't seem like an unreasonable ask. Not like people are saying 01 runs need an o-scope or anything ?
×
×
  • Create New...