Democritise HWBoints

[I.nfraR.ed]

Yes, but you're actually showing a "saved/previous" score and not the one you get just after the bench finishes. It could be easily avoided if the screenshot requires the score you get on the top left corner.

 

However, the bench needs to be investigated for other loopholes.

[I.M.O.G.]

I feel securing benches to be less important than other things actually. The cheats are exceptional cases, and there are a decent amount of people around here who I know, and who have reputations within their respective communities... That is the best security as its an effective technique for the vast majority of people. Checksums and wrappers do lend good confidence when you don't know the other people you are competing against though, so its good to see things progressing in that direction.

 

As far as improving the hwbot platform the most important item is figuring out how to effectively add and eliminate benchmarks. Eliminate the broken ones. Bring some new ones.

 

As we eliminate benches, anything that can't be competed in should no longer be worth points. Safe points go against the spirit of competition, that drives us to bench and rebench as we are beaten. Competition points should be addressed in that vane also - they are "safe" points I think, not ever expiring.

 

In order for anything to be valued, it must be possible to lose it. Experiencing loss makes us feel value for what we lost, and experiencing gain makes us feel rewarded. I just got a gold cup last night that boosted my points a bit and it felt good. Especially after last week when I lost a bunch of other cups and dropped points I had just earned in another bench.

 

The loss and gain of boints (and overall rankings) is what makes the whole thing tick. It's what hwbot has that no other benchmark platform does.

[GENiEBEN]

Yes, but you're actually showing a "saved/previous" score and not the one you get just after the bench finishes. It could be easily avoided if the screenshot requires the score you get on the top left corner.

 

However, the bench needs to be investigated for other loopholes.

 

[Bobnova]

World Record!

[I.nfraR.ed]

Yes, I agree, but that's kind of different issue - modifying the c4d file, which is not secured in any way.

[GENiEBEN]

Yes, I agree, but that's kind of different issue - modifying the c4d file, which is not secured in any way.

 

Can be done with the default scene Won't share the info on the thread until I build a secure version.

[reggiesanchez]

Wouldnt it be eaiser to just make new benchmarks that gained you hardware points only untill there was enough competition to warrant global points. If you do it this way I think it will be less of a problem then you think. For instance now there is enough people competing for memory clock to warrant global points, so just give it global points it dosent mean you have to take away points for other benches. Thats how I think benchmarks should earn the "right" to be counted for global points. Its not like your printing currency more global points dosent make current global points any less valuable.

 

Flawed or hackable benches are a diffrent story all together.

[IanCutress]

Read the thread, interesting stance. My take:

 

Pro OC needs a handful of benchmarks to focus on, so keeping the number of benchmarks with global points limited is a good idea. Through the year you can add X benchmarks with Global Points, but have a yearly vote that lasts 4 weeks on 'benchmark to drop global points' is a good idea, and take out X benchmarks from Global Point rankings.

 

Keep all benchmarks that have had hardware points with HW Points, but maintain security.

 

The OC League is a tricky one to deal with, being a combination of HW and GL points. You could do what we used to do, and limit HW points to 300 for the league - I know it's already limited to top 20 scores on the HW side, so maybe no change is needed.

 

So if you add Cinebench (ST + MT), Catzilla, 3D12, that leaves three benchmarks to get rid of Global Points at the end of the year, voted by the community.

 

Reggie has a good point regarding having a boundary which allows global points. Unless it's one demanded by all the sponsors, initially add in a benchmark, and when the popularity > 5000 (it that submissions in last 12 months?), give it global points automatically. Any benchmark that loses global points status won't be eligible again for globals for 24 months.

 

I don't think there's a need for an overall system benchmark. Any that are created either take too long or are exploitable. They're also not that fun to watch, especially for an audience. Technically a good GPU benchmark still abusively stresses the system (CPU, GPU, PSU, Memories).

[hokiealumnus]

FWIW, I agree with pretty much everything in Bobnova's post: http://hwbot.org/forum/showpost.php?p=221885&postcount=43 . Except Aquamark. Leave that alone. I heart Aquamark, always will. It might be better moved over to CPU benches though.

 

borandi, Pro OC doesn't need anything as it's about to go the way of the dodo.

[Rasparthe]

Just read through this entire thread, pretty interesting. I agree with some of the very first posts that changing the system so drastically with the hardware points will just create the impressions of instability and "always changing" structure. Also the amount of work that many people have put into the benches and hardware would disappoint alot of members. I also don't see any problem with 10 or 20 or 30 hardware only benchmarks, if anything it will force people to spend more time on a single platform or just submit on the ones that interest you. Those points are limited in the leagues anyway and give more value to benchers for each piece of equipment (something that would attract newer benchers with less hardware). I like the idea of adding more benchmarks and approve of how its being done with no points right now. After some time goes by (a year?), and the security is assured they probably should be added for hardware points, not unlike UC2011.

 

I do not think this is the case with global points. They are always changing anyway. Global points you earned last year have, most likely, 'leaked' away with the introduction of new and faster hardware. I think I would vote for a system where there is a set number of benchmarks in 2d and 3d, for argument sake say 10. Every year, perhaps in the first month of the year, every member has the option of voting for the 10 benchmarks that will serve the next year for globals. Feb 1, the globals are calculated using the existing scores for those benchmarks, and continue on for the next year. Of course, certain benches will probably always be popluar and may serve as globals for years while the other less popular will pop in and out of the list, sometimes providing globals, sometimes not. In this way, if a benchmark that finds a following for whatever reason will instantly become popular just based on the value to the leagues. There is no need to freeze or lock points then since the global points can always be recalculated if the bench comes back into the globals list.

 

Just my thoughts on how I would work it, but that being said, I don't see much problem with the system how it is. New benchmarks will always take some time to gather some steam and as long as HWBOT is open to adding them I don't see a problem. I don't think you should remove any from hardware point consideration unless there is a security issue with it.

[PAQd]

Some great thoughts and thought provoking comments here, however the last release of new benchmarks was poor to say the least. When we can manage the work associated with robust (secure as mentioned numerous times) and clear (rules and submission page) implimentation, then perhaps reconsideration of this thread intent should take place.

 

With the utmost respect, from what I read and see generally this "sport" and our beloved HWBOT is still attempting to run before we have learnt to crawl properly.

 

Yes some have a single focus on the boints, but others have a focus on cups, additionally, some focus on older hardware for no other reason than the pleasure and nostalgia. Does this mean that the poor get the picture? We must continue to offer benchmarks for all comers (age, race, knowledge or wealth/social status).

Some people spend thousands on new hardware, others thousands on old hardware, and for some their only desktop is it, but it means thousands to them. Plenty bench on the older benchmarks because they know that attempting without cold will be fruitless. The greatest advantage we have is the possibility to make our pleasure available to a diverse group of participants as possible.

 

Ask yourself what you/your team last did to attract new players to our wonderful game, and how would they view a sport/interest where the current rulebook is inconcise and ambiguous. Perhaps someone can check the real numbers but I would be surprised if more than 30% of the reistered HWBOT members in teams are active. What does this tell you?

 

We aren't selling the game well!

 

Lets devote our efforts to polishing the great that has been created in the past years before we go down this very generalistic thinking path. A working group should attack 2D and 3D sperately for a start, let alone the hardware points discussion.

 

As stated in this thread earlier the Pro League has very different drivers - let's make sure we don't confuse things. ....and keep up the good work

 

PAQd

Edited January 31, 2013 by OverPAQ'd

[Yamunsa]

having re-read the thread from the start p.1 reads like it has already been decided that implementation is going ahead and this is more a tweak discourse than actual discussion. p.2 starts to be a bit more understanding of what faults benchmarks have and their validity today but doesn't address the ramifications such a change could present.

 

personally speaking i'm not in favor of such a dramatic change at all. i just don't see the bot as broken so cannot understand why we need such a wonder fix. sure there are heaps of benches i could care less if they were binned or not. i mean i've not benched am3 since the first wrapper killed scores, don't really bother with pifast, maxmemm pfft. i'm not into 01 or 32m but love pc05, (which i benched legit from day dot and recognise the disgraceful abortion some members of the community made it.. and were held in high esteem for doing so by others).

 

noting that even though i'm not into 01 & spi 32m i have great respect for those that bench it proficiently as it is a true benchers benchmark and can be as much of a tweak fest as pc05. it'd be a sad day to see any of these go.

 

i also feel that opening up ideas to the hwbot community is a great idea in principle but the obvious is striking - the vast majority of the community are not 'every month' active, do not read the front/news page as their favorite cookie points to their own profile or team page, don't read let alone post on the forums, have certainly been confused of late and many would not anticipate such a change.

 

looking at this thread as a principle example, ultimately it'll be the voice of the few that have the loudest opinions heard and therefore have the most responsibility for the community as a whole.

 

surely it's worth highlighting the successes of the past and building on them rather than changing the game altogether? when i'm invited on consultancy jobs to proffer business sense i usually ask 'where's the 5 year plan'. so common powers that be - before any major change is implemented where does hwbot want to be in 5 years time and as a community how can we help you achieve that goal? (because this idea looks like it's been drawn up on the back of a beer mat and the fine-tune ideas offered read as trying to polish a turd).

[Massman]

Unlinke the Pro League change, this is not a plan set in stone. It's probably not going forward given the feedback from this thread. I should move it out the R5 subforum, I think.

[PAQd]

Yep, prime candidate to start a "Parking Lot" sticky in the development forum for future reference and revisiting by anyone.

[dinos22]

3DMark01: at global level, a bencher watches "999" for 6 tests and Chase High..... then hopes for the best at the end when the score comes up. It's not interesting to watch, it's 12 years old, newcomers won't understand why there's so much emphasis on such an old bench......that only runs well on an OS that was replaced 5 years ago. It's hard to bench in front of an audience because there's nothing to watch and there's every chance they'll look at the LOD and think "cheat." It's CPU-dependent and ORB verification has been dropped. A .jpg image is all that sets the noobs apart from the pros. IMO.... and I know a LOT of people will disagree..... at practical level, it is time for this benchmark to very slowly start moving out of the light.

 

What sets it apart from others is that level of tweaking and system tuning you cannot do on any other benchmark. While you are right about it scaling with CPU frequency there is certainly a hell of a lot of play to be able to nudge people out with higher clocking hardware if you know what you're doing. It is a time consuming benchmark and probably more frustrating than most others but there is absolutely nothing that is able to replace it. You remove that and you have yourself a bunch of click and run benchmarks.

 

If anything your sentiment probably applies more to something like 3DMARK05 which is way more CPU dependent than most benchmarks and you are only looking to get a lucky run for max frames without a great deal of skill involved. On the flip side it has a nice gain from tweaking system but yeah still nothing too exciting. Same with 3DMARK03 if you want to go that way and probably 06 as well.

 

I also reckon some of the newer 3DMARK benchmarks should get a cull before 01 is touched. I'd personally also drop Vantage before others as whatever new comes out is another "vantage" more or less. Keeping a couple of latest benchmarks is also fair enough as that's what tends to always get most interest because of the novelty users.

 

Unigine is now scaling with CPU as well but at the same time it's stupidly long, has very little tweakability, it's only there because we have bugger all alternatives to Futuremark. Same can probably be said about Aquamark as far as lack of alternatives and why we still hang onto it, even though on the tweakability side of things it is certainly better.

[I.M.O.G.]

it's only there because we have bugger all alternatives to Futuremark

 

I always wondered if HWBot ever pursued agreements with game companies. Lot's of games have built in benchmarks, and it would seem this would be one of the easier ways to diversify away from being as Futuremark-centric.

 

I don't know if HWBot could barter that kind of deal, but if the game benchmarks could be released in hwbot versions without buying the full game license, that'd make them viable candidates for us.

[GENiEBEN]

I don't know if HWBot could barter that kind of deal, but if the game benchmarks could be released in hwbot versions without buying the full game license, that'd make them viable candidates for us.

 

Already way ahead of you Problem is that you have to own the game, I can only provide the wrapper/launcher with a score mechanism.

[K404]

What sets it apart from others is that level of tweaking and system tuning you cannot do on any other benchmark. While you are right about it scaling with CPU frequency there is certainly a hell of a lot of play to be able to nudge people out with higher clocking hardware if you know what you're doing. It is a time consuming benchmark and probably more frustrating than most others but there is absolutely nothing that is able to replace it. You remove that and you have yourself a bunch of click and run benchmarks.

 

If anything your sentiment probably applies more to something like 3DMARK05 which is way more CPU dependent than most benchmarks and you are only looking to get a lucky run for max frames without a great deal of skill involved. On the flip side it has a nice gain from tweaking system but yeah still nothing too exciting. Same with 3DMARK03 if you want to go that way and probably 06 as well.

 

I also reckon some of the newer 3DMARK benchmarks should get a cull before 01 is touched. I'd personally also drop Vantage before others as whatever new comes out is another "vantage" more or less. Keeping a couple of latest benchmarks is also fair enough as that's what tends to always get most interest because of the novelty users.

 

Unigine is now scaling with CPU as well but at the same time it's stupidly long, has very little tweakability, it's only there because we have bugger all alternatives to Futuremark. Same can probably be said about Aquamark as far as lack of alternatives and why we still hang onto it, even though on the tweakability side of things it is certainly better.

 

I don't disagree with you 3D01 is a two-sided discussion for sure. 3D01 and SPi32M.... the two benchmarks where 0.5% score gain through tweaking is considered a day well-spent

[rbuass]

Regardless I never touch... for sure... if Hwbot want to increase public... is easy.

Add Crysis benchmark and benchmarks includded in the games.... so... the gamers that was the biggest public in the world will agree and like.

OMG.... 200 FPS in crysis?...

Each record or great score that you make public, the questions are:

Can it run crysis?... lol

[TerraRaptor]

games are too easy to cheat/hack. And popularity of games will lead to hundreds of cheaters here.

[Splave]

Unigine to the rescue

[Guest cowgut]

Too many benchmarks on a benchmark site?

What its letting some score too many points?

 

How about you limit the gobal scores to 20 benches? of the benchers choise,this way there is never Too many benches?

 

Just dont jump the shark and dont fix something thats not broken.

As for the security concerns since when can you trust ALL benchers.

I dont know do what your going to do

But dont be changing things just for the sake of changes,have good reasons to get rid(or add)benches.

 

And no the"my sponcers dont care about older benches" does not count.

Edited February 13, 2013 by cowgut

[PROBN4LYFE]

If you want to "legacy" benches...that's fine but you can't take points that have been earned.

[Vinster]

I like the thoughts, but I don't see any reason to change what has been working.

 

Old benches shouldn't be removed for the sake of being old. Many ppl including myself only bench old HW due to the cost of the hobby... I won't ever own the latest and greatest anymore, it's just too costly.

 

new benches are for new HW...

 

If you want some separation, create a Legacy HW/Benchmark section, leave the points structure as it is, if I get an old CPU/GPU I want to be able to compete with the submissions from 5 years ago. I wouldn't want to lose that right, and if you take that away you'll alienate myself and many other members in my position.

 

All hardware in existence have a place imo. don't take that away.

 

Vin