Jump to content
HWBOT Community Forums

How do HTPP work?

Recommended Posts

In data mining for the most valuable GPUs for hardware points, I came across something that surprised me. Coming in last among all teams yields pretty good HTPP - I thought it would yield nearly zero.


The most valuable GPU for 3DM05 HTPP is the 8800 GT 512MB: http://url.hwbot.org/YR42Zs


That didn't seem surprising. However the worst submission for the 8800 GT 512MB is worth 7.9HTPP. That is the lowest team high score ever submitted to hwbot for 3DM05 on the 8800 GT 512MB (ranked 165th out of 165 team submissions), and its worth 8 HTPP. Compare that to the 82nd out of 165, and that's worth 13.8 points. Only 6 extra points for beating half the team submissions, compared to the team who submitted the worst score on record.


Same thing with Radeon 4870 in 3D06. 7.4 HTPP for 183 out of 183 teams.


Compare that to an obscure card, and for being best out of 6 team submissions, 1st place gets you 6.1HTPP.


Bottom line: Shouldn't the HTPP range start at 0 or something? The rankings are supposed to kind of reflect level of difficulty... And it might be kind of hard to come in last out of hundreds of submissions, I don't think this is what was intended.

Edited by I.M.O.G.
Link to comment
Share on other sites

The range shouldn't really start at 0; the UHP also start at 0.1pts. 8 HTPP would be the equivalent of the 0.1 UHP.


Using the word "beating" to indicate 82th out of 165 teams is maybe not the right terminology because a lot of these scores are done with not-so competitive hardware. I should look deeper into the matter, but getting 82th place shouldn't be that difficult, right?


Are there any major complaints regarding the HTPP? Or more of a general note?

Link to comment
Share on other sites

Nope, no major complaints. I don't really think its a problem, I mean the algorithm works. I guess this is just an edge case where it looks sort of weird and made me wonder how HTPP worked. Mainly for instance, in popular categories the points for last place seems to vary but its usually more than than first place in an unpopular category. Just seems weird and I hadn't noticed before.


It makes sense though when you say 82nd out of 165 isn't really hard, so the reward makes sense when I think about it a little more in that situation.

Link to comment
Share on other sites

Why is 5/5 better than 165/165? Because the 5/5 submission has 160 chances to not become 165/165... Ya, it could actually be the worst score anyone could ever submit. But probability dictates that if 160 more submissions were made, at least one of them would likely be worse. I am not a betting man, but I would take that bet.


A 5/5 score has the potential to be a pretty good result, beaten by 4 better results. It could also just suck, but it could be pretty good.


A 165/165 score is terrible, either due to the choice of hardware, skills, or other problems with the rig. It cannot be a good result, because its been confirmed that everyone else who bothered to run it was able to run it better.

Link to comment
Share on other sites

The result that is ranked 5th out of 5 can not be compared to a result that is ranked 165th out of 165. The position of the one result does not say anything about the other result. You can be 165th on LN2 and 5th using stock clocks. You can be 165th because you used a Pentium 3 at 3GHz which is not enough to beat any 3770K and 5th because you didn't overclock your 3960X.


I can agree if you assume that the position indicate a ranking at HWBOT and those rankings have a normal spread in results. But without further information on the characteristics of the rankings and the individual results, you cannot make any exact statement about which is the better result. Not ever. Of course, you can talk in terms of probability and statistics, but we all know that statistics are useful for large sets of data, not individual datapoints.

Link to comment
Share on other sites

Whoa, that seems over complicated.


To be last out of over 100 submissions, you have to do something wrong. What can you do wrong to get last place?


-Bad platform choice

-bad OS

-didn't run the benchmark right

-et cetera



Pick any benchmark in the database where there are more than 100 submissions, look at the last place submission, and you can state exactly what was done wrong. Regardless of the reason when there is over 100 submissions and you get last place, it isn't open for debate - there is sufficient data to confidently say last place is a non-competitive submission.


With very few submissions like <10, last place may be competitive, but there is not sufficient data for the algorithm to be sure if last place sucks.


That said...


Any way you want to slice the pie, when you have >100 submissions in a certain ranking, last place never deserves more points compared to when you have <10 submissions and last place. One is definitely not competitive, one could still be competitive but the algorithm can't be sure... If they received equal points, that might be logical. If the confirmed non-competitive submission receives more points than the potentially competitive submission, that isn't logical.


As I mentioned before, I don't know that its a problem that needs to be fixed really. What difference does it make. I do think the algorithm isn't treating these cases logically however.

Link to comment
Share on other sites

Why is 5/5 better than 165/165 ?


Counterexample: if 5/5 equals 165/165, then 1/1 (or 2/2 if you like) equals 5/5, which again equals 165/165. Though, the opposite must also be true: 1/5 equals 1/165, but... which one gets the most points? ;) By that logic I should get 50 points for all my 1/x scores, where perhaps x must be different from 1 (those would get 50 and 0.1 at the same time, so perhaps 25 is reasonable?). Feel free to adjust this in the next update :P


If you think about it: say you're 5/5 a day or two after a new product hit the shelves. If another 160 people submit their scores your score has to be REALLY bad if all those 160 folks beat you, right? It's reasonable to assume at least a few of those would be even worse, unless your score is... garbage.

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

  • Create New...