Lucid Virtu MVP: Revolution in benching?

[Hondacity]

Decide on it after one month of release with the z77. Too premature to put out a decision.

[der8auer]

Then it's too late. We have to make a decision before the lauch or ppl will submit results and we don't know whether or how we accept them.

[Hondacity]

1000-10,000 results to moderate is easy

 

we could say in the headlines: "please indicate mvp assisted results, otherwise banned for 2 years"

[TaPaKaH]

Then it's too late. We have to make a decision before the lauch or ppl will submit results and we don't know whether or how we accept them.

not a perfect solution, if the technology lives for only a year (Lucid ) then some future scores are going to be handicapped.

 

I think a public poll is in order (add now / wait and see / never add)

[FM_Jarnis]

Then it's too late. We have to make a decision before the lauch or ppl will submit results and we don't know whether or how we accept them.

 

Note that you can (today) download the Virtu MVP software (asrock version is floating around on the net) and run it on any Z68 motherboard and (ab)use it for big numbers.

 

So it may already be too late. Basically any board that has iGPU support for Sandy Bridge is suspect and the software doesn't seem to be motherboard chipset model or even motherboard vendor locked.

 

We're working with Lucid to get detection to SystemInfo for FM benchmarks but it takes at least several more weeks to develop, test and deploy the update.

[der8auer]

Interesting. Thanks for this info. Actually I got a Z68 ASRock Board here. So will look for Virtu.

[TONY GAL]

can we post results with enable mvp in hwbot or not?

[IanCutress]

1. total fps - 140.000

2. rendered-fps=1 80.000

3. rendered-fps=0 60.000

 

if that is actually what lucid does...it is in no way usefull for benching. since when we count skipped fps??? if so we would have called the 03 blackscreen bug the best tweak ever but I can't remember we did so

 

I think you're getting confused about what MVP does. It doesn't 'skip frames', it negotiates which part of the previous frame can be kept for the next one.

 

So it isn't - 1+0+1+1+0 for 5 frames,

it would be 1+0.2+0.6+0.5+0.1

[Hondacity]

can we post results with enable mvp in hwbot or not?

 

needs a new thread in the forums for discussion. please share

[pro]

i think all the arguments about new software versions of lucid giving performance increases are debunked, remember vga driver updates often give big boosts in scores and we dont ban them,

 

i many reasons NOT to use Hydra, after all, scores eventually get passed, weather its by a new platform, betterh CPU, vga driver or some new tweak... not adopting something because we have to rebench to keep our points or ranking the same is certainly not a reason not to adopt something

 

the one negative thing i do see as a possibility of MVP is taking GPU clocking slightly out of the equation,

 

benchmarks like heaven that are totally GPU dependant arnt affected, they still scale nicely with gpu clocks and MVP, but other benchmarks like 3dmark03 perhaps, that are a nice balance of GPU and CPU get skewed more towards CPU and this could make benching even more dependant on having a good CPU

 

i dont want to see a benching community where you can take world records with a single card on air if you have a better CPU, but at the same time i dont want to be leaving behind 30%-100% performance that we could potentially be gaining, so its still a hard call for me

 

as a gaming prospect though it is simply amazing and a serious revolution for anyone considering building a gaming rig

[SoF]

So it isn't - 1+0+1+1+0 for 5 frames,

it would be 1+0.2+0.6+0.5+0.1

 

interstesting - but I highly doubt it is that way because you can't simply split and "re-puzzle" an image that fast considering the boosts of mvp.

if that would be so easy mvp would work better on 2 dedicated gpus.

one GPU to render actual image, one to decide for the parts that don't need to be recalculated.

 

The way it makes things CPU-related in 03 gives me a headache to understand...

 

Currently there is one main issue to me: 1 card air + MVP = WR 03 = benchmark useless

[Massman]

About HWBOT ranks/points and Virtu MVP

 

So, as you can all see from this thread, the discussion regarding Virtu MVP's legitimacy in competitive benchmarking is still going on. Although we've reached a point where we know in what situations Virtu MVP should and should not be considered legit, we do not have concrete information on how Virtu MVP works exactly. This piece of information is vital in this discussion.

 

For the time being, HWBOT will not award points or ranks to benchmark results obtained with the aid of Virtu MVP. In our current development sprint, we've added a feature that would allow users to indicate if they used Virtu MVP to obtain the score. In case Virtu MVP was used, the result will automatically be set to 'no points, no ranks'. Until this feature is installed on the production server, we would ask you to use the 'I do NOT want points for this submission'-checkbox.

 

Please note that no final decision about Virtu MVP has been taken yet. It's possible that Virtu MVP will be allowed in the normal rankings, or that it will get its own ranking or even that it will be completely disallowed. We hope to have all the pieces of the puzzle shortly so we can have a final conclusion on the legitimacy of this new technology.

[sumonpathak]

what if someone uses MVP and not disclose it?

will you guys be monitoring all Z77 submissions?

[Massman]

I reckon the difference in score is big enough for the staff to catch them?

[subaruwrc]

i think its fun to have new limits, but maybe dont drop it into the deep water the first time. let a bit of time, people make enough submissions, to see the differencies, and change the rules according to them. I mean a new wr with a single vga aircooled is a bit absurd to me.

[Mr.Scott]

So.......explain to me what is the difference between using MVP software in the 3D benches, and using software in PCM05 ?......which you allowed and is turning the bench upside down, which is exactly what you fear using MVP in the 3D marks.

There is no difference to me.

[ME4ME]

Without knowing the details, or tried the software myself.. one problem occurs to me.

If HWbot is to accept MVP, then SB-E, AMD Phenom, Bulldozer and future high-end CPU's will all be useless for 3D benchmarking since they have no IGP.

[I.M.O.G.]

About HWBOT ranks/points and Virtu MVP

 

So, as you can all see from this thread, the discussion regarding Virtu MVP's legitimacy in competitive benchmarking is still going on. Although we've reached a point where we know in what situations Virtu MVP should and should not be considered legit, we do not have concrete information on how Virtu MVP works exactly. This piece of information is vital in this discussion.

 

For the time being, HWBOT will not award points or ranks to benchmark results obtained with the aid of Virtu MVP. In our current development sprint, we've added a feature that would allow users to indicate if they used Virtu MVP to obtain the score. In case Virtu MVP was used, the result will automatically be set to 'no points, no ranks'. Until this feature is installed on the production server, we would ask you to use the 'I do NOT want points for this submission'-checkbox.

 

Please note that no final decision about Virtu MVP has been taken yet. It's possible that Virtu MVP will be allowed in the normal rankings, or that it will get its own ranking or even that it will be completely disallowed. We hope to have all the pieces of the puzzle shortly so we can have a final conclusion on the legitimacy of this new technology.

 

It would be cool if there was a page on hwbot where only announcements like this were made - like hwbot policy milestones page, it helps to keep track of what the current rules are, and keep track of pertinent issues that affect the community. Currently, the best list we have is "find all posts by massman". Takes a lot of digging or monitoring of the forum to keep up, and often times issues like this don't actually lead to a rule change immediately... But they might weeks or months down the road.

 

I'm not interested enough to follow the forum every day to catch a random thread or post like this, but I would follow a page dedicated to announcing hwbot's stance on issues like this that can impact the rules or interpretation of the rules.

Edited March 19, 2012 by I.M.O.G.

[Vinster]

^ I agree with this. that post is pretty definitive... also updating the different Bench Rules section would be handy as many people go and read those first and then posting a submission...

 

adding to this post, having an announcements section only imo isn't enough for new members only experienced and current members. the rules of the benches should be updated to reflect this.

 

Vin

[Gunslinger]

Without knowing the details, or tried the software myself.. one problem occurs to me.

If HWbot is to accept MVP, then SB-E, AMD Phenom, Bulldozer and future high-end CPU's will all be useless for 3D benchmarking since they have no IGP.

 

lol, awesome point.

[Mafio]

lol, that's good, at least we will not have to buy 1000 € CPUs

[Mikecdm]

So.......explain to me what is the difference between using MVP software in the 3D benches, and using software in PCM05 ?......which you allowed and is turning the bench upside down, which is exactly what you fear using MVP in the 3D marks.

There is no difference to me.

 

One big difference is that the software tweaks in pcmark can be used on every platform, and only affect one bench. MVP can only be used on certain platform and will render those that cannot use MVP, useless. But then again, the same can be said for every platform that isn't intels latest and greatest. Who actually benches 3d on amd .

 

I think you make a valid point here. The turning the standings around thing is a useless argument. Every new platform does this. Some benches can stand the test of time, some can't. Maybe it's to cater to the masses? Not that many people play PCmark compared to 3d. Just about anyone can participate in the 3d benches and fair well in the rankings. For PCmark, if you don't have a dedicated storage set up, you just aren't going to do well.

 

What if we see it as similar to Physx for vantage. It wasn't permitted because the red cards couldn't do it. Well, from what I know, which isn't much about MVP, not every platform can do it. At least with the PCmark tweaks, everyone has a fair shot, regardless of what cpu or motherboard you have. With MVP, only those with the necessary hardware will be able to heavily skew the results.

[gnidaol]

Am I wrong or MVP only won't work on X79, P67 and FX chipsets in the future?

I think that I've read somewhere that it will take part in every platform from now on... Am I wrong?

[I.M.O.G.]

I agree that turning the standings around is a bad argument for disallowing MVP.

 

Scores achieved with MVP are not directly comparable to scores without it, and the advantage of MVP is purely a technology based development that applies across multiple benchmarks. Its advantage is not tied to knowledge or supported by learning, and its a clear demarcation point. For that reason, I think it seems logical to separate the rankings for MVP based results. If we want to keep competing with traditional high end CPU/GPU combos, which is the most popular category currently, it seems separate rankings are the only way to remain competitive and draw valid hardware comparisons.

 

That leaves only 3 meaningful logical differences I can immediately identify from PCM05. First, is that PCM05 tweaks are tied to research, knowledge, and familiarity with the benchmark - these are good things to reward in a community founded on tweaking benchmarks. Second, there was no clear demarcation point with PCM05, tweaks were researched and evolved, and the rules evolved with them (few would argue the rules evolved perfectly, but rules were made in a best effort sense to keep it as interesting as possible). Third, tweaks in PCM05 are not platform specific - they apply across any mobo/cpu/storage setup, and there isn't one element that independently dominates all rankings... Each tweak in PCM05 has stretched the envelope for whats possible for total scores on a given hardware setup, but no single tweak, chip or piece of hardware have put relatively weak rigs in top spots. PCM05 tweaks make whatever hardware you have score better.If you want a top spot, everything must work together almost perfectly.

 

I think the comparison to PCM05 is a relevant one. OS and software tweaks have grown PCM05 scores incrementally over time regardless of CPU and GPU - it has learning and improving benchmark scores at it core. MVP is a disruptive instance of technology that independently renders scores not directly comparable, which is only supported on certain platforms - it has nothing to do with learning, though it does improve scores. I think MVP is an easier judgement call to make, because the situation appears different enough... HWBot is about competing against people with similar hardware who know how to run benchmarks well.

Edited March 20, 2012 by I.M.O.G.

[dinos22]

Am I wrong or MVP only won't work on X79, P67 and FX chipsets in the future?

I think that I've read somewhere that it will take part in every platform from now on... Am I wrong?

 

I heard rumours that one manufacturer will have licenses for all chipsets while others will only adopt it for Z77. There also might be some companies that don't take it up at all which used here on HWBOT...

 

This could be bad for OC community and HWBOT too if it turns out to be right and MVP is accepted for regular points.

 

This would be one concern but HWBOT may be able to restrict it and only accept it on a platform that everyone uses it on to make things even.

 

Second concern is the screen refresh frequency and if it impacts MVP. If so and it cannot be controlled, there will be a big problem figuring out who is putting out legit benchmarks and who isnt...

 

There are also some concerns about how heavy it is on GPU tests and it makes scoring insignificant unless you have the highest clocked CPU. After testing with monster CPU clocks and finding that i need even more monsterous CPU clocks to really make any impact on the score (say 3DMARK Vantage or 3DMARK11) and almost bugger all from GPU clocking, I'm thinking benchmarks would have to somehow account for that...It's a tough one this last one.

 

I am all for it personally but there are aspects that need to be right first before it is accepted.

Edited March 20, 2012 by dinos22