HWBOT x265 Benchmark

[Aleslammer]

Sweet bench, have problems with GPUPI & HWBPrime on my office box absolutely no problems what so ever with this bench, NICE JOB.

[sburnolo]

About two hours to complete but the important thing is that it works on old glories.

 

[/url

 

[/url]

Edited August 19, 2015 by sburnolo

[havli]

Thank you, I'm glad you like the benchmark.

 

It is good to hear it really works on old school hardware. I did some quick testing during development, but I didn't have the patience to wait this long.

 

 

Aleslammer:

Dual Socket 771 PC was one of my testing rigs... I did a lot of alpha testing there. I like dual-CPU boards very much, so I tried to optimize the benchmark frontend to work well on these things.

11.4 fps @ 1080p s really good score for "old" Core 2 architecture. My best score is around 4.5 fps for single Xeon L5410 and ~2 fps for 2x Xeon 5110.

[Trouffman]

Hey if the FPS counter is too low in term of digits, maybe you can use a `score derivation` per say :

22.73fps = 22730 score. But then it is weird.

 

If the bench output like 30FPS as score does that mean you could basically watch the video live as it encode ?

[trodas]

gigioracing -

how many time needed with pIII ?!

 

About 4 hours estiminated. I just run it and get to sleep and took the score in the morning

If there won't be the taking screens during the run and the screensaver won't kick it (that is the only time the machine start swapping during the test at 512MB of ram), the speed might be a bit faster. 3.5h maybe?

 

Hope that tell you something

[Trouffman]

Here is my submission :

http://hwbot.org/submission/2955489_trouffman_hwbot_x265_benchmark___4k_core_i7_3615qm_0.78_fps

 

Using 2 screens, x265 run on the second monitor, the popup was on the first monitor when clicking submit... so no score being shown ( as it is on the seond screen

 

Tested with bench screen on first monitor. And the Submit popup on the second monitor :

http://hwbot.org/submission/2955494_trouffman_hwbot_x265_benchmark___4k_core_i7_3615qm_0.78_fps

 

Screenshot always take on the first monitor then

[trodas]

This submission verification screen is a "bit" wrong, lol:

http://hwbot.org/submission/2955489_trouffman_hwbot_x265_benchmark___4k_core_i7_3615qm_0.78_fps

 

 

 

BTW, tried the last portable version and got this error:

 

 

Hardware detection error, please restart the program.

 

 

Log:

INFO -> Info: ------------------------------
INFO -> Info: 2015-08-22 00:16:37.75
INFO -> Info: MainFrame start
INFO -> Info: 2015-08-22 00:16:49.234
ERROR -> Error: CPU-Z info open error
INFO -> Info: ------------------------------
INFO -> Info: 2015-08-22 00:21:08.14
INFO -> Info: MainFrame start
INFO -> Info: 2015-08-22 00:21:19.468
ERROR -> Error: CPU-Z info open error
INFO -> Info: ------------------------------
INFO -> Info: 2015-08-22 00:21:25.578
INFO -> Info: MainFrame start
INFO -> Info: 2015-08-22 00:21:36.906
ERROR -> Error: CPU-Z info open error

 

...witch is weird, because latest CPU-Z works well for me ( http://valid.x86.fr/fygs46 ), same as all versions before it on this rather standard MSI P4 mobo

Edited August 21, 2015 by trodas

[havli]

Here is my submission :

http://hwbot.org/submission/2955489_trouffman_hwbot_x265_benchmark___4k_core_i7_3615qm_0.78_fps

 

Using 2 screens, x265 run on the second monitor, the popup was on the first monitor when clicking submit... so no score being shown ( as it is on the seond screen

 

Tested with bench screen on first monitor. And the Submit popup on the second monitor :

http://hwbot.org/submission/2955494_trouffman_hwbot_x265_benchmark___4k_core_i7_3615qm_0.78_fps

 

Screenshot always take on the first monitor then

Yeah, the java function to take screenshot only works on the primary monitor (by default). I'll do some research to see whether it is possible to extend it to capture the whole desktop.

 

-------

trodas -> sent PM.

[trodas]

x265 Benchmark works nicely on Win7 gaming machine, but absolutely fail to work on my WinXP SP3 main machine, claiming that:

 

INFO -> Info: ------------------------------

INFO -> Info: 2015-08-24 19:01:50.859

INFO -> Info: MainFrame start

INFO -> Info: 2015-08-24 19:02:02.187

ERROR -> Error: CPU-Z info open error

 

havli suggested to try older CPU-Z versions, but it does not do anything to help. I tried the latest CPU-Z 1.73, then I go by versions 1.72.1, 1.70, 1.68 and 1.66 and then I give up, it just fail to run all the time.

 

CPU-Z works w/o a glitch on this P4 machine, so any suggestions what could get wrong are more that welcome.

 

The config is really nothing special AT ALL:

 

Windows XP SP3 Czech

MSI PM8M3-V (VIA P4M800) - Thermalright SI-128 SE

Pentium 4 650 3400MHz (200x17) 1.375Vcore

2048MB OCZ4002048ELDCPE-K 2-3-2-5 200MHz 2.50V

Sapphire R9100 128MB 250/200MHz @ 293/248MHz

250G Samsung 850 PRO 512MB cache (MZ-7KE256BW) MLC chips

1024G Western Digital Black 64MB cache (WD1003FZEX)

 

...so, anyone get this standalove version to work on WinXP w/o Java installed?

[Aleslammer]

I’ve had no problems without a java install using XP SP3 english both socket 939 & 1366 although 1366 scores sucked and probably the 939 also just nothing to compare to.

[IanCutress]

My input:

 

Doesn't display properly on HiDPI displays - allow for 150% / 200% scaling.

The submit 'take a screenshot' takes a long time when running high resolution displays. If you're going to implement a result submission that relies on a checksum, I'd suggested limiting the screenshot to an ALT+PRTSCN application that just captures the benchmark window.

Show the time taken to process the benchmark at the end

Show the estimated time to complete during the benchmark

Allow a command line interface to run the benchmark which screenshots the result, saves the HWBOT file, and quits out automatically. That way someone could script up 50 runs and just choose to submit the best.

Automatically generate a result text file with a verbose output, or in the log include the benchmark result.

Include 8K. For LOLz.

 

 

Out of interest, have you spoken at all to MultiCoreWare in preparing this benchmark? They develop the x265 algorithm which is meant to be the toughest and most efficient in the business. I had a good long chat with their VP of product management about our own x265 encoding tests at AnandTech, relating to overclocked stability while transcoding, last week at IDF. We even discussed the potential of an open benchmark, like x264 HD 5.0 and the like. Looks like you're already ahead here, albeit with Java in tow. But it might be interesting if you did a cross collaboration on this. I can make intros where necessary, hit me up.

[trodas]

Report on machine with 512MB of ram:

1080 test is possible, lenghty, but possible: http://hwbot.org/submission/2959296_

4k test is impossible, machine run out of the swap (768MB specified, user size, no enlargement)

 

Conclusion: 1G ram requirments valid for 4k, 512MB possible for 1080p test.

[havli]

trodas:

Thank you, I'll update the RAM requirements section. P4 Celeron is really slow by the way

 

 

borandi:

I'm aware of the scaling issue. Java Swing doesn't work well with non-default DPI, so I disabled the scaling completely to avoid weird looking and broken GUI layout.

 

In the current version (1.1.1), screenshot is captured as a png. I realise now the size of lossless png is too big, especially for large screens. So I'm going to use jpg instead to keep the size reasonable in the next update.

 

Total time and ETA to finish should be possible to add if I find a free spot for it in the GUI.

 

Command line interface - I don't think this is a good idea, there ale enough options to to get the best score possible as it is. And most people are running default settings anyway.

 

Actually, the x265 console output is written to text file after each run even now. It looks like this:

yuv  [info]: 3840x2060 fps 23976/1000 i420p8 unknown frame count
raw  [info]: output file: run0-2160p.hevc
x265 [info]: HEVC encoder version 1.7+374-b015514a93868e2d
x265 [info]: build info [Windows][GCC 5.2.0][64 bit] 8bit
x265 [info]: Compiling by KG7x [x265.ru]
x265 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX
x265 [info]: Main profile, Level-5 (Main tier)
x265 [info]: Thread pool created using 4 threads
x265 [info]: frame threads / pool features       : 2 / wpp(33 rows)
x265 [info]: Coding QT: max CU size, min CU size : 64 / 8
x265 [info]: Residual QT: max TU size, max depth : 32 / 1 inter / 1 intra
x265 [info]: ME / range / subpel / merge         : hex / 57 / 2 / 2
x265 [info]: Keyframe min / max / scenecut       : 23 / 250 / 40
x265 [info]: Lookahead / bframes / badapt        : 15 / 4 / 0
x265 [info]: b-pyramid / weightp / weightb       : 1 / 1 / 0
x265 [info]: References / ref-limit  cu / depth  : 2 / 0 / 0
x265 [info]: AQ: mode / str / qg-size / cu-tree  : 1 / 1.0 / 64 / 1
x265 [info]: Rate Control / qCompress            : CRF-28.0 / 0.60
x265 [info]: tools: rd=2 psy-rd=0.30 signhide tmvp fast-intra
x265 [info]: tools: strong-intra-smoothing deblock sao
1 frames: 0.44 fps, 314.64 kb/s  
2 frames: 0.74 fps, 2176.72 kb/s  
3 frames: 1.02 fps, 3028.76 kb/s  
5 frames: 1.52 fps, 2940.84 kb/s  
6 frames: 1.47 fps, 2705.78 kb/s  
8 frames: 1.83 fps, 2589.36 kb/s  
9 frames: 1.94 fps, 2566.13 kb/s  
11 frames: 2.00 fps, 2455.78 kb/s  
13 frames: 2.23 fps, 2517.81 kb/s  
14 frames: 2.30 fps, 2522.64 kb/s  
16 frames: 2.34 fps, 2439.85 kb/s  
18 frames: 2.53 fps, 2796.70 kb/s  
19 frames: 2.57 fps, 2788.14 kb/s  
21 frames: 2.57 fps, 2933.98 kb/s  
23 frames: 2.71 fps, 3159.12 kb/s  
25 frames: 2.82 fps, 3180.12 kb/s  
26 frames: 2.72 fps, 3174.97 kb/s  
28 frames: 2.84 fps, 3494.32 kb/s  
29 frames: 2.81 fps, 3524.84 kb/s  
31 frames: 2.76 fps, 3747.71 kb/s  
34 frames: 2.91 fps, 4068.00 kb/s  
36 frames: 2.92 fps, 4213.74 kb/s  
38 frames: 2.82 fps, 4158.86 kb/s  
39 frames: 2.80 fps, 4177.12 kb/s  
41 frames: 2.87 fps, 4248.72 kb/s  
42 frames: 2.89 fps, 4270.79 kb/s
.
.
.
.
.   

 

8k - when quantum computers are ready.

 

No, I didn't speak to them. I just saw this http://x265.ru/en/x265-hd-benchmark/ , did some testing on various hardware and realized the x265 is very good for benchmarking both latest CPUs and legacy hardware as well. I thought - the encoder is opensource... so why not create my own GUI for it and use it on HWBOT.

[IanCutress]

High DPI: It's one of those things everyone is trying to get right. Perhaps offer two versions of the interface, one that fits nicely into a 4K screen

Screenshot: Awesome, thanks

Time left/time taken: Also, thanks!

CLI: I ask for this in the sense that say I'm testing a bunch of CPUs at stock for a review, and I want to automate the process but still have the results applicable for comparison. So you can disable HWBot submissions with CLI mode, but just so I could get a score out without having to babysit my benchmarking routine.

8K: Do it. Do it. Do it. If you can find an appropriate video to add to the download (1GB?). You'll be surprised and how the x265 algorithm deals with small frames and big frames, where L3 cache / eDRAM matters and whether it can exploit IPC. That's when the step ups to larger L3/core might be more significant.

 

MultiCoreWare: Like I said, I only discussed it as part of an IDF meeting and they hadn't given it much thought at that point so you wouldn't have seen any basic benchmark online from them yet. But we did discuss and they were keen - they'd work with anyone who is/was willing to develop one and spread it out.

[havli]

I'll see what I can do about the DPI scaling. If all goes good, it might get implemented after v1.2

Time labels are in place and working quite good. Of course the remaining time is not very accurate - encoding speed is not linear, so time sometimes go backwards, depending on current scene.

 

 

CLI: Oh, I understand now. I'm testing processors for web magazine too (retro hardware), unfortunately my methodology forces me to keep the benchmarking sequence running in person. It should be possible to implement batch run - using command line to start the benchmark and save results automatically... in theory at least. This will take some work, so no ETA on this one. Currently the code is not designed for operation like this.

 

8k: The download is big enough as it is (almost 500 MB). Adding another big video... uh, that would be too much IHMO. Also RAM capacity might become a limiting factor. 1080p ffmpeg + x265 needs at least 512 MB (no problem here), 4k = 1,5GB, 8k = who knows, I bet it would be a lot. Also when running "overkill mode" (aka multiple instances of encoder at the same time), RAM requirements grows very fast.

 

L4 cache can indeed make a huge difference in some applications, mostly games. I saw the Broadwell reviews. Huge Haswell-EP Xeons with large L3 cache (>20 MB) sometimes can pack similar boost. Apparently when performance critical part of the application can fit in L3 or L4 cache, performance increase is massive.

[Massman]

Why not make a special 8K benchmark application? So only if you really want to run 8K, you'll download the huge file.

 

I wonder if Tim has 8K footage though

[trodas]

...and if there is a good, 8k footage, then what will be the size to download Memory requirments around 4G is expected and not all that impossible to meet on hi-end machines and for nothing else is this 8k rendering loop aimed at. Clearly PIII is out of the question, unless someone get a PIII machine with at least 2G of ram _AND_ SSD as swap drive. That might produce result in 8k... but it will took ages.

[havli]

Just a little heads up.

Version 1.2.0 is ready. There are some security measures implemented as well a couple of improvements and bugfixes.

 

v1.2.0 (September 7th, 2015)

 

-Improved security

-Submit online - fixed CPU name (it was broken on single socket boards)

-Thumbnails now in zip file

-Partial optimization for high-dpi LCD

-Multiple screens captured on screenshot when using extended desktop

-95% jpg screenshot instead of png - faster upload to HWBOT

-Elapsed time and remaining time indicated during encoding

-Batch testing introduced - useful for stability check or detailed performance analysis

http://hw-museum.cz/hwbot_x265_benchmark.php

 

This version will be mandatory for submissions as of Monday 14th of September (a week from now). Until then v1.1.1 is still valid. Unfortunately it is not possible to use both versions simultaneously. I must switch them manually - which will happen next Monday.

Old saved data files will become invalid.

[QuantumX]

Hi guys, when submitting, is overkill and PMode scores valid?

 

I already seemed to score abnormally high http://hwbot.org/submission/2973796_quantumx_hwbot_x265_benchmark___4k_core_i7_6700k_5.04_fps

 

With 4x overkill mode I scored 5.51

[havli]

Yes, both are allowed.

[havli]

From now on only version 1.2.0 is allowed.

 

I made two submissions using v1.2.0 and all seems to be working well. If you find some bug, please report it here.

http://hwbot.org/submission/2976861_

http://hwbot.org/submission/2976863_

[Taloken]

Hi.

 

Just wanting to try x265 Bench.

double-click on exe, splash screen for 2 seconds, and black screen with only visible cursor, system freezed, had to reboot.

 

8350 @stock, M5A99X Evo R2.0, R9 390, Catalyst 15.7, W7 Pro 64 bits.

[GENiEBEN]

Hi.

 

Just wanting to try x265 Bench.

double-click on exe, splash screen for 2 seconds, and black screen with only visible cursor, system freezed, had to reboot.

 

8350 @stock, M5A99X Evo R2.0, R9 390, Catalyst 15.7, W7 Pro 64 bits.

 

Using the jar or exe? Any system services turned off, i.e WMI?

[havli]

Jar is no longer available in v1.2.0, only exe remains. And it is configured to use only the built-in java to avoid possible incompatibility with various java versions. HW/SW detection relies on cpu-z, so WMI shouldn't be an issue.

 

When the splash screen is displayed, the benchmark is starting in the background... there are two things that could cause crash.

 

1. Artificial CPU load is launched on n-1 cores/threads (7 in case of FX-8350) to determine correct frequency for full load.

2. When the load is active, cpu-z is launched to generate HW & SW report.

 

I guess the artificial CPU load part really shouldn't crash your PC. And if it does, it wouldn't be able to complete the benchmark anyway, as the x265 encoder is much more demanding. So maybe some sort of cpu-z incopatibility? You could try to use older version (1.72), maybe it will fix the problem. Just replace cpuz_x32.exe in the x265 directory.

[trodas]

it wouldn't be able to complete the benchmark anyway, as the x265 encoder is much more demanding

 

Agreed. Multithreaded Prime95 or SuperPi 32M or wPrime 1024M do it for my ASRock at 227x17, but x265 bench did not. At 225x17 it does pass. At 227x17 two cores (single it passed!) it does hang at about 87%...