Who washed my video? x264 vs x265 ‘placebo’ comparison

I wanted to know something about the actual x265 development state and since I didn’t find anything up to date I decided to do my own tests.
First of all I compiled a recent snapshot of x265:

./x265 --version
x265 [info]: HEVC encoder version 0.7+259-5e2043f89aa1
x265 [info]: build info [Linux][GCC 4.8.2][64 bit] 8bpp
x265 [info]: using cpu capabilities: MMX2 SSE SSE2Fast SSSE3 SSE4.2 AVX

x264 –version
x264 0.142.2389 956c8d8
built on Feb 21 2014, gcc: 4.8.2
configuration: –bit-depth=8 –chroma-format=all
x264 license: GPL version 2 or later

This is my 112s / 216MB test video:
https://mega.co.nz/#!eQhSjJQR!EEe8-taN5IspIu-RW0WQzmvKzc5fkCn282kS5ugZ_as

Let’s encode it using x265 first:

ffmpeg -v 0 -i ~/PlanetEarthBirds.mkv -threads 4 -vsync 0 -an -pix_fmt yuv420p -f yuv4mpegpipe - | ./x265 - --y4m --preset placebo --crf 28 -o prova.h265

As you can see I used ffmpeg to decode the video and then I piped the raw stream to the x265 encoder. I did a quality based encoding with the default crf (28) which gives low quality but low size output files (considering my source isn’t lossless this is exactly what I want to easily spot differences with x264).

Then I generated a file of the very same size using two pass encoding with x264:

ffmpeg -v 0 -i ~/PlanetEarthBirds.mkv -threads 4 -vsync 0 -an -pix_fmt yuv420p -f yuv4mpegpipe - | x264 - --demuxer y4m --preset placebo -o prova.h264 --pass 1 --bitrate 3310
ffmpeg -v 0 -i ~/PlanetEarthBirds.mkv -threads 4 -vsync 0 -an -pix_fmt yuv420p -f yuv4mpegpipe - | x264 - --demuxer y4m --preset placebo -o prova.h264 --pass 2 --bitrate 3310

Here is the result:

-rw-r--r-- 1 niko niko 46094455 22 feb 04.19 prova.h264
-rw-r--r-- 1 niko niko 46238768 21 feb 21.26 prova.h265

As you can see prova.h264 is 46094455 bytes (~45MB) while prova.h265 is 46238768 bytes (~45MB). The video encoded with x264 is just 0.14MB smaller which is a 0.3% difference.

Using the placebo presets I used the slowest settings which are supposed to give the best quality for both encoders.
Having to use two pass encoding for x264 to make a meaningful comparison there is no sense comparing encoding speed, so I will not do it.

Here are the encoded files:
prova.h264
prova.h265

Anyway we can compare DECODING speeds :)
Let’s decode them as fast as we can using ffh264 (FFmpeg H.264) and ffhevc (FFmpeg HEVC / H.265):

mplayer -benchmark -nosound -lavdopts threads=16 prova.h264 -vo null 2> /dev/null

BENCHMARKs: VC: 5.640s VO: 0.003s A: 0.000s Sys: 0.333s = 5.976s
BENCHMARK%: VC: 94.3730% VO: 0.0482% A: 0.0000% Sys: 5.5788% = 100.0000%

mplayer -benchmark -nosound -lavdopts threads=16 prova.h265 -vo null 2> /dev/null

BENCHMARKs: VC: 26.936s VO: 0.003s A: 0.000s Sys: 0.360s = 27.300s
BENCHMARK%: VC: 98.6680% VO: 0.0123% A: 0.0000% Sys: 1.3197% = 100.0000%

As you can see the video encoded with x264 took 5.976s to decode while the x265 one took 27.300s.
h265 actually takes 4.6x more time than h264 to decode. Not bad, really.

Let’s take some screenshots:

mplayer -lavdopts threads=16 -nosound -benchmark -fps 23.976 -vf framestep=275 -vo png:z=9 prova.h264; for i in `ls -l *png | awk '{ print $9 }' | grep -v h26 | grep -v source`; do mv $i x264-test1-$i; done
mplayer -lavdopts threads=16 -nosound -benchmark -fps 23.976 -vf framestep=275 -vo png:z=9 prova.h265; for i in `ls -l *png | awk '{ print $9 }' | grep -v h26 | grep -v source`; do mv $i x265-test1-$i; done
mplayer -lavdopts threads=16 -nosound -benchmark -fps 23.976 -vf framestep=275 -vo png:z=9 ~/PlanetEarthBirds.mkv; for i in `ls -l *png | awk '{ print $9 }' | grep -v h26 | grep -v source`; do mv $i source-test1-$i; done

Finally, let’s compare them:

x264 vs x265
http://files.linuxsystems.it/functions/comparison.php?test=test1&codec1=x264&codec2=x265&number=1
http://files.linuxsystems.it/functions/comparison.php?source=planetearthbirds&bitrate1=3310&codec1=x265-0.7-259-5e2043f89aa1-8bpp-placebo&bitrate2=3310&codec2=x264-0.142.2389-956c8d8-8bpp-placebo&n=10&current=1

source vs x265
http://files.linuxsystems.it/functions/comparison.php?test=test1&codec1=source&codec2=x265&number=1
http://files.linuxsystems.it/functions/comparison.php?source=planetearthbirds&bitrate1=source&codec1=x265-0.7-259-5e2043f89aa1-8bpp-placebo&bitrate2=3310&codec2=x265-0.7-259-5e2043f89aa1-8bpp-placebo&n=10&current=1

Press F11 to switch your browser to the full screen mode and click on the image to switch from the x265 screenshots to source or x264. I also put some previous/next buttons (there are 10 png screenshots in total).

I don’t want to be critical because x265 is still in early stages of development, but as you can see it completely washed the video: details are gone. Sometimes x265 did a better job but overall x264 definitely wins.
Keep up the good work devs 😉

10 comments to Who washed my video? x264 vs x265 ‘placebo’ comparison

  • Jackal

    This is a very inaccurate comparison, even for the older version of x265, you are comparing different rate factor methods, as well as presets that don’t directly carry over from x264 to x265. With appropriate encoder settings (not using crf) you should very consistently be able to achieve twice the bitrate at the same file size or half the file size with the same bitrate of an encoding with x264 using identical settings (where available). If your going to publish a comparison, get your facts straight first.

  • Emir Xristoph Paulichenko

    Totally useless and inaccurate comparison. Please don’t take it serious.

  • Bobo

    Emir Xristoph why dont you explain why its useless and inaccurate?

    Otherwise your comment is useless and inaccurate.

    If you can tell why I can take it seriuos

  • Fundamental error – encoding one at constant quality and the other at variable bitrate with variable (burst) quality. It’s pointless to compare the results.

    However, if you look at it closely, x265’s image has slightly softer contrast while maintaining more of the subtle details of the image overall. Look at the leg of the foreground bird and the feathers of its back. x264 blurred both. Contrast != preservation of details.

  • I figure I’d better describe what exactly the OP did wrong, in less technical terms.

    First off, the OP assumed that same filesize = same bitrate = fair comparison. But that’s a very naive assumption. Let me make it very clear: if you encoded the x264 with the first 50% at 200% bitrate and the last half at 0% bitrate and then compared frames from the first half against x265, it would be unfair. If you compared frames from the last half it would also be unfair. But the bitrate’s the same. See where you’re going wrong? x265 was encoded Constant Quality. That makes it attempt to maintain either a constnat PSNR or a constant SSIM or some other approximation model which is hopefully related to how we perceive images, so that every frame of the video appears to be about as good as any other frame. This is equivalent to encodeing a static picture at a constant bitrate for the whole clip. However images change through the clip, and the Variable Rate encoding gives extra bits to low-motion frames to keep up appearances where the eye has a better chance to see the differences, and discards some bits from high motion frames to make up the difference, which means that some images get more bits depending on what came before and after them (amount of movement) and therefore you cannot fairly compare frames from one encode with the other.

  • Thanks Wil, glad someone out there can explain the error without resorting to childish commentary.

  • Paul

    The comparison is not rigorous (as pointed out in the comments).

    The choice of planetearthbirds as source is asinine, as it’s a very low quality input with lots of noise – such sources are not meaningful, because they require preprocessing in order to have a reasonable output. It’s not clear if what appears to be retained in the x264 version is detail or noise. I also suspect that x264’s output for this source has very visible chroma noise, which is not so evident in the static shots.

    the hobbit is a appropriate source for comparison; in this, x265’s quality is better by far – x264 has some very visible blockiness.

  • Confirming your result TOTALLY!
    Today (3rd january 2016) I tried again,
    x264 –preset slower –tune film –crf 22
    vs. newest
    x265 –preset slower –tune grain –crf 22
    Result: Washed out like hell.
    My current fix for x265 washing machine is: –no-strong-intra-smoothing –high-tier
    BUT then the resulting video is about the same size as with x264, compared to five-to-ten times longer to encode.

    x265 works good on comic style, like Archer, but on real (grainy) video it is not yet better.

Leave a Reply

  

  

  

You can use these HTML tags

<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>