image-comparison-sources

This set of scripts allows to generate compressed images, calculate metrics for each quality, average the results and then plot graphs in order to to make a comparison of several codecs.

recipes.json

recipes.json is a JSON formatted files containing the settings for the codec formats to test. You can add new ones easily if you respect the structure:

• quality_start: the quality at which to stop decoding
• quality_end: the quality at which to stop encoding
• quality_step: the interval of quality between two settings
• encode_extension: the extension for encoded images
• decode_extension: the extension for decoded images
• encode_cmd: the command for encoding at a given quality
• lossless_cmd: the command for encoding at a lossles quality
• decode_cmd: the command for decoding the encoded image
• export_to_png: set to true if you want the images selected with rd_select.py to be exported in PNG. It is useful is the encoded format is not recognized by the browser.

Variables recognized:

• $quality: the quality parameter
• $target: the filename of the encoded image
• $target_dec: the filename of the decoded image
• $origpng, $origpng_y4m, $origpng_ppm: the original image to compress, in PNG, Y4M, or PPM format.

rd_collect.py

Generate compressed images from PNGs and calculate quality and speed metrics for a given format. It takes 3 arguments:

• Arg 1: the codec format to test.
• Arg 2: the name of the subset to test (e.g. 'subset1').
• Arg 3: the path to the subset to test (e.g. 'subset1/').

rd_select.py

Select images among the ones generated by rd_collect.py at fifth quality levels (lossless, large, medium, small and tiny) in order to publish it on the website. This step is optional.

It takes the same arguments as rd_collect.py.

rd_average.py

Calculate for each format the weighted averages for the metrics generated by rd_collect.py. It takes 2 arguments:

• Arg 1: Path to the results of a subset generated by rd_collect.py.

  For ex: rd_average.py 'results/subset1'.

• Arg 2: Reference format with which to compare other formats.

  If ommited, default to 'mozjpeg'.

rd_plot.py

Generate a plot for each quality metrics based on the results generated with rd_average.py. It takes 2 arguments:

• Arg 1: Path to a subset with results generated by rd_average.py.

  For ex: rd_plot.py 'results/subset1'.

• Arg 2: Comma-separated list of formats to plot.

  For ex: 'bpg,mozjpeg,flif,vp9'.

Dependencies

• ImageMagick
• ffmpeg
• pandas
• numpy
• matplotlib
• six
• pytablewriter

Licensing

The code is licensed under BSD Clause 3.

rd_collect.py is loosely based on a script written by Josh Aas, licensed under : https://github.com/bdaehlie/web_image_formats

vmaf_v0.6.1.pkl and vmaf_v0.6.1.pkl.model are licensed under the Apache Software License 2.0.