Image Compression Grand Challenge at ICIP 2016
November 26, 2015

By this challenge, the organizers of the event are keen to evaluate the state of the art image compression technologies. While the challenge is independent of any standardization body, it should be noted that the ISO working group JTC 1/SC 29/WG 1, known as JPEG, has an open Call for Information (CfI) on new image coding technologies and contributors to this challenge may want consider contribution of their technology as input to the ongoing JPEG activity. Contributors of new image coding technologies may find the ICIP to be a convenient and robust mechanism for promoting new approaches, and bringing new approaches into application. The objective of this call is to have an independent evaluation of the state of the art, and how contributions position themselves compared to existing still image compression standards.

Contributions – Input Requirements

Contributors are expected to provide

  • Contributions should provide a description of the basis of the compression algorithm as compression performance alone is not the sole evaluation criterion.
  • An implementation of algorithm in a form that allows stand-alone execution on a stand-alone computer, ideally in compilable source code form. Software is expected to implement a decoder to the PNM still image format.
  • Contributions are expected to be accompanied with detailed instructions, potentially including options to optimize to particular evaluation criteria such as PSNR, MS-SSIM or subjective quality or more.
  • Contributors are asked to identify how additional features such as spatial scalability, resolution accessibility, amneability to subjective optimization or representation of images of higher bitdepths, high dynamic range, hyperspectral data etc. might be addressed.
  • Contributors are expected to compress the provided test material (download here with their technology to bitrates that will be communicated by the organizers; the compressed codestreams along with the encoder options have to be included in the contribution.
  • Contributors may also provide a maximum of four (4) additional test images at their discretion. Images should be either grey-scale or sRGB with 8 or 12 bits per component, encoded in PNM. Contributed test images must not previously have been compressed using any lossy coding scheme. Contributors are also expected to provide the evaluation committee sufficient rights to allow usage of the provided software and provided input data for the purpose of evaluation of their own, and other contributions to the challenge. The evaluation process may need to crop and/or clip the provided images to make them suitable for subjective evaluation. This requires contributors to provide a suitable derivative license e.g. (CC BY-NC-SA 4.0) for any content contributed.

Input data for the compression algorithm will include at least 8-bits per component grey-scale and color image data from natural scenes.

Evaluation Criteria

The formal assessment process to be undertaken in this evaluation exercise is concerned only with compression performance. It should be noted, however, that compression performance alone is not the sole feature of interest for practical deployment of an image compression technology.

Objective evaluation will at least include the quality indices PSNR and MS-SSIM; a subjective evaluation will be performed by DSIS tests assessing image fidelity, collecting MOS values.

Test imagery for the evaluation will be drawn from a sizeable repository that is maintained by JPEG and a subset of which will be made available to contributors via this link. The test set will at least include 24bpp RGB photographic content.

In addition to the provided codecs, the evaluation will also include well-known standardized image compression solutions such as JPEG, JPEG 2000, JPEG XR and JPEG XT.

Submission Deadlines

The evaluation event will take the form of a ICIP Special Session. Contributions to this Special Session will be reviewed and potentially published as part of the proceedings of the conference following the regular paper submission and review process of the ICIP. All contributors are expected to produce a paper, formatted in accordance with the paper template for ICIP 2016, to be uploaded to the site specified on registration of intent with their final submission. Paper submissions, including any software, images and/or codestreams as an electronic attachment must be uploaded to the ICIP submission page prior February 15th 2016.

Contributors will be notified on the acceptance of their contribution by April 16th.


  • University of Stuttgart
  • EPFL
  • VUB
  • University of Patras

Participating Labs

The following parties will implement the procedures and tasks of the challenge:

  • UBI
  • EPFL
  • CVUT
  • VUB