JPEG Privacy & Security – a new activity launched by the JPEG committee – aims at developing a standard for realizing secure image information sharing, capable of ensuring privacy, maintaining data integrity, and protecting intellectual property rights. This activity is not only intended to protect private information carried by images – i.e. in the image itself or the associated metadata – but also to provide degrees of trust while sharing image content and metadata based on individually set policies. JPEG Privacy & Security will explore ways to design and implement the necessary functionality extension to current JPEG standards without significantly impacting on coding performance while ensuring simultaneously features such as scalability, interoperability, and forward and backward compatibility.

Rationale

Tremendous progress has been achieved in the way consumers and professionals capture, store, distribute, display and ultimately use images. We have been witnessing an ever-growing acceleration in creation and usage of images in all sectors, applications, products and services. This widespread and still growing use of images has brought new challenges related to the various new imaging modalities, increasingly larger data volumes, augmented reality support, privacy and security – to name a few – for which efficient and interoperable solutions have to be defined.

The efforts of the Joint Photographic Experts Group (JPEG) during the past 25 years has resulted in a series of successful and widely adopted coding technologies and file formats such as JPEG and JPEG 2000, and more recently the JPEG XR, JPSearch, JPEG XT and JPEG Systems families of imaging standards.

The JPEG format is today one of the most popular and widely used multimedia standards. Since cameras switched from analog to digital in the early years 2000 and not much later mobile phones integrated communication and image capturing in one device, nowadays several billion of JPEG encoded images are produced per day. Most of us are using JPEG codecs, on a daily basis – often unknowingly – in devices such as mobile phones, computers, tablets, television screens and of course digital cameras. This vast JPEG ecosystem is expected to continue its exponential growth and to generate additional economical value. In the last two decades, a large number of small, medium-sized and large companies have been relying on JPEG technology for their products, and this trend will likely continue.

More recently, we observed the trend to share images immediately via the Internet by means of social media and cloud-based image repositories (i.e. Facebook, Flickr, YouTube, and Pinterest) as reflected in the figure above. Moreover, electronic newspapers are publishing digital pictures from their photographers or image material provided by news agencies or even their readers.

This proliferation of use of digital images gave also rise to a number of conflicts in terms of non-intended release of privacy information, e.g. metadata associated to a published picture that still contained geographical information that allowed to identify persons that have given anonymous interviews to journalists, or pictures posted on social media only intended for a limited audience that went public. Moreover, images provided by commercial stock image repositories or news agencies have intellectual property rights (IPR) associated with them. Once used the content owners prefer that the IPR conditions continue to be applied, can be consulted and monitored as well.

Currently, these concerns are not well addressed and an inhibiting factor in the further proliferation of digital content distribution. The CEPIC Photographic association, the Plus Coalition, and the IPTC Photo-Metadata groups are all actively engaged on the preservation of the integrity of the metadata, the IPR and the distribution of rights to the private and public sectors.

Hence, the JPEG Privacy and Security initiative seeks for solutions that can provide a degree of trust while sharing image content and metadata, and simultaneous also allowing the signaling of the associated policies.

Technologies

In the context of JPEG 2000, a significant part of the required functionality for privacy and security support was already embedded in Part 8 of this standard: ISO/IEC 15444-8 | Recommendation ITU-T T.808.

The particular focus of the JPEG Privacy & Security activity is to provide codestream and file format syntax support to extend security and privacy functionality beyond JPEG 2000, i.e. for all JPEG standards. A high priority is put on JPEG legacy images, but also standards such as JPEG XT, JPSearch and JPEG Systems are within the scope of this activity.

Surprisingly, the necessary extension mechanisms to include similar functionalities in JPEG are to a large degree already present in JPEG XT. This recent standard that enables high dynamic range (HDR) image compression, lossless and near-lossless compression and opacity support for JPEG, separates the image into a base layer available to all legacy decoders and an extension layer only visible to applications conforming to the new standard. For that, it exploit the signaling mechanism of the APP marker segments in the codestream syntax that allows for embedding additional data, which will be interpreted if the JPEG decoder implementation is knowledgeable about the functionality extension and which otherwise will be simply ignored. Consequently, this mechanism ensures that none JPEG legacy decoder implementation will break down as long as the other information embedded in the JPEG codestream is correctly signaled.

Hence, in a simple extension of JPEG XT, it would be only necessary to encrypt the extension layer. In the simplest possible realization, both base and extension layer would be 8 bits per component images, both encoded in JPEG — such an arrangement is already covered by JPEG XT part 6. Image regions could be pixelated or of lower quality in the base layer, while the (encrypted) extension layer would carry the differential signal between intended full-quality image and the degraded base image.

As the extension layer is encapsulated in the marker’s carrying boxes, along with all other metadata, the JPEG privacy extension would only need to specify a method to encrypt selected markers or subsets of markers with a suitable cypher, and at the very same time additional metadata of the images could be encrypted all along with the image data. This could be used to secure Exif or IPCT metadata necessary for professional image applications while allowing some private use of the image content.

Preliminary requirements identified for JPEG Privacy & Security indicate that the standard preferably should carry support for — but not limited to:

• JPEG and JPEG 2000 backward compliant codestreams;
• encrypting metadata and image data independently;
• hierarchical levels of access and multiple protection levels for metadata and image protection;
• evaluation of privacy policies and controlled access to partial or complete metadata and image data;
• a set of default protection tools (cf. JPEG 2000’s JPSEC framework): ciphers such as AES, authentication, hashes, certification, and digital signature;
• a registration mechanism to enable for additional protection tools;
• storing of non-protected information in the APP11 marker;
• a box-based file format as specified in JPEG Systems (ISO/IEC 19566);
• JPEG native metadata schemes, Exif metadata, IPTC Photo metadata, IPR metadata schemes and other types of metadata formats;
• a provenance mechanism to allow tracking modifications made to an image and its accompanying metadata as well as guaranteeing the integrity of an image.

Summarized, JPEG Privacy and Security aims to provide technical solutions, for resolving privacy and security issues, which are compliant with legacy technology in the domain, i.e. both image coding technology as well as metadata standards that signal e.g. access policies and IPR conditions.

Action plan

The JPEG committee intends to interact closely with the actors enabling image privacy and security. This will be facilitated initially by focused workshops that are targeted to understand industry, governmental and user needs in terms of technology and supported functionalities. As a consequence, calls for evidence and/or contributions will be issued to launch new standards or extend existing JPEG standards. The latter consideration is extremely important since JPEG standards have been very successful and wherever feasible the JPEG committee will build upon existing legacy. A first workshop will be organized on October 13th, 2015 during the 70th JPEG Meeting in Brussels, Belgium.

Invitation for Participation

One of the strengths of ISO/IEC/ITU-T standards is that they are created by the people that need them. Industry, governmental and academic experts drive all aspects of the standard development process, from deciding whether a new standard is needed to defining all the technical content. Getting involved in this process can bring significant advantages to your business and research.

Standards are developed by groups of experts called technical committees. These experts are put forward by ISO’s national members. If you are interested in getting involved, contact your national member. The standardisation will follow IPR policies according ISO/IEC directives. Contact details can be found in the list of national members. For particular information related to JPEG standardization, you can contact as well and/or .

To stay posted on the action plan for JPEG Privacy & Security, please regularly consult our website or optionally subscribe to our e-mail reflector: (in order to subscribe to the mailing list, you simply have to send an empty email to – and – and follow the steps of the e-mail being received).