Applications for JPEG 2000

Digital Cinema

Digital Cinema describes the use of movies with a digital data representation in best quality. Traditionally movies are shot on film and projected with film. Today, this is done with digital cameras and digital projectors. Because of the huge amount of data within this application area data compression is necessary. In contrast to Electronic Cinema, which uses the digitization of the film for new commercialization pathes, Digital Cinema replaces only the film chain from the acquisition to the film theatres. Therefore Digital Cinema must achieve and surpass the traditional best film quality. The parameters for the digital representation of the movie have to be much more extensive than in standard videos.

Other compression standards have different limits for the use in Digital Cinema. This can be the maximum resolution, the compression possibilities (only lossy), the sampling type, the colorspace or the bit depth. However, JPEG2000 is an excellent compression standard for the use in Digital Cinema, because it delivers enough headroom in the description of digital movie data and has outstanding features, which can be used. Some features of JPEG2000 are intraframe coding for a simple editing access, lossless compression capabilities, metadata insertion, scalability in resolution and quality and so on. All features of the still Image JPEG2000 Standard 15444 - Part1 can also be used.

Requirements in data compression for Digital Cinema including high dynamic range, different color spaces, highest image resolutions, best compression quality upto lossless compression, and so were made possible by using JPEG 2000.

Broadcast Market

JPEG 2000 has been adopted by the broadcast industry as mezzanine compression in the live production workflows. The compression offers unique benefits suitable for video production as alternative to uncompressed video. Today JPEG 2000 is used for its high quality and low latency in video over IP applications such as Contribution Links (live events to studio transmission) and recent IP-based broadcast studio infrastructures. Moreover, it is also used as the master format for content storage. The broadcast implementations essentially rely on the JPEG 2000 Amd3 for Broadcast Contribution and the JPEG 2000 Amd8 for Interoperable Master Formats (IMF).

In 2014, several companies received an Emmy® Award for breakthroughs in the Standardization and Productization of transporting video with JPEG 2000 Broadcast Profile in MPEG-2 TS over IP Networks.

Several main benefits for JPEG 2000-based broadcast production workflows exists:

Image archives and databases

One early use of JPEG 2000 will be as a base file format in image archives and databases. Traditionally, image archives store multiple copies of an individual files at varying resolutions and quality levels so that they can supply appropriate image data on request. In addition, considerable metadata is held about each image to allow it to be easily classified and retrieved.

JPEG 2000 files typically can have extensive metadata stored with them, in a standard compliant XML environment. As well as allowing selected metadata from an image database to be distributed to its users, this does permit interchange of image files with metadata between databases, and removes the need for an extensive manual data entry stage when cataloguing new images. In addition, the files can be stored at high quality in a lossless, colour managed environment, with conversion to lower resolution or lower quality performed 'on the fly'. The ability of part of a JPEG 2000 file to be used for generation of such modified images also means that it becomes practical to provide other capabilities on demand.

One example might be to watermark each image as delivered, not only with details which communicate authorship or ownership, but also transactional information. This could include licensing restrictions, details of the customer, or information which would allow the image to be easily recognised through some automated process designed to test for breaches of copyright.

The new Part 8 of the JPEG 2000 standard (JPSEC) dealing with security addresses these possibilities, whilst Part 9 (JPIP) defines how interactive applications between a client and server can be created. This too will be very important in the image database arena - as examples it makes retrieval of selected parts of an image much faster and easier to control, permitting 'pan and zoom' operations on part of an image. Demonstrations of this technology already exist (for example using Kakadu) in which several areas of an image can be selected by a user and are delivered more rapidly that the remaining less interesting parts. A range of novel browsing opportunities exist therefore for remote client software, making the delivery of large high quality image information under user control a practical reality.

Medical Imaging

JPEG 2000 has many characteristics which are useful to one of its target market areas, medical imaging. Some background to this has been covered in a JPEG committee document (N2782) which also gives some useful information about how JPEG 2000 works. One key aspect which often concerns the medical profession is the need to ensure that images can be communicated losslessly, without any distortions introduced by a compression process that may lead to mis-diagnosis. This often results in huge files, which can be difficult to store, handle, and communicate. JPEG 2000 can be used to encode files completely (or partially) losslessly, and provides good compression performance for this purpose (similar, for example, to that offered by JPEG's optimised method for such compression, JPEG-LS (IS 14495)). It does however have several additional features which make JPEG 2000 particularly attractive for medical imaging:

Cultural Heritage

Many cultural heritage institutions such as Museums and Art Galleries have very extensive collections which are not visible to the public because of display capacity and other reasons. Projects, such as 'NOF-Digitise' in the UK with a budget in excess of $80M, are being created to try and provide on-line learning resources and other solutions for global access. Natural disasters such as fire, earthquake and flooding, as well as the problems created by war, vandalism and terrorism show the need to preserve this information in as accurate a form as possible lest the heritage be lost forever. In addition, it is critically important to use widely adopted standards that have some chance of longevity in the face of technological change. The UK 'Domesday' project, for example, in which the BBC helped many schools and individuals put together a comprehensive record of the UK (in November 1986) to celebrate the 900th anniversary of the Domesday Book was created using a BBC Master computer with a proprietary interface to an analogue videodisc. The problems less than 20 years later are self-evident, and the large expenditure on the project stands to be lost, unless emulators for the material can be created.

The original JPEG standard has been existence almost as long as the Domesday project videodiscs described above. Whilst there are only one or two working Domesday players, and some attempts to rescue the situation, there are hundreds of millions of devices which can image JPEG files. The JPEG 2000 standard has been designed from the ground up to try and address many of the areas which concern the users in the Cultural heritage sector. These include:

Wireless imaging

Wireless communications offers its own particular set of problems. More specifically, wireless networks are characterized by the frequent occurrence of transmission errors along with a low bandwidth. Hence, they put strong constraints on the transmission of digital images. Since JPEG2000 provides high compression efficiency, it is a good candidate for wireless multimedia applications. Moreover, due to its high scalability, JPEG2000 enables a wide range of Quality of Service (QoS) strategies for network operators.

To be widely adopted for wireless multimedia applications, JPEG 2000 has to be robust to transmission errors. To address this issue, the JPEG committee has established a new work item, JPEG 2000 Wireless (JPWL), as Part 11 of the standard. Its purpose is to standardise tools and methods to achieve the efficient transmission of JPEG 2000 imagery over an error-prone wireless network.

The main functionality of the JPWL system is to protect the codestream against transmission errors. More precisely, the protection technique modifies the codestream to make it more resilient to errors, e.g. by adding redundancy or interleaving the data. The decoding process detects the occurrence of errors and corrects them whenever possible.

A second functionality is to describe the degree of sensitivity of different parts of the codestream to transmission errors. This information can subsequently be used for unequal error protection. More specifically, sensitive parts of the codestream can be more heavily protected than less sensitive parts.

A third functionality is to describe the locations of residual errors in the codestream. This information can subsequently be used to make a decoder aware of the information loss and to prevent decoding corrupted parts of the stream.

Using the technologies standardised in JPWL, JPEG2000 becomes very resilient to transmission errors. Therefore, JPEG2000 is an ideal candidate for the efficient transmission of digital images and video in wireless applications. Indeed, recent studies have shown that Motion JPEG2000 is very well suited for video transmission over wireless channels. Specifically, it has been shown that Motion JPEG2000 outperforms the state-of-the-art MPEG-4 in terms of coding efficiency, error resilience, complexity, scalability and coding delay.

While the proposed solutions are not tuned to a specific network protocol, particular attention has been paid to three important use cases: 3rd generation wireless phone networks (3GPP/3GPP2), WLAN (IEEE 802.11 family of standards) and Digital Radio Mondiale (DRM).

Among potential killer applications for JPWL, Multimedia Messaging Service (MMS) knows a very rapid growth and is widely seen has one of the only bright spot in the wireless telecom industry. Other potential applications include video streaming and video conferencing.

Pre-Press

Pre-press is the process used when digital files are prepared for printing. Two key requirements of this process are fidelity and consistency. In the past, the pre-press industry has depended on lossless image compression (for example using EPS or TIFF file formats) and colour calibration of all components in the process, using defined lighting and viewing conditions in order to achieve optimum results.

JPEG 2000 offers opportunities to the pre-press industry to both substitute its traditional formats with the more advanced aspects inherent in JPEG 2000, and to re-purpose its content to allow it to be used in Internet publishing or other contexts. The same JPEG 2000 image can generate thumbnails, screen images and print ready material simply by truncating a prepared codestream at different points. In addition, the powerful metadata handling and association in JPEG 2000 files means that Digital Asset Management or workflow processes can be easily linked into pre-press delivery, providing security to the photographer, image creator, and printer alike.

A key facet is the ability of JPEG 2000 to deliver true lossless compression - in one possible operational mode even the colour transform from an defined colour profile such as sRGB is lossless. The ability exists within JPEG 2000 to use a number of well defined colour management profiles, and in particular ICC colour profiles, supporting the CMYK spaces used within the pre-press industry. As the file format can include full colour space definitions (at least in the extended version of JPEG 2000 defined in Part 2 of the standard), including the formula used for transforming to another colour space, proprietary and accurate colour representations can be transferred between systems (at least within the limitations of output devices to render them).

Remote sensing and GIS

Geographic Information Systems (GIS) allow the viewing and analysis of multiple layers of spatially related information associated with a geographical location or region. GIS enables companies and governments to easily analyze the development, maintenance, and impact of roads, vegetation, utilities (water, electrical, communication, sewage).

GIS includes maps, vector information, and imagery. The collection of imagery is commonly achieved through remote sensing. Remote sensing started with aerial photography in the late 1800's onboard a balloon. Airplanes were used to collect information from above in the early 1900's and the first image taken from space was aboard the Apollo spacecraft in 1969. In the early 1970s the first imaging satellite (ERTS-1) collected imagery of the Earth. Images continue to be collected form both space and aircraft and are available for commercial and personal use on the Internet. The challenge for Remote Sensing images for GIS and other applications is the size of the image. Currently, it is common to have images that are greater than 10,000 by 10,000 pixels, multiple bands, and greater than 8 bits per pixel per band. While JPEG DCT is currently being used for the collection, storage and delivery of several GIS applications and remote sensing systems, other compression and file format techniques have become popular because of greater efficiency of storing and accessing large images.

The original requirements for JPEG 2000 included requirements from the remote sensing and GIS community, which have been meet. Greater bit depths, tiles, resolution progression, quality progression, and fast access to spatial locations all contribute to the capability and functionality of JPEG 2000, which make it an ideal technology for the remote sensing and GIS applications. As an open standard, it is expected that JPEG 2000 will become more prevalent in the remote sensing and GIS applications.

Digital Photography

Photography has changed the way people record and remember images, events, and scientific information. Photography, which started in the mid 1800's, has continued to evolve over the last two centuries of scientific discovery. The development of flexible film in the late 1800s, color photography in the mid 1900s, and automatic cameras in the late 1900's changed the way photographs are taken and presented. The most recent addition of digital photography has also changed the way people collect, store, modify, disseminate and display images. Digital photography started with the advent of the first commercial digital cameras for consumers and professionals in the early 1990's, along with the first systems for digitizing film images. As the technology advanced, the cost of digital cameras and film digitization services has dropped, and the image quality has increased. The image size for professional portable digital cameras continues to grow, from about 1 Megapixel in 1993 to 10 Megapixels or more in 2003.

As digital cameras evolve, the requirements for the file format used to store the image data continue to evolve also. Digital cameras continue to increase the size and bit depth collected for an image to increase the resolution and extend the dynamic range and color gamut. Digital Photography requires the ability to compress three-band imagery of 8-to-16 bits per component. Digital photography requires efficient, high quality compression as well as rapid decoding of properly sized images for the camera's display screen. Metadata for the proper use and display of the image is a requirement for digital photography.

Scientific and Industrial

Many applications in the scientific and industrial sector are now turning to the use of image material to either replace or enhance existing data records. Examples include the use of satellite or aerial photography imagery to link to a mapping or GIS system, and the ease of use of digital cameras to provide evidence of satisfactory work completion - for example in road pipework excavation. To some extent these depend on the ubiquity and availability of the standard, and hence projects often use well tried and tested solutions. The widespread availability of digital cameras, and PC sofware to display and print the results are likely to keep costs down - especially important in sectors where deployment has to have a close link to company profitability.

However it is commonplace to have many different versions of images of the same item. As an example, a car manufacturer may have pictures of a vehicle engine used in service, marketing, quality assurance, training and testing applications. As digital asset management becomes recognised as an important control that a company should use, so more attention will need to be spent on reusability and re-purposing of digital assets such as images. JPEG 2000 offers many useful features in this context - proper colour management, compression that can include both lossy and lossless versions of an image in the same file, and extensive options to add user defined and standard metadata to an image file.

In addition, many aspects of scientific and industrial usage involve subsequent processing of a digital image, for example to enhance features or count items. Using any form of lossy compresion for images in this context may create problems - after all the information thrown away during lossy compression is generally that information that is imperceptible to a human eye - not necessarily showing the same characteristics as computer image processing software. It therefore becomes more important to ensure that archival material is stored at the highest fidelity possible - but is still rapidly searchable and viewable during a pre-processing stage for example. Again JPEG 2000 can offer significant advantages in this environment.

Extensive software toolkits are available from a number of vendors which support the new JPEG 2000 standard. These range from the freely available Jasper and JJ2000 software that is linked to Part 5 of the JPEG 2000 standard (Reference Software), to commercial alternatives from KakaduSoft, Aware, Algovision Luratech, Leadtools, Pegasus and others. These allow integration of the comprehensive features of JPEG 2000 into a wide range of products and systems.

Internet

Many key applications of the new JPEG 2000 standard will use the Internet, and Internet technologies to distribute images. JPEG 2000 images have a number of properties which make them very suitable for use with the Internet. Typically, Internet users are constrained from downloading large, high quality images because of their physical file size. Often providers of images must create three or more versions of an image, varying from a tiny thumbnail through to a page size image.

Digital cameras have improved in quality and resolution to a level where they are now competing effectively with traditional film. The images they generate are often no longer directly suitable for Internet deployment - the quality and size is wasted on traditional computer monitors. In part, this is because the monitor might show no more than a quarter of the captured image without scrolling, and in part because the colour fidelity of the monitor does not match that of the camera.

Both of these issues are addressed by JPEG 2000 standards. Images saved in JPEG 2000 format can be coded so that the data when transmitted and imaged gradually increases in resolution, starting with a thumbnail, or gradually increases in quality. A combination of these (and other) quality measures can also be achieved - and the user can stop the image transmission once they have enough detail to make their next choice, as the data is ordered in the file in the correct way to simplify its delivery by image servers.

Parts of JPEG 2000 that were created to facilitate these delivery methods are for example:

Surveillance

Traditional surveillance technology has been quite slow to embrace the advantages of digital image processing. In part this has been because the sheer volumes of data have required analogue storage methods such as lapsed time video recorders, and in part because the cost of moving to a digital base have been prohibitive. In the last few years however, costs have fallen dramatically, whilst processing power and capabilities have improved equally fast. This allows many of the shortcomings of traditional surveillance applications to be addressed, whilst also considering many of society's concerns about privacy and intrusion.

Movement detection, and many more sophisticated forms of image analysis can be coupled with new sensor technology to allow much more pro-active monitoring and alarms. Use of 'region of interest' enhancement allow accurate identification of suspects while excluding from analysis, and subsequent public exposure, the innocent bystander. Tight control can be exerted over the user of surveillance technology - for example showing sufficient details to allow recognition of an individual found to have passed a stolen cheque, whilst not permitting enough detail to allow a corrupt viewer of the scene to be able to view and copy a signature being made.

The need for stored evidence to be of a sufficient high quality however also raises concerns and a need for protection against tampering and fabrication. It is very easy within the digital environment to change either subtly or completely aspects of an image, and the metadata surrounding it. Techniques such as encryption and watermarking can be used to help protect against this risk, but there is a real need for well accepted media management techniques which can help reduce risks in this area, for example using trusted third parties and crypto-technology. In addition, it is important that evidence is not segmented, being kept in a single file to avoid the obvious risks of mis-information.

Many of these aspects point to the potential usefulness of JPEG 2000 in this environment:

Document imaging

Document imaging applications are often a trade-off between quality and compression. As technology has improved, and colour become the norm for many publication formats, so user quality expectations have also increased. As there is often a requirement for accurate on-screen viewing, as well as the ability to print high quality facsimiles of an original document, compression requirements are often conflicting. Many documents contain areas which are best communicated in character coded text format (to allow for optimum compression and indexing), together with photographic or half-toned images, graphics and other image types.