NEWS & PRESS

JPEG XE on Event-based Vision Activity Overview
November 21, 2024

Introduction

JPEG XE is an exploration activity of the JPEG Committee on Event-based vision. The goal of JPEG XE is the creation of an International Standard to define a format specification for the efficient representation of event data, to enable and improve interoperability between event-based vision applications and services.

Event-based vision revolves around a new and emerging image modality created by event-based visual sensors. Such sensors generate events – i.e. messages that signal the result of an observation at a precise point in time – at high and variable rates which need to be transmitted to either aprocessing node or storage device. In both cases, an efficient ands standardized representation format with either lossless or lossy compression will of benefit to the application towards interoperability, adoption, integration, etc.

Use Cases and Requirements

Like for all of its standards, the JPEG Committee started by creating the Use Cases and Requirements (download). This document defines the scope of JPEG XE, being the creation and development of a standard to represent Events in an efficient way allowing interoperability between sensing, storage, and processing, targeting machine vision and other relevant applications. This document also contains the terminology, definitions, and generic framework to construct a common technical language for discussing event-based vision.

Due to commonality w.r.t. applications and their requirements, all of the identified use cases are grouped into one of the following classes:

  • Embedded edge computing
  • Mobile devices
  • Industrial machine vision
  • Long-term storage
  • Movie production
  • Gaming
  • Scientific and engineering measurement

More details can be found in the Use Case and Requirements document.

Call for Proposals and Common Test Conditions

Following the creation of the Use Cases and Requirements for JPEG XE, the JPEG Committee saw that lossless event coding is most imminent for the industry. In this context, two documents were issued:

  • Common Test Conditions (download) for JPEG XE to evaluate lossless event-based coding solutions.
    • Defines a canonical reference format and a test dataset (in this format) used for evaluation.
    • Establishes the evaluation methodology for comparison of the proposed coding solutions.
    • Provides a formal definition of the key performance metrics used for evaluation.
  • Final Call for Proposals on JPEG XE for lossless coding of events (download).
    • The purpose of this Call for Proposals is to receive input contributions for technology to losslessly represent event data as an interchangeable format. Such an event data format, in addition to providing high compression efficiency, should also offer ultra-low latency and allow for low complexity implementations with low-power consumption. Additionally, with lower priority, the format may also facilitate classic image processing in the compressed domain to enhance the quality of the captured visual information for human consumption, like denoising, stabilization, and deblurring, as well as effectively performing computer vision tasks, like object recognition, object tracking, and motion segmentation.
    • In this context, JPEG standardization targets the design of a versatile and broadly applicable event-data format, based on a multitude of proposed technologies and close collaboration between industry and academics, while fitting the identified use cases and requirements. JPEG believes that a well-designed standard can cover a wide range of event-based use cases.

The Call for Proposals is an open call and the JPEG Committee welcomes all submissions!

Current timeline

The following timeline (lossless representation) is targeted (indicative):

  • 2024-04: Draft Call for Proposals
  • 2024-07: Final Call for Proposals and CTC (eventually including auxiliary software)
  • 2025-01-03: Registration of interest (not mandatory)
  • 2025-03-31: Deadline for submission of proposals
  • 2025-04: Presentation and discussion of proposals at the 107th JPEG meeting
  • 2025-07: Committee Draft (CD)
  • 2026: Draft International Standard (DIS)
  • 2026: International Standard (IS)

Active and relevant resources

More information

The JPEG XE section is available here.

Interested parties are invited to regularly consult the jpeg.org website for the latest news, or register on the mailing list of JPEG XE at https://listregistration.jpeg.org (jpeg-xe).