Call for Contributions on JPEG AR
March 1, 2014

1 Background and rationale

According to fast movement in the development of information technology, a breath demand for new services and development technologies related to synthesize two or more sources into a product framework is required. AR AdHoc group of WG1 has initiated an activity to study potential methodologies and standardization issues relating Augmented Reality (AR) which is a technology that provides a composite view of real world and artificial objects, which are generated with the support of computer software. It is expected that the definition of AR standard encourages activated and diverse distribution of AR contents providing of a new method of communication. Especially, as the capacity of mobile devices are being quite enhanced, the application of AR in mobile community needs to standardize the AR contents delivery framework and the interface approaches among the components in AR framework.

2 Overview of this document

According to the background and rationale, the JPEG committee starts to develop a standard, JPEG AR (ISO/IEC 19710). This part JPEG AR is an extension of relevant JPEG standards with image-based AR application, which is responsible for AR application’s behaviors. This document is newly written to call for contribution on the target applications used by the following JPEG AR framework architecture.

3 Augmented Reality Framework

Figure 1. Augmented Reality Framework Architecture (wg1n6603) In Figure 1, the major components of AR framework are illustrated.

3.1 User Device (Sensor and Display)

User device is being used on behalf of user’s perspective supporting image acquisition, sensor based value generation, compensation of acquired image or delivered AR objects.

3.1.1 Sensor

User device consists of camera and multiple sensors which are used for taking inputs from the real world. These components are:

  • GPS and sensors: GPS and sensors are used for acquisition of location information, direction of camera, and so on.

  • Camera: If an image for the AR system as input can be distorted, the quality of the final result will be affected. To resolve problems caused by such an inappropriate input, an image needs to be corrected for proper operation of AR system. Parameter calibration takes the role of such correction processing.

3.1.2 Display/UI

Display/UI consists of displays which show a composite scene of real world and objects. These hardware or software modules are used for present a composite view with the real world and the objects are displayed in the device’s screen.

3.2 AR Recognition/Track Module

User device provides a signal, which consists of image(s) from the real world and any interaction with the user.

This module provides the recognized/tracked data and/or their description to Event Handler to operate the augmentation and user’s event. The issues in this component are:

  • AR Recognizer: This module performs the marker detection in marker-based environment or image/object recognition in markerless environment, possibly using the images stored in the database. Then, the module provides the recognized data and its description to the event handler.

  • JPEG AR DB: To construct a JPEG AR application, an image that contains both the real world scene and objects through image registration is required. Also, specific AR application should be created with the customized user’s requirement, and this is dependent on the user application.

  • AR Tracker: This module performs a tracking of input signal and/or the tracked object from input stream. Then, the module provides the tracked data and its description, position, direction of the object to the event handler.

3.3 Event Handler

When an object (and/or scene) is recognized and tracked, a proper AR event should be activated or a certain event should be triggered as designed by the author or content creator. For proper event activation or triggering, information should be well designed and managed including the synchronization, harmonization, and support of transmission of information between user device and the backend server system.

3.4 AR Contents Repository

AR contents repository contains AR objects and corresponding operation-related information. The retrieval of information is done by using database item identification or surrogate information that is obtained from the image database. The repository can be created by the author independently, and/or this can use a public AR contents from other.

3.5 Renderer

To construct an image that contains both the real world scene and AR objects through AR application, the image may be displayed over the user’s device with composition of the view, interface for user interaction and an explanation of the objects.

4 Area of the Interests The major components of the framework in standard are interfaces, application description and JPEG file format. However, interested parties should be aware that JPEG AR is also interested in other constructs of AR system.

  • Interfaces:

    • between Sensor and AR Recognizer/Tracker;

    • between AR Recognizer/Tracker and Event Handler;

    • between Event Handler and Content Repository;

    • between Event Handler and Renderer

  • Application Description:

    • Input/output and behaviors of AR application
  • JPEG File Format:

    • Embedded scheme for application description and interface parameter.

    • An AR application may have pre-authored scenario of interactions between virtual objects and real-world objects as well as possible user interactions. This scenario and the possible user interactions with events triggered by the user interaction should be stored somewhere so that the application can have predictable behavior. JPEG AR Application Format can be a standardized format for the storage of the AR application. MPEG ARAF can be a reference.

5 Scope

Basically, scope definition can be derived from the analysis of AR architectures and requirements of AR service provider. However, the interested points are described in the previous section, and these main functional points are major scope of JPEG AR.

  • Interfaces:

    • Sensor - Recognizer/Tracker

    • Recognizer/Tracker - Event Handler

    • Event Handler - Content Repository

    • Event Handler - Renderer

  • Application Description:

    • AR Application’s Input/output and Behaviors
  • JPEG File Format:

    • Embedded scheme for Application Description

6 Call for Contribution for JPEG AR

WG1 requests the national bodies, affiliated entities, member companies and all parties that are interested in this topic to contribute with technology proposals, use cases and requirements of image-based AR.

7 Requested Action

The individuals or organizations with expertise in AR technologies are encouraged to provide inputs to the WG1 committee. Interested parties and individuals are invited to attend 66th (Sapporo) WG1 meetings where proposal can be presented. Participants who wish to attend this meeting are requested to contact WG1 Convener, their National Body or an organization in liaison with WG1.