D5.1 Requirements and Specifications for 3D Video

This deliverable first introduces different 3D video application scenarios on mobile phones, as for instance real-time 3D video communication or 3D video playback. Besides general low complexity requirements, which always apply for mobile phone video applications, the different 3D video application scenarios impose different requirements, which are presented. Then, different 3D video formats are introduced along with related algorithms, which are available and under development. The different 3D video formats have different advantages and drawbacks, which are analysed. An initial assessment of the 3D video formats versus applications scenarios is given, analysing which format might be suitable for which application scenario. Finally, a broad work plan for WP5 is presented.

D5.2 First Study Results for 3D Video Solutions

This document reports first study results for 3D video solutions. The different video formats that were defined in D5.1 are used for this, including three depth-based formats, namely single- and multiview video plus depth and layered depth video, and conventional stereo video. Study results and solutions presented in this deliverable address analysis and synthesis as well as coding algorithms for 3D video. Regarding conventional stereo video, coding approaches are analyzed. For video plus depth results on stereo video rendering for mobile phone video applications are evaluated. For MVD the report presents an extensive study on depth coding and appropriate evaluation methods as well as high quality rendering algorithms. Furthermore extraction and rendering algorithms for layered depth video are investigated. Finally, conclusions and future prospects of study results for 3D video solutions are presented.

D5.4 Interim progress report for 3D video solutions

This deliverable contains the interim progress report for 3D video solutions. Based on the achievements of the initial implementation the work on 3D video solutions has been continued and good progress has been made in all areas. Regarding the prototype platform with integrated 3D camera and 3D display, classical stereo video (CSV) is the most important 3D video representation format. Therefore, encoding and decoding of CSV has been studied in-depth for the various available coding methods with respect to coding efficiency, quality and complexity. This leads to the conclusion about which 3D video coding solutions will be implemented for the prototype platform. As one of the solutions will be based on H.264/MVC, the report presents the results of implementing and optimizing an MVC codec for the OMAP prototype platform with the help of DSP programming. Furthermore solutions for the 3D video application, regarding interfaces as well as coding and processing with the help of gstreamer, are investigated and developed. Finally, the MPEG 3D video coding standardization activities are reported.

D5.6 Final Report on 3D Video Solutions

This deliverable reports the final implementation of 3D video solutions and presents achievements for the different subtasks.

D6.1.1 Report on First Study Results for 3D UI / Direct Manipulation Solutions

This report describes the first study results of 3D User Interfaces and Direct Manipulation Interfaces. First, in Part 1, we survey the direct manipulation input approaches, including both the use of cameras and hardware sensors (such as accelerometers). Then, in Part 2, we explore 3D User Interface techniques, including the design of 3D widgets, 3D rendering, and perception studies.

D6.3.2 Application built on 3D rendering & 2D display, advanced interaction

This report describes the first interactive prototype built on 3D rendering on a 2D display with advanced interaction. The prototype is a wrapping, circular media browser. Several design alternatives have been considered and this report explains the design decisions that led to the final prototype, as well as alternatives that could be considered in future designs.

D6.3.1 - Mockups of Application For a First Loop of User Assessment

This report describes the WP6 activities relative to mock-ups. Three iterations are described: (i) Iteration 1: Development of scenarios, use cases, user analysis & requirements and initial sketches of storyboards, (ii) Iteration 2: Mock-up development; (iii) Iteration 3: Heuristic Evaluation of mock-ups and updated storyboards. The report describes the preparation of the mock-up that was presented at the NEM Summit, and provides information on how the development process continues after the mock-up version, including future plans.
The flash mockups of 3D User Interface Design.

D6.3.3 Application Built On 3D Rendering & 3D Display, Joystick Interaction

This report describes the second interactive prototype built on 3D rendering on a 3D display with joystick and touch screen interaction. The prototype is a combined clock and calendar widget, based on a spiral staircase metaphor. Several design alternatives have been considered and this report explains the current version of the prototype, as well as options that could be considered in future designs

D6.1.3 Initial implementation of 3DUI & Direct Manipulation Solutions(Amendment 1)

This document shows the implementation and the design work that has been done in the project. The design components and implemented classes that are explained in this document are related to the techniques that are in D6.1.2. The document is divided into several sections. The first two sections provide information about how the implementation procedure works and what software and hardware platforms we use. The next section that explains the high level design. The rest of the document covers the initial implementation and design. The report has been amended in response to comments received in the Technical Review Report, dated 15 April 2010. The text that is similar to deliverable D6.1.2 has been deleted, and a new section on applications, Section 10, has been added.

D6.3.4 Application built on 3D rendering & 3D display, advanced interaction

This report describes the third interactive prototype built on 3D rendering on a 3D display with joystick and touch screen interaction. The prototype is a stereoscopic media capture application and a media viewing application. These applications incorporate many different components developed within the 3Dphone project, such as the stereo camera, 3D multimedia framework, media compression algorithms and user interface components.

D6.1.4 Report on Prototype Implementation of 3D UI / Direct Manipulation Solutions

This document describes our efforts on developing prototypes for different 3D user interface and direct manipulation techniques. These efforts are categorized in three sections: Rendering, Direct Manipulation, 3D UI. We also gathered all the prototypal applications we developed under a separate section.

D4.1.3 3D Multimedia Framework Second Release Report

This document’s purpose is to demonstrate the 3D Multimedia Framework Second Release, explaining and showing with snapshots the status of its development (completed work, pending work...).

D4.1.1 3D Multimedia Framework Specification

This document’s purpose is to describe the 3D Multimedia Framework to be used in the 3DPhone. Such framework must provide support to the applications developed in the project, helping to manage all multimedia contents and data flows. Therefore, 3DPhone applications define functional requirements, namely use cases, that should satisfy the 3D Multimedia Framework.

D4.2.1 3D Software Phone Platform Specification

This document describes the software platform 3DPhone. This platform must provide support to all applications that run on the phone. Applications require access devices like the camera or sensors, rendering their windows in 3D, using 3D Multimedia Framework, receive incoming events, and so on. Hence, the platform SW form an intermediate layer between applications and hardware on which applications will run late.

D4.2.3-3D Soft Phone second release

This document's purpose is to demonstrate the 3D Soft Phone Second Release, explaining and showing the status of its development (completed work, pending work...).

D3.1.1 Specification of 3D display module

Present document has been prepared in fulfillment of Deliverable 3.1.1, required as an intermediate result of Work Package 3 of the EU Seventh Framework Program ALL 3D IMAGING PHONE Project. Specification of 3D display HW module means the description of the 3D display related hardware, low level software, mechanical and optical components which will be developed and integrated in the project.

D3.2.1 Specification of 3D camera HW module

Specification of 3D camera HW module means the description of the 3D camera related hardware, low level software, mechanical and optical components which will be developed and integrated in the project.

D3.5 Final hardware platform

Present document has been prepared in fulfillment of Deliverable 3.5, required as an intermediate result of Work Package 3 of the EU Seventh Framework Program ALL 3D IMAGING PHONE Project. Final hardware platform means the integrated phone device with the 3D display and 3D camera as specified in deliverables D3.1.3 and D3.2.3. This document presents the integrated final HW prototype devices delivered to the project partners. It will also compare the significance of the results compared to the current state of the art, and discuss improvement possibilities.

D7.3.1 User Requirements Report

This deliverable makes a user analysis including different user classes, user profiles and use case scenarios after presenting some important design principles of user interface and user interaction. Most important scenarios are chosen as the use case scenarios in this part. Lastly, the requirements for mobile users are categorized such as software, hardware, UI, performance, etc. and labeled as ‘must’, ‘should’ and ‘may’ according to their level of importance.

D7.3.2 Research protocol for ergonomics testing

This deliverable describes the research protocol for ergonomics testing. The purpose of the document is to:

• present the background and motivation for the evaluation criteria, distinguish between objective and subjective tests, and propose a diagnostic approach for 3D display evaluation.
• propose a test protocol consisting of objective and subjective visual ergonomics tests designed for testing the stereoscopic display itself and the display integrated to the final phone.

D7.1.2 PDA phone integrated with 3D camera HW module

This deliverable describes the handheld device prototype's integration for each 3D camera version in the frame of the 3DPhone project. The initial device enabled quick integration with the OMAP ZOOM device. The second version enabled stereo synchronization, and UXGA still image capture.

D7.1.1 PDA phone integrated with 3D display HW module

This deliverable describes the handheld device prototype's integration for each 3D display type in the frame of the 3DPhone project. Each device has a different glassless 3D display The list of prototypes consists of three devices: an OMAP ZOOM device integrated with a lenticular lens based 3D display, another one with a parallax barrier 3D display and a UMPC form factor Holovizio 3D display. Thus each display has specific hardware-software integration needs.

D7.1.3 In-lab test report

Present document has been prepared in fulfillment of Deliverable 7.1.3, required as an intermediate result of Work Package 7 of the EU Seventh Framework Program ALL 3D IMAGING PHONE Project. This deliverable contains an overview for the results of the tests conducted in the hardware lab. We had two hardware platforms, an OMAP ZOOM device, and a UMPC platform. We have tested the handheld platform with lenticular and parallax barrier 3D displays, and two types of stereo cameras. The UMPC platform has been tested with the stereo cameras and the Holovizio mobile display. This report also contains the results of the software testing on the software modules developed for the hardware layer to work.

D7.2.2-In-lab test report

This document's purpose is to describe those activities carried out to test Hardware- Software integration on the 3D-adapted Omap Zoom I, customized by Holografika. The goal of this report is, hence, to validate that software platform has a seamless support for every needed hardware feature available on the Omap Zoom I architecture, describing the results get from each test, even if they have an unsuccessful output.

D7.2.3.System test plan

This report describes the first study results of 3D User Interfaces and Direct Manipulation Interfaces. First, in Part 1, we survey the direct manipulation input approaches, including both the use of cameras and hardware sensors (such as accelerometers). Then, in Part 2, we explore 3D User Interface techniques, including the design of 3D widgets, 3D rendering, and perception studies.

Deliverable D7.2.4 - System Test Report

This document's purpose is to describe results get from those activities that has been carried out to test all the final functionalities included in the 3DPhone output device. These functionalities include applications developed by all the involved partners. The goal of this report is, hence, to validate that applications developed and deployed are properly working on the whole platform and they can be used as expected by a final user. With this report a final user can be sure that testing activities has been carried out and can see which ones have the expected results and which ones has some misbehaviour due to some concrete reason that, sometimes, could be fixed.

D2.2 Collection of Published Technical Papers

This deliverable lists the scientific publications as a result of the work done in the project

D7.3.3 Report of ergonomics testing

The effects of a small autostereoscopic display to the user experience of 40 participants were tested. The results indicate that the user interface and display of the tested device did not cause significantly more eye strain in the experiment group compared to the control group. The result is even more significant, because the use time in the experiment was 40 minutes of continuous use, which is fairly rare in typical mobile use cases. The disorientation symptoms were significantly more elevated in the experiment group. It is difficult to speculate of the reasons of increased disorientation symptoms and the issue should be further investigated. As there were also no significant differences in the functioning of the oculomotor system, one can safely conclude that the user interface that utilized small depth values was comfortable to use with the small lenticular display that was utilized in the experiment.

D7.3.4 Report on user trials

This deliverable reports the status of the tests, and summarizes the main findings of each test. The detailed descriptions of the tests are given in appendices or as published articles.