In this work we propose several full-color metagrating solutions for single waveguide-based Augmented and Virtual Reality near-eye display systems. The presented solutions are based on a combination of reflective and/or transmissive diffraction gratings inside or outside a waveguide. The proposed designs have high intensity across a wide angular range. Applying...
RESEARCH PAPER / Jan 2022
/
Immersive / AR/VR/MR,
Light Field,
Volumetric Imaging,
Machine learning/ Deep learning /Artificial Intelligence
Recently, learning methods have been designed to create Multiplane Images (MPIs) for view synthesis. While MPIs are extremely powerful and facilitate high quality renderings, a great amount of memory is required, making them impractical for many applications. In this paper, we propose a learning method that optimizes the available memory...
Facial caricature is the art of drawing faces in an exaggerated way to convey emotions such as humor or sarcasm. Automatic caricaturization has been explored both in the 2D and 3D domain. In this paper, we present the first study of facial mesh caricaturization techniques. In addition to a user...
Waveguide based optical combiners for Augmented Reality (AR) glasses are integrating several Surface Relief Gratings (SRG) whose pitch sizes can be as small as 200 nm for the blue wavelength. All SRG components exploit the first diffraction order to couple in and out or to deviate the light. We present...
We describe a multi-user system enabling instant messaging in Augmented Reality. A user can get in contact with another one without requiring his/her identification number and can easily localize the person initiating the contact. It is also possible to exchange various types of personal information in a private manner. This...
We introduce a system to assign navigation tasks to a self-moving robot using an Augmented Reality (AR) application running on a smartphone. The system relies on a robot controller and a central server hosted on a PC. The user points at a target location in the phone camera view and...
In visual effects it is usually required to process multiple film plates in order to reproduce the performance of real actors using a digital 3D character. One of the most challenging aspects of this process is the complexity of facial animation. Motivated by the work required to capture and digitally...
The upcoming MPEG Immersive Video (MIV) standard will enable storage and distribution of immersive video content over existing and future networks, for playback with 6 full or partial degrees of freedom of view position and orientation. The demo showcases a VOD server streaming MIV encoded immersive video contents up to...
We present a diminished reality application running live on consumer mobile devices. In our pre-observation-based approach, the clean 3D scene, free of undesired objects, is scanned beforehand and reconstructed as a high resolution textured 3D model. At runtime, objects added in a region of interest are efficiently removed by projecting...
Which parts or objects are interesting in a content? In this paper we first propose three computational models to automatically predict interestingness rankings of areas/objects inside a 2D picture. We based our modeling on previous experimental findings to ensure reliability of the prediction when compared to the human assessement of...
Augmented Reality (AR) is a concept and a set of technologies for merging of real and virtual elements to produce new visualizations – typically a video – where physical and digital objects co-exist and interact in real time. Most AR applications support real-time interaction with content (AR scene with virtual...
In this paper, we propose a new format for haptic texture mapping which is not dependent on the haptic rendering setup hardware. Our “haptic material” format encodes ten elementary haptic features in dedicated maps, similarly to “materials” used in computer graphics. These ten different features enable the expression of compliance,...
With the growth of virtual reality setups, digital sculpting tools become more and more immersive. It is now possible to create a piece of art within a virtual environment, directly with the controllers. However, these devices do not allow to touch the virtual material as a sculptor would do. To...
A Preliminary Investigation into the Impact of Training for Example-Based Facial Blendshape Creation
Our work is a perceptual study into the effects of training poses on the Example-Based Facial Rigging (EBFR) method. We analyse the output of EBFR given a set of training poses to see how well the results reproduced our ground truth actor scans compared to a Deformation Transfer approach. While...
Over the last few years, we watched head-worn virtual and augmented reality devices move from research laboratories to store shelves as the newest platform enabling novel virtual experiences. The big promise of emerging mixed reality (MR) technology is a far deeper immersion into imaginary worlds than is possible with the...
Photometric registration consists in blending real and virtual scenes in a visually coherent way. To achieve this goal, both reflectance and illumination properties must be estimated. These estimates are then used, within a rendering pipeline, to virtually simulate the real lighting's interaction with the scene. In this paper, we are...
Quadrotor drones equipped with high quality cameras have rapidly raised as novel, cheap and stable devices for filmmakers. While professional drone pilots can create aesthetically pleasing videos in short time, the smooth -- and cinematographic -- control of a camera drone remains challenging for most users, despite recent tools that...
Check out this report detailing the State of Art (SoA) of optical hardware currently used in the field of 3D display technology, as well as emerging optical hardware technologies that feature Light Field (LF) and Holographic techniques.
Color grading is a crucial step in film post-production for defining the creative intent and giving a particular look and feel to the content. In the context of VR, no adapted solutions exist yet for color grading 360° imagery, leading to cumbersome work-arounds and costing precious production time. To enable...
Virtual reality offers a new way of telling stories and engaging audiences. To create appealing content for this new imaging modality, content production and post-production workflows need to be adapted or completely rethought. One key aspect for ensuring image quality and preservation of the director's intent throughout the production workflow...
A multi-user experience extending a standard TV content with AR elements is presented. It runs with both a standard tablet and a premium MR headset, the Microsoft HoloLens. A virtual TV mosaic is displayed around the TV screen and used as a GUI to control both TV and MR content....
Presentation of Light Fields pipeline for Immersive Video Experiences
Technicolor has been investigating how Mixed Reality technology could impact the future of home entertainment. We have designed and implemented a system to extend a standard TV experience with AR content, using a consumer tablet or a headset. A virtual TV mosaic is displayed around the TV screen and used...
Augmented Reality (AR) scenarios aim to provide realistic blending between real world and virtual objects. A key factor for realistic AR is thus a correct illumination simulation. This consists in estimating the characteristics of real light sources and use them to model virtual lighting. In this paper, we briefly introduce...
Lighting is a key element in photography. Professional photographers often work with complex lighting setups to directly capture an image close to the targeted one. Some photographers reversed this traditional workflow. Indeed, they capture the scene under several lighting conditions, then combine the captured images to get the expected one....
360° video, supporting the ability to present views consistent with the rotation of the viewer's head along three axes (roll, pitch, yaw) is the current approach for creation of immersive video experiences. Nevertheless, a more fully natural, photorealistic experience - with support of visual cues that facilitate coherent psycho-visual sensory...
This paper investigates the role of the embodiment in an immersive video experience. A system allowing to play back omnidirectional videos enhanced with real-time 3D content is presented. It enables the user to be embodied in an avatar and to interact with 3D objects added to the video. A user...
With virtual reality (VR), the goal is to create experiences in which the user can be completely immersed; an alternative reality produced by a computer simulation and displayed to the user as a completely synthetic view generated by computer graphics. In both VR and AR, the reality that the physical...
Despite the relatively recent commercial availability of consumer devices, various market forecasts are expecting a rapid increase in the number of users and the market value of augmented reality (AR) content in the coming years. When it comes to media consumption, the quality of the experience is the major driving factor...
Immersive videos allow users to freely explore 4 π steradian scenes within head-mounted displays (HMD), leading to a strong feeling of immersion. However users may miss important elements of the narrative if not facing them. Hence, we propose four visual effects to guide the user's attention. After an informal pilot...
We introduce analytic approximations for accurate real-time rendering of surfaces lit by non-occluded area light sources. Our solution leverages the Irradiance Tensors developed by Arvo for the shading of Phong surfaces lit by a polygonal light source. Using a reformulation of the 1D boundary edge integral, we develop a general...
The acquisition of surface material properties and lighting conditions is a fundamental step for photo-realistic Augmented Reality (AR). In this paper, we present a new method for the estimation of diffuse and specular reflectance properties of indoor real static scenes. Using an RGB-D sensor, we further estimate the 3D position...
Virtual Reality enclosures are inexpensive devices that create a virtual reality experience using a mobile phone. For example, Google Cardboard lets a user put their phone inside the device and is made from cardboard, some simple lens and a magnet. Because the phone is encased, there is no way to...
Despite the rapid growth of wearable and mobile devices and associated availability of vibrotactile (VT) actuators, the design space of VT applications has remained primarily limited to event-based VT notifications. While useful in some cases, these notifications can be disruptive and cognitively-demanding. Further, the lack of perceptually salient VT sensations...
Monte-Carlo integration techniques for global illumination are popular on GPUs thanks to their massive parallel architecture, but efficient implementation remains challenging. The use of randomly decorrelated low-discrepancy sequences in the path-tracing algorithm allows faster visual convergence. However, the parallel tracing of incoherent rays often results in poor memory cache utilization,...
Adding the sense of touch to hearing and seeing would be necessary for a true immersive experience. This is the promise of the growing "4D-cinema" based on motion platforms and others sensory effects (water spray, wind, scent, etc.). Touch provides a new dimension for filmmakers and leads to a new...
The movie industry has been using Unmanned Aerial Vehicles as a new tool to produce more and more complex and aesthetic camera shots. However, the shooting process currently rely on manual control of the drones which makes it difficult and sometimes inconvenient to work with. In this paper we address...
We present in this paper a production-oriented technique designed to visualize contact in real-time between 3D objects. The motivation of this work is to provide integrated tools in the production workflow that help artists setting-up scenes and assets without undesired floating objects or inter-penetrations. Such issues can occur easily and...
The rise of Unmanned Aerial Vehicles and their increasing use in the cinema industry calls for the creation of dedicated tools. Though there is a range of techniques to automatically control drones for a variety of applications, none have considered the problem of producing cinematographic camera motion in real-time for...
Between the recent popularity of virtual reality (VR) and the development of 3D, immersion has become an integral part of entertainment concepts. Head-mounted Display (HMD) devices are often used to afford users a feeling of immersion in the environment. Another technique is to project additional material surrounding the viewer, as...
Facebook’s Oculus VR recently announced that it will launch a consumer version of its virtual reality headset in 2016. Shortly following the announcement, TechNewsWorld, an ECT News Network publication, featured an article that surrounded the release as well as the future of virtual reality – and InterDigital’s Senior Director of...
From the announcement of Facebook’s Oculus VR’s consumer headset in 2016 to the launch of Google Cardboard Expeditions, virtual reality has made its way into the topic of conversation in the tech world. VentureBeat, one of the premier online publications covering technology and startups, recently published an article surrounding virtual...
