Adding the sense of touch to hearing and seeing would be necessary for a true immersive experience. This is the promise of the growing "4D-cinema" based on motion platforms and others sensory effects (water spray, wind, scent, etc.). Touch provides a new dimension for filmmakers and leads to a new creative area, the haptic cinematography. However design rules are ...
Facial caricature is the art of drawing faces in an exaggerated way to convey emotions such as humor or sarcasm. Automatic caricaturization has been explored both in the 2D and 3D domain. In this paper, we present the first study of facial mesh caricaturization techniques. In addition to a user study, we propose two novel approaches to automatically caricaturize ...
360° video, supporting the ability to present views consistent with the rotation of the viewer's head along three axes (roll, pitch, yaw) is the current approach for creation of immersive video experiences. Nevertheless, a more fully natural, photorealistic experience - with support of visual cues that facilitate coherent psycho-visual sensory fusion without the ...
"Modeling digital human at high-fidelity has become key in many applications such as gaming, visual effects or virtual reality. To push further realism of human faces, we introduce a first data-driven generative model of eyelashes based on physical and semantic features. This model is derived from real data by introducing a new 3D hair reconstruction method from ...
With the growth of virtual reality setups, digital sculpting tools become more and more immersive. It is now possible to create a piece of art within a virtual environment, directly with the controllers. However, these devices do not allow to touch the virtual material as a sculptor would do. To tackle this issue we investigate in this paper the use of a tangible ...
Photometric registration consists in blending real and virtual scenes in a visually coherent way. To achieve this goal, both reflectance and illumination properties must be estimated. These estimates are then used, within a rendering pipeline, to virtually simulate the real lighting's interaction with the scene. In this paper, we are interested in indoor scenes w ...
Quadrotor drones equipped with high quality cameras have rapidly raised as novel, cheap and stable devices for filmmakers. While professional drone pilots can create aesthetically pleasing videos in short time, the smooth -- and cinematographic -- control of a camera drone remains challenging for most users, despite recent tools that either automate part of the p ...
Augmented Reality (AR) is a concept and a set of technologies for merging of real and virtual elements to produce new visualizations – typically a video – where physical and digital objects co-exist and interact in real time. Most AR applications support real-time interaction with content (AR scene with virtual objects) which has been produced in advance or offli ...
In this paper we present our on-going work on light sourceestimation in room-scale scenes for more photorealistic expe-riences. Our unique input is an up-to-date textured 3D meshof a real uncontrolled environment obtained using a consumermobile device. We base our approach on the detection of realshadows in a single RGB-D image rendered for a top view-point. Cont ...
Despite the rapid growth of wearable and mobile devices and associated availability of vibrotactile (VT) actuators, the design space of VT applications has remained primarily limited to event-based VT notifications. While useful in some cases, these notifications can be disruptive and cognitively-demanding. Further, the lack of perceptually salient VT sensations ...
Waveguide based optical combiners for Augmented Reality (AR) glasses are integrating several Surface Relief Gratings (SRG) whose pitch sizes can be as small as 200 nm for the blue wavelength. All SRG components exploit the first diffraction order to couple in and out or to deviate the light. We present SRG using higher diffraction orders featuring over-wavelength ...
Virtual reality offers a new way of telling stories and engaging audiences. To create appealing content for this new imaging modality, content production and post-production workflows need to be adapted or completely rethought. One key aspect for ensuring image quality and preservation of the director's intent throughout the production workflow is color managemen ...
In this work we propose several full-color metagrating solutions for single waveguide-based Augmented and Virtual Reality near-eye display systems. The presented solutions are based on a combination of reflective and/or transmissive diffraction gratings inside or outside a waveguide. The proposed designs have high intensity across a wide angular range. Applying o ...
research Paper / Jan 2022 / Immersive / AR/VR/MR, Light Field, Volumetric Imaging, Machine learning/ Deep learning /Artificial Intelligence
Recently, learning methods have been designed to create Multiplane Images (MPIs) for view synthesis. While MPIs are extremely powerful and facilitate high quality renderings, a great amount of memory is required, making them impractical for many applications. In this paper, we propose a learning method that optimizes the available memory to render compact and ada ...
Lighting is a key element in photography. Professional photographers often work with complex lighting setups to directly capture an image close to the targeted one. Some photographers reversed this traditional workflow. Indeed, they capture the scene under several lighting conditions, then combine the captured images to get the expected one. Acquiring such a set ...
Despite the relatively recent commercial availability of consumer devices, various market forecasts are expecting a rapid increase in the number of users and the market value of augmented reality (AR) content in the coming years. When it comes to media consumption, the quality of the experience is the major driving factor for success, as seen in the popular mobil ...
Virtual Reality enclosures are inexpensive devices that create a virtual reality experience using a mobile phone. For example, Google Cardboard lets a user put their phone inside the device and is made from cardboard, some simple lens and a magnet. Because the phone is encased, there is no way to interact with the touch screen. Cardboard uses its magnet, the phon ...
Immersive videos allow users to freely explore 4 π steradian scenes within head-mounted displays (HMD), leading to a strong feeling of immersion. However users may miss important elements of the narrative if not facing them. Hence, we propose four visual effects to guide the user's attention. After an informal pilot study, two of the most efficient effects were e ...
We introduce analytic approximations for accurate real-time rendering of surfaces lit by non-occluded area light sources. Our solution leverages the Irradiance Tensors developed by Arvo for the shading of Phong surfaces lit by a polygonal light source. Using a reformulation of the 1D boundary edge integral, we develop a general framework for approximating and eva ...
Check out this report detailing the State of Art (SoA) of optical hardware currently used in the field of 3D display technology, as well as emerging optical hardware technologies that feature Light Field (LF) and Holographic techniques. ...
"Immersive media applications offer experiences which immerse the user in a virtual or hybrid environment and offer more degrees of freedom than with traditional 2D video content. Platforms providing immersive media often enable the user to interact with the content and/or with other users in shared virtual or mixed reality spaces. To address the need for an inte ...
Monte-Carlo integration techniques for global illumination are popular on GPUs thanks to their massive parallel architecture, but efficient implementation remains challenging. The use of randomly decorrelated low-discrepancy sequences in the path-tracing algorithm allows faster visual convergence. However, the parallel tracing of incoherent rays often results in ...
Technicolor has been investigating how Mixed Reality technology could impact the future of home entertainment. We have designed and implemented a system to extend a standard TV experience with AR content, using a consumer tablet or a headset. A virtual TV mosaic is displayed around the TV screen and used as a GUI to control both TV and MR content. Using this inte ...
Between the recent popularity of virtual reality (VR) and the development of 3D, immersion has become an integral part of entertainment concepts. Head-mounted Display (HMD) devices are often used to afford users a feeling of immersion in the environment. Another technique is to project additional material surrounding the viewer, as is achieved using cave systems. ...
Over the last few years, we watched head-worn virtual and augmented reality devices move from research laboratories to store shelves as the newest platform enabling novel virtual experiences. The big promise of emerging mixed reality (MR) technology is a far deeper immersion into imaginary worlds than is possible with the flat 2D screens we currently use to consu ...
Which parts or objects are interesting in a content? In this paper we first propose three computational models to automatically predict interestingness rankings of areas/objects inside a 2D picture. We based our modeling on previous experimental findings to ensure reliability of the prediction when compared to the human assessement of interestingness. Our two fir ...
The acquisition of surface material properties and lighting conditions is a fundamental step for photo-realistic Augmented Reality (AR). In this paper, we present a new method for the estimation of diffuse and specular reflectance properties of indoor real static scenes. Using an RGB-D sensor, we further estimate the 3D position of light sources responsible for s ...
Presentation of Light Fields pipeline for Immersive Video Experiences ...
Color grading is a crucial step in film post-production for defining the creative intent and giving a particular look and feel to the content. In the context of VR, no adapted solutions exist yet for color grading 360° imagery, leading to cumbersome work-arounds and costing precious production time. To enable colorists to work on 360° content without modifying th ...
We present a diminished reality application running live on consumer mobile devices. In our pre-observation-based approach, the clean 3D scene, free of undesired objects, is scanned beforehand and reconstructed as a high resolution textured 3D model. At runtime, objects added in a region of interest are efficiently removed by projecting the previously captured ba ...
The upcoming MPEG Immersive Video (MIV) standard will enable storage and distribution of immersive video content over existing and future networks, for playback with 6 full or partial degrees of freedom of view position and orientation. The demo showcases a VOD server streaming MIV encoded immersive video contents up to a decoding client where an HMD is connected ...
Augmented Reality (AR) scenarios aim to provide realistic blending between real world and virtual objects. A key factor for realistic AR is thus a correct illumination simulation. This consists in estimating the characteristics of real light sources and use them to model virtual lighting. In this paper, we briefly introduce a novel method for recovering both 3D p ...
The rise of Unmanned Aerial Vehicles and their increasing use in the cinema industry calls for the creation of dedicated tools. Though there is a range of techniques to automatically control drones for a variety of applications, none have considered the problem of producing cinematographic camera motion in real-time for shooting purposes. In this paper we present ...
In this paper, we propose a new format for haptic texture mapping which is not dependent on the haptic rendering setup hardware. Our “haptic material” format encodes ten elementary haptic features in dedicated maps, similarly to “materials” used in computer graphics. These ten different features enable the expression of compliance, surface geometry and friction a ...
This paper investigates the role of the embodiment in an immersive video experience. A system allowing to play back omnidirectional videos enhanced with real-time 3D content is presented. It enables the user to be embodied in an avatar and to interact with 3D objects added to the video. A user study was conducted to understand the impact of this embodiment on the ...
With virtual reality (VR), the goal is to create experiences in which the user can be completely immersed; an alternative reality produced by a computer simulation and displayed to the user as a completely synthetic view generated by computer graphics. In both VR and AR, the reality that the physical environment has such a tight relationship with the experience s ...
We introduce a system to assign navigation tasks to a self-moving robot using an Augmented Reality (AR) application running on a smartphone. The system relies on a robot controller and a central server hosted on a PC. The user points at a target location in the phone camera view and the robot moves accordingly. The robot and the phone are independently located in ...
"This document gives an overview of the V3C Immersive Platform delivered to the Reference Tools 5G MAG An item ""V3C Immersive Platform"" will be added to the page https://5g-mag.github.io/Getting-Started/" ...
Our work is a perceptual study into the effects of training poses on the Example-Based Facial Rigging (EBFR) method. We analyse the output of EBFR given a set of training poses to see how well the results reproduced our ground truth actor scans compared to a Deformation Transfer approach. While EBFR produced better results overall, there were cases that did not s ...
We describe a multi-user system enabling instant messaging in Augmented Reality. A user can get in contact with another one without requiring his/her identification number and can easily localize the person initiating the contact. It is also possible to exchange various types of personal information in a private manner. This innovative type of social interaction ...
The movie industry has been using Unmanned Aerial Vehicles as a new tool to produce more and more complex and aesthetic camera shots. However, the shooting process currently rely on manual control of the drones which makes it difficult and sometimes inconvenient to work with. In this paper we address the lack of autonomous system to operate generic rotary-wing dr ...
In visual effects it is usually required to process multiple film plates in order to reproduce the performance of real actors using a digital 3D character. One of the most challenging aspects of this process is the complexity of facial animation.Motivated by the work required to capture and digitally augment the performances for the 2019 film Cats [Cats 2019], we ...
We present in this paper a production-oriented technique designed to visualize contact in real-time between 3D objects. The motivation of this work is to provide integrated tools in the production workflow that help artists setting-up scenes and assets without undesired floating objects or inter-penetrations. Such issues can occur easily and remain unnoticed unti ...