Learning light field synthesis from MPI: scene encoding as a segmentation task



Learning light field synthesis from MPI: scene encoding as a segmentation task

Learning light field synthesis from MPI: scene encoding as a segmentation task
Research Paper / ICIP 2020 / Oct 2020 / Light Field, Machine learning/ Deep learning /Artificial Intelligence

In this paper we address the problem of view synthesis from large baseline light fields, by turning a sparse set of input views into a Multi-plane Image (MPI). Because available datasets are scarce, we propose a lightweight network that does not require extensive training. Unlike latest approaches, our model does not learn to estimate RGB layers but only encodes the scene geometry within MPI alpha layers, which comes down to a segmentation task. A Learned Gradient Descent (LGD) framework is used to cascade the same convolutional network in a recurrent fashion in order to refine the volumetric representation obtained. Thanks to its low number of parameters, our model trains successfully on a small light field video dataset and provides visually appealing results. It also exhibits convenient generalization properties regarding both the number of input views, the number of depth planes in the MPI, and the number of refinement iterations

Download Now


Related Content

Deep View Synthesis with Compact and Adaptive Multiplane Images
RESEARCH PAPER / Jan 2022 / Immersive / AR/VR/MR, Light Field, Volumetric Imaging, Machine learning/ Deep learning /Artificial Intelligence
Recently, learning methods have been designed to create Multiplane Images (MPIs) for view synthesis. While MPIs are extremely powerful and facilitate high quality renderings, a great amount of memory is required, making them impractical for many applications. In this paper, we propose a learning method that optimizes the available memory...
Over-wavelength pitch sized diffraction gratings for augmented reality applications
RESEARCH PAPER / Jan 2022 / Immersive/AR/VR/MR, Optics, Light Field
Waveguide based optical combiners for Augmented Reality (AR) glasses are integrating several Surface Relief Gratings (SRG) whose pitch sizes can be as small as 200 nm for the blue wavelength. All SRG components exploit the first diffraction order to couple in and out or to deviate the light. We present...
Compressively samples light field reconstruction using orthogonal frequency selection and refinement
RESEARCH PAPER / Dec 2020 / Light Field
This paper considers the compressive sensing framework as a way of overcoming the spatio-angular trade-off inherent to light field acquisition devices. We present a novel method to reconstruct a full 4D light field from a sparse set of data samples or measurements. The approach relies on the assumption that sparse...
Defining the XR Experience
WEBINAR / Sep 2023
Defining the Evolution of Video Codecs: New Trends i...
WEBINAR / Jun 2023
How do We Capture the 6G Experience?
WEBINAR / Jun 2023
In high-profile speech InterDigital CEO emphasizes t...
BLOG / Sep 2023 / Posted By: InterDigital Comms
How InterDigital Leads in Video Sustainability
BLOG / Sep 2023 / Posted By: InterDigital Comms
Leading by Example: Alain Mourad, Head of InterDigit...
BLOG / Jun 2023 / Posted By: Milind Kulkarni, Head of Wireless Labs

ready to
learn more?

CONTACT INVESTORS
InterDigital logo in white

   © COPYRIGHT 2023 INTERDIGITAL, INC. ALL RIGHTS RESERVED.