3D location estimation of light sources in room-scale large scenes

In this paper we present our on-going work on light sourceestimation in room-scale scenes for more photorealistic expe-riences. Our unique input is an up-to-date textured 3D meshof a real uncontrolled environment obtained using a consumermobile device. We base our approach on the detection of realshadows in a single RGB-D image rendered for a top view-point. Contrary to prior art, our approach does not considerany object-based segmentation, neither simplifying assump-tions on the scene geometry or on poorly textured surfaces.The 3D locations of light sources are automatically estimated,and for now, the lighting model is completed with intensityvalues obtained interactively and subjectively through a GUIdisplaying augmentations on the scanned scenes. This light-ing model can then be reused to light the MR scene coher-ently during mobile experiences. Results on various indoorand outdoor scenes show the beginnings of a promising work.To illustrate the complexity of the problem and to make thecommunity aware of the importance of a correct lighting cueon user perception, we also fairly show how slightly inaccu-rate light estimation results or incomplete geometry knowl-edge can go completely unnoticed in some simple cases ormay deeply impact the final rendering photorealism in someother specific cases.