To realize the objectives of Integrated Sensing and
Communication (ISAC) in 6G, there is a need to introduce
new functionalities in 6G core (6GC) architecture that are
dynamic and resource-efficient. In ISAC, sensing signals are used by a Sensing Receiver (SRx) to measure and report Sensing Data Points (SDPs) to the network. However, a direct approach involving the measurement of all SDPs for each ISAC use case is resource-intensive and incurs significant signaling overhead. Addressing this challenge requires a shift towards a more dynamic and resource-efficient approach that selectively configures SDPs based on the Key Performance Indicators (KPI)
requirements of each use case. Building on recent research and standardization efforts, which envision a dedicated Sensing Coordination Function (SCF) to coordinate sensing tasks, this paper proposes enhancements to the SCF within the 6GC architecture.
We introduce a method for the selective configuration of SRx with SDPs, based on the KPI requirements of the sensing task. The proposed approach ensures optimized resource utilization while maintaining the flexibility required for diverse ISAC use cases

Dynamic and Resource-Efficient ISAC Operations in Sensing-Enabled 6G Systems

This paper proposes a method that enhances the compression performance of the current model under development for the upcoming MPEG standard on Feature Compression for Machines (FCM).
This standard aims at providing inter-operable compressed bitstreams of features in the context of split computing, i.e., when the inference of a large computer vision Neural-Network-based model is split between two devices.
Intermediate features correspond to tensors that can be reduced and entropy coded to limit the required bandwidth of such transmission.
In the envisioned design for the MPEG-FCM standard, feature tensors may be reduced using NN layers before being converted into 2D video frame to be compressed using existing video compression standards.
This paper introduces an additional channel truncation and packing method which enables the system to preserve only the relevant channels for a given content, while preserving the computer vision task performance at the receiver.
Implemented within the MPEG-FCM test model, the proposed method yields compression gains of 10.59\% on average in terms of bitrate vs. accuracy on multiple computer vision tasks and datasets.

Feature Compression for Machines with Range-Based Channel Truncation and Frame Packing

This paper presents a new coding mode deriving sample-based blending for bi-predictions with the reconstructed template. Several extensions of this regression-based geometry partitioning scheme are investigated. Finally, experimental results are provided and discussed.

Regression-based GPM coding

The creation of photorealistic avatar head models from multi-view or monocular images or videos of a face is a long-standing research topic. Besides novel view synthesis, the
construction of a 3D face model opens the way for emerging applications such as facial reenactment. Earlier approaches to this task relied on fitting to the input images the parameters of a 3D Morphable Model (3DMM) of the face, which was later improved to include non-Lambertian reflectance components. Recently, substantial gains in reconstruction quality were achieved using implicit neural radiance field (NeRF) and 3D Gaussian Splatting (3DGS). 3DGS represents a 3D scene using a distribution of 3D
Gaussian primitives whose geometries are defined by their 3D covariance matrices and whose viewpoint-dependent colors are paremeterized by a density and a set of Spherical Harmonics coefficients. The parameters of the Gaussian primitives are fitted to the input images using an optimization scheme. To allow the animation of the avatar model from facial expression cues, 3DGS models for avatar heads are further constrained to align to a mesh of the avatar head that is typically fitted to the input images in a separate process. This paper proposes a 3DGS approach that targets lightweight imaging configurations where the avatar head is captured from a sparse set of distant viewpoints and under a small set of static facial expressions, unlike video captures where a continuum of facial expressions is available for fitting an animatable 3DGS head model. The approach builds on a novel hybrid processing scheme that combines 3D Gaussian Splats, 3D Morphable Models and neural networks. It proceeds in two stages: first, an efficient geometry-aware density control of the Gaussian primitives, and second, an optimization scheme that leverages 3DMMs and neural networks to fit the 3DGS
parameters to the input images. We evaluate our method on a new dataset of high-resolution multi-view face images captured in a controlled environment, and show that it outperforms state-of-art approaches qualitatively and quantitively in rendering novel views of the face.

Geometry-aware 3D Gaussian Model for View-Dependent Rendering of 3D Morphable Models

User localization in wireless sensing and communication systems are essential for ensuring improved quality of service. To this end, the field of radio-frequency (RF) fingerprinting, extracting user location information from multiple transmit receive points (TRPs) using the wireless channel information between the user and different TRP, has been heavily studied and developed, both through model-based and deep neural network (DNN)-aided methods. However, conventional RF fingerprinting approaches face two major limitations (i) Dependence on fixed environment and TRP locations, and (ii) Relying on inputs from a fixed number of TRPs. Addressing these limitations, this work introduces the view independent localization (VIn-Loc) model - A transformer-based DNN framework that localizes users invariant to the number and location of the TRPs. This work presents the first step towards environment-agnostic user localization using both line-ofsight (LoS) and non-LoS channel information. This approach is rigorously validated on different statistical and ray tracing models, from wireless channels with only NLoS paths, to outdoor city blocks with both LOS and NLoS paths. Experimental results highlight the strength of the proposed model, VIn- Loc, over DNN-based RF fingerprinting.


Towards Deep Learning-Based Environment Agnostic Localization

Acquiring, encoding, transmitting, decoding, and displaying motion signals is an essential challenge in our new world of connected ecosystems, which includes immersive applications (AR, VR, XR) and video games. In addition to being potentially affected by multiple sources of disturbance (e.g., signal noise or packet loss), this motion data should be modifiable and customizable to fit the needs of the application. A possible solution would be to create a semantic representation, which could be modified using high-level and interpretable signals. Several approaches in the human motion modeling literature have successfully proposed separating the body into smaller parts. This representation facilitates deep-based modeling by explicitly integrating the hierarchy of the human body into the learning. However, this representation is an intermediate step before an arbitrary downstream task in the literature. Such a pipeline leads to a necessary entanglement of the data in the second process and the loss of the semantic information. To overcome the problem of streamed animation data, we thus propose to exploit this representation more deeply by creating fully independent body parts models, even at the cost of losing possible correlations between the body parts. We further compare the performances of distinct body parts decomposition using multiple standard reconstruction metrics. We argue that this Body Parts approach brings several advantages compared to a compact one, such as a perfectly partitioned motion data, which is also interpretable (body semantics) while obtaining comparable or better reconstruction results.

Evaluation of Body Parts-based Latent Representations for Skeletal Human Motion Reconstruction

A Comprehensive Suite of Open-Source Reference Tools for MPEG-I Applications

The split-inference paradigm divides an artificial intelligence (AI) model into two parts. This necessitates the transfer of intermediate feature data between the two halves. Here, effective compression of the feature data becomes vital. In this paper, we employ Z-score normalization to efficiently recover the compressed feature data at the decoder side. To examine the efficacy of our method, the proposed method is integrated into the latest Feature Coding for Machines (FCM) codec standard under development by the Moving Picture Experts Group (MPEG). Our method supersedes the existing scaling method used by the current standard under development. It both reduces the overhead bits and improves the end-task accuracy. To further reduce the overhead in certain circumstances, we also propose a simplified method. Experiments show that using our proposed method shows 17.09% reduction in bitrate on average across different tasks and up to 65.69% for object tracking without sacrificing the task accuracy.

Efficient feature compression for machines with global statistics preservation

In May 2021, MPEG issued a call for proposals
for the specification of a new coding format for haptic data.
Following this call, a baseline reference design and associated
software implementation were defined for the representation
and coding of haptic data. It resulted in a complete generic
framework, allowing the development of current and future
haptic applications in the mobile, gaming, and virtual reality
domains. This paper introduces the results of the first phase
targeting mainly temporal haptic effects, either in a parametric
or signal-based representation form. It includes the description
of the codec architecture, the current performance, and the
future plans for the coding representation and distribution of
haptics. The publication of the final ISO international standard
is expected at the end of 2024.

Adding Touch to Immersive Media: An Overview of the MPEG Haptics Coding Standard

Ubiquitous connectivity is essential for emerging edge-assisted machine-type communication (MTC) applications for net-worked robotics, intelligent transportation, smart surveillance, etc., which demand ultra-low latency, high data rate, reliability, and energy efficiency. These applications generate significant uplink traffic, often consisting of raw sensor data, where the utility of data is closely tied to its timeliness. The traffic patterns in these scenarios are event-driven, characterized by high variability and bursts, making them difficult to predict. While 5G networks provide Radio Resource Control (RRC) states-Idle, Inactive, and Connected-designed to manage latency and user equipment (UE) power consumption based on deterministic traffic patterns like video streaming, they may not be optimized for the unpredictable nature of MTC traffic. In this study, we evaluate the performance of legacy 5G systems in handling MTC traffic using the open-source 3GPP-compliant testbed, Open Air Interface (OAI). Our analysis reveals that current 5G UE connection setup mechanisms can adversely affect control latency and UE power consumption due to the irregular and event-driven nature of MTC traffic. To mitigate these issues, we propose an enhanced connection setup process that allows UEs to remain in idle mode during inactivity, thereby conserving power, and facilitating a rapid transition to connected mode with reduced signaling overhead and power consumption. We implement and validate this framework on the OAI testbed, demonstrating its compatibility with the 3GPP standard. Empirical results indicate that our proposed method can reduce UE control latency and power consumption by up to 4× and 3× respectively, compared to legacy 5G connection setup approaches.

Reducing 5G-NR Control Overhead for Swift UE Connection Setup with Low Power Consumption

End-to-end image and video codecs are becoming
increasingly competitive, compared to traditional compression
techniques that have been developed through decades of manual
engineering efforts. These trainable codecs have many advantages
over traditional techniques, such as their straightforward adap-
tation to perceptual distortion metrics and high performance in
specific fields thanks to their learning ability. However, current
state-of-the-art neural codecs do not fully exploit the benefits of
vector quantization and the existence of the entropy gradient in
decoding devices. In this paper, we propose to leverage these
two properties (vector quantization and entropy gradient) to
improve the performance of off-the-shelf codecs. Firstly, we
demonstrate that using non-uniform scalar quantization cannot
improve performance over uniform quantization. We thus suggest
using predefined optimal uniform vector quantization to improve
performance. Secondly, we show that the entropy gradient, avail-
able at the decoder, is correlated with the reconstruction error
gradient, which is not available at the decoder. We therefore use
the former as a proxy to enhance compression performance. Our
experimental results show that these approaches save between
2 to 4% of the rate for the same quality across various pre-
trained methods. In addition, the entropy gradient based solution
improves traditional codec performance significantly as well

Exploiting Latent Properties to Optimize Neural Codecs

Abstract—We provide an overview of the use cases for terahertz
(THz) communications systems. Starting from the work
recently finalized within the European Telecommunications Standards
Institute (ETSI) Industry Specification Group (ISG) THz,
we describe the available THz frequency bands and the distinct
propagation features of those bands. Next, we describe the key
identified THz use cases and group them according into nine
identified application categories. For each use case, we identify
the physical propagation environments where they are expected
to be deployed. We also specify which other enabling technologies
will be essential for successful deployment of each use case, and
provide initial considerations on target requirements and Key
Performance Indicators (KPIs). Finally, we summarize the past
and ongoing standardization efforts focusing on THz, while also
providing an outlook towards the inclusion of THz as an integral
part of the future 6G communication networks.

Use Cases for Terahertz Communications: An Industrial Perspective

Abstract—The film grain characteristics (FGC) Supplemental Enhancement Information (SEI) message was developed in 2003 and was incorporated into the H.264/AVC specification then ported into the HEVC, VVC and Versatile supplemental enhancement information (VSEI) specifications. The FGC SEI message is used to signal and convey to video decoders metadata characterizing a film grain signal, that can be synthetized and superposed to the decoded video as a post-decoding process. It is an efficient solution to mimic the native film grain present in the input video that may have been removed by the encoding, enabling artistic intent preservation. It is also considered as an efficient solution to bring perceptual quality improvements by mitigating visual artefacts resulting from the coding process. The FGC SEI message parameters relate to the entire picture and do not allow a fine local control of the film grain. This paper describes the Film Grain Regions characteristics (FGR) SEI message, submitted as a candidate new VSEI SEI message, that enhances the FGC SEI by adding region-based control parameters enabling to adjust the synthesized film grain per region. After introducing the syntax and semantics of the FGC SEI message, the paper describes the additional features brought by the FGR SEI message. The region-based grain adaptation is showcased on different use cases that illustrate the potential benefits of the technology.

Film Grain Regions characteristics SEI message

In this paper, a complexity- gains study is conducted for VVC intra partitioning to accelerate the exhaustive search in the Rate-Distortion Optimization process (RDO) . To address this problem, two techniques are proposed. The first method is a regression based technique that predicts normalized rate-distortions (RD) costs of a given Coding Unit (CU) then a threshold is applied to select a suitable split for the current CU. As partitioning is a dependent sequential decision-making problem, it can be modeled as Markov Decision Process (MDP) and solved by Reinforcement Learning (RL). The second approach is a RL based neural network trained on two partitioning depths. The RL agent is learned from trajectories of CU decision across two depths with Deep Q-Network (DQN) algorithm. The resulting agent is block size and depth independent that selects splits based on pre-determined thresholds.

Complexity Reduction Study Based on RD Costs Approximation for VVC Intra Partitioning

Learning human motion is a challenging task that involves modeling spatio-temporal correlations at different scales. Therefore, explicitly distinguishing these scales and their relative importance can simplify the problem at hand for a neural network. For this reason, several approaches have proposed to separate the human motion into smaller parts to ease the learning of local correlations, before modeling the more distant ones, often using different hierarchical divisions. As, the impact of this choice was never evaluated, we present in this paper a study comparing several body-parts-based decompositions with a straight-forward modeling of an entire skeleton motion. More precisely, we evaluate several models with varying number of body-parts using multiple standard reconstruction metrics in different validation contexts. We also propose to explicitly not model the correlations between the body parts, which leads to the production of several small independent models. We argue that this Body-Parts approach brings several benefits over a compact one, including the creation of a more semantic latent space and an overall more intuitive representation. Furthermore, such representations can also be leveraged to generate animations on skeletons with new topologies, or to enable the modeling of various body-parts motions using different dissociated datasets. Our results suggest that a 5-Body-Parts Model can obtain comparable, or better, reconstruction results than a single Whole-Body Model, while having a less entangled latent space and among others, a smaller amount of trainable parameters.

Evaluation of Body-parts-based Latent Representations for Skeletal Human Motion Reconstruction in a Video-conferencing Setting

Reconfigurable Intelligent Surfaces (RIS) are positioned as the key enabling technology for 6G networks as they provide ubiquitous coverage for areas with blocked LoS links. However, it is predicted that to be successful, such structures will require integration of sensors and radio network elements, therefore resulting in a Multi-Functional RIS. This Survey provides the up to date summary of state of the art and considers applications for such structures as well as the challenges associated with RIS deployment. A novel concept of Location Aware Smart Wireless Environment (LASWE) is proposed to satisfy the future demands of 6G, by utilising digitally-controlled low-cost metasurfaces with integrated sensing elements and energy harvesting techniques for power autonomous operation.

The Emergence of Multi-Functional and ”Truly Active” Reconfigurable Intelligent Surfaces for Location-Aware Communications. A Survey

The consumption of a video requires a considerable amount of energy during the various stages of its life-cycle. With a billion hours of video consumed daily, this contributes significantly to the greenhouse gas emission. Therefore, reducing the end-to-end carbon footprint of the video chain, while preserving the quality of experience at the user side, is of high importance. To contribute in an impactful manner, we propose 3R-INN, a single light invertible network that does three tasks at once: given a high-resolution grainy image, it Rescales it to a lower resolution, Removes film grain and Reduces its power consumption when displayed. Providing such a minimum viable quality content contributes to reducing the energy consumption during encoding, transmission, decoding and display. 3R-INN also offers the possibility to restore either the high-resolution grainy original image or a grain-free version, thanks to its invertibility and the disentanglement of the high frequency, and without transmitting auxiliary data. Experiments show that, while enabling significant energy savings for encoding (78%), decoding (77%) and rendering (5% to 20%), 3R-INN outperforms state-of-the-art film grain synthesis and energy-aware methods and achieves state-of-the-art performance on the rescaling task on different test-sets.

3R-INN: How to be climate friendly while delivering/consuming videos?

There have been recent efforts to learn more meaningful representations via fixed length codewords from mesh data, since a mesh serves as a complete model of underlying 3D shape compared to a point cloud. However, the mesh connectivity presents new difficulties when constructing a deep learning pipeline for meshes. Previous mesh unsupervised learning approaches typically assume category-specific templates, e.g., human face/body templates. It restricts the learned latent codes to only be meaningful for objects in a specific category, so the learned latent spaces are unable to be used across different types of objects. In this work, we present WrappingNet, the first mesh autoencoder enabling general mesh unsupervised learning over heterogeneous objects. It introduces a novel \emph{base graph} in the bottleneck dedicated to representing mesh connectivity, which is shown to facilitate learning a shared latent space representing object shape. The superiority of WrappingNet mesh learning is further demonstrated via improved reconstruction quality and competitive classification compared to point cloud learning, as well as latent interpolation between meshes of different categories.

WrappingNet: Mesh Autoencoder via Deep Sphere Deformation

Many service flows between client and server end- points traverse multiple computer networks even for a private network deployment. The challenge of reducing the complexity of such services is researched on by the European Commission- funded project PREDICT-6G, with special focus on deterministic communications. The provisioning of service flows over a multi-domain multi-technology group of computer networks is a particular challenge if each domain comes with its own set of deterministic communication support. To address that, an overarching End-to-End control plane is being developed in the project and is described herein. As the proposed innovations require a mix of technologies from Standardisation Development Organisations, this paper assesses the proposition of PREDICT- 6G against the ongoing work in 3GPP, IETF, ETSI and IEEE.

Standardisation Assessment of an Digital Twin-Based Multi-Domain Deterministic Communications System

Non-linear precodering outperforms linear precodering in a multi-user multi-input multi-output (MU-MIMO) system. The actual gain depends on multiple factors, including number of users, user distribution in the system, etc. Furthermore, to achieve the promised gain, non-linear precoding requires a higher implementation complexity which has prevented its adoption into the 3rd Generation Partnership Project (3GPP) standards. In many practical scenarios, the number of paired users for MU-MIMO remains low. When the number of paired scheduled users for MU-MIMO systems is low, the complexity of non-linear precodering becomes less significant. In this paper, we show that, even when, the number of paired users for MU-MIMO is not more than two, non-linear precoding can still offer a meaningful performance improvement over the linear precoding. For our study, under 3GPP new radio framework, we have evaluated and compared the performance of Tomlinson-Harashima non-linear precodering against a block diagonalized linear precoding.

Tomlinson-Harashima precoding for 3GPP

In learning-based compression systems, reproducibility is critical to guarantee interoperability across platforms. For instance, learning-based entropy coding or inter-prediction techniques may fail if the outcome of the encoder cannot be reproduced by the decoder. In practice, the problem of irreproducibility arises from the misalignments of low-level implementations among deep learning-based software/hardware platforms. This problem may be unable to be fully addressed in general, without eliminating the differences in the low-level implementations of the platforms. Differently, under our proposal, it becomes a tractable problem in the realm of learning-based compression. In this work, reproducibility in a learning-based compression system is analyzed and necessary conditions for a possible solution are introduced. A method under such conditions to address the challenge in compression is proposed. Experiments demonstrate the effectiveness of the proposed approach in example compression models.

Towards Reproducible Learning-based Compression

Abstract—Ubiquitous connectivity is vital for emerging appli cations like extended reality, factory automation, and robotics, necessitating low latency, high data rates, and reliability in both downlink and uplink. From the network protocol perspective, successfully supporting these new use cases hinges on the network being resilient enough to address the heterogeneous demand in dynamic channel conditions. To assess the performance of legacy 5G networks for these applications, we focus on the physical (PHY) layer and analyze the existing 5G time division duplexing (TDD) method in terms of throughput. Our preliminary experiments with 3rd Generation Partnership Protocol (3GPP) compliant Matlab 5G toolbox reveal limitations of the fixed config uration of the PHY frames, that are typically used by commercial 5G networks, hindering adaptability to heterogeneous demands and compromising quality of service (QoS). To overcome this, we propose a machine learning-enabled optimization framework facilitating proactive PHY frame reconfiguration based on real time prediction of wireless channel metrics computed at User Equipment (UE). Implementation and validation of our approach on the 3GPP-compliant Open Air Interface (OAI) 5G testbed demonstrate the practicality of our solution and its adherence to 3GPP standards. Overall, our dynamic PHY frame configuration approach consistently meets overall traffic demands better than any fixed configuration across various scenarios, while also having the lowest percentage of un-transmitted bytes in each scenario

Enabling emerging applications in 5G through UE assisted proactive PHY frame configuration

"JVET has developed a new Enhanced Compression Model (ECM) for testing future video coding algorithms on top of the Versatile Video Coding (VVC) standard. Reference Picture Resampling (RPR) is a powerful tool that improves video coding efficiency of next generation like Versatile Video Coding (VVC). This feature is well designed to support frame changing resolution without inserting intra refresh picture. Video streaming and low delay scenarios can take advantage of RPR to ensure a smooth frame-based bit-rate adaptation, compared to traditional techniques that can generates bitrate leaps. In this paper, a neural network regressor to predict RPR decision is discussed, and adaptation of downscaling factor is proposed to improve VVC coding efficiency in the context of random access and all intra modes configurations. "

NN-based decision predictor of Reference Picture Resampling for video coding

"This document gives an overview of the V3C Immersive Platform delivered to the Reference Tools 5G MAG An item ""V3C Immersive Platform"" will be added to the page https://5g-mag.github.io/Getting-Started/"

5G-MAG reference Tool presentation for V3C_Immersive Platform

In frequency division duplex (FDD) massive MIMO systems, each user equipment (UE) device typically feeds back downlink channel state information (CSI) to its serving base station (BS), which may then be utilized for beamforming and scheduling purposes. However, the large number of antennas increases the CSI signaling overhead in the uplink, thereby impacting the system's spectral efficiency. To address the feedback over-head problem, this paper proposes a low-complexity beam domain autoencoder that operates on low-dimensional beam domain channel samples. The key idea is to efficiently transform the channel from the antenna-domain to the beam-domain by selecting a small number of beams that preserve most of the channel information. To leverage the beam domain processor, we propose a beam selection method that enables high reconstruction quality with a small number of selected beams. The proposed framework offers a favorable balance between complexity, performance, and memory requirements. Simulations demonstrate the efficacy of the proposed approach in recovering the compressed channel at reduced overhead and complexity, relative to the state-of-the-art methods.

Low Complexity Beam Domain Processing for Autoencoder Based CSI Compression

A Haptic Experience may take many shapes or forms, ranging fromsimple UI notifications on a phone, to fully immersive multi-sensorial experiences.In this context, dedicated Haptic authoring tools have been flourishing and theliterature on Haptic design introduced many concepts and guidelines to improveHaptic Experiences. In this paper, we propose to go beyond haptic design andexplore the potential of end-user customization of Haptic Experiences. First,we discuss and extend a theoretical model of Haptic Experience. We revisitthe proposed design parameters to account for the spatialized notion of hapticfeedback and study the suitability of these parameters for end-user customization.Then, we detail a user study (n=52) exploring how users can customize differenthaptic effects (18 different effects) through haptic customization parameters, i.e.,intensity and spatial density. Finally, our results indicate significant variability inthese parameters depending on the context and provide directions for future userstudies regarding haptic profile design determined by subjective data.

Towards End-User Customization of Haptic Experiences - Poster

In recent video coding standards, the introduction of multiple transform selection (MTS) has significantly improved coding efficiency. The latest standard, Versatile Video Coding (VVC), adopted implicit MTS as an alternative tool to explicit MTS. This tool reduces the encoder complexity by inferring the optimal transform type from decoder-side information instead of performing an intensive rate-distortion search. However, implicit MTS became less efficient in the enhanced compression model (ECM) due to the introduction of new primary transforms to explicit MTS, which drastically increased the number of possible horizontal and vertical transform pairs compared to VVC. This paper presents a new implicit MTS method that leverages a lookup table (LUT) to infer the best transform pair based on the block size and intra-prediction mode. This LUT is used in a hybrid MTS scheme where explicit MTS is enabled only for MIP blocks. The proposed LUT-based hybrid method reduces the ECM complexity by 15% while retaining 60% and 83% of the MTS gain compared to the MTS method of ECM and VVC, respectively.

Low-Complexity Transform Design Using Hybrid Intra MTS

The futures of AI and wireless networks are intricately intertwined. On the one hand, AI is a potent tool for automating the deployment and management of wireless networks. The next-generation wireless network, on the other hand, can support the training and deployment of AI models by providing an ocean of multi-modal data and a distributed computation resource. However, at present, they are generally regarded as independent fields, preventing them from capitalizing on the superior performance of the other. To that end, we propose to facilitate the convergence of next-generation wireless networks and AI by prototyping a first-of-its-kind Pathway-of-Experts (PoE) AI system to enable user-centric applications (such as smart homes), which contain a variety of inter-connected expert models. To prove its superior performance, we propose a groundbreaking demonstration of a PoE system in a smart home environment. Our demo showcases how a variety of interconnected expert models can revolutionize smart home technology. This demo not only exemplifies the convergence of wireless networks and AI but also highlights the practical applications of this synergy in creating responsive, intelligent environments.

Toward the Convergence of the Next-Generation Wireless and AI

Deep learning based automatic modulation classification (AMC) has received significant attention owing to its potential applications in both military and civilian use cases. Recently, data-driven subsampling techniques have been utilized to overcome the challenges associated with computational complexity and training time for AMC. Beyond these direct advantages of data-driven subsampling, these methods also have regularizing properties that may improve the adversarial robustness of the modulation classifier. In this paper, we investigate the effects of an adversarial attack on an AMC system that employs deep learning models both for AMC and for subsampling. Our analysis shows that subsampling itself is an effective deterrent to adversarial attacks. We also uncover the most efficient subsampling strategy when an adversarial attack on both the classifier and the subsampler is anticipated.

Data-Driven Subsampling in the Presence of an Adversarial Actor

While advances in Machine Learning have revolutionized certain areas (computer vision, robotics, natural language processing, etc.), the application in wireless communications has been less dramatic. One limiting factor is the (potentially) high computational complexity. Yet another important inhibitor is the lack of realistic datasets. To fully understand the potential of deep learning based methods to solve wireless communication problems, it is critical to leverage representative datasets, i.e., datasets that are captured in a very wide variety of scenarios, conditions and configurations – that fully reflect the places where wireless networks are deployed and used. Collecting over-the-air data (OTA) at scale in all of these settings – the gold standard – is extremely challenging and impractical. In the absence of such, machine learning practitioners are forced to rely on a handful of indoor and outdoor simulation scenarios with limited variability for model training and validation. Adding new scenarios into simulation has been limited by the significant effort and time required to accurately capture the multipath characteristics in real environments. Having the capability to extract multipath parameters from OTA channel samples would allow for faster characterization of different scenarios and a quick turn-around time towards having them introduced in simulation. Taking a step towards this goal, in this paper, we explore the use of deep learning based inverse modeling frameworks for extracting multipath channel information from the channel impulse response. Specifically, we focus on extracting the multipath parameters from captured Multiple-Input-Multiple-Output (MIMO) channel matrices.

Channel Parameter Estimation in Wireless Communication: A Deep Learning Perspective

"Compressed beamforming algorithm is used in the current Wi-Fi standard to reduce the beamforming feedback overhead (BFO). However, with each new amendment of the standard the number of supported antennas in Wi-Fi devices increases, leading to increased BFO and hampering the throughput despite using compressed beamforming. In this paper, a novel index-based method is presented to reduce the BFO in Wi-Fi links. In particular, a k-means clustering based approach is presented to generate candidate beamforming feedback matrices, thereby reducing the BFO to only the index of the said candidate matrices. With extensive simulation results, we compare the newly proposed method with the IEEE 802.11be baseline and our previously published index-based method. We show approximately 54% gain in throughput at high SNR against the IEEE 802.11be baseline. Our comparison also shows approximately 4 dB gain compared to our previously published method at the packet-error-rate (PER) of 0:01 using MCS index 11. Additionally, we also discuss the impact of the distance metric chosen for clustering as well as candidate selection on the link performance."

Enhanced Index-Based Feedback Overhead Reduction for WLANs

"Immersive media applications offer experiences which immerse the user in a virtual or hybrid environment and offer more degrees of freedom than with traditional 2D video content. Platforms providing immersive media often enable the user to interact with the content and/or with other users in shared virtual or mixed reality spaces. To address the need for an interoperable cross-platform exchange format, and interactive and time evolving solution for such 3D environments, ISO/IEC JTC 1/SC29/WG03 MPEG Systems has standardized a Scene Description framework in ISO/IEC 23090-14, that serves as an entry point format to compose rich 3D scenes, referencing and positioning 2D and 3D assets in the scene, blending with the real world, rich interactivity and providing real-time media delivery. Carriage formats have also been defined for the delivery of the Scene Description data and of the linked assets, based on the well-known and ubiquitous ISOBMFF standard i.e., ISO/IEC 14496-12."

MPEG-I Scene Description: a scene description framework for interactive and Immersive media experiences

The challenging propagation environment, combined with the hardware limitations of mmWave systems, gives rise to the need for accurate initial access beam alignment strategies with low latency and high achievable beamforming gain. Much of the recent work in this area either focuses on onesided beam alignment, or, joint beam alignment methods where both sides of the link perform a sequence of fixed channel probing steps. Codebook-based non-adaptive beam alignment schemes have the potential to allow multiple user equipments (UEs) to perform initial access beam alignment in parallel whereas adaptive schemes are favourable in achievable beamforming gain. This work introduces a novel deep learning based joint beam alignment scheme that aims to combine the benefits of adaptive, codebook-free beam alignment at the UE side with the advantages of a codebook-sweep based scheme at the basestation (BS). The proposed end-to-end trainable scheme is compatible with current cellular standard signaling and can be readily integrated into the standard without requiring significant changes to it. Extensive simulations demonstrate superior performance of the proposed approach over purely codebook-based ones.

Deep Learning Based Adaptive Joint mmWave Beam Alignment

How to make images less power-hungry. An objective benchmark study

"Learning-based point cloud (PC) compression is a promising research avenue to reduce the transmission and storage costs for PC applications. Existing learning-based methods to compress PCs attributes employ variational autoencoders (VAE) or normalizing flows (NF) to learn compact signal representations. However, VAEs leverage a lower-dimensional bottleneck that limit the maximum reconstruction quality, while existing NF architectures have a big complexity in terms of number of coefficients in the network. In this paper, we propose an improved NF architecture combining two models to compress PC attributes. Our combined approach reduce the number of parameters of the existing NF architectures by over 6x and achieves state-of-the-art coding gains compared to previous learning-based methods, with, in some cases, a comparable performance to G-PCC v.21, showing the potential of this scheme for PC compression."

Reducing the Complexity of Normalizing Flow Architectures for Point Cloud Attribute Compression

"The last standard Versatile Video Codec (VVC), aims to im- prove the compression efficiency by saving around 50% of bitrate at the same quality compared to its predecessor High Efficiency Video Codec (HEVC). However, this comes with a significant rise in computational complexity due to the new added tools in the encoder side. This paper proposes a speed- up technique to accelerate the partitioning process in VVC based on multi-output regression model that predicts a suit- able split mode for coding unit (CU) of size 32 × 32. Expire- ment results shows that our approach can reduce the complex- ity up to 43% with 0.96% BD-rate loss."

RD-Cost Regression Speed Up Technique For vvc intra block Partitioning

Live production workflows, particularly those for sports, employ complex pipelines to deliver HD, UHD, SDR, and HDR video streams. Because content originates as HDR or SDR, effective and flexible conversions are required between formats. Today, diverse tone mapping and inverse tone mapping technologies are proposed to the industry, each with the promise to create premium HDR content while maintaining the quality of SDR versions. Look Up Tables (LUTs) are the chief conversion technique used today, which constrain the capabilities of HDR and/or compromise the SDR look. Based on fixed reference levels, their static nature exposes content to artifacts emerging from changing conditions during live capture (e.g., sunset, passing clouds, shadows), leading sometimes to a growing set of LUTs to adapt to each of the conditions. On the other hand, based on dynamic reference levels, dynamic techniques accommodate such changes. However, a problem emerging is that neither the different LUT solutions nor dynamic solutions are fully interoperable: each produces content with different properties, either different reference levels or different video signal levels, requiring different conversions to satisfy the critical requirement of seamlessly delivering final content. As the HDR technology becomes popular and more and more used in the industry, this interoperability issue is getting increasingly problematic, given the growing number of static and dynamic solutions. Indeed, when producing the final content, how does one ensure that the correct static LUT is used or that the dynamic solution is correctly set? Discussions have started in many organizations to address the problem. Experiments and demonstrations have been organized by the end of 2023 showcasing the industry appeal and concerns. This paper discusses the challenge and evaluation consensus. A generic solution is proposed, based on metadata, that intends to resolve this compatibility issue. By characterizing the produced HDR content with those new proposed metadata, we present how static and dynamic solutions can be interoperable, paving the way for industry to better control the content production while leveraging the full expansion capabilities of HDR, as well as a smooth transition from static to dynamic techniques ensuring delivery of premium HDR and SDR content.

HDR-SDR conversion: Live HDR Single Master Production Conversion Interoperability Challenges

"The future wireless networks look forwards to proving ubiquitous connection with reduced power consumption. Benefits from the development of programmable and softwaredefined metamaterials, the reconfigurable holographic surface (RHS) is of great potential"

Energy-Efficient Reconfigurable Holographic Surfaces Operating in the Presence of Realistic Hardware Impairments

Achieving successful variable bitrate compression with computationally simple algorithms from a single end-to-end learned image or video compression model remains a challenge. Many approaches have been proposed, including conditional auto-encoders, channel-adaptive gains for the latent tensor or uniformly quantizing all elements of the latent tensor. This paper follows the traditional approach to vary a single quantization step size to perform uniform quantization of all latent tensor elements. However, three modifications are proposed to improve the variable rate compression performance. First, multi objective optimization is used for (post) training. Second, a quantization-reconstruction offset is introduced into the quantization operation. Third, variable rate quantization is used also for the hyper latent. All these modifications can be made on a pre-trained single-rate compression model by performing post training. The algorithms are implemented into three well-known image compression models and the achieved variable rate compression results indicate negligible or minimal compression performance loss compared to training multiple models. (Codes will be shared at \href{https://github.com/InterDigitalInc/CompressAI}{https://github.com/InterDigitalInc/CompressAI})

Variable-Rate Learned Image Compression with Multi-Objective Optimization and Quantization-Reconstruction Offsets

The universality of the point cloud format enables many 3D applications, making the compression of point clouds a critical phase in practice. Sampled as discrete 3D points, a point cloud approximates 2D surface(s) embedded in 3D with a finite bit-depth. However, the point distribution of a practical point cloud changes drastically as its bit-depth increases, requiring different methodologies for effective consumption/analysis. In this regard, a heterogeneous point cloud compression (PCC) framework is proposed. We unify typical point cloud representations-pointbased, voxel-based, and tree-based representations-and their associated backbones under a learning-based framework to compress an input point cloud at different bit-depth levels. Having recognized the importance of voxel-domain processing, we augment the framework with a proposed context-aware upsampling for decoding and an enhanced voxel transformer for feature aggregation. Extensive experimentation demonstrates the state-of-the-art performance of our proposal on a wide range of point clouds.

PIVOT-Net: Heterogeneous Point-Voxel-Tree-based Framework for Point Cloud Compression

"The geographic data and timestamps of subscribers held by mobile carriers have long been used by law enforcement and intelligence agencies to conduct legal investigations, largescale surveillance, or even unconstitutional search. Meanwhile, with the surge in popularity of Location-based Services (LBS) over the past decade, selling the geolocation information of average mobile users has become a lucrative enterprise. Many mobile carriers, the leading aggregators of user location data, have repeatedly breached user privacy agreements and sold users’ location history to third parties without user consent. Location privacy is exacerbated by the reality that cellular-connected mobile devices will inevitably be localized more precisely as a result of continual enhancements to the positioning and localization technologies of 5G, 6G, and beyond networks. To address these challenges, we propose AAKA, a privacypreserving anonymous mutual authentication and key agreement scheme that is compatible with the cellular architecture and SIM standardized by 3GPP.1 At its core, AAKA leverages anonymous credentials (AC) to offer anonymity, unlinkability, unforgeability, and accountability in addition to meeting 3GPP’s fundamental security requirements for the User Equipment(UE) authentication and key agreement process. The evaluation demonstrates that the scheme is practical, with a credential presentation taking ∼52 ms on a constrained host device equipped with a standard cellular SIM card. "

AAKA: An Anti-Tracking Anonymous Authentication Scheme for Next Generation Mobile Networks

"Modeling digital human at high-fidelity has become key in many applications such as gaming, visual effects or virtual reality. To push further realism of human faces, we introduce a first data-driven generative model of eyelashes based on physical and semantic features. This model is derived from real data by introducing a new 3D hair reconstruction method from multi-view images. Thanks to an original dedicated extraction process, we determine the physical features of any given set of eyelashes, and present the most detailed and comprehensive statistical analysis of human eyelashes shape ever made. Compared to existing hair reconstruction techniques we show that our approach obtains much better result to reconstruct eyelashes. We show that this eyelashes model is also fully adapted to synthetic data generation. Results show that the proposed model enables interactive, intuitive and realistic eyelashes modeling for non experts, pushing the limits of avatar creation and synthetic data generation pipelines."

3D Reconstruction and Semantic Modeling of Eyelashes

The ubiquitous deployment of 4G/5G technology has made it a critical infrastructure for society that will facilitate the delivery and adoption of emerging applications and use cases (extended reality, automation, robotics, to name but a few). These new applications require high throughput and low latency in both uplink and downlink for optimal performance, while coexisting with traditional downlink-heavy consumer applications. Successfully supporting these new use cases hinges on the network being able to allocate resources as efficiently as possible. In this paper, we utilize a 3GPP-compliant 5G testbed to analyze the limitations of legacy network resource allocation methods, which are based on instantaneous channel measurements, and examine the effect on throughput – a key performance indicator. We then propose a framework that allows resource allocation decisions to leverage predictions of network quality (computed at the connected devices), and study two different prediction methods that provide different degrees of reliability. We further validate our framework with real-world cellular data and demonstrate that with accurate channel metric forecast knowledge, the mean network throughput can improve by a factor of ∼1.8 over the baseline reactive approach based on best CQI policy, for the considered scenario.

PROMPT: Prediction of Channel Metrics for Proactive Optimization in Cellular Networks

The production, transmission, and display of video content requires significant amounts of energy. Whether broadcasting or streaming a video, its display on modern televisions is responsible for a significant proportion of the energy consumption. This paper proposes a framework for analyzing and processing High Dynamic Range (HDR) video frames that allows a modern screen to use less energy when displaying these frames. The content is analyzed prior to encoding and transmission, generating metadata that are attached to the content. After reception by a compatible display, the metadata is then used along with display parameters and user settings to adapt the peak brightness of the content. Picture quality is assessed objectively and subjectively, while in the meantime, we analyze the influence of the peak brightness over the power consumption of the screen. All these analysis provide us new insights to propose an energy-aware HDR rendering.

End-to-end VVC encoding for energy-aware HDR rendering

"Industrial applications and Extended Reality vertical sectors have expressed the need for dedicated Quality of Service considerations by 3GPP to support time-sensitive and bursty high throughput communications, respectively. Consequently, 3GPP enabled support for Time-Sensitive Networking in Release 17 and started specifying the concept around Packet Data Unit Sets in Release 19. This paper presents a novel solution for any IP-based communication enabling time-sensitive communication while utilising 3GPP Packet Data Unit Set feature. This paper proposes extensions to the IP header for a more universal approach. The proposed solutions introduce the concept of a Data Unit Group to describe the entirety of an Application Data Unit and its fragmentation into individual IP packets to be delivered over a packet-switched network. The disclosure defines Data Unit Group Rules to communicate packet header detection and action rules in Time-Sensitive Networking switches to prioritise and re-order/pre-empt packets and to write Packet Data Unit Set Markings in User Equipments and User Plane Functions."

Formation and Assertion of Data Unit Groups in 3GPP Networks with TSN and PDU Set Support

"Abstract—3rd Generation Partnership Project (3GPP) Release 18 has initiated a comprehensive study of Artificial Intelligence (AI)/Machine Learning (ML) use cases for Air Interface, e.g., Channel State Information (CSI) feedback enhancement, beam management, and positioning accuracy enhancement. In order to advance the adoption of AI/ML in 5G and towards 6G, it is important to showcase the feasibility of real-time applications and data collection on platforms that closely simulate real-world deployment scenarios. In this paper, we develop the first-ofits-kind AI-enabled 5G CSI feedback enhancement platform in real-time. In our experimentation, we integrate a CSI autoencoder into the OpenAirInterface (OAI) 5G protocol stack. We demonstrate the real-time functionality and evaluate the performance of the CSI compression with the encoder running at the User Equipment (UE) and CSI reconstruction with the decoder running at the Next Generation Node Base (gNB), on an Over-the-Air (OTA) indoor experimental platform, ARENA, as well as, on an emulated environment using Colosseum, the worlds largest wireless network emulator."

Real-Time AI-Enabled CSI Feedback Experimentation with Open RAN

In this paper, we propose a deep-learning-based channel estimation scheme in an orthogonal frequency division multiplexing (OFDM) system. Our proposed method, named Single Slot Recurrence Along Frequency Network (SisRafNet), is based on a novel study of recurrent models for exploiting sequential behavior of channels across frequencies. Utilizing the fact that wireless channels have a high degree of correlation across frequencies, we employ recurrent neural network techniques within a single OFDM slot, thus overcoming the latency and memory constraints typically associated with recurrence based methods. The proposed SisRafNet delivers superior estimation performance compared to existing deep-learning-based channel estimation techniques and the performance has been validated on a wide range of 3rd Generation Partnership Project (3GPP) compliant channel scenarios at multiple signal-to-noise ratios.

Deep OFDM Channel Estimation: Capturing Frequency Recurrence

The main objective of 5G and beyond networks is to provide an optimal user experience in terms of throughput and reliability, regardless of location and time. To achieve this, traditional fixed macro base station deployments are being replaced by more innovative and flexible solutions, such as wireless backhaul and relays. This article focuses on the evolution and standardization of these advancements, which are shaping the electromagnetic (EM) landscape. Specifically, we will explore Integrated Access and Backhaul (IAB) nodes, which offer a cost-efficient and agile alternative to fiber backhaul. We will also discuss Network-Controlled Repeaters (NCRs) and the emergence of Reconfigurable Intelligent Surfaces (RIS), which actively adapt the wireless environment. The article provides an overview of the 5G features and ongoing developments in 3GPP Release 18 related to these intelligent EM entities, highlighting the expected evolution of future wireless networks in terms of architecture, operations, and control signals.

Shaping a Smarter Electromagnetic Landscape: An Overview of IAB, NCR, and RIS in 5G Standardization and Beyond

This paper describes the sixth edition of the Predicting Video Memorability task, part of the MediaEval multimedia evaluation benchmark initiative. Similar to previous editions, we use video data and annotations from two datasets, the Memento10k, and the VideoMem datasets. In light of the consistent performance plateau observed in previous iterations of the prediction task, in which participants were required to train and test on the same dataset, we have taken the decision to drop the prediction task from
this year’s competition. This modification allows participants the opportunity to redirect their efforts toward more challenging tasks. Therefore, for this edition we propose two tasks: the generalization task, where participants are required to train on one dataset and test their results on a different dataset, and the EEG task, where participants are required to predict memorability using EEG-related data. In this paper we present the main aspects of the 2023 Predicting Video Memorability task, exploring the proposed
tasks, the datasets, evaluation methods and metrics, as well as the requirements for participants.

Overview of The MediaEval 2023 Predicting Video Memorability Task

"In this paper, we theoretically formulate the ergodic sumrate of the simultaneous transmitting and reflecting reconfigurable intelligent surfaces (STAR-RIS) assisted uplink non-orthogonal multiple access (NOMA) scheme in face of channel estimation error, as well as the RIS and transceiver hardware impairments (HWI). Specifically, the two-timescale protocol is employed, where the STAR-RIS phase shift is configured based on the statistical channel state information (CSI), followed by the linear minimum mean square error (LMMSE) channel estimation of the equivalent channels from the user equipments (UEs) to the access point (AP). Afterwards, the successive interference cancellation (SIC) algorithm is operated at the AP based on the estimated instantaneous CSI. Theoretical analysis and simulation results show that the ergodic sum-rate is upper-bounded by the transceiver hardware quality in high power budget region. Furthermore, we show that the NOMA scheme outperforms the orthogonal multiple access (OMA) scheme in terms of the ergodic sum-rate and user rate fairness."

Achievable Rate Analysis for STAR-RIS Aided Uplink NOMA Systems

"In this paper, we discuss one aspect of the latest MPEG standard on Energy-Efficient Media Consumption, also known as Green Metadata, which is the interactive signaling for remote decoder-power reduction. We provide an overview on the latest findings from the literature, which motivate the integration of new syntax elements to render decoder-side energy and power savings more flexible. Based on these observations, we perform several experiments to validate the effectiveness of potential new syntax elements. Finally, we show that using the new syntax elements in the Green Metadata standard, up to 20% of energy can be saved for both software and hardware video decoding on versatile devices."

Improved Interactive Signaling for Remote Decoder Power Consumption in Green Metadata

Reconfigurable Intelligent Surface Aided Amplitude- and Phase-Modulated Downlink Transmission

"The new applications enabled by the 5G NR use cases require novel solutions capable of supporting high end- user data rates at low latency. To meet these needs, it is critical to have accurate channel state information at the receiver. This requires sending a large number of reference signals which scales up with the number of users and layers, thereby introducing significant overhead that eventually impacts the system’s spectral efficiency. This paper introduces a new downlink data structure that is free from demodulation reference signals, and does not require channel estimation at the receiver. The proposed structure involves a repetition step of part of the user data across the time-frequency grid. Exploiting the repetition structure at the receiver, it is shown that reliable recovery is possible via canonical correlation analysis (CCA). This paper also proposes two effective mechanisms for optimizing the CCA performance in OFDM systems, one for repetition pattern selection and another to deal with the severe frequency selectivity issues. The proposed ap- proach provides rigorous recovery guarantees, exhibits favorable complexity-performance tradeoff, and is shown to be robust to high interference scenarios, thus rendering it appealing for practical implementation. The paper provides simulation results, using a 3GPP link-level testbench, that demonstrate the feasibility of the proposed approach."

Pilot-free Transmission in 3GPP OFDM MIMO Systems via Canonical Correlation

The unprecedented increase in the number of wireless-connected devices requires novel solutions to improve the data rate at low latency. Reference signals overhead reduction is a powerful way to increase data rates. However, excessive reduction in the number of reference signals degrades the channel estimation performance with potential negative impacts on the data rates. Toward this end, this paper proposes a machine learning-based approach that enables reference signal-free data channel demodulation. This new approach involves a repetition of part of the data channel symbols across the slot. Invoking canonical correlation analysis on the repeated data at the user side yields high-quality combiners that are used to recover both the repeated data blocks and the rest of the data symbols in the slot, without the need for traditional channel estimation. This paper also proposes two effective and principled strategies; one for repetition pattern selection as a function of the channel parameters and the other addresses performance in highly frequency selective channels. The proposed approach offers considerable gains in throughput performance and complexity reduction. Simulation results using a 3GPP NR link-level test-bench, reveal the effectiveness of the proposed approach relative to the state-of-the-art methods.

Unsupervised Learning for Reference Signals Overhead Reduction in 3GPP MIMO Systems

"Deploying reconfigurable intelligent surfaces (RIS) is promising to enhance the transmission reliability of wireless communications by controlling the wireless environment, in which the beamforming at the base station and RIS are jointly designed based on the acquisition of channel state information (CSI). Hence, channel estimation is crucial for RIS-aided systems design. Due to the lack of active radio frequency (RF) chain at the RIS elements to process and transmit pilot sequences, only the cascaded transmitter-RIS-receiver channel can be estimated, which results in extremely high pilot overhead when a large number of RIS reflecting elements are deployed. In this paper, we propose a low pilot overhead channel estimation method, namely Eigen-decomposition linear minimal mean square error (LMMSE) estimator, by exploiting the spatial correlation of the RIS-cascaded channels, for the multi-cell multiple-input-multipleoutput (MIMO) RIS wireless communications. Furthermore, we extend our proposed low pilot overhead channel estimators while considering the effect of RIS hardware impairments. Additionally, we derive the theoretical mean square error (MSE) of our proposed channel estimators, verified by numerical simulations and compared to the state-of-art LMMSE estimator and various benchmark schemes. We show that the MSE performance of our proposed Eigen-decomposition LMMSE estimator is better than that of the state-of-the-art low pilot overhead channel estimators."

Low-Overhead Channel Estimation for RIS-Aided Multi-Cell Networks in the Presence of Phase Error

We propose Concavity-induced Distance (CID) as a novel way to measure the dissimilarity between a pair of points in an unoriented point cloud. CID indicates the likelihood of two points or two sets of points belonging to different convex parts of an underlying shape represented as a point cloud. After analyzing its properties, we demonstrate how CID can benefit point cloud analysis without the need for meshing or normal estimation, which is beneficial for robotics applications when dealing with raw point cloud observations. By randomly selecting very few points for manual labeling, a CID-based point cloud instance segmentation via label propagation achieves comparable average precision as recent supervised deep learning approaches, on S3DIS and ScanNet datasets. Moreover, CID can be used to group points into approximately convex parts whose convex hulls can be used as compact scene representations in robotics, and it outperforms the baseline method in terms of grouping quality

Concavity-Induced Distance for Unoriented Point Cloud Decomposition

"We present the Hybrid Polar Decoder (HyPD), a hybrid classical–quantum decoder design for Polar error correction codes, which are becoming widespread in today’s 5G and tomorrow’s 6G networks. HyPD considers CMOS processing for the Polar decoder’s binary tree traversal, and Quantum Annealing (QA) processing for executing a Quantum Polar Decoder (QPD)–a maximum-likelihood QA-based Polar decoder submodule. QPD’s design efficiently transforms a Polar decoder into a quadratic polynomial optimization form, then maps this polynomial on to the physical QA hardware via QPD-MAP, a customized problem mapping scheme that is flexible to code rates, block lengths, and backed by QA correction, making QPD amenable to the QA optimization process. We have experimentally evaluated HyPD on a state-of-the-art QA device with 5,627 qubits, for 5G-NR Polar codes with block length of 1,024 bits, in Rayleigh fading channels. Our results show that HyPD outperforms successive cancellation list decoders of list size eight by half an order of bit error rate magnitude, and achieves a 1,500-bytes frame delivery rate of 99.1%, at 1 dB signal to noise ratio. Further studies present parameters tuning and QA compute time considerations. We also propose QPD-HW, a novel QA hardware topology tailored for the task of decoding Polar codes. QPD-HW is sparse, flexible to code rates and block lengths, and may be of potential interest to the designers of tomorrow’s 6G wireless networks. "

A Quantum Annealer-Enabled Decoder and Hardware Topology for NextG Wireless Polar Codes

Abstract – Where content producers needed to make drastic choices to allocate the limited contrast and colors in SDR, HDR offers the possibility to show more - meaning greater freedom and flexibility for their storytelling. However, while future-proof, HDR content presents its own challenges, for example, the wide variation in HDR display capabilities and the desire to have any production, HDR or SDR, play optimally on any display, HDR or SDR. The general solution to this issue is tone mapping, whether a tone compression to map from HDR to a lower HDR or SDR luminance, or tone expansion to map from SDR or HDR to a higher HDR luminance. Various techniques of tone mapping, including HLG, static look-up tables, and dynamic metadata, are considered and the relative advantages and performance analyzed. A proposed solution that has been introduced in practice, “Advanced HDR by Technicolor,” is described.

HDR Problems and Solutions

"In the coming 6th Generation (6G) era, the pervasiveness of wireless communications in everyday life and vertical markets is expected to take a major leap, enabling revolutionary and high demanding use cases. In-X subnetworks are expected to be located at the edge of the 6G ’network of networks’ and provide short-range high performance wireless connectivity in entities like robots, production modules, vehicles, classrooms, overcoming the limitation of current wireless technologies in terms of data rate, or latency and reliability. In this paper, we present the vision the 6G SHort range extreme communication IN Entities (6G-SHINE) project, granted by the Smart Network and Services Joint Undertaking (SNS JU). 6G-SHINE is the first project exclusively focused on short-range in-X subneworks, and aims at providing a significant breakthough in multiple wireless communication fields, from physical layer to network architectures. We present the main project objectives, and an overview of the technology components being researched. Also, we discuss the long term beneficial impact of the project, and its potential for future 6G standardization."

Enhancing short-range wireless communications: the 6G-SHINE vision

"We propose a data-driven approach for capturing and modeling the correlation between gaze direction, eyelids aperture and eyelids shape. Subtle features such as inner eye corner stretching, local skin bulging and closed eyelid deformations are incorporated in a personalized trilinear animation rig. This work has led to a publication to ACM-SCA 2022"

Detailed Eye Region Capture and Animation

"While self-supervised 3D face reconstruction models have improved over the years, because their training loss is mainly based on the photometric loss, they struggle to predict a 3D face with a correct head pose. On the other hand, supervised methods can predict more accurate head pose but require a lot of annotated data. In this work, we propose to improve self-supervised methods by adding 3D information to their input to improve the predicted head pose. We encode the 3D information in the form of the Projected Normalized Coordinate Code (PNCC). To reduce the need for annotated data to generate the PNCCs, we use transfer learning to adapt a pre-trained face autoencoder to predict the PNCCs. Our PNCC predictor can be trained using only a few annotated samples. Our experiments on a self-supervised method shows that the addition of the PNCC improves the predicted head pose."

Improving self-supervised 3D face reconstruction with few-shot transfer learning

Following the Industry 4.0 vision of a full digitization of the industry, time-critical services and applications, allowing network infrastructures to deliver information with determinism and reliability, are becoming more and more relevant for a set of vertical sectors. As a consequence, deterministic network solutions are progressively emerging, albeit they are still bounded to specific technological domains. Even considering the existence of interconnected deterministic networks, the provision of an end-to-end (E2E) deterministic service over them must rely on a specific control plane architecture, capable of seamlessly integrate and control the underlying multi-technology data plane. In this work, we envision such a control plane solution, extending previous works and exploiting several innovations and novel architectural concepts. The proposed control architecture is service-centric, in order to provide the necessary flexibility, scalability, and modularity to deal with a heterogenous data plane. The architecture is hierarchical and encompasses a set of management platforms to interact with specific network technologies overarched by an E2E platform for the management, monitoring, and control of E2E deterministic services. Furthermore, Artificial Intelligence (AI) and Digital Twinning are used to enable network predictability and automation, as well as smart resource allocation, to ensure service reliability in dynamic scenarios where existing services may terminate and new ones may need to be deployed.

A hierarchical AI-based control plane solution for multi-technology deterministic networks

This paper provides extensive analysis of RAW (Reliable and Available Wireless) enhancements and solutions needed to manage industrial environments more effectively. Starting from the description of the industrial use case, an analysis of gaps and potential new extensions is performed. Namely, the need to (i) support multi-domain operation, at both technology and administrative levels; (ii) integrate RAW with edge architectures; and, (iii) the support for mobility support in RAW networks, are analyzed. The identified gaps are indeed not yet tackled by the relevant standardization development organizations, mainly the Internet Engineering Task Force, and are thus object of our future work.

Using RAW as Control Plane for Wireless Deterministic Networks: Challenges Ahead

"Aspects of the 5G ecosystem, now being deployed in 3GPP releases 17/18/19, include split processing and edge computing resources. These can be enlisted to offload manipulation and rendering of immersive datasets, thereby reducing the burden on the mobile device. Instead, the mobile device receives only the rendered video, for one or both eyes, ready-made for display by the headset. Key design criteria for 5G connectivity to near-edge compute resources have been established based, in large part, on augmented and mixed reality use cases that rely on network slicing and quality of service (QoS) management, impose limits on bidirectional communication latencies, and establish minimum requirements for the compute resources themselves. Additionally, 5G system is becoming ""XR aware"" with a specific set of features for XR offloading. These features provide not only the proper network requirements, but also additional intelligence to consider device power and capacity considerations critical to scaling the deployment of XR services. This paper describes the evolution of XR applications from merely delivering media over 5G to how to best utilize 5G cloud/edge infrastructure to process and distribute advanced immersive experiences and content to lightweight, wireless, head-mounted displays. Critical use cases impose clear network requirements and how those are met in 5G is examined. Lastly, the shape of potential relationships within the stakeholder ecosystem are presented, all with the goal of guiding content providers (developers and service providers) through the paradigm shift from device-centric to network-centric XR services. "

The evolution of delivering immersive media over 5G/Cloud

PDL for Trustworthy 6G

"End-to-end image/video codecs are getting competitive compared to traditional compression techniques that have been developed through decades of manual engineering efforts. These trainable codecs have many advantages over traditional techniques such as easy adaptation on perceptual distortion metrics and high performance on specific domains thanks to their learning ability. However, state of the art neural codecs does not take advantage of the existence of gradient of entropy in decoding device. In this research, we theoretically show that gradient of entropy (available at decoder side) is correlated with the gradient of the reconstruction error (which is not available at decoder side). Thus, we use the former as a proxy in order to improve the compression performance. Our proposal is complementary up to image specific finetuning solutions. According to our results, we save between 1-2% of rate for the same quality with this proposal, for various pre-trained models."

LATENT-SHIFT: GRADIENT OF ENTROPY HELPS NEURAL CODECS

To address the need for interoperable user representations (avatars) cross-platform exchange format for immersive realities, ISO/IEC JTC 1/SC29/WG03 MPEG Systems has standardized a Scene Description framework in ISO/IEC 23090-14 \cite{AMD2}, that serves as a baseline format for user representation format to enrich interactive experience between 3D objects in an immersive scene. This work presents the MPEG Original Reference Geometric Avatar Neutral (MORGAN), a humanoid avatar specified as informative content in the MPEG-I Scene Description (MPEG-I SD) standardization group. Morgan is a generic avatar representation that facilitates interactivity and manipulation in immersive realities and is accompanied by complete body mesh and realistic appearance, hierarchical skeletal representation, blend shapes, hair modeling, and semantical representation of human body parts.

MORGAN: MPEG Original Reference Geometric Avatar Neutral

"In order to meet mobile cellular users’ ever-increasing data demands, today’s 4G and 5G networks are designed mainly with the goal of maximizing spectral efficiency. While they have made progress in this regard, controlling the carbon footprint and operational costs of such networks remains a long-standing problem among network designers. This paper takes a long view on this problem, envisioning a NextG scenario where the network leverages quantum annealing for cellular baseband processing. We gather and synthesize insights on power consumption, computational throughput and latency, spectral efficiency, operational cost, and feasibility timelines surrounding quantum technology. Armed with these data, we analyze and project the quantitative performance targets future quantum annealing hardware must meet in order to provide a computational and power advantage over CMOS hardware, while matching its whole-network spectral efficiency. Our quantitative analysis predicts that with quantum annealing hardware operating at a 82.8 µs problem latency and 2.7M qubits, quantum annealing will achieve a spectral efficiency equal to CMOS computation while reducing power consumption by 41 kW (45% lower) in a 5G base station scenario with 400 MHz bandwidth and 64 antennas, and an 8 kW power reduction (16% lower) using 1.3M qubits in a 200 MHz-bandwidth 5G scenario."

A Cost and Power Feasibility Analysis of Quantum Annealing for NextG Cellular Wireless Networks

Displaying video content requires significant amounts of energy. In face of the energy crisis and climate emergency, we propose a new approach to produce energy-aware images. The purpose of such images is to consume less energy when displayed onscreen while maximising the quality of experience. A new invertible neural network called Invertible Energy-Aware Network (InvEAN) is proposed to produce invertible energy-aware images, allowing both to reduce the energy consumption of display devices and to recover the original image if required. Experimental results show that the InvEAN network outperforms two existing methods over three datasets

Invertible Energy-Aware Images

The growing use of wide angle image capture devices and need for fast and accurate image analysis in computer visions have enforced the need for dedicated under-representation approaches. Most recent decomposition methods segment an image into a small number of irregular homogeneous regions, called superpixels. Nevertheless, these approaches are generally designed to segment standard 2D planar images, i.e., captured with a 90o angle view without distortion. In this work, we introduce a new general superpixel method called SphSPS (for Spherical Shortest Path-based Superpixels), dedicated to wide 360o spherical or omnidirectional images. Our method respects the geometry of the 3D spherical acquisition space and generalizes the notion of shortest path between a pixel and a superpixel center to this context, to fastly extract relevant clustering features. We demonstrate that considering the geometry of the acquisition space to compute the shortest path enables to jointly improve the segmentation performance and the shape regularity of the superpixels. To evaluate this regularity aspect, we also generalize a planar global regularity metric to the spherical space to address the limitations of the only existing spherical compactness measure. Finally, the proposed SphSPS method is validated on the reference 360o spherical panorama segmentation dataset and on synthetic road omnidirectional images. Our method significantly outperforms both planar and spherical state-of-the-art approaches in terms of segmentation performance and regularity, and we also demonstrate its robustness to noise.

Generalization of the shortest path approach for superpixel segmentation of omnidirectional images

In this paper, we present a method to compute energy-aware images, that aims to reduce the energy consumption of displays. This method relies on a lightweight unsupervised deep model which finds out the best trade-off between visual quality and energy reduction. From an input image and a energy reduction rate, a dimming map is inferred. We show that the proposed model performs as good as state-of-the-art methods, while being much more simple. In addition, the dimming map computation is constrained in order to ease its distribution throughout the video chain.

DEEP-LEARNING-BASED ENERGY AWARE IMAGES

Live production workflows, particularly those for sports, employ complex pipelines to deliver HD, UHD, SDR, and HDR video streams. Because content originates as HDR or SDR, effective and flexible conversions are required between formats. Graphics insertion is extensively used, presenting scores and analytics. These graphics must remain stable to provide a consistent appearance throughout the conversions. Today, diverse tone mapping and inverse tone mapping technologies are proposed to industry, each with the promise to create premium HDR content while maintaining the quality of SDR versions. Look Up Tables (LUTs) are the chief conversion technique used today, but constrain the capabilities of HDR and/or compromise the SDR look. Further, their static nature exposes content to artifacts emerging from changing conditions during live capture (e.g., sunset, passing clouds, shadows). While dynamic techniques accommodate such changes, a problem emerging is that none of the different LUT solutions nor dynamic solutions are fully interoperable: Each produces content with different properties, requiring different conversions to satisfy the critical requirement of delivering a seamless final content. This paper presents a generic solution, based on metadata, that resolves the compatibility issue. We present an implementation, a dynamic solution, implementing this proposal and providing tools based on dynamic metadata generation that ensures viewers enjoy the highest HDR video quality with no compromise of the SDR stream. By leveraging an image-based adaptation having already-acknowledged benefits of dynamic conversion, this solution introduces a constraint to allow stable graphics management: a specific Static Diffuse White, which ensures compatibility with existing static workflows while offering resolution to their shortcomings and paves the way for industry to transition smoothly from static to dynamic techniques and ensure delivery of premium HDR and SDR content.

HDR production workflows: a dynamic solution that provides stable graphics management and compatibility with static solutions

The proposed demonstration combines several commercially viable state-of-the-art technologies to achieve significant energy reduction while delivering HDR video without compromising QoE. The components of the demo are: (i) The demo is based our solution on VVC, the most efficient, highest quality video codec available today. (ii) We leverage Advanced HDR by Technicolor, a solution that uniquely can deliver HDR and SDR video using a single stream. (iii) We introduce a solution for dynamically adjusting peak brightness of displays via display adaptation controlled through the network using dynamic metadata encapsulated in MPEG SEI messages. This solution has been tested and shows a capability of reducing power by 10% or more without any subjective quality loss, and far more with minimal subjective impact. (iv) While we demonstrate a streaming (unicast) scenario in our live demo, our system is also capable of working in a dynamic unicast/multicast environment based on DVB transmission standards. (v) The entire demonstration is resulting from an industrial collaboration and is based on ready to market technologies and prototypes

Energy-aware HDR content end-to-end VVC encoding

Immersive media applications offer experiences which immerse the user in a virtual or hybrid environment and offer more degrees of freedom than with traditional 2D video content. Platforms providing immersive media often enable the user to interact with the content and/or with other users in shared virtual or mixed reality spaces. To address the need for an interoperable cross-platform exchange format and interactive solution for such 3D environments, ISO/IEC JTC 1/SC29/WG03 MPEG Systems has standardized a Scene Description framework in ISO/IEC 23090-14, that serves as an entry point format to compose rich 3D scenes, referencing and positioning 2D and 3D assets in the scene, blending with the real world, rich interactivity and providing realtime media delivery. Carriage formats have also been defined for the delivery of the Scene Description data and of the linked assets, based on the well-known and ubiquitous ISOBMFF standard i.e., ISO/IEC 14496-12.

MPEG-I Scene Description: a dynamic scene description framework for Immersive media

6G-BRICKS: Building Reusable testbed Infrastructures for Cloud-to-device breaKthrough technologieS

"This paper presents a real-time implementation of a platform jointly developed by InterDigital and Philips that showcases use cases leveraging the MPEG volumetric (MPEG-I V3C) and 2D video standards (VVC, HEVC). We will detail how our platform enables interoperability within existing and emerging XR ecosystems, including the acquisition, adaptive streaming, and real-time interactive playback of volumetric video on current and future client devices, for use in applications like telelearning, free-viewpoint sport replays, and 3D telepresence in connected ecosystems like the metaverse. MPEG’s Visual Volumetric Video-based Coding (V3C) standard is an extensive framework for the coding of volumetric video, from dynamic point clouds (V-PCC) to multi-view plus depth and multi-plane image (MIV) representations, to offer a single bitstream structure with a uniform bridge to systems-level standards. The V3C carriage standard defines how volumetric content can be stored, transported, and delivered to the end-user, and it repurposes existing 2D video hardware decoding capabilities and GPUs to decode and render volumetric video. We will present the architecture of InterDigital and Philips' collaborative multi-platform implementation, which leverages Unity 3D, and is comprised of the V3C, HEVC and VVC decoders and rendering pipelines. We will evaluate the real-time performance on self-captured and MPEG-provided content to demonstrate that this multi-platform implementation enables efficient streaming of volumetric and 2D video to phones, tablets, and head-mounted displays, and is approaching readiness for distribution through cloud infrastructure. "

Efficient delivery and rendering on client devices via MPEG-I standards for emerging volumetric video experiences

In this paper, deep learning-based techniques for film grain removal and synthesis that can be applied in video coding are proposed. Film grain is inherent in analog film content because of the physical process of capturing images and video on film. It can also be present in digital content where it is purposely added to reflect the era of analog film and to evoke certain emotions in the viewer or enhance the perceived quality. In the context of video coding, the random nature of film grain makes it both difficult to preserve and very expensive to compress. To better preserve it while compressing the content efficiently, film grain is removed and modeled before video encoding and then restored after video decoding. In this paper, a film grain removal model based on an encoder-decoder architecture and a film grain synthesis model based on a conditional generative adversarial network (cGAN) are proposed. Both models are trained on a large dataset of pairs of clean (grain-free) and grainy images. Quantitative and qualitative evaluations of the developed solutions were conducted and showed that the proposed film grain removal model is effective in filtering film grain at different intensity levels using two configurations: 1) a non-blind configuration where the film grain level of the grainy input is known and provided as input; and 2) a blind configuration where the film grain level is unknown. As for the film grain synthesis task, the experimental results show that the proposed model is able to reproduce realistic film grain with a controllable intensity level specified as input.

Deep-based film grain removal and synthesis

Kalman Filter Based Channel Tracking for RIS-Assisted Multi-User Networks

"The 3GPP has standardized the 5G New Radio (NR) sidelink (SL) technology that enables direct device-to-device communications. SL will be key in realizing several high-speed, low-latency V2X applications, such as automated driving. To meet these applications‚Äô high data rate requirements, operating SL at mmWave and sub-THz frequencies will be essential. However, the SL design primarily caters to sub-6 GHz frequencies and does not consider the directionality of transmissions at high frequencies. For Mode 2 of SL, where SL UEs perform sensing-based autonomous resource selection, this results in hidden node interference due to the Tx UE‚Äôs inability to sense transmissions not aligned with the primary direction of communication. We propose paired transmission and sensing of sidelink control information (SCI), whereby SL Tx UEs transmit and receive SCI in an additional paired direction directly opposite to the primary direction. This helps eliminate hidden node interference while avoiding too many exposed nodes. Simulations in NR V2X highway deployments show that the paired scheme improves the average packet reception ratio (PRR) by 27% over the state-of-the-art at the highest traffic loads. We also propose enhanced transmit power control strategies to minimize interference between concurrent SL transmissions. This enables better resource reuse and further improves performance, with our combined solution achieving at least 95% system average PRR in all scenarios. Finally, we present a stochastic geometry-based analytical model for a single-lane highway V2X network and validate it against simulation results. Our model provides insights into the reliability and capacity of high-frequency SL networks."

Enhanced Distributed Resource Selection and Power Control for High Frequency NR V2X Sidelink

"The limited contrast and colors in SDR have historically compelled content producers to make hard choices, whereas HDR offers them the possibility to show more - meaning telling stories with greater freedom and flexibility. Even so, HDR content presents a challenge of its own: The wide variation in capabilities among HDR displays conflicts with the desire to have any HDR or SDR production play optimally on any display, regardless of whether it is HDR or SDR. Tone mapping is the generic term for the operation that addresses this issue, whether tone compression, which maps from HDR to a lower-luminance HDR or SDR, or tone expansion, which maps from SDR or HDR to a higher- luminance HDR. Though the trend is to produce the highest quality content in HDR, legacy content is ever-present, and lots of new content is still produced in SDR. Mixing feeds in production requires tone expansion for SDR content to match the HDR. Distribution paths can then require tone compression back to SDR without degradation. In this context, a perfect SDR ‚Äì HDR ‚Äì SDR round-trip conversion is required to preserve artistic intent for all content. Further, in a live context, such tools need to be automatic and low latency. This paper discusses various tone mapping techniques, including static solutions such as look-up tables and dynamic solutions based on metadata. The relative advantages are considered and performance of each is analyzed. Particular attention is paid to the round-trip performance, which is important to content producers mastering both SDR and HDR sources yet wanting to exploit the full range of storytelling capabilities an HDR stream can offer. Test results using a proposed solution that has been introduced in practice, ‚ÄúAdvanced HDR by Technicolor¬Æ‚Äù, are described "

HDR Production ‚Äì Tone Mapping Techniques and Round-Trip Conversion Performance

"The media and entertainment industry consumes a significant amount of energy to create, distribute as well as to present video content to consumers. A large proportion of this energy is used by display devices, notably by the 1.9+ billion displays installed in consumers' homes globally. The production of light within a display is the main reason for this energy use, independent of display technology. Reducing the global energy use of display devices is therefore a topic of prime concern, by any means possible. Due to the very large number of displays currently in use, even small reductions per display may end up globally saving enormous amounts of energy. In this spirit, we present a video processing technique that is able to reduce the energy requirements without altering the visual quality of the displayed material. For use cases whereby a viewer would accept a certain loss of visual quality, a user parameter could be exposed that further reduces display energy requirements. "

Pixel value adjustment tor reduce the energy requirements of display devices

This keynote speech will talk about current research using AI technologies for immersive media. Will promote IDC research in this domain. The talk will include both AI-PCC and some long-term research, e.g., point cloud/mesh autoencoder.

Learning Technologies for Immersive Media

"Challenges of Ultra-Massive MIMO at THz frequencies (The document is a section contribution to Sensors journal publication)"

Challenges of Ultra-Massive MIMO at THz frequencies

This paper proposes a novel orchestration scheme to achieve optimal network resource allocation in Application Aware Networking (AAN) using wireless contextual information. We prove AAN to be NP-hard and show how our formulation enhances remote driving of a robot in a real wifi-based scenario, meeting latency constraints.

Application Aware Networking for Robotic Applications

"This paper proposes a novel framework to compress face-centric videos for both human viewing and machine analysis. Our system uses three coding branches to combine the power of generic face prior learning with data-dependent detail recovery. The generic branch embeds faces into a discrete code space described by a learned high-quality (HQ) codebook, based on which an HQ baseline face is reconstructed. The domain-adaptive branch adds domain-specific information through a supplementary domain-specific codebook, which adjusts reconstruction to the current data domain. The task-adaptive branch derives assistive details from a low-quality (LQ) input to help machine analysis over the restored face. The adaptive weights guide the importance of the domain-adaptive feature and the task-adaptive feature in reconstruction, which enable adaptive control to balance the perceptual quality, the fidelity, the bitrate, and the task accuracy. An online learning mechanism automatically adjusts the adaptive weights according to the actual compression needs. By sharing the main generic branch, our framework can better scale to multiple data domains and multiple tasks comparing to conventional coding schemes. Experiments demonstrate that with very low bitrates, our approach can restore faces with high perceptual quality for human use and can maintain high recognition accuracy for machine use."

Adaptive Face-Centric Video Compression for both Human and Machine

Film grain which used to be a by-product of the chemical processing in the analog film stock, is a desirable feature in the era of digital cameras. Besides participating to the artistic intent during content creation, film grain has also interesting properties in the video compression chain such as its ability to mask compression artifacts. In this paper, we use a deep learning-based framework for film grain analysis, generation and synthesis. Our framework consists of three modules: a style encoder performing film grain style analysis, a mapping network responsible for film grain style generation, and a synthesis network that generates and blends a specific grain style to a given content in a content-adaptive manner. All modules are trained jointly, thanks to dedicated loss functions, on a new large and diverse dataset of pairs of grain-free and grainy images that we made publicly available to the community1. Quantitative and qualitative evaluations show that fidelity to the reference grain, diversity of grain styles as well as a perceptually pleasant grain synthesis are achieved, demonstrating that each module outperforms the state-of-the-art in the task it was designed for.

Style-based film grain analysis and synthesis

"Efficient objective and perceptual metrics are valuable tools to evaluate the visual impact of com- pression artifacts on the visual quality of volumetric videos (VVs). In this paper we present some of the MPEG group efforts to create, benchmark and calibrate objective quality assessment metrics for volumetric videos represented as textured meshes. We created a challenging dataset of 176 vol- umetric videos impaired with various distortions and conducted a subjective experiment to gather human opinions (more than 5896 subjective scores were collected). We adapted some state of the art model-based metrics for point cloud evaluation to our context of textured mesh evaluation by selecting efficient sampling methods. We also present a new image-based metric for the evaluation of such VVs whose purpose is to reduce the cumbersome computation times inherent to the point based metrics due to their use of multiple kd-tree searches. Each metric presented above is calibrated (i.e., selection of best values for parameters such as the number of views or grid sampling density) and evaluated on our new ground-truth subjective dataset. For each metric, the optimal selection and combination of features is determined by logistic regression through cross-validation. This perfor- mance analysis, combined with MPEG experts‚Äô requirements, lead to the validation of two selected metrics and recommendations on the features of most importance through learned feature weights."

Crafting the MPEG metrics for objective and perceptual quality assessment of Volumetric Videos

"With the advent of new 3D scanning technologies, point clouds have become a crucial way to depict real and virtual objects/scenes. Point clouds represent the continuous surfaces of underlying object/scene through a collection (usually millions) of discrete, irregular, and often sparsely distributed 3D samples on the surface of the objects. This nature of point cloud data presents a considerable challenge to not only store but also understand and extract the topology of object(s) from the point cloud data. In this regard, our work presents a point cloud compression procedure that leverages \emph{sparse 3D convolutions} to extract features at various octree scales to cater for several quantization levels. For hierarchical flow of information between octree levels, our proposed method named SparseContextNet (SCN) also propagates features from a lower resolution scale to higher resolution scale via 3D upsampling convolutions. Additionally, based on our novel proposal for compression, we also propose new architectures for quality enhancement of quantized point cloud reconstruction. The proposed methods digest the already reconstructed quantized point clouds for simultaneous upsampling and refinement, and can seamlessly integrate with any octree-based compression method at no additional bitrate costs. Our experiments with LiDAR datasets reveal superior performance of our proposals compared to the state-of-the-art methods."

SPARSE CONVOLUTION BASED OCTREE FEATURE PROPAGATION FOR POINT CLOUD COMPRESSION AND QUALITY ENHANCEMENT

Point cloud compression is necessary to enable the deployment of virtual and mixed reality applications, since the number of points in a single character can be in the order of millions. There exist different learning-based approaches to compress the geometry of point clouds, on the other hand the compression of attributes has been less explored. In this paper we present a first normalizing flow architecture to compress point cloud attributes and the strategies to enable its use. The proposed method outperforms all other learning-based methods for point cloud attributes in the proposed quantitative measurement and has comparable results to the latest G-PCC, version 14.

NF-PCAC: Normalizing Flow based Point Cloud Attribute Compression

"Already approved PID1408 The increasing popularity of XR applications is driving the media industry to explore the creation and delivery of new immersive experiences, addressing the key challenge of real video content manipulation. The proposed session addresses challenges and presents key technologies leveraging open standards for large scale distribution. A volumetric video consists of a sequence of frames, where each frame is a static 3D representation of a real-world object or scene capture at a different point in time. A volumetric video can be presented as dynamic point clouds, multi-view plus depth or multi-plane image representations. Such bandwidth heavy content requires dedicated compression schemes to obtain data rates and files sizes that are economically viable in the industry. Standardization is required to ensure interoperability. The Moving Picture Experts Group (MPEG) Visual Volumetric Video-based Coding (V3C) standard is presented in this talk along with the MPEGV3C system part as an open standard solution for volumetric video efficient streaming. To enrich an XR experience, additional media and interactions mechanisms need to be created and distributed in a synchronized way with all the volumetric video components. For that purpose, the MPEG-I Haptics and MPEG-I Scene description standards are under development and propose solutions to the industry. On the real content creation side, enabling technologies covering 3D capturing, calibration, depth processing, format conversion, transmission and rendering are crucial and challenging. The talk presents two creation pipelines: (1) a real-time pipeline for telepresence using depth and color cameras; (2) an offline pipeline for sports and media applications using color cameras and prior geometric information. Both pipelines leverage the above standard codecs. "

Immersive Technologies From Capture to XR Experiences

Deep neural networks have been recently applied to point cloud compression (PCC). The features extracted via deep neural networks are essential for compression performance. Different from high level tasks such as point cloud classification or segmentation which homogenizes descriptors within same classes, PCC requires low level features discriminative for point level 3D reconstructions. With this motivation, we first adopt Gaussian distribution to model the shape of feature elements. Then, we propose a deep distribution-aware network (DDA-Net) which manipulates distributions of feature elements on-the-fly to favor the point cloud reconstruction with high fidelity. Moreover, a residual network is integrated to enhance the modification of the Gaussian models. The proposed DDA-Net is incorporated into an end-to-end PCC system. Experimental results show that our DDA-Net significantly improves the compression performance across a wide range of point clouds.

DDA-Net: Deep Distribution-Aware Network for Point Cloud Compression

The impact of randomized layer-2 addresses on privacy and applications

The contributions of the paper are listed below. First, the paper gives an insightful comparison of both use cases and their underlying technologies. Beyond the comparison of the two proposed sets of approaches, a thorough analysis of two existing energy-aware technologies is performed over different OLED screens both objectively and subjectively. Second, the paper also contributes to the research community with the release of a new dataset of energy measures on display screens. The last contribution concerns the presentation of an emerging activities in JVET standardization.

Energy-aware images: Quality of Experience vs Energy Reduction

"In the context of climate change, reducing the environmental impact of the video chain is a requirement for any actor in the field. Each step, from production, encoding, transmission, to decoding and display on screens participates to the overall energy consumption, but it is commonly agreed that displays are the most energy-intensive devices. Modifying videos and images so that they consume less energy when displayed has been already proposed in the literature and devices, via e.g., reducing the overall luminance level. Most of these technologies reduce the energy consumption at the expense of the quality of experience. In this paper, we explore two different use cases which claim for different compromises between QoE and consumed energy. Indeed, in the context of broadcast, the final QoE will prevail over the energy savings: content creators and their first intent should be prioritized and could not be deteriorated by visible modifications of the content. On the contrary, streaming content providers already adapt and prepare different versions of content with different levels of quality. These could be enlarged with different versions dependent on their energy consumption levels. Depending of these two use cases, broadcast and streaming, we propose to investigate and compare different sets of technologies that create energy aware images capable of reducing their energy consumption while displayed. Interestingly, depending on the use case, the technologies have different specificities. A first set of technologies dedicated to broadcast are conservative in terms of QoE while decreasing the energy consumption of images, so that they stay below a Just-Noticeable Difference threshold in terms of quality. On the contrary, a second set of technologies are more energy-driven, while providing images with different energy reduction rates and qualities, which could in turn be proposed in a streaming service to an end user willing to decrease its energy consumption. The paper will give an insightful comparison of both use cases and their underlying technologies. Beyond the comparison of the two proposed sets of approaches, a comprehensive and thorough analysis is performed over different OLED screens both objectively and subjectively. The paper also contributes to the research community with the release of a new dataset of energy measures on display screens. "

ETSI Multi-access Edge computing (MEC) main purpose is to improve latency and bandwidth consumption by keeping local traffic local while providing computing resources near the end-user. Despite its clear benefits, the next-generation of hyper-distributed applications (e.g., edge robotics, augmented environments, or smart agriculture) will exacerbate latency and bandwidth requirements, posing significant challenges to today's MEC deployments. In this work, we leverage on the current study item ETSI GR MEC 036, introducing a lightweight constrained version of a MEC platform that can be deployed in a mobile end terminal or in its closed locality. This work presents design options for cMEC, and how it can untangle the aforementioned gaps while being architectural compatible with a full-fledged MEC framework. Finally, key use cases and still open challenges are discussed, including recommendations to extend the current MEC standard towards constrained environments.

Beyond Multi-access Edge Computing: Keeping the Essentials for Constrained Devices

Recent advances in the area of long document matching have primarily focused on using transformer-based models for long document encoding and matching. There are two primary challenges associated with these models. Firstly, the performance gain provided by transformer-based models comes at a steep cost - both in terms of the required training time and the resource (memory and energy) consumption. The second major limitation is their inability to handle more than a pre-defined input token length at a time. In this work, we empirically demonstrate the effectiveness of simple neural models (such as feed-forward networks, and CNNs) and simple embeddings (like GloVe, and Paragraph Vector) over transformer-based models on the task of document matching. We show that simple models outperform the more complex BERT-based models while taking significantly less training time, energy, and memory. The simple models are also more robust to variations in document length and text perturbations.

Transformer-based Models for Long-Form Document Matching: Challenges and Empirical Analysis

"The spectrum access system (SAS) designated by the FCC follows a centralized server-client model where a spectrum user registers with one SAS service provider for spectrum allocation and other spectrum management functions. This model, however, is vulnerable to adversarial influence on individual SAS servers, causing concerns over system reliability and trustworthiness, especially when the ecosystem embraces a growing base of SAS service providers and heterogeneous user requirements. In this paper, we propose a blockchainbased decentralized SAS architecture called BD-SAS to provide SAS services securely and efficiently, without assuming trust in individual SAS servers. BD-SAS is backward compatible with the existing SAS infrastructure and supports the automatic execution of key spectrum management functions. A global blockchain network (G-Chain) is used for spectrum regulatory tasks while localized blockchain networks (L-Chains) are instantiated in individual spectrum zones for automating spectrum access assignment and other spectrum management activities. To further achieve security against an adaptive adversary, BDSAS integrates a SAS server reshuffle scheme to resist adaptive corruptions on individual SAS servers. We implemented a BDSAS prototype with practical blockchain platforms. Evaluation results demonstrate the feasibility and responsiveness of our system, wherein a spectrum access assignment can be finalized at the level of seconds."

BD-SAS: Enabling Dynamic Spectrum Sharing in Low-trust Environment

We explore the future of synchronous, multiplayer mobile AR gaming through our game V-Light, which extends current mobile AR game capacities using edge computing. Mobile AR games are currently limited by on-board processing power, while offloading operations to the cloud introduces high latency costs. This is a critical issue for games needing real-time response to player input. V-Light demonstrates how mobile AR games can leverage the power of edge computing, bringing computational resources closer to the user, keeping latency low and bandwidth high. We share our development toolkit, analyze the design and development of V-Light through the lens of an existing model for shared-world mobile AR, and demonstrate that edge computing can provide a ‚Äútime machine‚Äù that lets game designers prototype mobile AR games for devices that do not yet exist.

V-Light: Leveraging Edge Computing For The Design of Mobile Augmented Reality Games

"This paper presents a demonstration showcasing the capabilities of the upcoming MPEG-I standards on Haptics and Scene Description. The demonstration is a sensorial immersive experience using a Virtual Reality headset, haptic enabled controllers and a Haptic vest. The proposed experience is an interactive go-kart race-like game where interactions with the environment will trigger spatialized haptic feedback to provide an enhance sense of immersion. By implementing the MPEG-I standards on Haptics and Scene Description, and by designing this demonstration using exclusively these standards, this work showcases part of the research and standardization efforts produced through the MPEG ecosystem."

Sensorial Immersive Experiences using MPEG Haptic and Scene Description Standards

"To compensate for the path losses in millimeter wave (mmWave) communication, multiple-input-multiple-output (MIMO) systems leverage beamforming based solutions to boost the receive SNR. Since beamforming requires knowledge of the wireless channel, its performance is also closely tied with the channel estimation accuracy. In mmWave communication, wireless channels are known to have only a few dominant paths. Several channel estimation methods leverage this characteristic by either using low rank methods or by modeling element-wise sparsity of channels in angular domain. In this work we propose that the channel in angular domain is better characterized as group sparse rather than element wise sparse. We further leverage the recent advancements in implicit regularization with gradient descent to develop a matrix factorization formula- tion that implicitly enforces group sparsity without additional regularization terms. We further showcase that the proposed approach outperforms existing low rank or sparsity based matrix completion methods for channel estimation."

RESEARCH PAPERS