Evaluation of Body Parts-based Latent Representations for Skeletal Human Motion Reconstruction

Acquiring, encoding, transmitting, decoding, and displaying motion signals is an essential challenge in our new world of connected ecosystems, which includes immersive applications (AR, VR, XR) and video games. In addition to being potentially affected by multiple sources of disturbance (e.g., signal noise or packet loss), this motion data should be modifiable and customizable to fit the needs of the application. A possible solution would be to create a semantic representation, which could be modified using high-level and interpretable signals. Several approaches in the human motion modeling literature have successfully proposed separating the body into smaller parts. This representation facilitates deep-based modeling by explicitly integrating the hierarchy of the human body into the learning. However, this representation is an intermediate step before an arbitrary downstream task in the literature. Such a pipeline leads to a necessary entanglement of the data in the second process and the loss of the semantic information. To overcome the problem of streamed animation data, we thus propose to exploit this representation more deeply by creating fully independent body parts models, even at the cost of losing possible correlations between the body parts. We further compare the performances of distinct body parts decomposition using multiple standard reconstruction metrics. We argue that this Body Parts approach brings several advantages compared to a compact one, such as a perfectly partitioned motion data, which is also interpretable (body semantics) while obtaining comparable or better reconstruction results.