Search the repository

Abstract: Radar-based gesture recognition has emerged as a promising approach for unobtrusive interaction. Unlike camera-based systems, radar sensors can detect gestures through opaque materials, enabling seamless embedding into various everyday objects. However, it remains unclear how to train models efficiently for robust gesture recognition through diverse materials. To investigate this, we collected a dataset of 17,520 gesture recordings performed through 73 everyday materials. By comparing several material-sampling and data-augmentation strategies, we found that a small carefully selected representative subset of training materials was sufficient to match performance of a classifier trained on the full material dataset. Our results showed that the models trained on 14 quota-sampled materials achieved accuracies of 95.8% and 91.2%, comparable to training on all 73 materials (96.8% and 91.6%) and significantly better than training without material data (66.8% and 65.8%). Among the evaluated sampling approaches, Quota sampling also provided the best overall trade-off between performance and practicality. In contrast, classifiers trained on augmented data performed worse than those trained on actual material-specific data. Taken together, these findings indicate that, for the tested sensor, gesture set, and material collection, carefully selected real-material data offer a practical route to reducing material-specific data collection in radar-based gesture recognition while preserving generalisation. Code, models, and data are available in the public repository, with additional details provided in the supplementary materials: https://gitlab.com/hicuplab/seeing-through.
Keywords: deep learning, training optimization for neural networks, millimetre-wave radar, gesture recognition, material selection, sampling, synthetic noise
Published in RUP: 19.06.2026; Views: 309; Downloads: 12
Full text (15,56 MB)
This document has more files! More...

Abstract: This chapter investigates the feasibility and challenges of using millimetre-wave radar for gesture recognition on deformable objects, such as plush toys or other objects made of flexible materials, which are typically not instrumented with sensors. Unlike vision-based systems, which are limited by occlusion and require clear line of sight, radar sensing can detect gestures through non-conductive materials. The authors compare prior work on gesture recognition performance across mid-air, on-object and on-deformable-object contexts using different radar signal representations and deep learning models. In addition, the authors conduct an experiment demonstrating that object deformations do not negatively impact recognition accuracy. These findings open new possibilities for contactless interaction with soft materials in everyday environments without requiring embedded instrumentation.
Keywords: radar, gesture recognition, deformable objects
Published in RUP: 18.05.2026; Views: 447; Downloads: 19
Full text (41,39 MB)
This document has more files! More...

Abstract: The ability to sense mid-air gestures with miniaturised radars embedded in everyday objects opens up new opportunities for interaction. Applications include integration into wearable devices, automotive dashboards, smart furniture, and other components within smart environments. Despite this potential, the lack of studies on how various occluding materials affect gesture recognition performance hinders progress in this area. Previous studies have primarily focused on evaluating only one type of radar signal representation, despite the fact that several other representations exist and were proved effective. To address this, the chapter presents a comparative evaluation of four radar signal representations: In-phase and Quadrature (IQ) representations in the frequency domain and magnitude and range-angle (including both elevation and azimuth components) and range-Doppler. The goal is to assess their robustness against signal distortions introduced by occluding materials. Preliminary results indicate that recognition performance tends to improve with a higher transmission coefficient. Moreover, range-Doppler and range-angle representations exhibit significantly greater robustness to distortion compared to IQ representations
Keywords: comparative testing, radar signal representations, sensing through materials
Published in RUP: 18.05.2026; Views: 415; Downloads: 11
Full text (41,62 MB)
This document has more files! More...

Abstract: Although deformable objects are not typically designed for digital interaction, they offer a largely unexplored potential—any such object could be repurposed as a medium for controlling digital content. While existing approaches embed sensors into deformable objects to enable interaction, this limits scalability and practicality of such systems. An alternative is to perform gesture recognition on deformable objects using a wrist-worn radar sensor. However, when analysing reflected radar signals it is difficult to separate reflections originating from the continues deformations of the object shape and those from the user’s hand and fingers. Additionally, the continuous shape changes of deformable objects introduce changes in radar cross-section, affecting signal variability. Furthermore, user ergonomics—such as variations in hand size, finger dexterity, and strength—are likely to influence the degree of object deformation during interaction. In this paper, we explore whether radar sensing can be used for robust gesture detection on deformable objects, focusing on how well does a system generalize to previously unseen users and what can we do to improve such generalisability. In pursuit of this goal, we record a dataset of 4.3k labelled gestures with Google Soli millimeter-wave radar sensor on a plush toy and demonstrates robust classification performance, achieving accuracy of up to 90% on a five-gesture set. Furthermore, we investigate model generalizability and show that transfer learning improves recognition for previously unseen users, yielding performance gains of up to 20%. These findings highlight the potential of radar-based sensing for spontaneous and practical interaction with deformable objects.
Keywords: gesture recognition, deformable objects, millimeter-wave radar
Published in RUP: 23.06.2025; Views: 1378; Downloads: 12
Full text (9,65 MB)
This document has more files! More...

Keywords: radar, wearables, fabrics, Soli, gestures, deep learning
Published in RUP: 06.05.2021; Views: 3854; Downloads: 36
Link to full text