Search the repository

Abstract: Radar-based gesture recognition has emerged as a promising approach for unobtrusive interaction. Unlike camera-based systems, radar sensors can detect gestures through opaque materials, enabling seamless embedding into various everyday objects. However, it remains unclear how to train models efficiently for robust gesture recognition through diverse materials. To investigate this, we collected a dataset of 17,520 gesture recordings performed through 73 everyday materials. By comparing several material-sampling and data-augmentation strategies, we found that a small carefully selected representative subset of training materials was sufficient to match performance of a classifier trained on the full material dataset. Our results showed that the models trained on 14 quota-sampled materials achieved accuracies of 95.8% and 91.2%, comparable to training on all 73 materials (96.8% and 91.6%) and significantly better than training without material data (66.8% and 65.8%). Among the evaluated sampling approaches, Quota sampling also provided the best overall trade-off between performance and practicality. In contrast, classifiers trained on augmented data performed worse than those trained on actual material-specific data. Taken together, these findings indicate that, for the tested sensor, gesture set, and material collection, carefully selected real-material data offer a practical route to reducing material-specific data collection in radar-based gesture recognition while preserving generalisation. Code, models, and data are available in the public repository, with additional details provided in the supplementary materials: https://gitlab.com/hicuplab/seeing-through.
Keywords: deep learning, training optimization for neural networks, millimetre-wave radar, gesture recognition, material selection, sampling, synthetic noise
Published in RUP: 19.06.2026; Views: 281; Downloads: 12
Full text (15,56 MB)
This document has more files! More...

Abstract: This chapter investigates the feasibility and challenges of using millimetre-wave radar for gesture recognition on deformable objects, such as plush toys or other objects made of flexible materials, which are typically not instrumented with sensors. Unlike vision-based systems, which are limited by occlusion and require clear line of sight, radar sensing can detect gestures through non-conductive materials. The authors compare prior work on gesture recognition performance across mid-air, on-object and on-deformable-object contexts using different radar signal representations and deep learning models. In addition, the authors conduct an experiment demonstrating that object deformations do not negatively impact recognition accuracy. These findings open new possibilities for contactless interaction with soft materials in everyday environments without requiring embedded instrumentation.
Keywords: radar, gesture recognition, deformable objects
Published in RUP: 18.05.2026; Views: 429; Downloads: 16
Full text (41,39 MB)
This document has more files! More...

Abstract: The ability to sense mid-air gestures with miniaturised radars embedded in everyday objects opens up new opportunities for interaction. Applications include integration into wearable devices, automotive dashboards, smart furniture, and other components within smart environments. Despite this potential, the lack of studies on how various occluding materials affect gesture recognition performance hinders progress in this area. Previous studies have primarily focused on evaluating only one type of radar signal representation, despite the fact that several other representations exist and were proved effective. To address this, the chapter presents a comparative evaluation of four radar signal representations: In-phase and Quadrature (IQ) representations in the frequency domain and magnitude and range-angle (including both elevation and azimuth components) and range-Doppler. The goal is to assess their robustness against signal distortions introduced by occluding materials. Preliminary results indicate that recognition performance tends to improve with a higher transmission coefficient. Moreover, range-Doppler and range-angle representations exhibit significantly greater robustness to distortion compared to IQ representations
Keywords: comparative testing, radar signal representations, sensing through materials
Published in RUP: 18.05.2026; Views: 399; Downloads: 11
Full text (41,62 MB)
This document has more files! More...

Abstract: While many digital distractions can be managed, real-world interruptions, such as phone calls, notifications, and office noise, are harder to control and can harm productivity, well-being, and learning. Mixed reality systems like Augmented Reality (AR) are often described as immersive—a property which might protect users from such disruptions. We tested this assumption by comparing a head-mounted AR interface that overlays digital annotations on physical objects with a traditional flat screen during vocabulary learning under common office distractions. In a user study (n = 32), AR users reported feeling less distracted and recalled less task-irrelevant information, but their learning performance did not improve. Instead, distraction-related performance decline was greater in AR. Physiological and self-report measures showed no reduction in effort or workload, and participants with higher auditory distractibility did not benefit. Overall, AR annotation alone may not sufficiently shield learners from real-world distractions, motivating new design approaches.
Keywords: distractions, auditory distractibility, augmented reality, immersion and presence, learning
Published in RUP: 15.05.2026; Views: 400; Downloads: 12
Full text (3,52 MB)
This document has more files! More...

Abstract: This work explores how computer vision and robotics can support remote, gesture-based embodied signals for expressing presence and emotion in remote communication. We present an initial proof-of-concept in which users interact through robotic hands placed on their desks: one user’s hand gestures are captured in real time by a camera, transmitted over a network, and reproduced by a robotic hand at the remote location. The prototype uses the InMoov robotic hand and MediaPipe Hands for gesture tracking across varied lighting conditions, viewing angles, and backgrounds. Our preliminary tests demonstrate that gestures can be reliably recognised and consistently reproduced through stable network communication. While still at an early stage, this project illustrates the potential of combining affordable robotics with computer vision to create accessible alternatives to voice communication and new forms of remote communication.
Keywords: robotic hand, gesture transmission, embodied signals, non-verbal communication, remote communication, computer vision
Published in RUP: 30.01.2026; Views: 854; Downloads: 6
Full text (642,14 KB)