1. Efficient material selection for training occluded mmWave radar-based gesture recognisersNuwan Attygalle, Luis A. Leiva, Matjaž Kljun, Klen Čopič Pucihar, 2026, izvirni znanstveni članek Opis: Radar-based gesture recognition has emerged as a promising approach for unobtrusive interaction. Unlike camera-based systems, radar sensors can detect gestures through opaque materials, enabling seamless embedding into various everyday objects. However, it remains unclear how to train models efficiently for robust gesture recognition through diverse materials. To investigate this, we collected a dataset of 17,520 gesture recordings performed through 73 everyday materials. By comparing several material-sampling and data-augmentation strategies, we found that a small carefully selected representative subset of training materials was sufficient to match performance of a classifier trained on the full material dataset. Our results showed that the models trained on 14 quota-sampled materials achieved accuracies of 95.8% and 91.2%, comparable to training on all 73 materials (96.8% and 91.6%) and significantly better than training without material data (66.8% and 65.8%). Among the evaluated sampling approaches, Quota sampling also provided the best overall trade-off between performance and practicality. In contrast, classifiers trained on augmented data performed worse than those trained on actual material-specific data. Taken together, these findings indicate that, for the tested sensor, gesture set, and material collection, carefully selected real-material data offer a practical route to reducing material-specific data collection in radar-based gesture recognition while preserving generalisation. Code, models, and data are available in the public repository, with additional details provided in the supplementary materials: https://gitlab.com/hicuplab/seeing-through.
Ključne besede: deep learning, training optimization for neural networks, millimetre-wave radar, gesture recognition, material selection, sampling, synthetic noise
Objavljeno v RUP: 19.06.2026; Ogledov: 288; Prenosov: 12
 Celotno besedilo (15,56 MB)
Gradivo ima več datotek! Več... |
2. Radar-based gesture recognition on deformable objectsKlen Čopič Pucihar, Matjaž Kljun, Nuwan Attygalle, 2026, samostojni znanstveni sestavek ali poglavje v monografski publikaciji Opis: This chapter investigates the feasibility and challenges of using millimetre-wave radar for gesture recognition on deformable objects, such as plush toys or other objects made of flexible materials, which are typically not instrumented with sensors. Unlike vision-based systems, which are limited by occlusion and require clear line of sight, radar sensing can detect gestures through non-conductive materials. The authors compare prior work on gesture recognition performance across mid-air, on-object and on-deformable-object contexts using different radar signal representations and deep learning models. In addition, the authors conduct an experiment demonstrating that object deformations do not negatively impact recognition accuracy. These findings open new possibilities for contactless interaction with soft materials in everyday environments without requiring embedded instrumentation.
Ključne besede: radar, gesture recognition, deformable objects
Objavljeno v RUP: 18.05.2026; Ogledov: 435; Prenosov: 16
Celotno besedilo (41,39 MB)
Gradivo ima več datotek! Več... |
3. Radar-based human-computer interaction when sensing through materialsNuwan Attygalle, Matjaž Kljun, Arthur Sluÿters, Klen Čopič Pucihar, 2026, samostojni znanstveni sestavek ali poglavje v monografski publikaciji Opis: We increasingly interact with computing devices that are either worn on the body (e.g. smartphones, smartwatches) or are embedded within our surroundings (e.g. smart homes, smart offices). These interactions occur through a variety of input methods, including physical buttons and knobs, mid-air gestures, touch and voice commands. With the exception of voice, these modalities require direct or near-direct physical contact with the device, typically involving line-of-sight or proximity for touch or grasp. However, interaction becomes constrained or entirely infeasible (i) when the interaction device is covered (e.g. when a smartphone is inside a pocket or a smartwatch is covered by a jacket sleeve), (ii) in sterile environments requiring separation between the user and device (e.g. when wearing personal protective equipment) or (iii) when interfaces are deliberately concealed for aesthetic, safety or functional reasons. Enabling interaction in such contexts is important to leverage the computational capabilities embedded in our environments. Yet, current technologies remain limited when interaction through occluding materials is needed. In this chapter, we examine existing research on the use of radar-based systems for interaction through materials, focusing on how materials affect system performance, principles for designing such interfaces and strategies to advance these systems.
Ključne besede: radar, human-computer interaction, sensing through materials
Objavljeno v RUP: 18.05.2026; Ogledov: 411; Prenosov: 10
Celotno besedilo (53,62 MB)
Gradivo ima več datotek! Več... |
4. Comparative testing of radar signal representations when sensing through materialsNuwan Attygalle, Matjaž Kljun, Una Vuletić, Klen Čopič Pucihar, 2026, samostojni znanstveni sestavek ali poglavje v monografski publikaciji Opis: The ability to sense mid-air gestures with miniaturised radars embedded in everyday objects opens up new opportunities for interaction. Applications include integration into wearable devices, automotive dashboards, smart furniture, and other components within smart environments. Despite this potential, the lack of studies on how various occluding materials affect gesture recognition performance hinders progress in this area. Previous studies have primarily focused on evaluating only one type of radar signal representation, despite the fact that several other representations exist and were proved effective. To address this, the chapter presents a comparative evaluation of four radar signal representations: In-phase and Quadrature (IQ) representations in the frequency domain and magnitude and range-angle (including both elevation and azimuth components) and range-Doppler. The goal is to assess their robustness against signal distortions introduced by occluding materials. Preliminary results indicate that recognition performance tends to improve with a higher transmission coefficient. Moreover, range-Doppler and range-angle representations exhibit significantly greater robustness to distortion compared to IQ representations
Ključne besede: comparative testing, radar signal representations, sensing through materials
Objavljeno v RUP: 18.05.2026; Ogledov: 405; Prenosov: 11
Celotno besedilo (41,62 MB)
Gradivo ima več datotek! Več... |
5. Digital signal processing tools for radar-based human-computer interactionNuwan Attygalle, Matjaž Kljun, Klen Čopič Pucihar, 2026, samostojni znanstveni sestavek ali poglavje v monografski publikaciji Opis: In recent years, miniature radar-on-chip sensors have been explored for HCI by both academia and industry. This is driven by the availability of affordable radar hardware and advances in signal processing and machine learning. However, comparative evaluation of radar-based gesture interaction systems is challanging and rear. One important dimension for comparison is the set of radar signal representa- tions derived from raw voltage data. These representations commonly include range- Doppler, range-azimuth-angle, range-elevation-angle, point cloud and In-phase and Quadrature (IQ) radar cube formats. However, existing studies often restrict com- parative analysis to a single signal representation type, typically focusing on gesture recognition algorithms or minor variations within Digital Signal Processing (DSP) pipelines. To promote comparative evaluation of radar signal representations, this chapter develops an open-source application designed to facilitate efficient and reli- able dataset preparation of various radar signal representations. The application sup- ports visualisation of generated signal representations and includes a command-line interface for batch processing, thereby streamlining the dataset preparation workflow.
Ključne besede: digital signal processing, radar, human-computer interaction
Objavljeno v RUP: 18.05.2026; Ogledov: 416; Prenosov: 8
Celotno besedilo (41,69 MB)
Gradivo ima več datotek! Več... |
6. Can immersion protect us from distraction? : The impact of real-world distractions on learning in augmented realityKlen Čopič Pucihar, Karolina Trajkovska, Anuradhi Maheshya W. Weerasinghe Arachchillage, Ali Waqas, Douglas A. Parry, Daniel B. Le Roux, Matjaž Kljun, 2026, izvirni znanstveni članek Opis: While many digital distractions can be managed, real-world interruptions, such as phone calls, notifications, and office noise, are harder to control and can harm productivity, well-being, and learning. Mixed reality systems like Augmented Reality (AR) are often described as immersive—a property which might protect users from such disruptions. We tested this assumption by comparing a head-mounted AR interface that overlays digital annotations on physical objects with a traditional flat screen during vocabulary learning under common office distractions. In a user study (n = 32), AR users reported feeling less distracted and recalled less task-irrelevant information, but their learning performance did not improve. Instead, distraction-related performance decline was greater in AR. Physiological and self-report measures showed no reduction in effort or workload, and participants with higher auditory distractibility did not benefit. Overall, AR annotation alone may not sufficiently shield learners from real-world distractions, motivating new design approaches.
Ključne besede: distractions, auditory distractibility, augmented reality, immersion and presence, learning
Objavljeno v RUP: 15.05.2026; Ogledov: 401; Prenosov: 12
Celotno besedilo (3,52 MB)
Gradivo ima več datotek! Več... |
7. |
8. |
9. |
10. Real-time gesture transmission with a robotic hand : embodied signals for non-verbal remote communicationLea Pajnič, Matjaž Kljun, Anuradhi Maheshya W. Weerasinghe Arachchillage, Klen Čopič Pucihar, 2025, objavljeni znanstveni prispevek na konferenci Opis: This work explores how computer vision and robotics can support remote, gesture-based embodied signals for expressing presence and emotion in remote communication. We present an initial proof-of-concept in which users interact through robotic hands placed on their desks: one user’s hand gestures are captured in real time by a camera, transmitted over a network, and reproduced by a robotic hand at the remote location. The prototype uses the InMoov robotic hand and MediaPipe Hands for gesture tracking across varied lighting conditions, viewing angles, and backgrounds. Our preliminary tests demonstrate that gestures can be reliably recognised and consistently reproduced through stable network communication. While still at an early stage, this project illustrates the potential of combining affordable robotics with computer vision to create accessible alternatives to voice communication and new forms of remote communication.
Ključne besede: robotic hand, gesture transmission, embodied signals, non-verbal communication, remote communication, computer vision
Objavljeno v RUP: 30.01.2026; Ogledov: 865; Prenosov: 6
Celotno besedilo (642,14 KB) |
