SearcharxivSearch

arXiv subjects

Matjaž Kljun

Publications and source records attributed to Matjaž Kljun.

14 recordsLinked to original sources

Assessing Medical Training Skills via Eye and Head Movements

We examined eye and head movements to gain insights into skill development in clinical settings. A total of 24 practitioners participated in simulated baby delivery training sessions. We calculated key metrics, including pupillary response rate, fixation duration, or angular velocity. Our findings indicate that eye and head tracking can effectively differentiate between trained and untrained practitioners, particularly during labor tasks. For example, head-related features achieved an F1 score of 0.85 and AUC of 0.86, whereas pupil-related features achieved F1 score of 0.77 and AUC of 0.85. The results lay the groundwork for computational models that support implicit skill assessment and training in clinical settings by using commodity eye-tracking glasses as a complementary device to more traditional evaluation methods such as subjective scores.

cs.HC

Text-to-Image Generation for Vocabulary Learning Using the Keyword Method

The 'keyword method' is an effective technique for learning vocabulary of a foreign language. It involves creating a memorable visual link between what a word means and what its pronunciation in a foreign language sounds like in the learner's native language. However, these memorable visual links remain implicit in the people's mind and are not easy to remember for a large set of words. To enhance the memorisation and recall of the vocabulary, we developed an application that combines the keyword method with text-to-image generators to externalise the memorable visual links into visuals. These visuals represent additional stimuli during the memorisation process. To explore the effectiveness of this approach we first run a pilot study to investigate how difficult it is to externalise the descriptions of mental visualisations of memorable links, by asking participants to write them down. We used these descriptions as prompts for text-to-image generator (DALL-E2) to convert them into images and asked participants to select their favourites. Next, we compared different text-to-image generators (DALL-E2, Midjourney, Stable and Latent Diffusion) to evaluate the perceived quality of the generated images by each. Despite heterogeneous results, participants mostly preferred images generated by DALL-E2, which was used also for the final study. In this study, we investigated whether providing such images enhances the retention of vocabulary being learned, compared to the keyword method only. Our results indicate that people did not encounter difficulties describing their visualisations of memorable links and that providing corresponding images significantly improves memory retention.

cs.HC

Teaching Shortest Path Algorithms With a Robot and Overlaid Projections

Robots have the potential to enhance teaching of advanced computer science topics, making abstract concepts more tangible and interactive. In this paper, we present Timmy-a GoPiGo robot augmented with projections to demonstrate shortest path algorithms in an interactive learning environment. We integrated a JavaScript-based application that is projected around the robot, which allows users to construct graphs and visualise three different shortest path algorithms with colour-coded edges and vertices. Animated graph exploration and traversal are augmented by robot movements. To evaluate Timmy, we conducted two user studies. An initial study (n=10) to explore the feasibility of this type of teaching where participants were just observing both robot-synced and the on-screen-only visualisations. And a pilot study (n=6) where participants actively interacted with the system, constructed graphs and selected desired algorithms. In both studies we investigated the preferences towards the system and not the teaching outcome. Initial findings suggest that robots offer an engaging tool for teaching advanced algorithmic concepts, but highlight the need for further methodological refinements and larger-scale studies to fully evaluate their effectiveness.

cs.RO

Gaze2AOI: Open Source Deep-learning Based System for Automatic Area of Interest Annotation with Eye Tracking Data

Eye gaze is considered an important indicator for understanding and predicting user behaviour, as well as directing their attention across various domains including advertisement design, human-computer interaction and film viewing. In this paper, we present a novel method to enhance the analysis of user behaviour and attention by (i) augmenting video streams with automatically annotating and labelling areas of interest (AOIs), and (ii) integrating AOIs with collected eye gaze and fixation data. The tool provides key features such as time to first fixation, dwell time, and frequency of AOI revisits. By incorporating the YOLOv8 object tracking algorithm, the tool supports over 600 different object classes, providing a comprehensive set for a variety of video streams. This tool will be made available as open-source software, thereby contributing to broader research and development efforts in the field.

cs.SE

Retzzles: Do Jigsaw Puzzle Actions on Interactive Display Maps Increase the Retention of Map Information?

While maps provide upfront content, this might not always be the most effective way for users to remember information. With the proliferation of interactive displays for tourists and visitors in public spaces, we can create a more playful user experience with maps than just exploring them. Adding interactions with the map could also help users retain more information as they use them. In this paper, we investigated whether completing a jigsaw puzzle of a map supports users in retaining more information about a specific map. The results of a between-subject study with a sample of n=28 indicate that additional interaction helped improve mean scores of textual and spatial recall but not visual recall. However, the results are not statistically significant, and the topic is subject to further investigation. Our findings contribute to discussions on using interactive touchscreen displays in similar learning scenarios involving memory retention.

cs.HC

Teach Me How to ImproVISe: Co-Designing an Augmented Piano Training System for Improvisation

Improvisation is a vital but often neglected aspect of traditional piano teaching. Challenges such as difficulty in assessment and subjectivity have hindered its effective instruction. Technological approaches, including augmentation, aim to enhance piano instruction, but the specific application of digital augmentation for piano improvisation is under-explored. This paper outlines a co-design process developing an Augmented Reality (AR) Piano Improvisation Training System, ImproVISe, involving improvisation teachers. The prototype, featuring basic improvisation concepts, was created and refined through expert interaction. Their insights guided the identification of objectives, tools, interaction metaphors, and software features. The findings offer design guidelines and recommendations to address challenges in assessing piano improvisation in a learning context.

cs.HC

A Survey of Augmented Piano Prototypes: Has Augmentation Improved Learning Experiences?

Humans have been developing and playing musical instruments for millennia. With technological advancements, instruments were becoming ever more sophisticated. In recent decades computer-supported innovations have also been introduced in hardware design, usability, and aesthetics. One of the most commonly digitally augmented instruments is the piano. Besides electronic keyboards, several prototypes augmenting pianos with different projections providing various levels of interactivity on and around the keyboard have been implemented in order to support piano players. However, it is still not understood if these solutions are indeed supporting the learning process. In this paper we present a systematic review of augmented piano prototypes focusing on instrument learning, which is based on the four themes derived from interviews of piano experts to better understand the problems of teaching the piano. These themes are: (i) synchronised movement and body posture, (ii) sight-reading, (iii) ensuring motivation, and (iv) encouraging improvisation. We found that prototypes are saturated on the synchronisation themes, and there are opportunities for sight-reading, motivation, and improvisation themes. We conclude by presenting recommendations on augmenting piano systems towards enriching the piano learning experience as well as on possible directions to expand knowledge in the area.

cs.HC

VocabulARy replicated: comparing teenagers to young adults

A critical component of user studies is gaining access to a representative sample of the population researches intend to investigate. Nevertheless, the vast majority of human-computer interaction (HCI)studies, including augmented reality (AR) studies, rely on convenience sampling. The outcomes of these studies are often based on results obtained from university students aged between 19 and 26 years. In order to investigate how the results from one of our studies are affected by convenience sampling, we replicated the AR-supported language learning study called VocabulARy with 24 teenagers, aged between 14 and 19 years. The results verified most of the outcomes from the original study. In addition, it also revealed that teenagers found learning significantly less mentally demanding compared to young adults, and completed the study in a significantly shorter time. All this at no cost to learning outcomes.

cs.HC

Arigatō: Effects of Adaptive Guidance on Engagement and Performance in Augmented Reality Learning Environments

Experiential learning (ExL) is the process of learning through experience or more specifically "learning through reflection on doing". In this paper, we propose a simulation of these experiences, in Augmented Reality (AR), addressing the problem of language learning. Such systems provide an excellent setting to support "adaptive guidance", in a digital form, within a real environment. Adaptive guidance allows the instructions and learning content to be customised for the individual learner, thus creating a unique learning experience. We developed an adaptive guidance AR system for language learning, we call Arigatō (Augmented Reality Instructional Guidance & Tailored Omniverse), which offers immediate assistance, resources specific to the learner's needs, manipulation of these resources, and relevant feedback. Considering guidance, we employ this prototype to investigate the effect of the amount of guidance (fixed vs. adaptive-amount) and the type of guidance (fixed vs. adaptive-associations) on the engagement and consequently the learning outcomes of language learning in an AR environment. The results for the amount of guidance show that compared to the adaptive-amount, the fixed-amount of guidance group scored better in the immediate and delayed (after 7 days) recall tests. However, this group also invested a significantly higher mental effort to complete the task. The results for the type of guidance show that the adaptive-associations group outperforms the fixed-associations group in the immediate, delayed (after 7 days) recall tests, and learning efficiency. The adaptive-associations group also showed significantly lower mental effort and spent less time to complete the task.

cs.HC

VocabulARy: Learning Vocabulary in AR Supported by Keyword Visualisations

Learning vocabulary in a primary or secondary language is enhanced when we encounter words in context. This context can be afforded by the place or activity we are engaged with. Existing learning environments include formal learning, mnemonics, flashcards, use of a dictionary or thesaurus, all leading to practice with new words in context. In this work, we propose an enhancement to the language learning process by providing the user with words and learning tools in context, with VocabulARy. VocabulARy visually annotates objects in AR, in the user's surroundings, with the corresponding English (first language) and Japanese (second language) words to enhance the language learning process. In addition to the written and audio description of each word, we also present the user with a keyword and its visualisation to enhance memory retention. We evaluate our prototype by comparing it to an alternate AR system that does not show an additional visualisation of the keyword, and, also, we compare it to two non-AR systems on a tablet, one with and one without visualising the keyword. Our results indicate that AR outperforms the tablet system regarding immediate recall, mental effort and task completion time. Additionally, the visualisation approach scored significantly higher than showing only the written keyword with respect to immediate and delayed recall and learning efficiency, mental effort and task-completion time.

cs.HC

Quantifying the Effects of Working in VR for One Week

Virtual Reality (VR) provides new possibilities for modern knowledge work. However, the potential advantages of virtual work environments can only be used if it is feasible to work in them for an extended period of time. Until now, there are limited studies of long-term effects when working in VR. This paper addresses the need for understanding such long-term effects. Specifically, we report on a comparative study (n=16), in which participants were working in VR for an entire week -- for five days, eight hours each day -- as well as in a baseline physical desktop environment. This study aims to quantify the effects of exchanging a desktop-based work environment with a VR-based environment. Hence, during this study, we do not present the participants with the best possible VR system but rather a setup delivering a comparable experience to working in the physical desktop environment. The study reveals that, as expected, VR results in significantly worse ratings across most measures. Among other results, we found concerning levels of simulator sickness, below average usability ratings and two participants dropped out on the first day using VR, due to migraine, nausea and anxiety. Nevertheless, there is some indication that participants gradually overcame negative first impressions and initial discomfort. Overall, this study helps lay the groundwork for subsequent research, by clearly highlighting current shortcomings and identifying opportunities for improving the experience of working in VR.

cs.HC

The Vision of a Human-Centered Piano

For around 300 years, humans have been learning to play the modern piano either with a teacher or on their own. In recent years teaching and learning have been enhanced using augmented technologies that support novices. Other technologies have also tried to improve different use cases with the piano, such as composing and performing. Researchers and practitioners have showcased several forms of augmentation, from hardware improvements, sound quality, rendering projected visualizations to gesture-based and immersive technologies. Today, the landscape of piano augmentations is very diverse, and it is unclear how to describe the ideal piano and its features. In this work, we discuss how the human-centered piano -- the piano that has been designed with humans in the center of the design process and that effectively supports tasks performed on it -- can support pianists. In detail, we present the three tasks of learning, composing, and improvising in which a human-centered piano would be beneficial for the pianist.

cs.HC

PoVRPoint: Authoring Presentations in Mobile Virtual Reality

Virtual Reality (VR) has the potential to support mobile knowledge workers by complementing traditional input devices with a large three-dimensional output space and spatial input. Previous research on supporting VR knowledge work explored domains such as text entry using physical keyboards and spreadsheet interaction using combined pen and touch input. Inspired by such work, this paper probes the VR design space for authoring presentations in mobile settings. We propose PoVRPoint -- a set of tools coupling pen- and touch-based editing of presentations on mobile devices, such as tablets, with the interaction capabilities afforded by VR. We study the utility of extended display space to, for example, assist users in identifying target slides, supporting spatial manipulation of objects on a slide, creating animations, and facilitating arrangements of multiple, possibly occluded, shapes. Among other things, our results indicate that 1) the wide field of view afforded by VR results in significantly faster target slide identification times compared to a tablet-only interface for visually salient targets; and 2) the three-dimensional view in VR enables significantly faster object reordering in the presence of occlusion compared to two baseline interfaces. A user study further confirmed that the interaction techniques were found to be usable and enjoyable.

cs.HC

Accuracy Evaluation of Touch Tasks in Commodity Virtual and Augmented Reality Head-Mounted Displays

An increasing number of consumer-oriented head-mounted displays (HMD) for augmented and virtual reality (AR/VR) are capable of finger and hand tracking. We report on the accuracy of off-the-shelf VR and AR HMDs when used for touch-based tasks such as pointing or drawing. Specifically, we report on the finger tracking accuracy of the VR head-mounted displays Oculus Quest, Vive Pro and the Leap Motion controller, when attached to a VR HMD, as well as the finger tracking accuracy of the AR head-mounted displays Microsoft HoloLens 2 and Magic Leap. We present the results of two experiments in which we compare the accuracy for absolute and relative pointing tasks using both human participants and a robot. The results suggest that HTC Vive has a lower spatial accuracy than the Oculus Quest and Leap Motion and that the Microsoft HoloLens 2 provides higher spatial accuracy than Magic Leap One. These findings can serve as decision support for researchers and practitioners in choosing which systems to use in the future.

cs.HC