arXiv subjects

Tommy Nilsson

Publications and source records attributed to Tommy Nilsson.

At least 19 recordsLinked to original sources

arXiv2024

Lunar Subterra: a Self-Integrative Unit with an Automated Drilling System

As humans venture deeper into space, the need for a lunar settlement, housing the first group of settlers, grows steadily. By means of new technologies such as in situ resource utilisation (ISRU) as well as computational design, this goal can be implemented in present years. Providing the first arrivals with an immediate underground habitat safe from radiation and other environmental constraints is of crucial importance to initialise a prolonged mission on the Moon. The project's proposal revolves around the idea of establishing a base which provides an immediately habitable space with the possibility for future expansion. Advanced construction methods and sustainable practices lay the groundwork for a permanent human presence, predominantly based on ISRU. This paper outlines a two-phase initiative aimed at the foundation of the Lunar Subterra, followed by an extension of the habitat above ground. Following our collaboration with the PoliSpace Sparc Student Association group, a Virtual Reality (VR) reproduction of the proposed habitat enabled quick iterative testing of the habitable space with the use of a Meta Quest 2 headset. This not only allowed an evaluation of the environment and its impact on human residents but also eradicated the need for tangible models to conceptualise the idea, enabling rapid user-centred design and implementation in the future of space exploration.

cs.HC↗

arXiv2024

Towards a Reliable Offline Personal AI Assistant for Long Duration Spaceflight

As humanity prepares for new missions to the Moon and Mars, astronauts will need to operate with greater autonomy, given the communication delays that make real-time support from Earth difficult. For instance, messages between Mars and Earth can take up to 24 minutes, making quick responses impossible. This limitation poses a challenge for astronauts who must rely on in-situ tools to access the large volume of data from spacecraft sensors, rovers, and satellites, data that is often fragmented and difficult to use. To bridge this gap, systems like the Mars Exploration Telemetry-Driven Information System (METIS) are being developed. METIS is an AI assistant designed to handle routine tasks, monitor spacecraft systems, and detect anomalies, all while reducing the reliance on mission control. Current Generative Pretrained Transformer (GPT) Models, while powerful, struggle in safety-critical environments. They can generate plausible but incorrect responses, a phenomenon known as "hallucination," which could endanger astronauts. To overcome these limitations, this paper proposes enhancing systems like METIS by integrating GPTs, Retrieval-Augmented Generation (RAG), Knowledge Graphs (KGs), and Augmented Reality (AR). The idea is to allow astronauts to interact with their data more intuitively, using natural language queries and visualizing real-time information through AR. KGs will be used to easily access live telemetry and multimodal data, ensuring that astronauts have the right information at the right time. By combining AI, KGs, and AR, this new system will empower astronauts to work more autonomously, safely, and efficiently during future space missions.

cs.AI↗

arXiv2024

Testing and validation of innovative eXtended Reality technologies for astronaut training in a partial-gravity parabolic flight campaign

The use of eXtended Reality (XR) technologies in the space domain has increased significantly over the past few years as it can offer many advantages when simulating complex and challenging environments. Space agencies are currently using these disruptive tools to train astronauts for Extravehicular Activities (EVAs), to test equipment and procedures, and to assess spacecraft and hardware designs. With the Moon being the current focus of the next generation of space exploration missions, simulating its harsh environment is one of the key areas where XR can be applied, particularly for astronaut training. Peculiar lunar lighting conditions in combination with reduced gravity levels will highly impact human locomotion especially for movements such as walking, jumping, and running. In order to execute operations on the lunar surface and to safely live on the Moon for an extended period of time, innovative training methodologies and tools such as XR are becoming paramount to perform pre-mission validation and certification. This research work presents the findings of the experiments aimed at exploring the integration of XR technology and parabolic flight activities for astronaut training. In addition, the study aims to consolidate these findings into a set of guidelines that can assist future researchers who wish to incorporate XR technology into lunar training and preparation activities, including the use of such XR tools during long duration missions.

cs.HC↗

arXiv2024

AI Assistants for Spaceflight Procedures: Combining Generative Pre-Trained Transformer and Retrieval-Augmented Generation on Knowledge Graphs With Augmented Reality Cues

This paper describes the capabilities and potential of the intelligent personal assistant (IPA) CORE (Checklist Organizer for Research and Exploration), designed to support astronauts during procedures onboard the International Space Station (ISS), the Lunar Gateway station, and beyond. We reflect on the importance of a reliable and flexible assistant capable of offline operation and highlight the usefulness of audiovisual interaction using augmented reality elements to intuitively display checklist information. We argue that current approaches to the design of IPAs in space operations fall short of meeting these criteria. Therefore, we propose CORE as an assistant that combines Knowledge Graphs (KGs), Retrieval-Augmented Generation (RAG) for a Generative Pre-Trained Transformer (GPT), and Augmented Reality (AR) elements to ensure an intuitive understanding of procedure steps, reliability, offline availability, and flexibility in terms of response style and procedure updates.

cs.AI↗

arXiv2024

Applied User Research in Virtual Reality: Tools, Methods, and Challenges

This chapter explores the practice of conducting user research studies and design assessments in virtual reality (VR). An overview of key VR hardware and software tools is provided, including game engines, such as Unity and Unreal Engine. Qualitative and quantitative research methods, along with their various synergies with VR, are likewise discussed, and some of the challenges associated with VR, such as limited sensory stimulation, are reflected upon. VR is proving particularly useful in the context of space systems development, where its utilisation offers a cost-effective and secure method for simulating extraterrestrial environments, allowing for rapid prototyping and evaluation of innovative concepts under representative operational conditions. To illustrate this, we present a case study detailing the application of VR to aid aerospace engineers testing their ideas with end-users and stakeholders during early design stages of the European Space Agency's (ESA) prospective Argonaut lunar lander. This case study demonstrates the effectiveness of VR simulations in gathering important feedback concerning the operability of the Argonaut lander in poor lighting conditions as well as surfacing relevant ergonomics considerations and constraints. The chapter concludes by discussing the strengths and weaknesses associated with VR-based user studies and proposes future research directions, emphasising the necessity for novel VR interfaces to overcome existing technical limitations.

cs.HC↗

arXiv2024

Touching the Moon: Leveraging Passive Haptics, Embodiment and Presence for Operational Assessments in Virtual Reality

Space agencies are in the process of drawing up carefully thought-out Concepts of Operations (ConOps) for future human missions on the Moon. These are typically assessed and validated through costly and logistically demanding analogue field studies. While interactive simulations in Virtual Reality (VR) offer a comparatively cost-effective alternative, they have faced criticism for lacking the fidelity of real-world deployments. This paper explores the applicability of passive haptic interfaces in bridging the gap between simulated and real-world ConOps assessments. Leveraging passive haptic props (equipment mockup and astronaut gloves), we virtually recreated the Apollo 12 mission procedure and assessed it with experienced astronauts and other space experts. Quantitative and qualitative findings indicate that haptics increased presence and embodiment, thus improving perceived simulation fidelity and validity of user reflections. We conclude by discussing the potential role of passive haptic modalities in facilitating early-stage ConOps assessments for human endeavours on the Moon and beyond.

cs.HC↗

arXiv2024

Designing for Human Operations on the Moon: Challenges and Opportunities of Navigational HUD Interfaces

Future crewed missions to the Moon will face significant environmental and operational challenges, posing risks to the safety and performance of astronauts navigating its inhospitable surface. Whilst head-up displays (HUDs) have proven effective in providing intuitive navigational support on Earth, the design of novel human-spaceflight solutions typically relies on costly and time-consuming analogue deployments, leaving the potential use of lunar HUDs largely under-explored. This paper explores an alternative approach by simulating navigational HUD concepts in a high-fidelity Virtual Reality (VR) representation of the lunar environment. In evaluating these concepts with astronauts and other aerospace experts (n=25), our mixed methods study demonstrates the efficacy of simulated analogues in facilitating rapid design assessments of early-stage HUD solutions. We illustrate this by elaborating key design challenges and guidelines for future lunar HUDs. In reflecting on the limitations of our approach, we propose directions for future design exploration of human-machine interfaces for the Moon.

cs.HC↗

arXiv2023

Well-being in isolation: Exploring artistic immersive virtual environments in a simulated lunar habitat to alleviate asthenia symptoms

Revived interest in lunar and planetary exploration is heralding a new era for human spaceflight, characterized by frequent strain on astronaut's mental well-being, which stems from increased exposure to isolated, confined, and extreme (ICE) conditions. Whilst Immersive Virtual Reality (IVR) has been employed to facilitate self-help interventions to mitigate challenges caused by isolated environments in several domains, its applicability in support of future space expeditions remains largely unexplored. To address this limitation, we administered the use of distinct IVR environments to crew members (n=5) partaking in a simulated lunar habitat study. Utilizing a Bayesian approach to scrutinize small group data, we discovered a significant relationship between IVR usage and a reduction in perceived stress-related symptoms, particularly those associated with asthenia (syndrome often linked to chronic fatigue and weakness; a condition characterized by feelings of energy depletion or exhaustion that can be amplified in ICE conditions). The reductions were most prominent with the use of interactive virtual environments. The 'Aesthetic Realities' - virtual environments conceived as art exhibits - received exceptional praise from our participants. These environments mark a fascinating convergence of art and science, holding promise to mitigate effects related to isolation in spaceflight training and beyond.

cs.HC↗

arXiv2023

Augmented Reality in Service of Human Operations on the Moon: Insights from a Virtual Testbed

Future astronauts living and working on the Moon will face extreme environmental conditions impeding their operational safety and performance. While it has been suggested that Augmented Reality (AR) Head-Up Displays (HUDs) could potentially help mitigate some of these adversities, the applicability of AR in the unique lunar context remains underexplored. To address this limitation, we have produced an accurate representation of the lunar setting in virtual reality (VR) which then formed our testbed for the exploration of prospective operational scenarios with aerospace experts. Herein we present findings based on qualitative reflections made by the first 6 study participants. AR was found instrumental in several use cases, including the support of navigation and risk awareness. Major design challenges were likewise identified, including the importance of redundancy and contextual appropriateness. Drawing on these findings, we conclude by outlining directions for future research aimed at developing AR-based assistive solutions tailored to the lunar setting.

cs.HC↗

arXiv2023

Using Virtual Reality to Shape Humanity's Return to the Moon: Key Takeaways from a Design Study

Revived interest in lunar exploration is heralding a new generation of design solutions in support of human operations on the Moon. While space system design has traditionally been guided by prototype deployments in analogue studies, the resource-intensive nature of this approach has largely precluded application of proficient user-centered design (UCD) methods from human-computer interaction (HCI). This paper explores possible use of Virtual Reality (VR) to simulate analogue studies in lab settings and thereby bring to bear UCD in this otherwise engineering-dominated field. Drawing on the ongoing development of the European Large Logistics Lander, we have recreated a prospective lunar operational scenario in VR and evaluated it with a group of astronauts and space experts (n=20). Our qualitative findings demonstrate the efficacy of VR in facilitating UCD, enabling efficient contextual inquiries and improving project team coordination. We conclude by proposing future directions to further exploit VR in lunar systems design.

cs.HC↗

arXiv2023

Telerobotic Mars Mission for Lava Tube Exploration and Examination of Life

The general profile and overarching goal of this proposed mission is to pioneer potentially highly beneficial (even vital) and cost-effective techniques for the future human colonization of Mars. Adopting radically new and disruptive solutions untested in the Martian context, our approach is one of high risk and high reward. The real possibility of such a solution failing has prompted us to base our mission architecture around a rover carrying a set of 6 distinct experimental payloads, each capable of operating independently on the others, thus substantially increasing the chances of the mission yielding some valuable findings. At the same time, we sought to exploit available synergies by assembling a combination of payloads that would together form a coherent experimental ecosystem, with each payload providing potential value to the others. Apart from providing such a testbed for evaluation of novel technological solutions, another aim of our proposed mission is to help generate scientific know-how enhancing our understanding of the Red Planet. To this end, our mission takes aim at the Nili-Fossae region, rich in natural resources (and carbonates in particular), past water repositories and signs of volcanic activity. With our proposed experimental payloads, we intend to explore existing lava-tubes, search for signs of past life and assess their potentially valuable geological features for future base building. We will evaluate biomatter in the form of plants and fungi as possible food and base-building materials respectively. Finally, we seek to explore a variety of novel power generation techniques using the Martian atmosphere and gravity. As detailed throughout the remainder of this chapter, this assemblage of experimental payloads, then, constitutes the backbone of our proposed telerobotic mission to Mars.

cs.HC↗

arXiv2022

Categorisation of future applications for Augmented Reality in human lunar exploration

The European Space Agency (ESA) has a clear mission to go forward to the Moon in preparation of human presence on Mars. One of the technologies looked at to increase safety and efficiency of astronauts in this context is Augmented Reality (AR). This technology allows digital visual information to be overlaid onto the user's environment through some type of display or projector. In recent years separate studies have been conducted to test the potential value of AR for astronauts by implementing a few functionalities on an AR display followed by testing in terrestrial analogue environments. One of the groups contributing to these investigations is Spaceship EAC (SSEAC). SSEAC is a group of interns and trainees at the European Astronaut Centre (EAC) focusing on emerging technologies for human space exploration. This paper presents an outcome of SSEAC's activities related to AR for lunar extravehicular activities (EVAs), in which an approach similar to design thinking was used to explore, identify, and structure the opportunities offered by this technology. The resulting categorization of AR use cases can be used to identify new functionalities to test through prototyping and usability tests and can also be used to relate individual studies to each other to gain insight into the overall potential value AR has to offer to human lunar exploration. The approach adopted in this paper is based on the Fuzzy Front End (FFE) model from the innovation management domain. Utilising a user-driven instead of technology-driven method resulted in findings that are relevant irrespective of the hardware and software implementation. Instead, the outcome is an overview of use cases in which some type of AR system could provide value by contributing to increased astronaut safety, efficiency and/or efficacy.

cs.HC↗

arXiv2022

Holistic Outpost Design for Lunar Lava Tubes

As the space industry continues its rapid development, humanity is poised to expand beyond Low Earth Orbit (LEO), seeking to establish permanent presence on the Moon and beyond. While space travel has traditionally been the domain of a small number of highly specialized professionals, a new era of human exploration, involving non-space actors and stakeholders, is now becoming a reality. In spite of this development, most space habitats are still designed for a narrow target group. This paper seeks to address this deficit by rethinking the established design approaches, typically limited to tackling engineering and challenges of human space exploration (such as radiation or hypogravity), by instead adopting an interdisciplinary "big picture" perspective encompassing social, psychological and cultural aspects of future space habitats. By elaborating and reflecting on our concept, this paper seeks to demonstrate the importance of a trans-disciplinary approach to designing thriving sustainable colonies beyond LEO. We demonstrate the potentially key role of design as mediator in advancing macro-strategies promoting thriving existence and sustainable growth. With this approach we tackle big-picture questions about humanity's future and prospects amongst the stars.

cs.HC↗

arXiv2022

Electroencephalography (EEG), electromyography (EMG) and eye-tracking for astronaut training and space exploration

The ongoing push to send humans back to the Moon and to Mars is giving rise to a wide range of novel technical solutions in support of prospective astronaut expeditions. Against this backdrop, the European Space Agency (ESA) has recently launched an investigation into unobtrusive interface technologies as a potential answer to such challenges. Three particular technologies have shown promise in this regard: EEG-based brain-computer interfaces (BCI) provide a non-invasive method of utilizing recorded electrical activity of a user's brain, electromyography (EMG) enables monitoring of electrical signals generated by the user's muscle contractions, and finally, eye tracking enables, for instance, the tracking of user's gaze direction via camera recordings to convey commands. Beyond simply improving the usability of prospective technical solutions, our findings indicate that EMG, EEG, and eye-tracking could also serve to monitor and assess a variety of cognitive states, including attention, cognitive load, and mental fatigue of the user, while EMG could furthermore also be utilized to monitor the physical state of the astronaut. In this paper, we elaborate on the key strengths and challenges of these three enabling technologies, and in light of ESA's latest findings, we reflect on their applicability in the context of human space flight. Furthermore, a timeline of technological readiness is provided. In so doing, this paper feeds into the growing discourse on emerging technology and its role in paving the way for a human return to the Moon and expeditions beyond the Earth's orbit.

cs.HC↗

arXiv2022

Using Virtual Reality to Design and Evaluate a Lunar Lander: The EL3 Case Study

The European Large Logistics Lander (EL3) is being designed to carry out cargo delivery missions in support of future lunar ground crews. The capacity of virtual reality (VR) to visualize and interactively simulate the unique lunar environment makes it a potentially powerful design tool during the early development stages of such solutions. Based on input from the EL3 development team, we have produced a VR-based operational scenario featuring a hypothetical configuration of the lander. Relying on HCI research methods, we have subsequently evaluated this scenario with relevant experts (n=10). Qualitative findings from this initial pilot study have demonstrated the usefulness of VR as a design tool in this context, but likewise surfaced a number of limitations in the form of potentially impaired validity and generalizability. We conclude by outlining our future research plan and reflect on the potential use of physical stimuli to improve the validity of VR-based simulations in forthcoming design activities.

cs.HC↗

arXiv2020

Visions, Values, and Videos: Revisiting Envisionings in Service of UbiComp Design for the Home

UbiComp has been envisioned to bring about a future dominated by calm computing technologies making our everyday lives ever more convenient. Yet the same vision has also attracted criticism for encouraging a solitary and passive lifestyle. The aim of this paper is to explore and elaborate these tensions further by examining the human values surrounding future domestic UbiComp solutions. Drawing on envisioning and contravisioning, we probe members of the public (N=28) through the presentation and focus group discussion of two contrasting animated video scenarios, where one is inspired by "calm" and the other by "engaging" visions of future UbiComp technology. By analysing the reasoning of our participants, we identify and elaborate a number of relevant values involved in balancing the two perspectives. In conclusion, we articulate practically applicable takeaways in the form of a set of key design questions and challenges.

cs.HC↗

arXiv2019

Immersive virtual worlds: Multi-sensory virtual environments for health and safety training

Virtual environments (VEs) offer potential benefits to health and safety training: exposure to dangerous (virtual) environments; the opportunity for experiential learning; and a high level of control over the training, in that aspects can be repeated or reviewed based on the trainee's performance. However, VEs are typically presented as audiovisual (AV) systems, whereas engagement of other senses could increase the immersion in the virtual experience. Moreover, other senses play a key role in certain health and safety contexts, for example the feel of heat and smell in a fire or smell in a fuel leak. A multisensory (MS) VE was developed, which provided simulated heat and smell in accordance with events in a virtual world. As users approached a virtual fire, they felt heat from three 2 kW heaters and smelled smoke from a scent diffuser. Behaviours in the MS VE demonstrated higher validity than those in a comparable AV VE, which ratings and verbatim responses indicated was down to a greater belief that participants were in a real fire. However, a study of the effectiveness of the MS VE as a training tool demonstrated that it did not offer benefits over AV as measured by a written knowledge test and subjective ratings of engagement, attitude towards health and safety and desire to repeat. However, the study found further evidence for the use of AV VEs in health and safety training, particularly as the subjective ratings were generally better than for PowerPoint based training. Despite the lack of evidence for MS simulation on traditional measures of training, the different attitudes and experiences of users suggest that it may have value as a system for changing trainees' attitudes towards their personal safety and awareness. This view was supported by feedback from industrial partners.

cs.HC↗

arXiv2019

Breaching the Future: Understanding Human Challenges of Autonomous Systems for the Home

The domestic environment is a key area for the design and deployment of autonomous systems. Yet research indicates their adoption is already being hampered by a variety of critical issues including trust, privacy and security. This paper explores how potential users relate to the concept of autonomous systems in the home and elaborates further points of friction. It makes two contributions. One methodological, focusing on the use of provocative utopian and dystopian scenarios of future autonomous systems in the home. These are used to drive an innovative workshop-based approach to breaching experiments, which surfaces the usually tacit and unspoken background expectancies implicated in the organisation of everyday life that have a powerful impact on the acceptability of future and emerging technologies. The other contribution is substantive, produced through participants efforts to repair the incongruity or "reality disjuncture" created by utopian and dystopian visions, and highlights the need to build social as well as computational accountability into autonomous systems, and to enable coordination and control.

cs.HC↗