arXiv subjects

Andry Rakotonirainy

Publications and source records attributed to Andry Rakotonirainy.

17 recordsLinked to original sources

arXiv2026

NARRATE: A Multimodal Real-World Australian Driving Dataset for Human-Centred Explanations in Automated Driving

Automated vehicles must explain their decisions in ways that passengers can understand, monitor, and trust. Existing language-annotated driving datasets are mostly observer-written, post-hoc, simulation-based, or generated from sensor inputs, rather than elicited from the driver performing the action. We introduce NARRATE, a multimodal real-world Australian driving dataset comprising 2,050 annotated events from 35 experienced drivers and driving instructors on public roads. Each event is grounded in synchronised visual, localisation, motion, and LiDAR streams and paired with in-vehicle and/or post-drive free-text explanations. NARRATE provides action labels, scenario-context labels spanning six high-level and 32 fine-grained categories, and span-level Situational Awareness (SA) annotations over driver explanations for Perception, Comprehension and Projection. Four benchmark tasks (SA, scenario-context, driver-action classification, and explanation generation) show that this structure is learnable from driver language, while fine-grained context recognition and explanation generation remain challenging. NARRATE paves a path towards more human-centred and domain-aware explanation models for automated driving.

cs.CV↗

arXiv2026

Small Talk, Big Impact? LLM-based Conversational Agents to Mitigate Passive Fatigue in Conditional Automated Driving

Passive fatigue during conditional automated driving can compromise driver readiness and safety. This paper presents findings from a test-track study with 40 participants in a real-world automated driving scenario. In this scenario, a Large Language Model (LLM) based conversational agent (CA) was designed to check in with drivers and re-engage them with their surroundings. Drawing on in-car video recordings, sleepiness ratings and interviews, we analysed how drivers interacted with the agent and how these interactions shaped alertness. Results show the CA is helpful for supporting vigilance during passive fatigue. Thematic analysis of acceptability further revealed three user preference profiles that implicate future intention to use CAs. Positioning empirically observed profiles within existing CA archetype frameworks highlights the need for adaptive design sensitive to diverse user groups. This work underscores the potential of CAs as proactive Human-Machine Interface (HMI) interventions, demonstrating how natural language can support context-aware interaction during automated driving.

cs.HC↗

arXiv2026

X-Blocks: Linguistic Building Blocks of Natural Language Explanations for Automated Vehicles

Natural language explanations play a critical role in establishing trust and acceptance of automated vehicles (AVs), yet existing approaches lack systematic frameworks for analysing how humans linguistically construct driving rationales across diverse scenarios. This paper introduces X-Blocks (eXplanation Blocks), a hierarchical analytical framework that identifies the linguistic building blocks of natural language explanations for AVs at three levels: context, syntax, and lexicon. At the context level, we propose RACE (Reasoning-Aligned Classification of Explanations), a multi-LLM ensemble framework that combines Chain-of-Thought reasoning with self-consistency mechanisms to robustly classify explanations into 32 scenario-aware categories. Applied to human-authored explanations from the Berkeley DeepDrive-X dataset, RACE achieves 91.45 percent accuracy and a Cohens kappa of 0.91 against cases with human annotator agreement, indicating near-human reliability for context classification. At the lexical level, log-odds analysis with informative Dirichlet priors reveals context-specific vocabulary patterns that distinguish driving scenarios. At the syntactic level, dependency parsing and template extraction show that explanations draw from a limited repertoire of reusable grammar families, with systematic variation in predicate types and causal constructions across contexts. The X-Blocks framework is dataset-agnostic and task-independent, offering broad applicability to other automated driving datasets and safety-critical domains. Overall, our findings provide evidence-based linguistic design principles for generating scenario-aware explanations that support transparency, user trust, and cognitive accessibility in automated driving systems.

cs.AI↗

arXiv2025

Zero-Shot Scene Understanding with Multimodal Large Language Models for Automated Vehicles

Scene understanding is critical for various downstream tasks in autonomous driving, including facilitating driver-agent communication and enhancing human-centered explainability of autonomous vehicle (AV) decisions. This paper evaluates the capability of four multimodal large language models (MLLMs), including relatively small models, to understand scenes in a zero-shot, in-context learning setting. Additionally, we explore whether combining these models using an ensemble approach with majority voting can enhance scene understanding performance. Our experiments demonstrate that GPT-4o, the largest model, outperforms the others in scene understanding. However, the performance gap between GPT-4o and the smaller models is relatively modest, suggesting that advanced techniques such as improved in-context learning, retrieval-augmented generation (RAG), or fine-tuning could further optimize the smaller models' performance. We also observe mixed results with the ensemble approach: while some scene attributes show improvement in performance metrics such as F1-score, others experience a decline. These findings highlight the need for more sophisticated ensemble techniques to achieve consistent gains across all scene attributes. This study underscores the potential of leveraging MLLMs for scene understanding and provides insights into optimizing their performance for autonomous driving applications.

cs.CV↗

arXiv2025

Vision-Language Models for Autonomous Driving: CLIP-Based Dynamic Scene Understanding

Scene understanding is essential for enhancing driver safety, generating human-centric explanations for Automated Vehicle (AV) decisions, and leveraging Artificial Intelligence (AI) for retrospective driving video analysis. This study developed a dynamic scene retrieval system using Contrastive Language-Image Pretraining (CLIP) models, which can be optimized for real-time deployment on edge devices. The proposed system outperforms state-of-the-art in-context learning methods, including the zero-shot capabilities of GPT-4o, particularly in complex scenarios. By conducting frame-level analysis on the Honda Scenes Dataset, which contains a collection of about 80 hours of annotated driving videos capturing diverse real-world road and weather conditions, our study highlights the robustness of CLIP models in learning visual concepts from natural language supervision. Results also showed that fine-tuning the CLIP models, such as ViT-L/14 and ViT-B/32, significantly improved scene classification, achieving a top F1 score of 91.1%. These results demonstrate the ability of the system to deliver rapid and precise scene recognition, which can be used to meet the critical requirements of Advanced Driver Assistance Systems (ADAS). This study shows the potential of CLIP models to provide scalable and efficient frameworks for dynamic scene understanding and classification. Furthermore, this work lays the groundwork for advanced autonomous vehicle technologies by fostering a deeper understanding of driver behavior, road conditions, and safety-critical scenarios, marking a significant step toward smarter, safer, and more context-aware autonomous driving systems.

cs.CV↗

arXiv2024

Visual Reasoning and Multi-Agent Approach in Multimodal Large Language Models (MLLMs): Solving TSP and mTSP Combinatorial Challenges

Multimodal Large Language Models (MLLMs) harness comprehensive knowledge spanning text, images, and audio to adeptly tackle complex problems, including zero-shot in-context learning scenarios. This study explores the ability of MLLMs in visually solving the Traveling Salesman Problem (TSP) and Multiple Traveling Salesman Problem (mTSP) using images that portray point distributions on a two-dimensional plane. We introduce a novel approach employing multiple specialized agents within the MLLM framework, each dedicated to optimizing solutions for these combinatorial challenges. Our experimental investigation includes rigorous evaluations across zero-shot settings and introduces innovative multi-agent zero-shot in-context scenarios. The results demonstrated that both multi-agent models. Multi-Agent 1, which includes the Initializer, Critic, and Scorer agents, and Multi-Agent 2, which comprises only the Initializer and Critic agents; significantly improved solution quality for TSP and mTSP problems. Multi-Agent 1 excelled in environments requiring detailed route refinement and evaluation, providing a robust framework for sophisticated optimizations. In contrast, Multi-Agent 2, focusing on iterative refinements by the Initializer and Critic, proved effective for rapid decision-making scenarios. These experiments yield promising outcomes, showcasing the robust visual reasoning capabilities of MLLMs in addressing diverse combinatorial problems. The findings underscore the potential of MLLMs as powerful tools in computational optimization, offering insights that could inspire further advancements in this promising field. Project link: https://github.com/ahmed-abdulhuy/Solving-TSP-and-mTSP-Combinatorial-Challenges-using-Visual-Reasoning-and-Multi-Agent-Approach-MLLMs-.git

cs.AI↗

arXiv2024

Eyeballing Combinatorial Problems: A Case Study of Using Multimodal Large Language Models to Solve Traveling Salesman Problems

Multimodal Large Language Models (MLLMs) have demonstrated proficiency in processing di-verse modalities, including text, images, and audio. These models leverage extensive pre-existing knowledge, enabling them to address complex problems with minimal to no specific training examples, as evidenced in few-shot and zero-shot in-context learning scenarios. This paper investigates the use of MLLMs' visual capabilities to 'eyeball' solutions for the Traveling Salesman Problem (TSP) by analyzing images of point distributions on a two-dimensional plane. Our experiments aimed to validate the hypothesis that MLLMs can effectively 'eyeball' viable TSP routes. The results from zero-shot, few-shot, self-ensemble, and self-refine zero-shot evaluations show promising outcomes. We anticipate that these findings will inspire further exploration into MLLMs' visual reasoning abilities to tackle other combinatorial problems.

cs.AI↗

arXiv2024

Impact of Connected and Automated Vehicles on Transport Injustices

Connected and automated vehicles are poised to transform the transport system. However, significant uncertainties remain about their impact, particularly regarding concerns that this advanced technology might exacerbate injustices, such as safety disparities for vulnerable road users. Therefore, understanding the potential conflicts of this technology with societal values such as justice and safety is crucial for responsible implementation. To date, no research has focused on what safety and justice in transport mean in the context of CAV deployment and how the potential benefits of CAVs can be harnessed without exacerbating the existing vulnerabilities and injustices VRUs face. This paper addresses this gap by exploring car drivers' and pedestrians' perceptions of safety and justice issues that CAVs might exacerbate using an existing theoretical framework. Employing a qualitative approach, the study delves into the nuanced aspects of these concepts. Interviews were conducted with 30 participants aged between 18 and 79 in Queensland, Australia. These interviews were recorded, transcribed, organised, and analysed using reflexive thematic analysis. Three main themes emerged from the participants' discussions: CAVs as a safety problem for VRUs, CAVs as a justice problem for VRUs, and CAVs as an alignment with societal values problem. Participants emphasised the safety challenges CAVs pose for VRUs, highlighting the need for thorough evaluation and regulatory oversight. Concerns were also raised about CAVs potentially marginalising vulnerable groups within society. Participants advocated for inclusive discussions and a justice-oriented approach to designing a comprehensive transport system to address these concerns.

cs.HC↗

arXiv2024

An Eye Gaze Heatmap Analysis of Uncertainty Head-Up Display Designs for Conditional Automated Driving

This paper reports results from a high-fidelity driving simulator study (N=215) about a head-up display (HUD) that conveys a conditional automated vehicle's dynamic "uncertainty" about the current situation while fallback drivers watch entertaining videos. We compared (between-group) three design interventions: display (a bar visualisation of uncertainty close to the video), interruption (interrupting the video during uncertain situations), and combination (a combination of both), against a baseline (video-only). We visualised eye-tracking data to conduct a heatmap analysis of the four groups' gaze behaviour over time. We found interruptions initiated a phase during which participants interleaved their attention between monitoring and entertainment. This improved monitoring behaviour was more pronounced in combination compared to interruption, suggesting pre-warning interruptions have positive effects. The same addition had negative effects without interruptions (comparing baseline & display). Intermittent interruptions may have safety benefits over placing additional peripheral displays without compromising usability.

cs.HC↗

arXiv2022

A Review on Drivers Red Light Running Behavior Predictions and Technology Based Countermeasures

Red light running at signalised intersections is a growing road safety issue worldwide, leading to the rapid development of advanced intelligent transportation technologies and countermeasures. However, existing studies have yet to summarise and present the effect of these technology based innovations in improving safety. This paper represents a comprehensive review of red light running behaviour prediction methodologies and technology-based countermeasures. Specifically, the major focus of this study is to provide a comprehensive review on two streams of literature targeting red light running and stop and go behaviour at signalised intersection (1) studies focusing on modelling and predicting the red light running and stop and go related driver behaviour and (2) studies focusing on the effectiveness of different technology based countermeasures which combat such unsafe behaviour. The study provides a systematic guide to assist researchers and stakeholders in understanding how to best identify red light running and stop and go associated driving behaviour and subsequently implement countermeasures to combat such risky behaviour and improve the associated safety.

cs.AI↗

arXiv2021

Hybrid Pointer Networks for Traveling Salesman Problems Optimization

In this work, a novel idea is presented for combinatorial optimization problems, a hybrid network, which results in a superior outcome. We applied this method to graph pointer networks [1], expanding its capabilities to a higher level. We proposed a hybrid pointer network (HPN) to solve the travelling salesman problem trained by reinforcement learning. Furthermore, HPN builds upon graph pointer networks which is an extension of pointer networks with an additional graph embedding layer. HPN outperforms the graph pointer network in solution quality due to the hybrid encoder, which provides our model with a verity encoding type, allowing our model to converge to a better policy. Our network significantly outperforms the original graph pointer network for small and large-scale problems increasing its performance for TSP50 from 5.959 to 5.706 without utilizing 2opt, Pointer networks, Attention model, and a wide range of models, producing results comparable to highly tuned and specialized algorithms. We make our data, models, and code publicly available [2].

cs.LG↗

arXiv2021

ECG-Based Driver Stress Levels Detection System Using Hyperparameter Optimization

Stress and driving are a dangerous combination which can lead to crashes, as evidenced by the large number of road traffic crashes that involve stress. Motivated by the need to address the significant costs of driver stress, it is essential to build a practical system that can classify driver stress level with high accuracy. However, the performance of an accurate driving stress levels classification system depends on hyperparameter optimization choices such as data segmentation (windowing hyperparameters). The configuration setting of hyperparameters, which has an enormous impact on the system performance, are typically hand-tuned while evaluating the algorithm. This tuning process is time consuming and often depends on personal experience. There are also no generic optimal values for hyperparameters values. In this work, we propose a meta-heuristic approach to support automated hyperparameter optimization and provide a real-time driver stress detection system. This is the first systematic study of optimizing windowing hyperparameters based on Electrocardiogram (ECG) signal in the domain of driving safety. Our approach is to propose a framework based on Particle Swarm Optimization algorithm (PSO) to select an optimal/near optimal windowing hyperparameters values. The performance of the proposed framework is evaluated on two datasets: a public dataset (DRIVEDB dataset) and our collected dataset using an advanced simulator. DRIVEDB dataset was collected in a real time driving scenario, and our dataset was collected using an advanced driving simulator in the control environment. We demonstrate that optimising the windowing hyperparameters yields significant improvement in terms of accuracy. The most accurate built model applied to the public dataset and our dataset, based on the selected windowing hyperparameters, achieved 92.12% and 77.78% accuracy, respectively.

eess.SP↗

arXiv2020

Developing a Novel Crowdsourcing Business Model for Micro-Mobility Ride-Sharing Systems: Methodology and Preliminary Results

Micro-mobility ride-sharing is an emerging technology that provides access to the transit system with minimum environmental impacts. Significant research is required to ensure that micro-mobility ride-sharing provides a better fulfilment of user needs. In this study, we propose a novel business model for the micro-mobility ride-sharing system where light vehicles such as electric scooters and electric bikes are crowdsourced. This new model consists of three entities, the suppliers, the customers, and a management party, which is responsible for receiving, renting, booking, and demand matching with offered resources. The proposed model has the potential to allow the suppliers to define the location of their private e-scooter/e-bike and the period of time they are available for rent, match it with a particular demand, and then offer suppliers the opportunity to get their e-scooters/e-bikes rented and returned at the end of the renting period to the same (nearby) location. The management party will need to match the e-scooter/e-bike to a series of renting demands with the last demand as a destination very close to the initial location of the e-scooter/e-bike at the start of the renting period. One potential advantage of the proposed model is that it shifts the charging and maintenance efforts to a crowd of suppliers.

cs.CY↗

arXiv2020

Vulnerable Road User Detection Using Smartphone Sensors and Recurrence Quantification Analysis

With the fast advancements of the Autonomous Vehicle (AV) industry, detection of Vulnerable Road Users (VRUs) using smartphones is critical for safety applications of Cooperative Intelligent Transportation Systems (C-ITSs). This study explores the use of low-power smartphone sensors and the Recurrence Quantification Analysis (RQA) features for this task. These features are computed over a thresholded similarity matrix extracted from nine channels: accelerometer, gyroscope, and rotation vector in each direction (x, y, and z). Given the high-power consumption of GPS, GPS data is excluded. RQA features are added to traditional time domain features to investigate the classification accuracy when using binary, four-class, and five-class Random Forest classifiers. Experimental results show a promising performance when only using RQA features with a resulted accuracy of 98. 34% and a 98. 79% by adding time domain features. Results outperform previous reported accuracy, demonstrating that RQA features have high classifying capability with respect to VRU detection.

cs.CY↗

arXiv2020

A Comparative Analysis of E-Scooter and E-Bike Usage Patterns: Findings from the City of Austin, TX

E-scooter-sharing and e-bike-sharing systems are accommodating and easing the increased traffic in dense cities and are expanding considerably. However, these new micro-mobility transportation modes raise numerous operational and safety concerns. This study analyzes e-scooter and dockless e-bike sharing system user behavior. We investigate how average trip speed change depending on the day of the week and the time of the day. We used a dataset from the city of Austin, TX from December 2018 to May 2019. Our results generally show that the trip average speed for e-bikes ranges between 3.01 and 3.44 m/s, which is higher than that for e-scooters (2.19 to 2.78 m/s). Results also show a similar usage pattern for the average speed of e-bikes and e-scooters throughout the days of the week and a different usage pattern for the average speed of e-bikes and e-scooters over the hours of the day. We found that users tend to ride e-bikes and e-scooters with a slower average speed for recreational purposes compared to when they are ridden for commuting purposes. This study is a building block in this field, which serves as a first of its kind, and sheds the light of significant new understanding of this emerging class of shared-road users.

cs.CY↗

arXiv2019

Topological Stability: a New Algorithm for Selecting The Nearest Neighbors in Non-Linear Dimensionality Reduction Techniques

In the machine learning field, dimensionality reduction is an important task. It mitigates the undesired properties of high-dimensional spaces to facilitate classification, compression, and visualization of high-dimensional data. During the last decade, researchers proposed many new (non-linear) techniques for dimensionality reduction. Most of these techniques are based on the intuition that data lies on or near a complex low-dimensional manifold that is embedded in the high-dimensional space. New techniques for dimensionality reduction aim at identifying and extracting the manifold from the high-dimensional space. Isomap is one of widely-used low-dimensional embedding methods, where geodesic distances on a weighted graph are incorporated with the classical scaling (metric multidimensional scaling). The Isomap chooses the nearest neighbours based on the distance only which causes bridges and topological instability. In this paper, we propose a new algorithm to choose the nearest neighbours to reduce the number of short-circuit errors and hence improves the topological stability. Because at any point on the manifold, that point and its nearest neighbours form a vector subspace and the orthogonal to that subspace is orthogonal to all vectors spans the vector subspace. The prposed algorithmuses the point itself and its two nearest neighbours to find the bases of the subspace and the orthogonal to that subspace which belongs to the orthogonal complementary subspace. The proposed algorithm then adds new points to the two nearest neighbours based on the distance and the angle between each new point and the orthogonal to the subspace. The superior performance of the new algorithm in choosing the nearest neighbours is confirmed through experimental work with several datasets.

cs.LG↗

arXiv2019

Sustainable Business Models: A Review

The concept of the sustainable business model describes the rationale of how an organization creates, delivers, and captures value, in economic, social, cultural, or other contexts, in a sustainable way. The process of sustainable business model construction forms an innovative part of a business strategy. Different industries and businesses have utilized sustainable business models concept to satisfy their economic, environmental, and social goals simultaneously. However, the success, popularity, and progress of sustainable business models in different application domains are not clear. To explore this issue, this research provides a comprehensive review of sustainable business models literature in various application areas. Notable sustainable business models are identified and further classified in fourteen unique categories, and in every category, the progress -- either failure or success -- has been reviewed, and the research gaps are discussed. Taxonomy of the applications includes innovation, management and marketing, entrepreneurship, energy, fashion, healthcare, agri-food, supply chain management, circular economy, developing countries, engineering, construction and real estate, mobility and transportation, and hospitality. The key contribution of this study is that it provides an insight into the state of the art of sustainable business models in the various application areas and future research directions. This paper concludes that popularity and the success rate of sustainable business models in all application domains have been increased along with the increasing use of advanced technologies.

econ.GN↗