arXiv subjects

Le Mao

Publications and source records attributed to Le Mao.

5 recordsLinked to original sources

arXiv2025

CDE: Concept-Driven Exploration for Reinforcement Learning

Intelligent exploration remains a critical challenge in reinforcement learning (RL), especially in visual control tasks. Unlike low-dimensional state-based RL, visual RL must extract task-relevant structure from raw pixels, making exploration inefficient. We propose Concept-Driven Exploration (CDE), which leverages a pre-trained vision-language model (VLM) to generate object-centric visual concepts from textual task descriptions as weak, potentially noisy supervisory signals. Rather than directly conditioning on these noisy signals, CDE trains a policy to reconstruct the concepts via an auxiliary objective, learning general representations of the concepts and using reconstruction accuracy as an intrinsic reward to guide exploration toward task-relevant objects. Across five challenging simulated visual manipulation tasks, CDE achieves efficient, targeted exploration and remains robust to both synthetic errors and noisy VLM predictions. Finally, we demonstrate real-world transfer by deploying CDE on a Franka arm, attaining an 80\% success rate in a real-world manipulation task.

cs.RO↗

arXiv2023

Multi-Robot Cooperative Socially-Aware Navigation Using Multi-Agent Reinforcement Learning

In public spaces shared with humans, ensuring multi-robot systems navigate without collisions while respecting social norms is challenging, particularly with limited communication. Although current robot social navigation techniques leverage advances in reinforcement learning and deep learning, they frequently overlook robot dynamics in simulations, leading to a simulation-to-reality gap. In this paper, we bridge this gap by presenting a new multi-robot social navigation environment crafted using Dec-POSMDP and multi-agent reinforcement learning. Furthermore, we introduce SAMARL: a novel benchmark for cooperative multi-robot social navigation. SAMARL employs a unique spatial-temporal transformer combined with multi-agent reinforcement learning. This approach effectively captures the complex interactions between robots and humans, thus promoting cooperative tendencies in multi-robot systems. Our extensive experiments reveal that SAMARL outperforms existing baseline and ablation models in our designed environment. Demo videos for this work can be found at: https://sites.google.com/view/samarl

cs.RO↗

arXiv2023

NaviSTAR: Socially Aware Robot Navigation with Hybrid Spatio-Temporal Graph Transformer and Preference Learning

Developing robotic technologies for use in human society requires ensuring the safety of robots' navigation behaviors while adhering to pedestrians' expectations and social norms. However, maintaining real-time communication between robots and pedestrians to avoid collisions can be challenging. To address these challenges, we propose a novel socially-aware navigation benchmark called NaviSTAR, which utilizes a hybrid Spatio-Temporal grAph tRansformer (STAR) to understand interactions in human-rich environments fusing potential crowd multi-modal information. We leverage off-policy reinforcement learning algorithm with preference learning to train a policy and a reward function network with supervisor guidance. Additionally, we design a social score function to evaluate the overall performance of social navigation. To compare, we train and test our algorithm and other state-of-the-art methods in both simulator and real-world scenarios independently. Our results show that NaviSTAR outperforms previous methods with outstanding performance\footnote{The source code and experiment videos of this work are available at: https://sites.google.com/view/san-navistar

cs.RO↗

arXiv2021

Fluid structure interaction: Insights into biomechanical implications of endograft after thoracic endovascular aortic repair

Thoracic endovascular aortic repair (TEVAR) has developed to be the most effective treatment for aortic diseases. This study aims to evaluate the biomechanical implications of the implanted endograft after TEVAR. We present a novel image-based, patient-specific, fluid-structure computational framework. The geometries of blood, endograft, and aortic wall were reconstructed based on clinical images. Patient-specific measurement data was collected to determine the parameters of the three-element Windkessel. We designed three postoperative scenarios with rigid wall assumption, blood-wall interaction, blood-endograft-wall interplay, respectively, where a two-way fluid-structure interaction (FSI) method was applied to predict the deformation of the composite stent-wall. Computational results were validated with Doppler ultrasound data. Results show that the rigid wall assumption fails to predict the waveforms of blood outflow and energy loss (EL). The complete storage and release process of blood flow energy, which consists of four phases is captured by the FSI method. The endograft implantation would weaken the buffer function of the aorta and reduce mean EL by 19.1%. The closed curve area of wall pressure and aortic volume could indicate the EL caused by the interaction between blood flow and wall deformation, which accounts for 68.8% of the total EL. Both the FSI and endograft have a slight effect on wall shear stress-related-indices. The deformability of the composite stent-wall region is remarkably limited by the endograft. Our results highlight the importance of considering the interaction between blood flow, the implanted endograft, and the aortic wall to acquire physiologically accurate hemodynamics in post-TEVAR computational studies and the deformation of the aortic wall is responsible for the major EL of the blood flow.

physics.med-ph↗

arXiv2021

Hemodynamic effects of stent-graft introducer sheath during thoracic endovascular aortic repair

Thoracic endovascular aortic repair (TEVAR) has become the standard treatment of a variety of aortic pathologies. The objective of this study is to evaluate the hemodynamic effects of stent-graft introducer sheath during TEVAR. Three idealized representative diseased aortas of aortic aneurysm, coarctation of the aorta, and aortic dissection were designed. Computational fluid dynamics studies were performed in the above idealized aortic geometries. An introducer sheath routinely used in the clinic was virtually-delivered into diseased aortas. Comparative analysis was carried out to evaluate the hemodynamic effects of the introducer sheath. Results show that the blood flow to the supra-aortic branches would increase above 9% due to the obstruction of the introducer sheath. The region exposed to high endothelial cell activation potential (ECAP) expands in the scenarios of coarctation of the aorta and aortic dissection, which indicates that the probability of thrombus formation may increase during TEVAR. The pressure magnitude in peak systole shows an obvious rise and a similar phenomenon is not observed in early diastole. The blood viscosity in the aortic arch and descending aorta is remarkably altered by the introducer sheath. The uneven viscosity distribution confirms the necessity of using non-Newtonian models and high viscosity region with high ECAP further promotes thrombosis. Our results highlight the hemodynamic effects of stent-graft introducer sheath during TEVAR, which may associate with perioperative complications.

physics.med-ph↗