arXiv subjects

Inseok Hwang

Publications and source records attributed to Inseok Hwang.

At least 19 recordsLinked to original sources

arXiv2026

Reachability-based Temporal Logic Verification for Reliable LLM-guided Human-Autonomy Teaming

We propose a reachability-based framework for reliable LLM-guided human-autonomy teaming (HAT) using signal temporal logic (STL). In the proposed framework, LLM is leveraged as a translator that transfers natural language commands given by a human operator into corresponding STL specifications or vice versa. An STL feasibility filter (SFF) is proposed to check the feasibility of the generated STL. The SFF first decomposes the complex and nested LLM translation into a set of simpler subformulas for parallelization and informative feedback generation. The reachability analysis method is then applied to verify if each subformula is feasible for a target dynamical system: if feasible, perform mission planning, otherwise, reject it. The proposed SFF can identify infeasible subformulas, more than simply providing the boolean verification results for the whole STL, thereby facilitating the feedback generation of LLM to request modification of the command to the human. Consequently, the proposed framework can allow more reliable HAT by enabling safe and informative communication between the human operator and the autonomous agent. Our experiments demonstrate that the proposed framework can successfully filter out infeasible subformulas and generate informative feedback based on such information.

eess.SY↗

arXiv2026

Signal Temporal Logic Verification and Synthesis Using Deep Reachability Analysis and Layered Control Architecture

We propose a signal temporal logic (STL)-based framework that rigorously verifies the feasibility of a mission described in STL and synthesizes control to safely execute it. The proposed framework ensures safe and reliable operation through two phases. First, the proposed framework assesses the feasibility of STL by computing a backward reachable tube (BRT), which captures all states that can satisfy the given STL, regardless of the initial state. The proposed framework accommodates the multiple reach-avoid (MRA) problem to address more general STL specifications and leverages a deep neural network to alleviate the computation burden for reachability analysis, reducing the computation time by about 1000 times compared to a baseline method. We further propose a layered planning and control architecture that combines mixed-integer linear programming (MILP) for global planning with model predictive control (MPC) as a local controller for the verified STL. Consequently, the proposed framework can robustly handle unexpected behavior of obstacles that are not described in the environment information or STL, thereby providing reliable mission performance. Our numerical simulations demonstrate that the proposed framework can successfully compute BRT for a given STL and perform the mission.

eess.SY↗

arXiv2025

Next-Generation LLM for UAV: From Natural Language to Autonomous Flight

With the rapid advancement of Large Language Models (LLMs), their capabilities in various automation domains, particularly Unmanned Aerial Vehicle (UAV) operations, have garnered increasing attention. Current research remains predominantly constrained to small-scale UAV applications, with most studies focusing on isolated components such as path planning for toy drones, while lacking comprehensive investigation of medium- and long-range UAV systems in real-world operational contexts. Larger UAV platforms introduce distinct challenges, including stringent requirements for airport-based take-off and landing procedures, adherence to complex regulatory frameworks, and specialized operational capabilities with elevated mission expectations. This position paper presents the Next-Generation LLM for UAV (NeLV) system -- a comprehensive demonstration and automation roadmap for integrating LLMs into multi-scale UAV operations. The NeLV system processes natural language instructions to orchestrate short-, medium-, and long-range UAV missions through five key technical components: (i) LLM-as-Parser for instruction interpretation, (ii) Route Planner for Points of Interest (POI) determination, (iii) Path Planner for waypoint generation, (iv) Control Platform for executable trajectory implementation, and (v) UAV monitoring. We demonstrate the system's feasibility through three representative use cases spanning different operational scales: multi-UAV patrol, multi-POI delivery, and multi-hop relocation. Beyond the current implementation, we establish a five-level automation taxonomy that charts the evolution from current LLM-as-Parser capabilities (Level 1) to fully autonomous LLM-as-Autopilot systems (Level 5), identifying technical prerequisites and research challenges at each stage.

cs.RO↗

arXiv2025

AutiHero: Engaging Parents in Creating Personalized, Multi-path Social Narratives for Autistic Children

Social narratives help autistic children understand and navigate social situations through stories. To ensure effective practice, however, they often require significant time and effort from parents in customizing the narrative materials and delivering repeated instructions on them. We present AutiHero, a generative AI (GenAI)-based social narrative system, which supports parents to create personalized, multi-path stories targeting specific behavior of their autistic children, while enabling them to explore behavioral choices and causal consequences together in reading. A two-week deployment study with 16 autistic child-parent dyads showed that parents actively created, adapted, and read stories with their children, with increased confidence in everyday behavioral guidance. Our work contributes real-world-contextualized text+image content creation approaches harnessing GenAI, ensuring user-aligned application in sensitive contexts involving autistic children and their parents.

cs.HC↗

arXiv2025

Chameleon: A Surface-Anchored Smartphone AR Prototype with Visually Blended Mobile Display

Augmented reality (AR) is often realized through head-mounted displays, offering immersive but egocentric experiences. While smartphone-based AR is more accessible, it remains limited to handheld, single-user interaction. We introduce Chameleon, a prototype AR system that transforms smartphones into surface-anchored displays for co-located use. When placed flat, the phone creates a transparency illusion and anchors digital content visible to multiple users. Chameleon supports natural repositioning on the surface without external hardware by combining two techniques: (1) Background Acquisition uses opportunistic sensing and language model-assisted pattern generation to blend with surrounding surfaces, and (2) Real-Time Position Tracking augments inertial sensing to maintain spatial stability. This work shows how lightweight sensing can support casual, collaborative AR experiences using existing devices.

cs.HC↗

arXiv2025

A Dynamically Weighted ADMM Framework for Byzantine Resilience

The alternating direction of multipliers method (ADMM) is a popular method to solve distributed consensus optimization utilizing efficient communication among various nodes in the network. However, in the presence of faulty or attacked nodes, even a small perturbation (or sharing false data) during the communication can lead to divergence of the solution. To address this issue, in this work we consider ADMM under the effect of Byzantine threat, where an unknown subset of nodes is subject to Byzantine attacks or faults. We propose Dynamically Weighted ADMM (DW-ADMM), a novel variant of ADMM that uses dynamic weights on the edges of the network, thus promoting resilient distributed optimization. We establish that the proposed method (i) produces a nearly identical solution to conventional ADMM in the error-free case, and (ii) guarantees a bounded solution with respect to the global minimizer, even under Byzantine threat. Finally, we demonstrate the effectiveness of our proposed algorithm using an illustrative numerical simulation.

math.OC↗

arXiv2025

Energy-Constrained Resilient Multi-Robot Coverage Control

The problem of multi-robot coverage control becomes significantly challenging when multiple robots leave the mission space simultaneously to charge their batteries, disrupting the underlying network topology for communication and sensing. To address this, we propose a resilient network design and control approach that allows robots to achieve the desired coverage performance while satisfying energy constraints and maintaining network connectivity throughout the mission. We model the combined motion, energy, and network dynamics of the multirobot systems (MRS) as a hybrid system with three modes, i.e., coverage, return-to-base, and recharge, respectively. We show that ensuring the energy constraints can be transformed into designing appropriate guard conditions for mode transition between each of the three modes. Additionally, we present a systematic procedure to design, maintain, and reconfigure the underlying network topology using an energy-aware bearing rigid network design, enhancing the structural resilience of the MRS even when a subset of robots departs to charge their batteries. Finally, we validate our proposed method using numerical simulations.

cs.RO↗

arXiv2025

Chatperone: An LLM-Based Negotiable Scaffolding System for Mediating Adolescent Mobile Interactions

Adolescents' uncontrolled exposure to digital content can negatively impact their development. Traditional regulatory methods, such as time limits or app restrictions, often take a rigid approach, ignoring adolescents' decision-making abilities. Another issue is the lack of content and services tailored for adolescents. To address this, we propose Chatperone, a concept of a system that provides adaptive scaffolding to support adolescents. Chatperone fosters healthy mobile interactions through three key modules: Perception, Negotiation, and Moderation. This paper outlines these modules' functionalities and discusses considerations for real-world implementation.

cs.HC↗

arXiv2025

SAFE-TAXI: A Hierarchical Multi-UAS Safe Auto-Taxiing Framework with Runtime Safety Assurance and Conflict Resolution

We present a hierarchical safe auto-taxiing framework to enhance the automated ground operations of multiple unmanned aircraft systems (multi-UAS). The auto-taxiing problem becomes particularly challenging due to (i) unknown disturbances, such as crosswind affecting the aircraft dynamics, (ii) taxiway incursions due to unplanned obstacles, and (iii) spatiotemporal conflicts at the intersections between multiple entry points in the taxiway. To address these issues, we propose a hierarchical framework, i.e., SAFE-TAXI, combining centralized spatiotemporal planning with decentralized MPC-CBF-based control to safely navigate the aircraft through the taxiway while avoiding intersection conflicts and unplanned obstacles (e.g., other aircraft or ground vehicles). Our proposed framework decouples the auto-taxiing problem temporally into conflict resolution and motion planning, respectively. Conflict resolution is handled in a centralized manner by computing conflict-aware reference trajectories for each aircraft. In contrast, safety assurance from unplanned obstacles is handled by an MPC-CBF-based controller implemented in a decentralized manner. We demonstrate the effectiveness of our proposed framework through numerical simulations and experimentally validate it using Night Vapor, a small-scale fixed-wing test platform.

cs.RO↗

arXiv2025

On Enhancing Structural Resilience of Multirobot Coverage Control with Bearing Rigidity

The problem of multi-robot coverage control has been widely studied to efficiently coordinate a team of robots to cover a desired area of interest. However, this problem faces significant challenges when some robots are lost or deviate from their desired formation during the mission due to faults or cyberattacks. Since a majority of multi-robot systems (MRSs) rely on communication and relative sensing for their efficient operation, a failure in one robot could result in a cascade of failures in the entire system. In this work, we propose a hierarchical framework for area coverage, combining centralized coordination by leveraging Voronoi partitioning with decentralized reference tracking model predictive control (MPC) for control design. In addition to reference tracking, the decentralized MPC also performs bearing maintenance to enforce a rigid MRS network, thereby enhancing the structural resilience, i.e., the ability to detect and mitigate the effects of localization errors and robot loss during the mission. Furthermore, we show that the resulting control architecture guarantees the recovery of the MRS network in the event of robot loss while maintaining a minimally rigid structure. The effectiveness of the proposed algorithm is validated through numerical simulations.

cs.RO↗

arXiv2024

Bridging the Gap between Expert and Language Models: Concept-guided Chess Commentary Generation and Evaluation

Deep learning-based expert models have reached superhuman performance in decision-making domains such as chess and Go. However, it is under-explored to explain or comment on given decisions although it is important for model explainability and human education. The outputs of expert models are accurate, but yet difficult to interpret for humans. On the other hand, large language models (LLMs) can produce fluent commentary but are prone to hallucinations due to their limited decision-making capabilities. To bridge this gap between expert models and LLMs, we focus on chess commentary as a representative task of explaining complex decision-making processes through language and address both the generation and evaluation of commentary. We introduce Concept-guided Chess Commentary generation (CCC) for producing commentary and GPT-based Chess Commentary Evaluation (GCC-Eval) for assessing it. CCC integrates the decision-making strengths of expert models with the linguistic fluency of LLMs through prioritized, concept-based explanations. GCC-Eval leverages expert knowledge to evaluate chess commentary based on informativeness and linguistic quality. Experimental results, validated by both human judges and GCC-Eval, demonstrate that CCC generates commentary which is accurate, informative, and fluent.

cs.LG↗

arXiv2024

An Algorithm for Distributed Computation of Reachable Sets for Multi-Agent Systems

In this paper, we consider the problem of distributed reachable set computation for multi-agent systems (MASs) interacting over an undirected, stationary graph. A full state-feedback control input for such MASs depends no only on the current agent's state, but also of its neighbors. However, in most MAS applications, the dynamics are obscured by individual agents. This makes reachable set computation, in a fully distributed manner, a challenging problem. We utilize the ideas of polytopic reachable set approximation and generalize it to a MAS setup. We formulate the resulting sub-problems in a fully distributed manner and provide convergence guarantees for the associated computations. The proposed algorithm's convergence is proved for two cases: static MAS graphs, and time-varying graphs under certain restrictions.

eess.SY↗

arXiv2024

Application of Log-Linear Dynamic Inversion Control to a Multi-rotor

This paper presents an approach that employs log-linearization in Lie group theory and the Newton-Euler equations to derive exact linear error dynamics for a multi-rotor model, and applies this model with a novel log-linear dynamic inversion controller to simplify the nonlinear distortion and enhance the robustness of the log-linearized system. In addition, we utilize Linear Matrix Inequalities (LMIs) to bound the tracking error for the log-linearization in the presence of bounded disturbance input and use the exponential map to compute the invariant set of the nonlinear system in the Lie group. We demonstrate the effectiveness of our method via an illustrative example of a multi-rotor system with a reference trajectory, and the result validates the safety guarantees of the tracking error in the presence of bounded disturbance.

eess.SY↗

arXiv2024

Range-based Multi-Robot Integrity Monitoring Against Cyberattacks and Faults: An Anchor-Free Approach

Coordination of multi-robot systems (MRSs) relies on efficient sensing and reliable communication among the robots. However, the sensors and communication channels of these robots are often vulnerable to cyberattacks and faults, which can disrupt their individual behavior and the overall objective of the MRS. In this work, we present a multi-robot integrity monitoring framework that utilizes inter-robot range measurements to (i) detect the presence of cyberattacks or faults affecting the MRS, (ii) identify the affected robot(s), and (iii) reconstruct the resulting localization error of these robot(s). The proposed iterative algorithm leverages sequential convex programming and alternating direction of multipliers method to enable real-time and distributed implementation. Our approach is validated using numerical simulations and demonstrated using PX4-SiTL in Gazebo on an MRS, where certain agents deviate from their desired position due to a GNSS spoofing attack. Furthermore, we demonstrate the scalability and interoperability of our algorithm through mixed-reality experiments by forming a heterogeneous MRS comprising real Crazyflie UAVs and virtual PX4-SiTL UAVs working in tandem.

cs.RO↗

arXiv2024

MIXED-SENSE: A Mixed Reality Sensor Emulation Framework for Test and Evaluation of UAVs Against False Data Injection Attacks

We present a high-fidelity Mixed Reality sensor emulation framework for testing and evaluating the resilience of Unmanned Aerial Vehicles (UAVs) against false data injection (FDI) attacks. The proposed approach can be utilized to assess the impact of FDI attacks, benchmark attack detector performance, and validate the effectiveness of mitigation/reconfiguration strategies in single-UAV and UAV swarm operations. Our Mixed Reality framework leverages high-fidelity simulations of Gazebo and a Motion Capture system to emulate proprioceptive (e.g., GNSS) and exteroceptive (e.g., camera) sensor measurements in real-time. We propose an empirical approach to faithfully recreate signal characteristics such as latency and noise in these measurements. Finally, we illustrate the efficacy of our proposed framework through a Mixed Reality experiment consisting of an emulated GNSS attack on an actual UAV, which (i) demonstrates the impact of false data injection attacks on GNSS measurements and (ii) validates a mitigation strategy utilizing a distributed camera network developed in our previous work. Our open-source implementation is available at \href{https://github.com/CogniPilot/mixed\_sense}{\texttt{https://github.com/CogniPilot/mixed\_sense}}

cs.RO↗

arXiv2024

Human Behavior Modeling via Identification of Task Objective and Variability

Human behavior modeling is important for the design and implementation of human-automation interactive control systems. In this context, human behavior refers to a human's control input to systems. We propose a novel method for human behavior modeling that uses human demonstrations for a given task to infer the unknown task objective and the variability. The task objective represents the human's intent or desire. It can be inferred by the inverse optimal control and improve the understanding of human behavior by providing an explainable objective function behind the given human behavior. Meanwhile, the variability denotes the intrinsic uncertainty in human behavior. It can be described by a Gaussian mixture model and capture the uncertainty in human behavior which cannot be encoded by the task objective. The proposed method can improve the prediction accuracy of human behavior by leveraging both task objective and variability. The proposed method is demonstrated through human-subject experiments using an illustrative quadrotor remote control example.

cs.RO↗

arXiv2024

Multi-Agent Based Transfer Learning for Data-Driven Air Traffic Applications

Research in developing data-driven models for Air Traffic Management (ATM) has gained a tremendous interest in recent years. However, data-driven models are known to have long training time and require large datasets to achieve good performance. To address the two issues, this paper proposes a Multi-Agent Bidirectional Encoder Representations from Transformers (MA-BERT) model that fully considers the multi-agent characteristic of the ATM system and learns air traffic controllers' decisions, and a pre-training and fine-tuning transfer learning framework. By pre-training the MA-BERT on a large dataset from a major airport and then fine-tuning it to other airports and specific air traffic applications, a large amount of the total training time can be saved. In addition, for newly adopted procedures and constructed airports where no historical data is available, this paper shows that the pre-trained MA-BERT can achieve high performance by updating regularly with little data. The proposed transfer learning framework and MA-BERT are tested with the automatic dependent surveillance-broadcast data recorded in 3 airports in South Korea in 2019.

cs.LG↗

arXiv2023

Distributed Error-Identification and Correction using Block-Sparse Optimization

The conventional solutions for fault-detection, identification, and reconstruction (FDIR) require centralized decision-making mechanisms which are typically combinatorial in their nature, necessitating the design of an efficient distributed FDIR mechanism that is suitable for multi-agent applications. To this end, we develop a general framework for efficiently reconstructing a sparse vector being observed over a sensor network via nonlinear measurements. The proposed framework is used to design a distributed multi-agent FDIR algorithm based on a combination of the sequential convex programming (SCP) and the alternating direction method of multipliers (ADMM) optimization approaches. The proposed distributed FDIR algorithm can process a variety of inter-agent measurements (including distances, bearings, relative velocities, and subtended angles between agents) to identify the faulty agents and recover their true states. The effectiveness of the proposed distributed multi-agent FDIR approach is demonstrated by considering a numerical example in which the inter-agent distances are used to identify the faulty agents in a multi-agent configuration, as well as reconstruct their error vectors.

eess.SY↗