Searcharxiv⌕ Search

arXiv subjects

Xinyao Zhang

Publications and source records attributed to Xinyao Zhang.

18 recordsLinked to original sources
arXiv2026

READ More than What You See: Reinforcement Learning for Accurate and Coherent Audio Description Generations

Audio Description aims to generate concise narrations of essential visual content in audio-visual media for blind and low-vision audiences. Existing methods either rely on prompting off-the-shelf multimodal models, which often mismatch AD style, or partially optimize training-based systems with next-token prediction, which under-explores model capacity and biases generation toward generic expressions. We present READ, the first reinforcement-learning (RL) framework for training-based AD generation. READ formulates AD as sequence-level optimization with reference-matching, length, and format rewards, and further introduces a dedicated coherence reward under context-aware supervision to promote narratively coherent descriptions. Experiments on MAD-Eval, CMD-AD, and TV-AD show that READ substantially outperforms prior methods across diverse evaluation metrics. Our results highlight RL as a promising paradigm for accurate and coherent AD generation. Our codes, models, and benchmark results will be publicly available.

cs.CV↗
arXiv2026

Machine Learning Modeling for Real-Time Melt Pool Monitoring in Laser Powder Bed Fusion Additive Manufacturing: A Hybrid Approach

This work investigates the implementation of artificial intelligence and machine learning (AI/ML) for real-time monitoring in laser powder bed fusion (LPBF) additive manufacturing. We developed a binary image classification framework for distinguishing normal and abnormal melt pool images using a balanced dataset of 1,200 images collected from Nickel superalloy 625 on the NIST AMMT platform. The study evaluates accuracy and inference time based on control requirements and hardware limitations of open-architecture LPBF machines. We benchmark three transfer learning architectures (ResNet50, EfficientNetB0, and MobileNetV2) against two Random Forest approaches: one trained on EfficientNetB0 feature embeddings (hybrid) and one trained on raw pixel features (baseline). Images are stratified into 80/20 train-test splits, with a further 90/10 validation split on the training set, and undergo standardized resizing, normalization, and label-preserving data augmentation to emulate realistic process variability. Each model is evaluated using accuracy, precision, recall, F1 score, and area under the receiver operating characteristic curve (AUC), along with training time, inference latency, and CPU & GPU usage to capture deployability constraints relevant to factory-floor monitoring. The hybrid EfficientNetB0-plus-Random Forest approach achieves the best performance on the held-out test set, with an F1 score of 0.9451, accuracy of 0.9458, and AUC of 0.9904, while maintaining sub-millisecond per-image inference (1.15 ms). In contrast, purely deep learning models exhibit significantly higher inference times with lower accuracy. These results demonstrate that combining pre-trained convolutional features with classical ensemble methods provides a robust, computationally efficient route to real-time melt pool anomaly detection in data-limited additive manufacturing environments.

cs.LG↗
arXiv2026

Predictive Repair Management Using a Multi-Head Attention Transformer and Online Learning

Accurate prediction of repair duration is an important challenge in product maintenance due to its implications for resource allocation, customer satisfaction, and operational performance. This study aims to develop a deep learning framework to help fleet repair shops accurately categorize repair time given product historical data. The study uses an automobile repair and maintenance dataset and creates an end-to-end predictive framework by employing a multi-head attention network designed for tabular data. The developed framework combines categorical information, transformed through embeddings and attention mechanisms, with numerical historical data to facilitate integration and learning from diverse data features. A weighted loss function is introduced to overcome class imbalance issues in large datasets. Moreover, an online learning strategy is used for continuous incremental model updates to maintain predictive accuracy in evolving operational environments. Our empirical findings demonstrate that the multi-head attention mechanism extracts meaningful interactions between vehicle identifiers and repair types compared to a feed-forward neural network and a random forest model. Also, combining historical maintenance data with an online learning strategy facilitates real-time adjustments to changing patterns and increases the model's predictive performance on new data. The model is tested on real-world repair data spanning 2013 to 2020 and achieves an accuracy of 78%, with attention weight analyses illustrating feature interactions.

cs.LG↗
arXiv2026

MapNav: A Novel Memory Representation via Annotated Semantic Maps for Vision-and-Language Navigation

Vision-and-language navigation (VLN) is a key task in Embodied AI, requiring agents to navigate diverse and unseen environments while following natural language instructions. Traditional approaches rely heavily on historical observations as spatio-temporal contexts for decision making, leading to significant storage and computational overhead. In this paper, we introduce MapNav, a novel end-to-end VLN model that leverages Annotated Semantic Map (ASM) to replace historical frames. Specifically, our approach constructs a top-down semantic map at the start of each episode and update it at each timestep, allowing for precise object mapping and structured navigation information. Then, we enhance this map with explicit textual labels for key regions, transforming abstract semantics into clear navigation cues and generate our ASM. MapNav agent using the constructed ASM as input, and use the powerful end-to-end capabilities of VLM to empower VLN. Extensive experiments demonstrate that MapNav achieves state-of-the-art (SOTA) performance in both simulated and real-world environments, validating the effectiveness of our method. Moreover, we will release our ASM generation source code and dataset to ensure reproducibility, contributing valuable resources to the field. We believe that our proposed MapNav can be used as a new memory representation method in VLN, paving the way for future research in this field.

cs.RO↗
arXiv2026

Proactive Detection of GUI Defects in Multi-Window Scenarios via Multimodal Reasoning

Multi-window mobile scenarios, such as split-screen and foldable modes, make GUI display defects more likely by forcing applications to adapt to changing window sizes and dynamic layout reflow. Existing detection techniques are limited in two ways: they are largely passive, analyzing screenshots only after problematic states have been reached, and they are mainly designed for conventional full-screen interfaces, making them less effective in multi-window settings.We propose an end-to-end framework for GUI display defect detection in multi-window mobile scenarios. The framework proactively triggers split-screen, foldable, and window-transition states during app exploration, uses Set-of-Mark (SoM) to align screenshots with widget-level interface elements, and leverages multimodal large language models with chain-of-thought prompting to detect, localize, and explain display defects. We also construct a benchmark of GUI display defects using 50 real-world Android applications.Experimental results show that multi-window settings substantially increase the exposure of layout-related defects, with text truncation increasing by 184% compared with conventional full-screen settings. At the application level, our method detects 40 defect-prone apps with a false positive rate of 10.00% and a false negative rate of 11.11%, outperforming OwlEye and YOLO-based baselines. At the fine-grained level, it achieves the best F1 score of 87.2% for widget occlusion detection.

cs.SE↗
arXiv2026

Employing General-Purpose and Biomedical Large Language Models with Advanced Prompt Engineering for Pharmacoepidemiologic Study Design

Background: The potential of large language models (LLMs) to automate and support pharmacoepidemiologic study design is an emerging area of interest, yet their reliability remains insufficiently characterized. General-purpose LLMs often display inaccuracies, while the comparative performance of specialized biomedical LLMs in this domain remains unknown. Methods: This study evaluated general-purpose LLMs (GPT-4o and DeepSeek-R1) versus biomedically fine-tuned LLMs (QuantFactory/Bio-Medical-Llama-3-8B-GGUF and Irathernotsay/qwen2-1.5B-medical_qa-Finetune) using 46 protocols (2018-2024) from the HMA-EMA Catalogue and Sentinel System. Performance was assessed across relevance, logic of justification, and ontology-code agreement across multiple coding systems using Least-to-Most (LTM) and Active Prompting strategies. Results: GPT-4o and DeepSeek-R1 paired with LTM prompting achieved the highest relevance and logic of justification scores, with GPT-4o-LTM reaching a median relevance score of 4 in 8 of 9 questions for HMA-EMA protocols. Biomedical LLMs showed lower relevance overall and frequently generated insufficient justification. All LLMs demonstrated limited proficiency in ontology-code mapping, although LTM provided the most consistent improvements in reasoning stability. Conclusion: Off-the-shelf general-purpose LLMs currently offer superior support for pharmacoepidemiologic design compared to biomedical LLMs. Prompt strategy strongly influenced LLM performance.

cs.CL↗
arXiv2026

Precise Robot Command Understanding Using Grammar-Constrained Large Language Models

Human-robot collaboration in industrial settings requires precise and reliable communication to enhance operational efficiency. While Large Language Models (LLMs) understand general language, they often lack the domain-specific rigidity needed for safe and executable industrial commands. To address this gap, this paper introduces a novel grammar-constrained LLM that integrates a grammar-driven Natural Language Understanding (NLU) system with a fine-tuned LLM, which enables both conversational flexibility and the deterministic precision required in robotics. Our method employs a two-stage process. First, a fine-tuned LLM performs high-level contextual reasoning and parameter inference on natural language inputs. Second, a Structured Language Model (SLM) and a grammar-based canonicalizer constrain the LLM's output, forcing it into a standardized symbolic format composed of valid action frames and command elements. This process guarantees that generated commands are valid and structured in a robot-readable JSON format. A key feature of the proposed model is a validation and feedback loop. A grammar parser validates the output against a predefined list of executable robotic actions. If a command is invalid, the system automatically generates corrective prompts and re-engages the LLM. This iterative self-correction mechanism allows the model to recover from initial interpretation errors to improve system robustness. We evaluate our grammar-constrained hybrid model against two baselines: a fine-tuned API-based LLM and a standalone grammar-driven NLU model. Using the Human Robot Interaction Corpus (HuRIC) dataset, we demonstrate that the hybrid approach achieves superior command validity, which promotes safer and more effective industrial human-robot collaboration.

cs.RO↗
arXiv2026

Evaluating Large and Lightweight Vision Models for Irregular Component Segmentation in E-Waste Disassembly

Precise segmentation of irregular and densely arranged components is essential for robotic disassembly and material recovery in electronic waste (e-waste) recycling. This study evaluates the impact of model architecture and scale on segmentation performance by comparing SAM2, a transformer-based vision model, with the lightweight YOLOv8 network. Both models were trained and tested on a newly collected dataset of 1,456 annotated RGB images of laptop components including logic boards, heat sinks, and fans, captured under varying illumination and orientation conditions. Data augmentation techniques, such as random rotation, flipping, and cropping, were applied to improve model robustness. YOLOv8 achieved higher segmentation accuracy (mAP50 = 98.8%, mAP50-95 = 85%) and stronger boundary precision than SAM2 (mAP50 = 8.4%). SAM2 demonstrated flexibility in representing diverse object structures but often produced overlapping masks and inconsistent contours. These findings show that large pre-trained models require task-specific optimization for industrial applications. The resulting dataset and benchmarking framework provide a foundation for developing scalable vision algorithms for robotic e-waste disassembly and circular manufacturing systems.

cs.CV↗
arXiv2026

SAMA: Factorized Semantic Anchoring and Motion Alignment for Instruction-Guided Video Editing

Current instruction-guided video editing models struggle to simultaneously balance precise semantic modifications with faithful motion preservation. While existing approaches rely on injecting explicit external priors (e.g., VLM features or structural conditions) to mitigate these issues, this reliance severely bottlenecks model robustness and generalization. To overcome this limitation, we present SAMA (factorized Semantic Anchoring and Motion Alignment), a framework that factorizes video editing into semantic anchoring and motion modeling. First, we introduce Semantic Anchoring, which establishes a reliable visual anchor by jointly predicting semantic tokens and video latents at sparse anchor frames, enabling purely instruction-aware structural planning. Second, Motion Alignment pre-trains the same backbone on motion-centric video restoration pretext tasks (cube inpainting, speed perturbation, and tube shuffle), enabling the model to internalize temporal dynamics directly from raw videos. SAMA is optimized with a two-stage pipeline: a factorized pre-training stage that learns inherent semantic-motion representations without paired video-instruction editing data, followed by supervised fine-tuning on paired editing data. Remarkably, the factorized pre-training alone already yields strong zero-shot video editing ability, validating the proposed factorization. SAMA achieves state-of-the-art performance among open-source models and is competitive with leading commercial systems (e.g., Kling-Omni). Code, models, and datasets will be released.

cs.CV↗
arXiv2026

Zariski density of modular points in the Eisenstein case

In this article, we study the Zariski closure of modular points in the two-dimensional universal deformation space when the residual Galois representation is reducible. Unlike the previous approaches in the residually irreducible case from Gouvêa-Mazur, Böckle and Allen, our method relies on local-global compatibility results, potential pro-modularity arguments and a non-ordinary finiteness result between the local deformation ring at $p$ and the global deformation ring. This allows us to construct sufficiently many non-ordinary regular de Rham points whose modularity is guaranteed by the recent progress on the Fontaine-Mazur conjecture. Also, we will discuss some applications of our main results, including the equidimensionality of certain big Hecke algebras and big $R=\mathbb{T}$ theorems in the residually reducible case.

math.NT↗
arXiv2025

On the pro-modularity in the residually reducible case for some totally real fields

In this article, we study the relation between the universal deformation rings and big Hecke algebras in the residually reducible case. Following the strategy of Skinner-Wiles and Pan's proof of the Fontaine-Mazur conjecture, we prove a pro-modularity result. Based on this result, we also give a conditional big $R=\mathbb{T}$ theorem over some totally real fields, which is a generalization of Deo's result.

math.NT↗
arXiv2025

On the Fontaine-Mazur conjecture for $p=3$

In this article, we prove the remaining open cases of the Fontaine-Mazur conjecture on two-dimensional regular Galois representations over $\Gal(\overline{\Q}/\Q)$ when $p=3$, hence concluding the conjecture in the regular case for all odd primes. Our result is a sequel to Pan's work, based on some recent progress on $p$-adic Langlands correspondence, Galois deformation theory and a potential pro-modularity result.

math.NT↗
arXiv2025

Improving the Solution of Indefinite Quadratic Programs and Linear Programs with Complementarity Constraints by a Progressive MIP Method

Indefinite quadratic programs (QPs) are known to be very difficult to be solved to global optimality, so are linear programs with linear complementarity constraints. Treating the former as a subclass of the latter, this paper presents a progressive mixed integer linear programming method for solving a general linear program with linear complementarity constraints (LPCC). Instead of solving the LPCC with a full set of integer variables expressing the complementarity conditions, the presented method solves a finite number of mixed integer subprograms by starting with a small fraction of integer variables and progressively increasing this fraction. After describing the PIP (for progressive integer programming) method and its various implementations, we demonstrate, via an extensive set of computational experiments, the superior performance of the progressive approach over the direct solution of the full-integer formulation of the LPCCs. It is also shown that the solution obtained at the termination of the PIP method is a local minimizer of the LPCC, a property that cannot be claimed by any known non-enumerative method for solving this nonconvex program. In all the experiments, the PIP method is initiated at a feasible solution of the LPCC obtained from a nonlinear programming solver, and with high likelihood, can successfully improve it. Thus, the PIP method can improve a stationary solution of an indefinite QP, something that is not likely to be achievable by a nonlinear programming method. Finally, some analysis is presented that provides a better understanding of the roles of the LPCC suboptimal solutions in the local optimality of the indefinite QP.

math.OC↗
arXiv2024

Multi-Floor Zero-Shot Object Navigation Policy

Object navigation in multi-floor environments presents a formidable challenge in robotics, requiring sophisticated spatial reasoning and adaptive exploration strategies. Traditional approaches have primarily focused on single-floor scenarios, overlooking the complexities introduced by multi-floor structures. To address these challenges, we first propose a Multi-floor Navigation Policy (MFNP) and implement it in Zero-Shot object navigation tasks. Our framework comprises three key components: (i) Multi-floor Navigation Policy, which enables an agent to explore across multiple floors; (ii) Multi-modal Large Language Models (MLLMs) for reasoning in the navigation process; and (iii) Inter-Floor Navigation, ensuring efficient floor transitions. We evaluate MFNP on the Habitat-Matterport 3D (HM3D) and Matterport 3D (MP3D) datasets, both include multi-floor scenes. Our experiment results demonstrate that MFNP significantly outperforms all the existing methods in Zero-Shot object navigation, achieving higher success rates and improved exploration efficiency. Ablation studies further highlight the effectiveness of each component in addressing the unique challenges of multi-floor navigation. Meanwhile, we conducted real-world experiments to evaluate the feasibility of our policy. Upon deployment of MFNP, the Unitree quadruped robot demonstrated successful multi-floor navigation and found the target object in a completely unseen environment. By introducing MFNP, we offer a new paradigm for tackling complex, multi-floor environments in object navigation tasks, opening avenues for future research in visual-based navigation in realistic, multi-floor settings.

cs.RO↗
arXiv2024

RoboGPT: an intelligent agent of making embodied long-term decisions for daily instruction tasks

Robotic agents must master common sense and long-term sequential decisions to solve daily tasks through natural language instruction. The developments in Large Language Models (LLMs) in natural language processing have inspired efforts to use LLMs in complex robot planning. Despite LLMs' great generalization and comprehension of instruction tasks, LLMs-generated task plans sometimes lack feasibility and correctness. To address the problem, we propose a RoboGPT agent\footnote{our code and dataset will be released soon} for making embodied long-term decisions for daily tasks, with two modules: 1) LLMs-based planning with re-plan to break the task into multiple sub-goals; 2) RoboSkill individually designed for sub-goals to learn better navigation and manipulation skills. The LLMs-based planning is enhanced with a new robotic dataset and re-plan, called RoboGPT. The new robotic dataset of 67k daily instruction tasks is gathered for fine-tuning the Llama model and obtaining RoboGPT. RoboGPT planner with strong generalization can plan hundreds of daily instruction tasks. Additionally, a low-computational Re-Plan module is designed to allow plans to flexibly adapt to the environment, thereby addressing the nomenclature diversity challenge. The proposed RoboGPT agent outperforms SOTA methods on the ALFRED daily tasks. Moreover, RoboGPT planner exceeds SOTA LLM-based planners like ChatGPT in task-planning rationality for hundreds of unseen daily tasks, and even other domain tasks, while keeping the large model's original broad application and generality.

cs.RO↗
arXiv2024

Manipulable compact many-body localization and absence of superfluidity in geometrically frustrated systems

Geometric frustration is known to completely damage kinetic processes of some of the orbitals (and their associated quantum coherence) as to produce flat bands in the non-interacting systems. The impact of introducing additional interaction to the system in such frustrated systems is, however, a highly controversial issue. On the one hand, numerical studies on geometrically frustrated systems of hard-core boson (equivalent to a spin-1/2 systems) typically lead to glass or solid phases containing only local many-body coherence, indicating the persistence of the damage in quantum coherence. On the other, there continues to be noticeable claims of development of superfluidity that implies kinetic flow of particles. To resolve this apparent contradiction of great significance, we present a rigorous proof showing that density-density interaction is incapable of defeating the geometric frustration to allow propagation of those immobile particles, let alone sustaining a superfluidity. Instead, the frustrated systems develop many $\textit{compact}$ many-body localized states as "many-body scars" that do not thermalize, making them good candidates for storing $\textit{robust}$ and $\textit{manipulable}$ quantum information.

cond-mat.str-el↗
arXiv2024

Geometric inhibition of superflow in single-layer graphene suggests a staggered-flux superconductivity in bilayer and trilayer graphene

In great contrast to the numerous discoveries of superconductivity in layer-stacked graphene systems, the absence of superconductivity in the simplest and cleanest monolayer graphene remains a big puzzle. Here, through realistic computation of electronic structure, we identify a systematic trend that superconductivity appears to emerge only upon alteration of the low-energy electronic lattice from the underlying honeycomb atomic structure. We then demonstrate that this inhibition can result from from geometric frustration of the bond lattice that disables quantum phase coherence of the order parameter residing on it. In comparison, upon deviating from the honeycomb lattice, relief of geometric frustration allows robust superfluidity with non-trivial spatial structure. For the specific examples of bilayer and trilayer graphene under an external electric field, such bond centered order parameter would develop superfluidity with staggered flux that breaks the time-reversal symmetry. Our study also suggests the possible realization of the long-sought superconductivity in single-layer graphene via the application of uni-directional strain.

cond-mat.supr-con↗
arXiv2023

On the modularity of elliptic curves over the cyclotomic $\mathbb{Z}_p$-extension of some real quadratic fields

The modularity of elliptic curves always intrigues number theorists. Recently, Thorne had proved a marvelous result that for a prime $ p $, every elliptic curve defined over a $ p $-cyclotomic extension of $ \mathbb{Q} $ is modular. The method is to use some automorphy lifting theorems and study non-cusp points on some specific elliptic curves by Iwasawa theory for elliptic curves. Since the modularity of elliptic curves over real quadratic was proved, one may ask whether it is possible to replace $ \mathbb{Q} $ with a real quadratic field $ K $. Following Thorne's idea, we give some assumptions first and prove the modularity of elliptic curves over the $\mathbb{Z}_p$-extension of some real quadratic fields.

math.NT↗