arXiv subjects

Sheng Yin

Publications and source records attributed to Sheng Yin.

17 recordsLinked to original sources

arXiv2026

GLoRI: Closed-Loop Whole-Body Tracking with Global-Local Reference Interaction for Humanoid Loco-Manipulation

Humanoid loco-manipulation requires accurate whole-body motion tracking in the world frame for physical interaction. While local references preserve motion structure, they lack explicit constraints on absolute spatial placement, leading to accumulated global errors. Existing globally aware approaches augment teleoperation policies with global observations but do not explicitly integrate global correction with local motion guidance, limiting autonomous tracking accuracy. We present GLoRI, a closed-loop whole-body controller that integrates structured global reference and feedback with local motion guidance. Its GLoRI-Net uses Global-Local Cross Attention(GLCA) to refine local keypoint features with global target and pose-difference features, preserving motion structure while correcting world-frame placement. GLoRI achieves 100% completion and a g-MPJPE of 6.44cm on held-out HuMoTo motions. This accuracy remains robust under direct Isaac Gym-to-MuJoCo transfer without fine-tuning, demonstrating strong generalization. Furthermore, such accuracy and generalization enable autonomous loco-manipulation with a single policy on a real Unitree G1 interacting with diverse unseen objects, extending beyond prior systems that primarily rely on teleoperation or focus on single-object interactions.

cs.RO↗

arXiv2026

LLM-Guided Safety Agent for Edge Robotics with an ISO-Compliant Perception-Compute-Control Architecture

Ensuring functional safety in human-robot interaction is challenging because AI perception is inherently probabilistic, whereas industrial standards require deterministic behavior. We present an LLM-guided safety agent for edge robotics, built on an ISO-compliant low-latency perception-compute-control architecture. Our method translates natural-language safety regulations into executable predicates and deploys them through a redundant heterogeneous edge runtime. For fault-tolerant closed-loop execution under edge constraints, we adopt a symmetric dual-modular redundancy design with parallel independent execution for low-latency perception, computation, and control. We prototype the system on a dual-RK3588 platform and evaluate it in representative human-robot interaction scenarios. The results demonstrate a practical edge implementation path toward ISO 13849 Category 3 and PL d using cost-effective hardware, supporting practical deployment of safety-critical embodied AI.

cs.RO↗

arXiv2025

EmboMatrix: A Scalable Training-Ground for Embodied Decision-Making

Embodied decision-making enables agents to translate high-level goals into executable actions through continuous interactions within the physical world, forming a cornerstone of general-purpose embodied intelligence. Large language models (LLMs), with their general decision-making capabilities, offer a promising path to realize this potential; however, LLMs trained solely on language lack exposure to physical environments, limiting their true embodied understanding. To bridge this gap, we propose the concept of a training ground: a comprehensive infrastructure that provides task and scene simulation, embodied interaction, and feedback signals, offering a one-stop solution for LLM acquire genuine embodied decision-making skills. In this work, we present EmboMatrix, the first training ground of its kind, providing massive and diverse tasks with efficient simulation and precise rewards. EmboMatrix incorporates a series of novel techniques: a multi-agent data engine for large-scale task and scene generation, a distributed heterogeneous-hardware system for scalable simulation, and a multi-level reward architecture for precise supervision. Leveraging EmboMatrix, we cultivate EmboBrain, an LLM whose embodied decision-making abilities emerge from extensive embodied interactions. Experiments show that EmboBrain-7B surpasses the 671B DeepSeek-R1 baseline by 9.5\% on two challenging embodied decision-making benchmarks, demonstrating the power of interactive, environment-grounded learning for building truly intelligent embodied agents.

cs.AI↗

arXiv2025

PolySim: Bridging the Sim-to-Real Gap for Humanoid Control via Multi-Simulator Dynamics Randomization

Humanoid whole-body control (WBC) policies trained in simulation often suffer from the sim-to-real gap, which fundamentally arises from simulator inductive bias, the inherent assumptions and limitations of any single simulator. These biases lead to nontrivial discrepancies both across simulators and between simulation and the real world. To mitigate the effect of simulator inductive bias, the key idea is to train policies jointly across multiple simulators, encouraging the learned controller to capture dynamics that generalize beyond any single simulator's assumptions. We thus introduce PolySim, a WBC training platform that integrates multiple heterogeneous simulators. PolySim can launch parallel environments from different engines simultaneously within a single training run, thereby realizing dynamics-level domain randomization. Theoretically, we show that PolySim yields a tighter upper bound on simulator inductive bias than single-simulator training. In experiments, PolySim substantially reduces motion-tracking error in sim-to-sim evaluations; for example, on MuJoCo, it improves execution success by 52.8 over an IsaacSim baseline. PolySim further enables zero-shot deployment on a real Unitree G1 without additional fine-tuning, showing effective transfer from simulation to the real world. We will release the PolySim code upon acceptance of this work.

cs.RO↗

arXiv2025

Price Aware Power Split Control in Heterogeneous Battery Storage Systems

This paper presents a unified framework for the optimal scheduling of battery dispatch and internal power allocation in Battery energy storage systems (BESS). This novel approach integrates both market-based (price-aware) signals and physical system constraints to simultaneously optimize (1) external energy dispatch and (2) internal heterogeneity management of BESS, enhancing its operational economic value and performance. This work compares both model-based Linear Programming (LP) and model-free Reinforcement Learning (RL) approaches for optimization under varying forecast assumptions, using a custom Gym-based simulation environment. The evaluation considers both long-term and short-term performance, focusing on economic savings, State of Charge (SOC) and temperature balancing, and overall system efficiency. In summary, the long-term results show that the RL approach achieved 10% higher system efficiency compared to LP, whereas the latter yielded 33% greater cumulative savings. In terms of internal heterogeneity, the LP approach resulted in lower mean SOC imbalance, while the RL approach achieved better temperature balance between strings. This behavior is further examined in the short-term evaluation, which indicates that LP delivers strong optimization under known and stable conditions, whereas RL demonstrates higher adaptability in dynamic environments, offering potential advantages for real-time BESS control.

eess.SY↗

arXiv2024

SafeAgentBench: A Benchmark for Safe Task Planning of Embodied LLM Agents

With the integration of large language models (LLMs), embodied agents have strong capabilities to understand and plan complicated natural language instructions. However, a foreseeable issue is that those embodied agents can also flawlessly execute some hazardous tasks, potentially causing damages in the real world. Existing benchmarks predominantly overlook critical safety risks, focusing solely on planning performance, while a few evaluate LLMs' safety awareness only on non-interactive image-text data. To address this gap, we present SafeAgentBench -- the first comprehensive benchmark for safety-aware task planning of embodied LLM agents in interactive simulation environments, covering both explicit and implicit hazards. SafeAgentBench includes: (1) an executable, diverse, and high-quality dataset of 750 tasks, rigorously curated to cover 10 potential hazards and 3 task types; (2) SafeAgentEnv, a universal embodied environment with a low-level controller, supporting multi-agent execution with 17 high-level actions for 9 state-of-the-art baselines; and (3) reliable evaluation methods from both execution and semantic perspectives. Experimental results show that, although agents based on different design frameworks exhibit substantial differences in task success rates, their overall safety awareness remains weak. The most safety-conscious baseline achieves only a 10% rejection rate for detailed hazardous tasks. Moreover, simply replacing the LLM driving the agent does not lead to notable improvements in safety awareness. Dataset and codes are available in https://github.com/shengyin1224/SafeAgentBench and https://huggingface.co/datasets/safeagentbench/SafeAgentBench.

cs.CR↗

arXiv2024

Ethical-Lens: Curbing Malicious Usages of Open-Source Text-to-Image Models

The burgeoning landscape of text-to-image models, exemplified by innovations such as Midjourney and DALLE 3, has revolutionized content creation across diverse sectors. However, these advancements bring forth critical ethical concerns, particularly with the misuse of open-source models to generate content that violates societal norms. Addressing this, we introduce Ethical-Lens, a framework designed to facilitate the value-aligned usage of text-to-image tools without necessitating internal model revision. Ethical-Lens ensures value alignment in text-to-image models across toxicity and bias dimensions by refining user commands and rectifying model outputs. Systematic evaluation metrics, combining GPT4-V, HEIM, and FairFace scores, assess alignment capability. Our experiments reveal that Ethical-Lens enhances alignment capabilities to levels comparable with or superior to commercial models like DALLE 3, ensuring user-generated content adheres to ethical standards while maintaining image quality. This study indicates the potential of Ethical-Lens to ensure the sustainable development of open-source text-to-image tools and their beneficial integration into society. Our code is available at https://github.com/yuzhu-cai/Ethical-Lens.

cs.CV↗

arXiv2023

Malicious Agent Detection for Robust Multi-Agent Collaborative Perception

Recently, multi-agent collaborative (MAC) perception has been proposed and outperformed the traditional single-agent perception in many applications, such as autonomous driving. However, MAC perception is more vulnerable to adversarial attacks than single-agent perception due to the information exchange. The attacker can easily degrade the performance of a victim agent by sending harmful information from a malicious agent nearby. In this paper, we extend adversarial attacks to an important perception task -- MAC object detection, where generic defenses such as adversarial training are no longer effective against these attacks. More importantly, we propose Malicious Agent Detection (MADE), a reactive defense specific to MAC perception that can be deployed by each agent to accurately detect and then remove any potential malicious agent in its local collaboration network. In particular, MADE inspects each agent in the network independently using a semi-supervised anomaly detector based on a double-hypothesis test with the Benjamini-Hochberg procedure to control the false positive rate of the inference. For the two hypothesis tests, we propose a match loss statistic and a collaborative reconstruction loss statistic, respectively, both based on the consistency between the agent to be inspected and the ego agent where our detector is deployed. We conduct comprehensive evaluations on a benchmark 3D dataset V2X-sim and a real-road dataset DAIR-V2X and show that with the protection of MADE, the drops in the average precision compared with the best-case "oracle" defender against our attack are merely 1.28% and 0.34%, respectively, much lower than 8.92% and 10.00% for adversarial training, respectively.

cs.CR↗

arXiv2022

Dynamic-Group-Aware Networks for Multi-Agent Trajectory Prediction with Relational Reasoning

Demystifying the interactions among multiple agents from their past trajectories is fundamental to precise and interpretable trajectory prediction. However, previous works mainly consider static, pair-wise interactions with limited relational reasoning. To promote more comprehensive interaction modeling and relational reasoning, we propose DynGroupNet, a dynamic-group-aware network, which can i) model time-varying interactions in highly dynamic scenes; ii) capture both pair-wise and group-wise interactions; and iii) reason both interaction strength and category without direct supervision. Based on DynGroupNet, we further design a prediction system to forecast socially plausible trajectories with dynamic relational reasoning. The proposed prediction system leverages the Gaussian mixture model, multiple sampling and prediction refinement to promote prediction diversity, training stability and trajectory smoothness, respectively. Extensive experiments show that: 1)DynGroupNet can capture time-varying group behaviors, infer time-varying interaction category and interaction strength during trajectory prediction without any relation supervision on physical simulation datasets; 2)DynGroupNet outperforms the state-of-the-art trajectory prediction methods by a significant improvement of 22.6%/28.0%, 26.9%/34.9%, 5.1%/13.0% in ADE/FDE on the NBA, NFL Football and SDD datasets and achieve the state-of-the-art performance on the ETH-UCY dataset.

cs.LG↗

arXiv2022

One Dimensional Wormhole Corrosion in Metals

Corrosion is a ubiquitous failure mode of materials in extreme environments. The more localized it is, the more difficult it is to detect and more deleterious its effects. Often, the progression of localized corrosion is accompanied by the evolution of porosity in materials, creating internal void-structures that facilitate the ingress of the external environment into the interior of the material, further accelerating the internal corrosion. Previously, the dominant morphology of such void-structures has been reported to be either three-dimensional (3D) or two-dimensional (2D). Here, we report a more localized form of corrosion, which we call 1D wormhole corrosion. Using electron tomography, we show multiple examples of this 1D and percolating morphology that manifests a significantly high aspect ratio differentiable from 2D and 3D corrosion. To understand the origin of this mechanism in a Ni-Cr alloy corroded by molten salt, we combined energy-filtered four-dimensional scanning transmission electron microscopy (EF-4D-STEM) and ab initio density functional theory (DFT) calculations to develop a vacancy mapping method with nanometer-resolution, identifying a remarkably high vacancy concentration in the diffusion-induced grain boundary migration (DIGM) zone, up to 100 times the equilibrium value at the melting point. These vacancy supersaturation regions act as the precursors of wormholes, and lead to the asymmetrical growth of voids along GBs. We show that similar 1D penetrating corrosion morphologies could also occur in other materials or corrosion conditions, implying the broad impact of this extremely localized corrosion mechanism. Deciphering the origins of 1D corrosion is an important step towards designing structural materials with enhanced corrosion resistance, and also offers new pathways to create ordered-porous materials for functional applications.

cond-mat.mtrl-sci↗

arXiv2021

Trap of Feature Diversity in the Learning of MLPs

In this paper, we focus on a typical two-phase phenomenon in the learning of multi-layer perceptrons (MLPs), and we aim to explain the reason for the decrease of feature diversity in the first phase. Specifically, people find that, in the training of MLPs, the training loss does not decrease significantly until the second phase. To this end, we further explore the reason why the diversity of features over different samples keeps decreasing in the first phase, which hurts the optimization of MLPs. We explain such a phenomenon in terms of the learning dynamics of MLPs. Furthermore, we theoretically explain why four typical operations can alleviate the decrease of the feature diversity.

cs.LG↗

arXiv2021

Universality of free random variables: atoms for non-commutative rational functions

Consider a tuple $(Y_1,\dots,Y_d)$ of normal operators in a tracial operator algebra setting with prescribed sizes of the eigenspaces for each $Y_i$. We address the question what one can say about the sizes of the eigenspaces for any non-commutative polynomial $P(Y_1,\dots,Y_d)$ in those operators? We show that for each polynomial $P$ there are unavoidable eigenspaces, which occur in $P(Y_1,\dots,Y_d)$ for any $(Y_1,\dots,Y_d)$ with the prescribed eigenspaces for the marginals. We will describe this minimal situation both in algebraic terms - where it is given by realizations via matrices over the free skew field and via rank calculations - and in analytic terms - where it is given by freely independent random variables with prescribed atoms in their distributions. The fact that the latter situation corresponds to this minimal situation allows to draw many new conclusions about atoms in polynomials of free variables. In particular, we give a complete description of atoms in the free commutator and the free anti-commutator. Furthermore, our results do not only apply to polynomials, but much more general also to non-commutative rational functions. Since many random matrix models become asymptotically free in the large $N$ limit, our results allow us to calculate the location and size of atoms in the asymptotic eigenvalue distribution of polynomials and rational functions in randomly rotated matrices.

math.OA↗

arXiv2021

Convergence for noncommutative rational functions evaluated in random matrices

One of the main applications of free probability is to show that for appropriately chosen independent copies of $d$ random matrix models, any noncommutative polynomial in these $d$ variables has a spectral distribution that converges asymptotically and can be described with the help of free probability. This paper aims to show that this can be extended to noncommutative rational functions, answering an open question by Roland Speicher. This paper also provides a noncommutative probability approach to approximating the free field. At the algebraic level, its construction relies on the approximation by generic matrices. On the other hand, it admits many embeddings in the algebra of operators affiliated with a $II_1$ factor. A consequence of our result is that, as soon as the generators admit a random matrix model, the approximation of any self-adjoint noncommutative rational function by generic matrices can be upgraded at the level of convergence in distribution.

math.OA↗

arXiv2019

Ab initio modeling of the energy landscape for screw dislocations in body-centered cubic high-entropy alloys

In traditional body-centered cubic (bcc) metals, the core properties of screw dislocations play a critical role in plastic deformation at low temperatures. Recently, much attention has been focused on refractory high-entropy alloys (RHEAs), which also possess bcc crystal structures. However, unlike face-centered cubic high-entropy alloys (HEAs), there have been far fewer investigations on bcc HEAs, specifically on the possible effects of chemical short-range order (SRO) in these multiple principal element alloys on dislocation mobility. Here, using density functional theory, we investigate the distribution of dislocation core properties in MoNbTaW RHEAs alloys, and how they are influenced by SRO. The average values of the core energies in the RHEA are found to be larger than those in the corresponding pure constituent bcc metals, and are relatively insensitive to the degree of SRO. However, the presence of SRO is shown to have a large effect on narrowing the distribution of dislocation core energies and decreasing the spatial heterogeneity of dislocation core energies in the RHEA. It is argued that the consequences for the mechanical behavior of HEAs is a change in the energy landscape of the dislocations which would likely heterogeneously inhibit their motion.

cond-mat.mtrl-sci↗

arXiv2019

The free field: realization via unbounded operators and Atiyah property

Let $X_1,\dots,X_n$ be operators in a finite von Neumann algebra and consider their division closure in the affiliated unbounded operators. We address the question when this division closure is a skew field (aka division ring) and when it is the free skew field. We show that the first property is equivalent to the strong Atiyah property and that the second property can be characterized in terms of the non-commutative distribution of $X_1,\dots,X_n$. More precisely, $X_1,\dots,X_n$ generate the free skew field if and only if there exist no non-zero finite rank operators $T_1,\dots,T_n$ such that $\sum_i[T_i,X_i]=0$. Sufficient conditions for this are the maximality of the free entropy dimension or the existence of a dual system of $X_1,\dots,X_n$. Our general theory is not restricted to selfadjoint operators and thus does also include and recover the result of Linnell that the generators of the free group give the free skew field. We give also consequences of our result for the question of atoms in the distribution of rational functions in free variables or in the asymptotic eigenvalue distribution of matrices over polynomials in asymptotically free random matrices. This solves in particular a conjecture of Charlesworth and Shlyakhtenko.

math.OA↗

arXiv2018

The free field: zero divisors, Atiyah property and realizations via unbounded operators

We consider noncommutative rational functions as well as matrices in polynomials in noncommuting variables in two settings: in an algebraic context the variables are formal variables, and their rational functions generate the "free field"; in an analytic context the variables are given by operators from a finite von Neumann algebra and the question of rational functions is treated within the affiliated unbounded operators. Our main result shows that for a "good" class of operators - namely those for which the free entropy dimension is maximal - the analytic and the algebraic theory are isomorphic. This means in particular that any non-trivial rational function can be evaluated as an unbounded operator for any such good tuple and that those operators don't have zero divisors. On the matrix side, this means that matrices of polynomials which are invertible in the free field are also invertible as matrices over unbounded operators when we plug in our good operator tuples. We also address the question how this is related to the strong Atiyah property. The above yields a quite complete picture for the question of zero divisors (or atoms in the corresponding distributions) for operator tuples with maximal free entropy dimension. We give also some partial results for the question of existence and regularity of a density of the distribution.

math.OA↗

arXiv2017

Non-commutative rational function in strongly convergent random variables

Random matrices like GUE, GOE and GSE have been studied for decades and have been shown that they possess a lot of nice properties. In 2005, a new property of independent GUE random matrices is discovered by Haagerup and Thorbj{\o}rnsen in their paper [18], it is called strong convergence property and then more random matrices with this property are followed (see [27], [5], [1], [24], [10] and [3]). In general, the definition can be stated for a sequence of tuples over some \text{C}^{\ast}-algebras. And in this general setting, some stability property under reduced free product can be achieved (see Skoufranis [30] and Pisier [26]), as an analogy of the result by Camille Male [24] for random matrices. In this paper, we want to show that, for a sequence of strongly convergent random variables, non-commutative polynomials can be extended to non-commutative rational functions under certain assumptions. Roughly speaking, the strong convergence property is stable under taking the inverse. As a direct corollary, we can conclude that for a tuple (X_{1}^{\left(n\right)},\cdots,X_{m}^{\left(n\right)}) of independent GUE random matrices, r(X_{1}^{\left(n\right)},\cdots,X_{m}^{\left(n\right)}) converges in trace and in norm to r(s_{1},\cdots,s_{m}) almost surely, where r is a rational function and (s_{1},\cdots,s_{m}) is a tuple of freely independent semi-circular elements which lies in the domain of r.

math.OA↗