arXiv subjects

Hao Wang

Publications and source records attributed to Hao Wang.

At least 577 records · Page 32Linked to original sources

arXiv2025

Learnable Behavior Control: Breaking Atari Human World Records via Sample-Efficient Behavior Selection

The exploration problem is one of the main challenges in deep reinforcement learning (RL). Recent promising works tried to handle the problem with population-based methods, which collect samples with diverse behaviors derived from a population of different exploratory policies. Adaptive policy selection has been adopted for behavior control. However, the behavior selection space is largely limited by the predefined policy population, which further limits behavior diversity. In this paper, we propose a general framework called Learnable Behavioral Control (LBC) to address the limitation, which a) enables a significantly enlarged behavior selection space via formulating a hybrid behavior mapping from all policies; b) constructs a unified learnable process for behavior selection. We introduce LBC into distributed off-policy actor-critic methods and achieve behavior control via optimizing the selection of the behavior mappings with bandit-based meta-controllers. Our agents have achieved 10077.52% mean human normalized score and surpassed 24 human world records within 1B training frames in the Arcade Learning Environment, which demonstrates our significant state-of-the-art (SOTA) performance without degrading the sample efficiency.

cs.LG↗

arXiv2025

Prompting is not Enough: Exploring Knowledge Integration and Controllable Generation

Open-domain question answering (OpenQA) represents a cornerstone in natural language processing (NLP), primarily focused on extracting answers from unstructured textual data. With the rapid advancements in Large Language Models (LLMs), LLM-based OpenQA methods have reaped the benefits of emergent understanding and answering capabilities enabled by massive parameters compared to traditional methods. However, most of these methods encounter two critical challenges: how to integrate knowledge into LLMs effectively and how to adaptively generate results with specific answer formats for various task situations. To address these challenges, we propose a novel framework named GenKI, which aims to improve the OpenQA performance by exploring Knowledge Integration and controllable Generation on LLMs simultaneously. Specifically, we first train a dense passage retrieval model to retrieve associated knowledge from a given knowledge base. Subsequently, we introduce a novel knowledge integration model that incorporates the retrieval knowledge into instructions during fine-tuning to intensify the model. Furthermore, to enable controllable generation in LLMs, we leverage a certain fine-tuned LLM and an ensemble based on text consistency incorporating all coherence, fluency, and answer format assurance. Finally, extensive experiments conducted on the TriviaQA, MSMARCO, and CMRC2018 datasets, featuring diverse answer formats, have demonstrated the effectiveness of GenKI with comparison of state-of-the-art baselines. Moreover, ablation studies have disclosed a linear relationship between the frequency of retrieved knowledge and the model's ability to recall knowledge accurately against the ground truth. Our code of GenKI is available at https://github.com/USTC-StarTeam/GenKI

cs.CL↗

arXiv2025

Game-TARS: Pretrained Foundation Models for Scalable Generalist Multimodal Game Agents

We present Game-TARS, a generalist game agent trained with a unified, scalable action space anchored to human-aligned native keyboard-mouse inputs. Unlike API- or GUI-based approaches, this paradigm enables large-scale continual pre-training across heterogeneous domains, including OS, web, and simulation games. Game-TARS is pre-trained on over 500B tokens with diverse trajectories and multimodal data. Key techniques include a decaying continual loss to reduce causal confusion and an efficient Sparse-Thinking strategy that balances reasoning depth and inference cost. Experiments show that Game-TARS achieves about 2 times the success rate over the previous sota model on open-world Minecraft tasks, is close to the generality of fresh humans in unseen web 3d games, and outperforms GPT-5, Gemini-2.5-Pro, and Claude-4-Sonnet in FPS benchmarks. Scaling results on training-time and test-time confirm that the unified action space sustains improvements when scaled to cross-game and multimodal data. Our results demonstrate that simple, scalable action representations combined with large-scale pre-training provide a promising path toward generalist agents with broad computer-use abilities.

cs.AI↗

arXiv2025

SN 2024iss: A Double-peaked Type IIb Supernova with Evidence of Circumstellar Interaction

We present optical, ultraviolet, and X-ray observations of supernova (SN) 2024iss, a Type IIb SN that shows a prominent double-peaked light curve. We modeled the first peak with a semianalytical shock-cooling model and the X-ray emission with a free-free model. We compare the envelope radius and mass-loss rate with other Type IIb SNe to explore the relationships between the progenitor envelope and the circumstellar material (CSM). The shock-cooling peak in the $V$-band light curve reached $M_V = -17.33\pm 0.26$mag, while the $^{56}$Ni-powered second peak attained $M_V = -17.43\pm 0.26$mag. Early spectra show an photospheric velocity of $\sim19,400\,km\,s^{-1}$ at 3.82days from the H$α$ P~Cygni profile. The Balmer lines persist at least +87 days after the explosion, characterizing hydrogen-rich ejecta. Modeling the first light-curve peak suggests an extended envelope with a mass of $0.11\pm0.04\,M_{\odot}$ and a radius of $244\pm43~R_{\odot}$. Fitting the second light-curve peak with an Arnett-like model indicates a typical $^{56}$Ni mass of $ 0.117\pm0.013~M_{\odot}$ and a relatively low ejecta mass of $1.272\pm0.343\,M_{\odot}$. X-ray observations reveal bright thermal bremsstrahlung emission and indicate a mass-loss rate of $1.6\times10^{-5}\ M_{\odot} \ \rm{yr}^{-1}$. SN 2024iss occupies a transitional position between the two subclasses of extended (eIIb) and compact (cIIb) Type IIb SNe. Its envelope radius and pre-explosion mass-loss rate appear to be correlated as theoretically predicted. The observational properties of SN 2024iss are compatible with a binary interaction scenario being the dominant mechanism for envelope stripping. Furthermore, the low column density of neutral hydrogen suggests a compact CSM with an outer radius of $\lesssim1.3\times10^{14}$ cm, indicating that the progenitor star experienced eruptive mass loss within $\sim4\,yr$ of its terminal explosion.

astro-ph.HE↗

arXiv2025

Exploiting Electrolyzer Flexibility via Multiscale Model Predictive Control Cross Heterogeneous Energy Markets

Green hydrogen production via electrolysis is crucial for decarbonization but faces significant economic hurdles primarily due to the high cost of the electricity. However, current electrolyzer-based hydrogen production processes predominantly rely on the single-scale Day-Ahead Market (DAM) for electricity procurement, failing to fully exploit the economic benefits offered by multi-scale electricity market that integrates both the DAM and the Real-Time Market (RTM), thereby eliminating the opportunity to reduce the overall cost. To mitigate this technical gap, this research investigates a dynamic operational strategy enabling electrolyzers to strategically navigate between the DAM and RTM to minimize net operation costs. Using a rolling horizon optimization framework to coordinate bidding and operation, we demonstrate a strategy where electrolyzers secure primary energy via exclusive DAM purchases, then actively engage the RTM to buy supplemental energy cheaply or, critically, sell procured DAM energy back at a profit during high RTM price periods. Our analysis reveals that this coordinated multi-scale electricity market participation strategy can dramatically reduce net electricity expenditures, achieving near-zero or even negative effective electricity costs for green hydrogen production under realistic market scenarios, effectively meaning the operation can profit from its electricity market interactions. By transforming electrolyzers from simple price-takers into active participants capable of arbitrage between market timescales, this approach unlocks a financially compelling pathway for green hydrogen, accelerating its deployment while simultaneously enhancing power grid flexibility.

math.OC↗

arXiv2025

TerraGen: A Unified Multi-Task Layout Generation Framework for Remote Sensing Data Augmentation

Remote sensing vision tasks require extensive labeled data across multiple, interconnected domains. However, current generative data augmentation frameworks are task-isolated, i.e., each vision task requires training an independent generative model, and ignores the modeling of geographical information and spatial constraints. To address these issues, we propose \textbf{TerraGen}, a unified layout-to-image generation framework that enables flexible, spatially controllable synthesis of remote sensing imagery for various high-level vision tasks, e.g., detection, segmentation, and extraction. Specifically, TerraGen introduces a geographic-spatial layout encoder that unifies bounding box and segmentation mask inputs, combined with a multi-scale injection scheme and mask-weighted loss to explicitly encode spatial constraints, from global structures to fine details. Also, we construct the first large-scale multi-task remote sensing layout generation dataset containing 45k images and establish a standardized evaluation protocol for this task. Experimental results show that our TerraGen can achieve the best generation image quality across diverse tasks. Additionally, TerraGen can be used as a universal data-augmentation generator, enhancing downstream task performance significantly and demonstrating robust cross-task generalisation in both full-data and few-shot scenarios.

cs.CV↗

arXiv2025

Toward a Metrology for Artificial Intelligence: Hidden-Rule Environments and Reinforcement Learning

We investigate reinforcement learning in the Game Of Hidden Rules (GOHR) environment, a complex puzzle in which an agent must infer and execute hidden rules to clear a 6$\times$6 board by placing game pieces into buckets. We explore two state representation strategies, namely Feature-Centric (FC) and Object-Centric (OC), and employ a Transformer-based Advantage Actor-Critic (A2C) algorithm for training. The agent has access only to partial observations and must simultaneously infer the governing rule and learn the optimal policy through experience. We evaluate our models across multiple rule-based and trial-list-based experimental setups, analyzing transfer effects and the impact of representation on learning efficiency.

cs.LG↗

arXiv2025

EmbodiedBrain: Expanding Performance Boundaries of Task Planning for Embodied Intelligence

The realization of Artificial General Intelligence (AGI) necessitates Embodied AI agents capable of robust spatial perception, effective task planning, and adaptive execution in physical environments. However, current large language models (LLMs) and multimodal LLMs (MLLMs) for embodied tasks suffer from key limitations, including a significant gap between model design and agent requirements, an unavoidable trade-off between real-time latency and performance, and the use of unauthentic, offline evaluation metrics. To address these challenges, we propose EmbodiedBrain, a novel vision-language foundation model available in both 7B and 32B parameter sizes. Our framework features an agent-aligned data structure and employs a powerful training methodology that integrates large-scale Supervised Fine-Tuning (SFT) with Step-Augumented Group Relative Policy Optimization (Step-GRPO), which boosts long-horizon task success by integrating preceding steps as Guided Precursors. Furthermore, we incorporate a comprehensive reward system, including a Generative Reward Model (GRM) accelerated at the infrastructure level, to improve training efficiency. For enable thorough validation, we establish a three-part evaluation system encompassing General, Planning, and End-to-End Simulation Benchmarks, highlighted by the proposal and open-sourcing of a novel, challenging simulation environment. Experimental results demonstrate that EmbodiedBrain achieves superior performance across all metrics, establishing a new state-of-the-art for embodied foundation models. Towards paving the way for the next generation of generalist embodied agents, we open-source all of our data, model weight, and evaluating methods, which are available at https://zterobot.github.io/EmbodiedBrain.github.io.

cs.CV↗

arXiv2025

Tibetan Language and AI: A Comprehensive Survey of Resources, Methods and Challenges

Tibetan, one of the major low-resource languages in Asia, presents unique linguistic and sociocultural characteristics that pose both challenges and opportunities for AI research. Despite increasing interest in developing AI systems for underrepresented languages, Tibetan has received limited attention due to a lack of accessible data resources, standardized benchmarks, and dedicated tools. This paper provides a comprehensive survey of the current state of Tibetan AI in the AI domain, covering textual and speech data resources, NLP tasks, machine translation, speech recognition, and recent developments in LLMs. We systematically categorize existing datasets and tools, evaluate methods used across different tasks, and compare performance where possible. We also identify persistent bottlenecks such as data sparsity, orthographic variation, and the lack of unified evaluation metrics. Additionally, we discuss the potential of cross-lingual transfer, multi-modal learning, and community-driven resource creation. This survey aims to serve as a foundational reference for future work on Tibetan AI research and encourages collaborative efforts to build an inclusive and sustainable AI ecosystem for low-resource languages.

cs.CL↗

arXiv2025

Micro-Doppler Energy-Based Robust Multi-Target Vital Signs Monitoring Using 77-GHz FMCW Radar with Spatiotemporal Adaptive Processing

This paper presents a novel micro-Doppler energy-based framework for robust multi-target vital signs monitoring using 77-GHz Frequency-Modulated Continuous-Wave (FMCW) radar. Unlike conventional phase-based methods that are susceptible to environmental noise, random body movements, and stringent calibration requirements, our approach exploits the energy variations in radar returns induced by cardiopulmonary activities. The proposed system integrates a comprehensive processing pipeline including space-time adaptive processing (STAP) for target detection and tracking, MUSIC algorithm for high-resolution angle estimation, and an innovative adaptive spectral filtering technique for vital signs extraction. We establish a rigorous mathematical framework that formalizes the relationship between micro-Doppler energy variations and physiological activities, enabling robust separation of closely spaced targets. The key innovation lies in the micro-Doppler energy extraction methodology that provides inherent robustness to phase noise and motion artifacts. Experimental results using millimeter-wave radar datasets demonstrate that the system can accurately detect and separate vital signs of up to four targets within \SI{5}{\meter} range, achieving mean absolute errors of \SI{1.2}beats per minute and \SI{2.3} beats per minute for respiration and heart rates, respectively. The proposed approach demonstrates superior performance compared to traditional phase-based methods, particularly in challenging multi-target scenarios with environmental noise and subject movement.

eess.SP↗

arXiv2025

VisuRiddles: Fine-grained Perception is a Primary Bottleneck for Multimodal Large Language Models in Abstract Visual Reasoning

Recent strides in multimodal large language models (MLLMs) have significantly advanced their performance in many reasoning tasks. However, Abstract Visual Reasoning (AVR) remains a critical challenge, primarily due to limitations in perceiving abstract graphics. To tackle this issue, we investigate the bottlenecks in current MLLMs and synthesize training data to improve their abstract visual perception. First, we propose VisuRiddles, a benchmark for AVR, featuring tasks meticulously constructed to assess models' reasoning capacities across five core dimensions and two high-level reasoning categories. Second, we introduce the Perceptual Riddle Synthesizer (PRS), an automated framework for generating riddles with fine-grained perceptual descriptions. PRS not only generates valuable training data for abstract graphics but also provides fine-grained perceptual description, crucially allowing for supervision over intermediate reasoning stages and thereby improving both training efficacy and model interpretability. Our extensive experimental results on VisuRiddles empirically validate that fine-grained visual perception is the principal bottleneck and our synthesis framework markedly enhances the performance of contemporary MLLMs on these challenging tasks. Our code and dataset will be released at https://github.com/yh-hust/VisuRiddles

cs.CV↗

arXiv2025

POLAR: Policy-based Layerwise Reinforcement Learning Method for Stealthy Backdoor Attacks in Federated Learning

Federated Learning (FL) enables decentralized model training across multiple clients without exposing local data, but its distributed feature makes it vulnerable to backdoor attacks. Despite early FL backdoor attacks modifying entire models, recent studies have explored the concept of backdoor-critical (BC) layers, which poison the chosen influential layers to maintain stealthiness while achieving high effectiveness. However, existing BC layers approaches rely on rule-based selection without consideration of the interrelations between layers, making them ineffective and prone to detection by advanced defenses. In this paper, we propose POLAR (POlicy-based LAyerwise Reinforcement learning), the first pipeline to creatively adopt RL to solve the BC layer selection problem in layer-wise backdoor attack. Different from other commonly used RL paradigm, POLAR is lightweight with Bernoulli sampling. POLAR dynamically learns an attack strategy, optimizing layer selection using policy gradient updates based on backdoor success rate (BSR) improvements. To ensure stealthiness, we introduce a regularization constraint that limits the number of modified layers by penalizing large attack footprints. Extensive experiments demonstrate that POLAR outperforms the latest attack methods by up to 40% against six state-of-the-art (SOTA) defenses.

cs.LG↗

arXiv2025

QKCV Attention: Enhancing Time Series Forecasting with Static Categorical Embeddings for Both Lightweight and Pre-trained Foundation Models

In real-world time series forecasting tasks, category information plays a pivotal role in capturing inherent data patterns. This paper introduces QKCV (Query-Key-Category-Value) attention, an extension of the traditional QKV framework that incorporates a static categorical embedding C to emphasize category-specific information. As a versatile plug-in module, QKCV enhances the forecasting accuracy of attention-based models (e.g., Vanilla Transformer, Informer, PatchTST, TFT) across diverse real-world datasets. Furthermore, QKCV demonstrates remarkable adaptability in fine-tuning univariate time series foundation model by solely updating the static embedding C while preserving pretrained weights, thereby reducing computational overhead and achieving superior fine-tuning performance.

cs.LG↗

arXiv2025

Anomalous terahertz nonlinearity in disordered s-wave superconductor close to the superconductor-insulator transition

Detection of the Higgs mode in superconductors using nonlinear terahertz spectroscopy is a key area of interest in condensed matter physics. We investigate the influence of disorder on the nonlinear terahertz response and the Higgs mode in NbN thin films with varying Ioffe-Regel parameters ($k_Fl$). In strongly disordered films near the superconductor-insulator transition (SIT), we observe an anomalous third-harmonic generation (THG) signal above $T_c$, which is absent in both cleaner superconducting and non-superconducting counterparts. The persistence of this normal-state THG signal in a high magnetic field excludes superconducting fluctuations as its origin. Below $T_c$, the THG intensity increases sharply, indicating a dominant contribution from the driven Higgs mode. The THG spectrum of the strongly disordered sample exhibits a broadened, multi-peak structure, which we attribute to quantum path interference between distinct channels involving unpaired electrons and Cooper pairs within emergent superconducting islands. Our findings not only demonstrate how disorder tunes the nonlinear terahertz response but also uncover a strong coupling between electrons responsible for normal-state THG and the superconducting Higgs mode below $T_c$ in strongly disordered samples.

cond-mat.supr-con↗

arXiv2025

Comprehensive Dynamic Modeling and Constraint-Aware Air Supply Control for Localized Water Management in Automotive Polymer Electrolyte Membrane Fuel Cells

In this paper, a predictive constraint-aware control scheme is formulated within the Command Governor (CG) framework for localized hydration management of a proton exchange membrane (PEM) fuel cell system. First, a comprehensive nonlinear dynamic model of the fuel cell system is presented which includes a pseudo 2-dimensional (P2D) model of the stack, reactant supply and cooling subsystems. The model captures the couplings among the various subsystems and serves as the basis for designing output feedback controllers to track the optimal set-points of the air supply and cooling systems for power optimization. The closed-loop nonlinear model is then used to analyze the dynamic behavior of membrane hydration near the anode inlet, the driest region of the membrane in a counter-flow configuration, under various operating conditions. A reduced-order linearized model is then derived to approximate hydration behavior with sufficient fidelity for constraint enforcement. This model is used within the CG framework to adjust the air supply set-points when necessary to prevent membrane dry-out. The effectiveness of the proposed approach in maintaining local membrane hydration while closely tracking the requested net power is demonstrated through realistic drive-cycle simulations.

eess.SY↗

arXiv2025

Chronos-2: From Univariate to Universal Forecasting

Pretrained time series models have enabled inference-only forecasting systems that produce accurate predictions without task-specific training. However, existing approaches largely focus on univariate forecasting, limiting their applicability in real-world scenarios where multivariate data and covariates play a crucial role. We present Chronos-2, a pretrained model capable of handling univariate, multivariate, and covariate-informed forecasting tasks in a zero-shot manner. Chronos-2 employs a group attention mechanism that facilitates in-context learning (ICL) through efficient information sharing across multiple time series within a group, which may represent sets of related series, variates of a multivariate series, or targets and covariates in a forecasting task. These general capabilities are achieved through training on synthetic datasets that impose diverse multivariate structures on univariate series. Chronos-2 delivers state-of-the-art performance across three comprehensive benchmarks: fev-bench, GIFT-Eval, and Chronos Benchmark II. On fev-bench, which emphasizes multivariate and covariate-informed forecasting, Chronos-2's universal ICL capabilities lead to substantial improvements over existing models. On tasks involving covariates, it consistently outperforms baselines by a wide margin. Case studies in the energy and retail domains further highlight its practical advantages. The in-context learning capabilities of Chronos-2 establish it as a general-purpose forecasting model that can be used "as is" in real-world forecasting pipelines.

cs.LG↗

arXiv2025

Hierarchical Fuel-Cell Airpath Control: an Efficiency-Aware MIMO Control Approach Combined with a Novel Constraint-Enforcing Reference Governor

This paper presents a hierarchical multivariable control and constraint management approach for an air supply system for a proton exchange membrane fuel cell (PEMFC) system. The control objectives are to track desired compressor mass airflow and cathode inlet pressure, maintain a minimum oxygen excess ratio (OER), and run the system at maximum net efficiency. A multi-input multi-output (MIMO) internal model controller (IMC) is designed and simulated to track flow and pressure set-points, which showed high performance despite strongly coupled plant dynamics. A new set-point map is generated to compute the most efficient cathode inlet pressure from the stack current load. To enforce OER constraints, a novel reference governor (RG) with the ability to govern multiple references (the cascade RG) and the ability to speed up as well as slow down a reference signal (the cross-section RG) is developed and tested. Compared with a single-input single-output (SISO) air-flow control approach, the proposed MIMO control approach shows up to 7.36 percent lower hydrogen fuel consumption. Compared to a traditional load governor, the novel cascaded cross-section RG (CC-RG) shows up to 3.68 percent less mean absolute percent error (MAPE) on net power tracking and greatly improved worst-case OER on realistic drive-cycle simulations. Control development and validations were conducted on two fuel cell system (FCS) models, a nonlinear open-source model and a proprietary Ford high-fidelity model

eess.SY↗

arXiv2025

DynaSearcher: Dynamic Knowledge Graph Augmented Search Agent via Multi-Reward Reinforcement Learning

Multi-step agentic retrieval systems based on large language models (LLMs) have demonstrated remarkable performance in complex information search tasks. However, these systems still face significant challenges in practical applications, particularly in generating factually inconsistent intermediate queries and inefficient search trajectories, which can lead to reasoning deviations or redundant computations. To address these issues, we propose DynaSearcher, an innovative search agent enhanced by dynamic knowledge graphs and multi-reward reinforcement learning (RL). Specifically, our system leverages knowledge graphs as external structured knowledge to guide the search process by explicitly modeling entity relationships, thereby ensuring factual consistency in intermediate queries and mitigating biases from irrelevant information. Furthermore, we employ a multi-reward RL framework for fine-grained control over training objectives such as retrieval accuracy, efficiency, and response quality. This framework promotes the generation of high-quality intermediate queries and comprehensive final answers, while discouraging unnecessary exploration and minimizing information omissions or redundancy. Experimental results demonstrate that our approach achieves state-of-the-art answer accuracy on six multi-hop question answering datasets, matching frontier LLMs while using only small-scale models and limited computational resources. Furthermore, our approach demonstrates strong generalization and robustness across diverse retrieval environments and larger-scale models, highlighting its broad applicability.

cs.LG↗