arXiv subjects

Zihao Wang

Publications and source records attributed to Zihao Wang.

At least 145 records · Page 8Linked to original sources

arXiv2025

DefenSee: Dissecting Threat from Sight and Text -- A Multi-View Defensive Pipeline for Multi-modal Jailbreaks

Multi-modal large language models (MLLMs), capable of processing text, images, and audio, have been widely adopted in various AI applications. However, recent MLLMs integrating images and text remain highly vulnerable to coordinated jailbreaks. Existing defenses primarily focus on the text, lacking robust multi-modal protection. As a result, studies indicate that MLLMs are more susceptible to malicious or unsafe instructions, unlike their text-only counterparts. In this paper, we proposed DefenSee, a robust and lightweight multi-modal black-box defense technique that leverages image variants transcription and cross-modal consistency checks, mimicking human judgment. Experiments on popular multi-modal jailbreak and benign datasets show that DefenSee consistently enhances MLLM robustness while better preserving performance on benign tasks compared to SOTA defenses. It reduces the ASR of jailbreak attacks to below 1.70% on MiniGPT4 using the MM-SafetyBench benchmark, significantly outperforming prior methods under the same conditions.

cs.CR↗

arXiv2025

Mavors: Multi-granularity Video Representation for Multimodal Large Language Model

Long-context video understanding in multimodal large language models (MLLMs) faces a critical challenge: balancing computational efficiency with the retention of fine-grained spatio-temporal patterns. Existing approaches (e.g., sparse sampling, dense sampling with low resolution, and token compression) suffer from significant information loss in temporal dynamics, spatial details, or subtle interactions, particularly in videos with complex motion or varying resolutions. To address this, we propose $\mathbf{Mavors}$, a novel framework that introduces $\mathbf{M}$ulti-gr$\mathbf{a}$nularity $\mathbf{v}$ide$\mathbf{o}$ $\mathbf{r}$epre$\mathbf{s}$entation for holistic long-video modeling. Specifically, Mavors directly encodes raw video content into latent representations through two core components: 1) an Intra-chunk Vision Encoder (IVE) that preserves high-resolution spatial features via 3D convolutions and Vision Transformers, and 2) an Inter-chunk Feature Aggregator (IFA) that establishes temporal coherence across chunks using transformer-based dependency modeling with chunk-level rotary position encodings. Moreover, the framework unifies image and video understanding by treating images as single-frame videos via sub-image decomposition. Experiments across diverse benchmarks demonstrate Mavors' superiority in maintaining both spatial fidelity and temporal continuity, significantly outperforming existing methods in tasks requiring fine-grained spatio-temporal reasoning.

cs.CV↗

arXiv2025

Chemical vapor deposition growth of continuous monolayer antiferromagnetic CrOCl films

The discovery of two-dimensional magnetic materials has provided an ideal platform for exploring physical phenomena in the two-dimensional limit. However, intrinsic two-dimensional antiferromagnetic materials have been rarely reported, limiting systematic studies of their electronic properties. The discovery of novel intrinsic two-dimensional antiferromagnets and the development of robust synthesis strategies, therefore, remain significant challenges. Here, we report the chemical vapor deposition synthesis of CrOCl monolayer films and nanosheets that exhibit excellent air stability. The CrOCl morphology is tunable, ranging from two-dimensional nanosheets to three-dimensional flower-like structures, with lateral sizes ranging from several microns to continuous monolayer films. Structural characterization confirms the materials composition and high crystalline quality. Furthermore, magnetic measurements, supported by theoretical calculations, reveal a Néel temperature for CrOCl of ~14 K. This work provides a reliable route for preparing two-dimensional antiferromagnetic materials.

cond-mat.mes-hall↗

arXiv2025

PRBench: Large-Scale Expert Rubrics for Evaluating High-Stakes Professional Reasoning

Frontier model progress is often measured by academic benchmarks, which offer a limited view of performance in real-world professional contexts. Existing evaluations often fail to assess open-ended, economically consequential tasks in high-stakes domains like Legal and Finance, where practical returns are paramount. To address this, we introduce Professional Reasoning Bench (PRBench), a realistic, open-ended, and difficult benchmark of real-world problems in Finance and Law. We open-source its 1,100 expert-authored tasks and 19,356 expert-curated criteria, making it, to our knowledge, the largest public, rubric-based benchmark for both legal and finance domains. We recruit 182 qualified professionals, holding JDs, CFAs, or 6+ years of experience, who contributed tasks inspired by their actual workflows. This process yields significant diversity, with tasks spanning 114 countries and 47 US jurisdictions. Our expert-curated rubrics are validated through a rigorous quality pipeline, including independent expert validation. Subsequent evaluation of 20 leading models reveals substantial room for improvement, with top scores of only 0.39 (Finance) and 0.37 (Legal) on our Hard subsets. We further catalog associated economic impacts of the prompts and analyze performance using human-annotated rubric categories. Our analysis shows that models with similar overall scores can diverge significantly on specific capabilities. Common failure modes include inaccurate judgments, a lack of process transparency and incomplete reasoning, highlighting critical gaps in their reliability for professional adoption.

cs.CL↗

arXiv2025

Diff-V2M: A Hierarchical Conditional Diffusion Model with Explicit Rhythmic Modeling for Video-to-Music Generation

Video-to-music (V2M) generation aims to create music that aligns with visual content. However, two main challenges persist in existing methods: (1) the lack of explicit rhythm modeling hinders audiovisual temporal alignments; (2) effectively integrating various visual features to condition music generation remains non-trivial. To address these issues, we propose Diff-V2M, a general V2M framework based on a hierarchical conditional diffusion model, comprising two core components: visual feature extraction and conditional music generation. For rhythm modeling, we begin by evaluating several rhythmic representations, including low-resolution mel-spectrograms, tempograms, and onset detection functions (ODF), and devise a rhythmic predictor to infer them directly from videos. To ensure contextual and affective coherence, we also extract semantic and emotional features. All features are incorporated into the generator via a hierarchical cross-attention mechanism, where emotional features shape the affective tone via the first layer, while semantic and rhythmic features are fused in the second cross-attention layer. To enhance feature integration, we introduce timestep-aware fusion strategies, including feature-wise linear modulation (FiLM) and weighted fusion, allowing the model to adaptively balance semantic and rhythmic cues throughout the diffusion process. Extensive experiments identify low-resolution ODF as a more effective signal for modeling musical rhythm and demonstrate that Diff-V2M outperforms existing models on both in-domain and out-of-domain datasets, achieving state-of-the-art performance in terms of objective metrics and subjective comparisons. Demo and code are available at https://Tayjsl97.github.io/Diff-V2M-Demo/.

cs.SD↗

arXiv2025

CryptPEFT: Efficient and Private Neural Network Inference via Parameter-Efficient Fine-Tuning

Publicly available large pretrained models (i.e., backbones) and lightweight adapters for parameter-efficient fine-tuning (PEFT) have become standard components in modern machine learning pipelines. However, preserving the privacy of both user inputs and fine-tuned adapters -- often trained on sensitive data -- during inference remains a significant challenge. Applying cryptographic techniques, such as multi-party computation (MPC), to PEFT settings still incurs substantial encrypted computation across both the backbone and adapter, mainly due to the inherent two-way communication between them. To address this limitation, we propose CryptPEFT, the first PEFT solution specifically designed for private inference scenarios. CryptPEFT introduces a novel one-way communication (OWC) architecture that confines encrypted computation solely to the adapter, significantly reducing both computational and communication overhead. To maintain strong model utility under this constraint, we explore the design space of OWC-compatible adapters and employ an automated architecture search algorithm to optimize the trade-off between private inference efficiency and model utility. We evaluated CryptPEFT using Vision Transformer backbones across widely used image classification datasets. Our results show that CryptPEFT significantly outperforms existing baselines, delivering speedups ranging from $20.62\times$ to $291.48\times$ in simulated wide-area network (WAN) and local-area network (LAN) settings. On CIFAR-100, CryptPEFT attains 85.47% accuracy with just 2.26 seconds of inference latency. These findings demonstrate that CryptPEFT offers an efficient and privacy-preserving solution for modern PEFT-based inference.

cs.CR↗

arXiv2025

Game-TARS: Pretrained Foundation Models for Scalable Generalist Multimodal Game Agents

We present Game-TARS, a generalist game agent trained with a unified, scalable action space anchored to human-aligned native keyboard-mouse inputs. Unlike API- or GUI-based approaches, this paradigm enables large-scale continual pre-training across heterogeneous domains, including OS, web, and simulation games. Game-TARS is pre-trained on over 500B tokens with diverse trajectories and multimodal data. Key techniques include a decaying continual loss to reduce causal confusion and an efficient Sparse-Thinking strategy that balances reasoning depth and inference cost. Experiments show that Game-TARS achieves about 2 times the success rate over the previous sota model on open-world Minecraft tasks, is close to the generality of fresh humans in unseen web 3d games, and outperforms GPT-5, Gemini-2.5-Pro, and Claude-4-Sonnet in FPS benchmarks. Scaling results on training-time and test-time confirm that the unified action space sustains improvements when scaled to cross-game and multimodal data. Our results demonstrate that simple, scalable action representations combined with large-scale pre-training provide a promising path toward generalist agents with broad computer-use abilities.

cs.AI↗

arXiv2025

The Early Bird Catches the Leak: Unveiling Timing Side Channels in LLM Serving Systems

The wide deployment of Large Language Models (LLMs) has given rise to strong demands for optimizing their inference performance. Today's techniques serving this purpose primarily focus on reducing latency and improving throughput through algorithmic and hardware enhancements, while largely overlooking their privacy side effects, particularly in a multi-user environment. In our research, for the first time, we discovered a set of new timing side channels in LLM systems, arising from shared caches and GPU memory allocations, which can be exploited to infer both confidential system prompts and those issued by other users. These vulnerabilities echo security challenges observed in traditional computing systems, highlighting an urgent need to address potential information leakage in LLM serving infrastructures. In this paper, we report novel attack strategies designed to exploit such timing side channels inherent in LLM deployments, specifically targeting the Key-Value (KV) cache and semantic cache widely used to enhance LLM inference performance. Our approach leverages timing measurements and classification models to detect cache hits, allowing an adversary to infer private prompts with high accuracy. We also propose a token-by-token search algorithm to efficiently recover shared prompt prefixes in the caches, showing the feasibility of stealing system prompts and those produced by peer users. Our experimental studies on black-box testing of popular online LLM services demonstrate that such privacy risks are completely realistic, with significant consequences. Our findings underscore the need for robust mitigation to protect LLM systems against such emerging threats.

cs.CR↗

arXiv2025

Online Rubrics Elicitation from Pairwise Comparisons

Rubrics provide a flexible way to train LLMs on open-ended long-form answers where verifiable rewards are not applicable and human preferences provide coarse signals. Prior work shows that reinforcement learning with rubric-based rewards leads to consistent gains in LLM post-training. Most existing approaches rely on rubrics that remain static over the course of training. Such static rubrics, however, are vulnerable to reward-hacking type behaviors and fail to capture emergent desiderata that arise during training. We introduce Online Rubrics Elicitation (OnlineRubrics), a method that dynamically curates evaluation criteria in an online manner through pairwise comparisons of responses from current and reference policies. This online process enables continuous identification and mitigation of errors as training proceeds. Empirically, this approach yields consistent improvements of up to 8% over training exclusively with static rubrics across AlpacaEval, GPQA, ArenaHard as well as the validation sets of expert questions and rubrics. We qualitatively analyze the elicited criteria and identify prominent themes such as transparency, practicality, organization, and reasoning.

cs.CL↗

arXiv2025

SHAPE. I. A SOM-SED hybrid approach for efficient galaxy parameter estimation leveraging JWST

With the launch and application of next-generation ground- and space-based telescopes, astronomy has entered the era of big data, necessitating more efficient and robust data analysis methods. Most traditional parameter estimation methods are unable to reconcile differences between photometric systems. Ideally, we would like to optimally rely on high-quality observation data provided by, e.g., JWST, for calibrating and improving upcoming wide-field surveys such as the China Space Station Telescope (CSST) and Euclid. To this end, we introduce a new approach (SHAPE, SOM-SED Hybrid Approach for efficient Parameter Estimation) that can bridge different photometric systems and efficiently estimate key galaxy parameters, such as stellar mass ($M_\star$) and star formation rate (SFR), leveraging data from a large and deep JWST/NIRCam and MIRI survey (PRIMER). As a test of the methodology, we focus on galaxies at $z\sim 1.5-2.5$. To mitigate discrepancies between input colors and the training set, we replace the default SOM weights with stacked SEDs from each cell, extending the applicability of our model to other photometric catalogs (e.g., COSMOS2020). By incorporating a SED library (SED Lib), we apply this JWST-calibrated model to the COSMOS2020 catalog. Despite the limited sample size and potential template-related uncertainties, SOM-derived parameters exhibit a good agreement with results from SED-fitting using extended photometry. Under identical photometric constraints from CSST and Euclid bands, our method outperforms traditional SED-fitting techniques in SFR estimation, exhibiting both a reduced bias (-0.01 vs. 0.18) and a smaller $σ_{\rm NMAD}$ (0.25 vs. 0.35). With its computational efficiency capable of processing $10^6$ sources per CPU per hour during the estimation phase, this JWST-calibrated estimator holds significant promise for next-generation wide-field surveys.

astro-ph.GA↗

arXiv2025

Fabrication of hydrogen-bonded metal inorganic-organic complex glasses by ligand-tuning approach

Metal inorganic-organic complex (MIOC) crystals are a new category of hybrid glass formers. However, the glass-forming compositions of MIOC crystals are limited due to lack of both a general design principle for such compositions and a deep understanding of the structure and formation mechanism for MIOC glasses. This work reports a general approach for synthesizing glass-forming MIOC crystals. In detail, the principle of this approach is based on the creation of hydrogen-bonded structural network by substituting acid anions for imidazole or benzimidazole ligands in the tetrahedral units of zeolitic imidazolate framework crystals. By tuning the metal centers, anions, and organic ligands of MIOCs, supramolecular unit structures can be designed to construct supramolecular networks and thereby enable property modulation. Furthermore, mixed-ligand synthesis yielded a mixed-crystal system in which the glass-transition temperature (Tg) can be linearly tuned from 282 K to 360 K through gradual substitution of benzimidazole for imidazole. Interestingly, upon vitrification, MIOCs were observed to undergo reorganization of hydrogen-bonded networks, with retention of tetrahedral units, short-range disorder, and the freezing of multiple conformations. This work offers a new strategy to systematically expand the glass-forming compositional range of MIOCs and to develop functional MIOC glasses.

cond-mat.mtrl-sci↗

arXiv2025

Discretization form of the continuity condition at the polar axis, with application to the gyrokinetic simulation in a magnetic fusion torus

A new computational method to solve the hyperbolic (Vlasov) equation and the elliptic (Poisson-like) equation at the polar axis is proposed. It is shown that the value of a scalar function at the polar axis can be predicted by its neighbouring values based on the continuity condition. This continuity condition systematically solves the pole problems including the singular factor 1/r in the hyperbolic equation and the inner boundary in the elliptic equation. The proposed method is applied to the global gyrokinetic simulation of the tokamak plasma with the magnetic axis included.

physics.comp-ph↗

arXiv2025

Comb-Driven Coherent Optical Transmitter for Scalable DWDM Interconnects

Driven by the growing demand for large-scale artificial intelligence applications, disaggregated compute nodes and high-radix switches in next-generation computing clusters are set to surpass the capacity of current optical interconnect technologies. Such a surge turns several aspects of transmitters into critical bottlenecks: shoreline bandwidth density and energy efficiency are effectively limiting the scalability. We present a comb-driven coherent optical transmitter architecture on a Si/SiN platform that provides the bandwidth density, energy efficiency, and compact footprint required for such co-packaged-enabled optical interconnects. We evaluate scalability through critical building blocks, including ultra-compact microring-assisted Mach--Zehnder modulators (MRA-MZMs) and dense wavelength-division multiplexing (DWDM) interleavers. Single-tone experiments demonstrate a net line rate of 400 Gbps per polarization (16-QAM, 120 GBd) in silicon within the O-band, achieving a record shoreline density of 4 Tbps/mm while consuming only 10 fJ/bit for modulation. We also demonstrate transmission rates of up to 160 GBd QPSK in back-to-back and 100 GBd over 7 km of fiber without dispersion compensation. Using a quantum-dot frequency comb, six 100 GHz-spaced WDM channels transmit 1.08 Tbps over 5 km. System-level analyses show that by leveraging advanced modulation formats through the integration of wavelength and polarization multiplexing, our proposed architecture can realistically support combined transmission rates exceeding 10 Tbps per fiber within practical limits of power consumption and packaging, outlining a clear path toward future petabit-scale interconnects.

physics.optics↗

arXiv2025

Losing is for Cherishing: Data Valuation Based on Machine Unlearning and Shapley Value

The proliferation of large models has intensified the need for efficient data valuation methods to quantify the contribution of individual data providers. Traditional approaches, such as game-theory-based Shapley value and influence-function-based techniques, face prohibitive computational costs or require access to full data and model training details, making them hardly achieve partial data valuation. To address this, we propose Unlearning Shapley, a novel framework that leverages machine unlearning to estimate data values efficiently. By unlearning target data from a pretrained model and measuring performance shifts on a reachable test set, our method computes Shapley values via Monte Carlo sampling, avoiding retraining and eliminating dependence on full data. Crucially, Unlearning Shapley supports both full and partial data valuation, making it scalable for large models (e.g., LLMs) and practical for data markets. Experiments on benchmark datasets and large-scale text corpora demonstrate that our approach matches the accuracy of state-of-the-art methods while reducing computational overhead by orders of magnitude. Further analysis confirms a strong correlation between estimated values and the true impact of data subsets, validating its reliability in real-world scenarios. This work bridges the gap between data valuation theory and practical deployment, offering a scalable, privacy-compliant solution for modern AI ecosystems.

cs.AI↗

arXiv2025

AIM 2025 challenge on Inverse Tone Mapping Report: Methods and Results

This paper presents a comprehensive review of the AIM 2025 Challenge on Inverse Tone Mapping (ITM). The challenge aimed to push forward the development of effective ITM algorithms for HDR image reconstruction from single LDR inputs, focusing on perceptual fidelity and numerical consistency. A total of \textbf{67} participants submitted \textbf{319} valid results, from which the best five teams were selected for detailed analysis. This report consolidates their methodologies and performance, with the lowest PU21-PSNR among the top entries reaching 29.22 dB. The analysis highlights innovative strategies for enhancing HDR reconstruction quality and establishes strong benchmarks to guide future research in inverse tone mapping.

cs.CV↗

arXiv2025

LogiDynamics: Unraveling the Dynamics of Inductive, Abductive and Deductive Logical Inferences in LLM Reasoning

Modern large language models (LLMs) employ diverse logical inference mechanisms for reasoning, making the strategic optimization of these approaches critical for advancing their capabilities. This paper systematically investigate the comparative dynamics of inductive (System 1) versus abductive/deductive (System 2) inference in LLMs. We utilize a controlled analogical reasoning environment, varying modality (textual, visual, symbolic), difficulty, and task format (MCQ / free-text). Our analysis reveals System 2 pipelines generally excel, particularly in visual/symbolic modalities and harder tasks, while System 1 is competitive for textual and easier problems. Crucially, task format significantly influences their relative advantage, with System 1 sometimes outperforming System 2 in free-text rule-execution. These core findings generalize to broader in-context learning. Furthermore, we demonstrate that advanced System 2 strategies like hypothesis selection and iterative refinement can substantially scale LLM reasoning. This study offers foundational insights and actionable guidelines for strategically deploying logical inference to enhance LLM reasoning. Resources are available at https://github.com/HKUST-KnowComp/LogiDynamics.

cs.CL↗

arXiv2025

From Automation to Autonomy: A Survey on Large Language Models in Scientific Discovery

Large Language Models (LLMs) are catalyzing a paradigm shift in scientific discovery, evolving from task-specific automation tools into increasingly autonomous agents and fundamentally redefining research processes and human-AI collaboration. This survey systematically charts this burgeoning field, placing a central focus on the changing roles and escalating capabilities of LLMs in science. Through the lens of the scientific method, we introduce a foundational three-level taxonomy-Tool, Analyst, and Scientist-to delineate their escalating autonomy and evolving responsibilities within the research lifecycle. We further identify pivotal challenges and future research trajectories such as robotic automation, self-improvement, and ethical governance. Overall, this survey provides a conceptual architecture and strategic foresight to navigate and shape the future of AI-driven scientific discovery, fostering both rapid innovation and responsible advancement. Github Repository: https://github.com/HKUST-KnowComp/Awesome-LLM-Scientific-Discovery.

cs.CL↗

arXiv2025

YuE: Scaling Open Foundation Models for Long-Form Music Generation

We tackle the task of long-form music generation--particularly the challenging \textbf{lyrics-to-song} problem--by introducing YuE, a family of open foundation models based on the LLaMA2 architecture. Specifically, YuE scales to trillions of tokens and generates up to five minutes of music while maintaining lyrical alignment, coherent musical structure, and engaging vocal melodies with appropriate accompaniment. It achieves this through (1) track-decoupled next-token prediction to overcome dense mixture signals, (2) structural progressive conditioning for long-context lyrical alignment, and (3) a multitask, multiphase pre-training recipe to converge and generalize. In addition, we redesign the in-context learning technique for music generation, enabling versatile style transfer (e.g., converting Japanese city pop into an English rap while preserving the original accompaniment) and bidirectional generation. Through extensive evaluation, we demonstrate that YuE matches or even surpasses some of the proprietary systems in musicality and vocal agility. In addition, fine-tuning YuE enables additional controls and enhanced support for tail languages. Furthermore, beyond generation, we show that YuE's learned representations can perform well on music understanding tasks, where the results of YuE match or exceed state-of-the-art methods on the MARBLE benchmark. Keywords: lyrics2song, song generation, long-form, foundation model, music generation

eess.AS↗