arXiv subjects

Zhe Hu

Publications and source records attributed to Zhe Hu.

At least 37 records · Page 2Linked to original sources

arXiv2026

Guided by the Plan: Enhancing Faithful Autoregressive Text-to-Audio Generation with Guided Decoding

Autoregressive (AR) models excel at generating temporally coherent audio by producing tokens sequentially, yet they often falter in faithfully following complex textual prompts, especially those describing complex sound events. We uncover a surprising capability in AR audio generators: their early prefix tokens implicitly encode global semantic attributes of the final output, such as event count and sound-object category, revealing a form of implicit planning. Building on this insight, we propose Plan-Critic, a lightweight auxiliary model trained with a Generalized Advantage Estimation (GAE)-inspired objective to predict final instruction-following quality from partial generations. At inference time, Plan-Critic enables guided exploration: it evaluates candidate prefixes early, prunes low-fidelity trajectories, and reallocates computation to high-potential planning seeds. Our Plan-Critic-guided sampling achieves up to a 10-point improvement in CLAP score over the AR baseline-establishing a new state of the art in AR text-to-audio generation-while maintaining computational parity with standard best-of-N decoding. This work bridges the gap between causal generation and global semantic alignment, demonstrating that even strictly autoregressive models can plan ahead.

cs.CL↗

arXiv2025

Constraint-Free Coherent Diffraction Imaging via Physics-Guided Neural Fields

CDI is a lensless imaging technique that enables atomic-resolution imaging of non-crystalline specimens and their dynamics. However, its broader implementation has been hindered by the instability and ill-posedness of its reconstruction process, known as phase retrieval, which relies heavily on handcrafted, object-specific constraints. To overcome the key limitations, we propose CDIP, a robust phase-retrieval framework that eliminates the need for such constraints by combining untrained coordinate-based neural fields for static and dynamic reconstructions and a physics-consistent forward model. We evaluate CDIP on simulated and experimental datasets that involve both static samples and dynamic processes, demonstrating that it substantially outperforms classical iterative algorithms and deep-learning baselines in terms of fidelity and stability. These results highlight a paradigm shift in both static and time-resolved CDI reconstruction, providing a broadly applicable framework for coherent imaging modalities such as ptychography and holography, across X-ray, electron, and optical probes.

physics.optics↗

arXiv2025

Time-resolved 3D imaging opportunities with XMPI at ForMAX

X-rays are commonly used in imaging experiments due to their penetration power, which enables non-destructive resolution of internal structures in samples that are opaque to visible light. Time-resolved X-ray tomography is the state-of-the-art method for obtaining volumetric 4D (3D + time) information by rotating the sample and acquiring projections from different angular viewpoints over time. This method enables studies to address a plethora of research questions across various scientific disciplines. However, it faces several limitations, such as incompatibility with single-shot experiments, challenges in rotating complex sample environments that restrict the achievable rotation speed or range, and the introduction of centrifugal forces that can affect the sample's dynamics. These limitations can hinder and even preclude the study of certain dynamics. Here, we present an implementation of an alternative approach, X-ray Multi-Projection Imaging (XMPI), which eliminates the need for sample rotation. Instead, the direct incident X-ray beam is split into beamlets using beam splitting X-ray optics. These beamlets intersect at the sample position from different angular viewpoints, allowing multiple projections to be acquired simultaneously. We commissioned this setup at the ForMAX beamline at MAX IV. We present projections acquired from two different sample systems - fibers under mechanical load and particle suspension in multi-phase flow - with distinct spatial and temporal resolution requirements. We demonstrate the capabilities of the ForMAX XMPI setup using the detector's full dynamical range for the relevant sample-driven spatiotemporal resolutions: i) at least 12.5 kHz framerates with 4 micrometer pixel sizes (fibers) and ii) 40 Hz acquisitions with 1.3 micrometer pixel sizes (multi-phase flows), setting the basis for a permanent XMPI endstation at ForMAX.

physics.app-ph↗

arXiv2025

Exploring Scale Shift in Crowd Localization under the Context of Domain Generalization

Crowd localization plays a crucial role in visual scene understanding towards predicting each pedestrian location in a crowd, thus being applicable to various downstream tasks. However, existing approaches suffer from significant performance degradation due to discrepancies in head scale distributions (scale shift) between training and testing data, a challenge known as domain generalization (DG). This paper aims to comprehend the nature of scale shift within the context of domain generalization for crowd localization models. To this end, we address four critical questions: (i) How does scale shift influence crowd localization in a DG scenario? (ii) How can we quantify this influence? (iii) What causes this influence? (iv) How to mitigate the influence? Initially, we conduct a systematic examination of how crowd localization performance varies with different levels of scale shift. Then, we establish a benchmark, ScaleBench, and reproduce 20 advanced DG algorithms to quantify the influence. Through extensive experiments, we demonstrate the limitations of existing algorithms and underscore the importance and complexity of scale shift, a topic that remains insufficiently explored. To deepen our understanding, we provide a rigorous theoretical analysis on scale shift. Building on these insights, we further propose an effective algorithm called Causal Feature Decomposition and Anisotropic Processing (Catto) to mitigate the influence of scale shift in DG settings. Later, we also provide extensive analytical experiments, revealing four significant insights for future research. Our results emphasize the importance of this novel and applicable research direction, which we term Scale Shift Domain Generalization.

cs.CV↗

arXiv2025

Praxis-VLM: Vision-Grounded Decision Making via Text-Driven Reinforcement Learning

Vision Language Models exhibit impressive performance for various tasks, yet they often lack the sophisticated situational reasoning required for complex decision-making. This paper shows that VLMs can achieve surprisingly strong decision-making performance when visual scenes are replaced by textual descriptions, suggesting foundational reasoning can be effectively learned from language. Motivated by this insight, we propose Praxis-VLM, a reasoning VLM for vision-grounded decision-making. Praxis-VLM employs the GRPO algorithm on textual scenarios to instill robust reasoning capabilities, where models learn to evaluate actions and their consequences. These reasoning skills, acquired purely from text, successfully transfer to multimodal inference with visual inputs, significantly reducing reliance on scarce paired image-text training data. Experiments across diverse decision-making benchmarks demonstrate that Praxis-VLM substantially outperforms standard supervised fine-tuning, exhibiting superior performance and generalizability. Further analysis confirms that our models engage in explicit and effective reasoning, underpinning their enhanced performance and adaptability.

cs.CL↗

arXiv2025

Language Model Based Text-to-Audio Generation: Anti-Causally Aligned Collaborative Residual Transformers

While language models (LMs) paired with residual vector quantization (RVQ) tokenizers have shown promise in text-to-audio (T2A) generation, they still lag behind diffusion-based models by a non-trivial margin. We identify a critical dilemma underpinning this gap: incorporating more RVQ layers improves audio reconstruction fidelity but exceeds the generation capacity of conventional LMs. To address this, we first analyze RVQ dynamics and uncover two key limitations: 1) orthogonality of features across RVQ layers hinders effective LMs training, and 2) descending semantic richness in tokens from deeper RVQ layers exacerbates exposure bias during autoregressive decoding. Based on these insights, we propose Siren, a novel LM-based framework that employs multiple isolated transformers with causal conditioning and anti-causal alignment via reinforcement learning. Extensive experiments demonstrate that Siren outperforms both existing LM-based and diffusion-based T2A systems, achieving state-of-the-art results. By bridging the representational strengths of LMs with the fidelity demands of audio synthesis, our approach repositions LMs as competitive contenders against diffusion models in T2A tasks. Moreover, by aligning audio representations with linguistic structures, Siren facilitates a promising pathway toward unified multi-modal generation frameworks.

cs.SD↗

arXiv2025

Authenticated Private Set Intersection: A Merkle Tree-Based Approach for Enhancing Data Integrity

Private Set Intersection (PSI) enables secure computation of set intersections while preserving participant privacy, standard PSI existing protocols remain vulnerable to data integrity attacks allowing malicious participants to extract additional intersection information or mislead other parties. In this paper, we propose the definition of data integrity in PSI and construct two authenticated PSI schemes by integrating Merkle Trees with state-of-the-art two-party volePSI and multi-party mPSI protocols. The resulting two-party authenticated PSI achieves communication complexity $\mathcal{O}(n λ+n \log n)$, aligning with the best-known unauthenticated PSI schemes, while the multi-party construction is $\mathcal{O}(n κ+n \log n)$ which introduces additional overhead due to Merkle tree inclusion proofs. Due to the incorporation of integrity verification, our authenticated schemes incur higher costs compared to state-of-the-art unauthenticated schemes. We also provide efficient implementations of our protocols and discuss potential improvements, including alternative authentication blocks.

cs.CR↗

arXiv2025

VIVA+: Human-Centered Situational Decision-Making

Multimodal Large Language Models (MLLMs) show promising results for embodied agents in operating meaningfully in complex, human-centered environments. Yet, evaluating their capacity for nuanced, human-like reasoning and decision-making remains challenging. In this work, we introduce VIVA+, a cognitively grounded benchmark for evaluating the reasoning and decision-making of MLLMs in human-centered situations. VIVA+ consists of 1,317 real-world situations paired with 6,373 multiple-choice questions, targeting three core abilities for decision-making: (1) Foundational Situation Comprehension, (2) Context-Driven Action Justification, and (3) Reflective Reasoning. Together, these dimensions provide a systematic framework for assessing a model's ability to perceive, reason, and act in socially meaningful ways. We evaluate the latest commercial and open-source models on VIVA+, where we reveal distinct performance patterns and highlight significant challenges. We further explore targeted training and multi-step reasoning strategies, which yield consistent performance improvements. Finally, our in-depth analysis highlights current model limitations and provides actionable insights for advancing MLLMs toward more robust, context-aware, and socially adept decision-making in real-world settings.

cs.CL↗

arXiv2025

Authenticated Sublinear Quantum Private Information Retrieval

This paper introduces a novel lower bound on communication complexity using quantum relative entropy and mutual information, refining previous classical entropy-based results. By leveraging Uhlmann's lemma and quantum Pinsker inequalities, the authors establish tighter bounds for information-theoretic security, demonstrating that quantum protocols inherently outperform classical counterparts in balancing privacy and efficiency. Also explores symmetric Quantum Private Information Retrieval (QPIR) protocols that achieve sub-linear communication complexity while ensuring robustness against specious adversaries: A post-quantum cryptography based protocol that can be authenticated for the specious server; A ring-LWE-based protocol for post-quantum security in a single-server setting, ensuring robustness against quantum attacks; A multi-server protocol optimized for hardware practicality, reducing implementation overhead while maintaining sub-linear efficiency. These protocols address critical gaps in secure database queries, offering exponential communication improvements over classical linear-complexity methods. The work also analyzes security trade-offs under quantum specious adversaries, providing theoretical guarantees for privacy and correctness.

quant-ph↗

arXiv2025

Super time-resolved tomography

Understanding 3D fundamental processes is crucial for academic and industrial applications. Nowadays, X-ray time-resolved tomography, or tomoscopy, is a leading technique for in-situ and operando 4D (3D+time) characterization. Despite its ability to achieve 1000 tomograms per second at large-scale X-ray facilities, its applicability is limited by the centrifugal forces exerted on samples and the challenges of developing suitable environments for such high-speed studies. Here, we introduce STRT, an approach that has the potential to enhance the temporal resolution of tomoscopy by at least an order of magnitude while preserving spatial resolution. STRT exploits a 4D DL reconstruction algorithm to produce high-fidelity 3D reconstructions at each time point, retrieved from a significantly reduced angular range of a few degrees compared to the 0-180 degrees of traditional tomoscopy. Thus, STRT enhances the temporal resolution compared to tomoscopy by a factor equal to the ratio between 180 degrees and the angular ranges used by STRT. In this work, we validate the 4D capabilities of STRT through simulations and experiments on droplet collision simulations and additive manufacturing processes. We anticipate that STRT will significantly expand the capabilities of 4D X-ray imaging, enabling previously unattainable studies in both academic and industrial contexts, such as materials formation and mechanical testing.

physics.optics↗

arXiv2025

Physics-informed 4D X-ray image reconstruction from ultra-sparse spatiotemporal data

The unprecedented X-ray flux density provided by modern X-ray sources offers new spatiotemporal possibilities for X-ray imaging of fast dynamic processes. Approaches to exploit such possibilities often result in either i) a limited number of projections or spatial information due to limited scanning speed, as in time-resolved tomography, or ii) a limited number of time points, as in stroboscopic imaging, making the reconstruction problem ill-posed and unlikely to be solved by classical reconstruction approaches. 4D reconstruction from such data requires sample priors, which can be included via deep learning (DL). State-of-the-art 4D reconstruction methods for X-ray imaging combine the power of AI and the physics of X-ray propagation to tackle the challenge of sparse views. However, most approaches do not constrain the physics of the studied process, i.e., a full physical model. Here we present 4D physics-informed optimized neural implicit X-ray imaging (4D-PIONIX), a novel physics-informed 4D X-ray image reconstruction method combining the full physical model and a state-of-the-art DL-based reconstruction method for 4D X-ray imaging from sparse views. We demonstrate and evaluate the potential of our approach by retrieving 4D information from ultra-sparse spatiotemporal acquisitions of simulated binary droplet collisions, a relevant fluid dynamic process. We envision that this work will open new spatiotemporal possibilities for various 4D X-ray imaging modalities, such as time-resolved X-ray tomography and more novel sparse acquisition approaches like X-ray multi-projection imaging, which will pave the way for investigations of various rapid 4D dynamics, such as fluid dynamics and composite testing.

eess.IV↗

arXiv2025

Synchronized Video-to-Audio Generation via Mel Quantization-Continuum Decomposition

Video-to-audio generation is essential for synthesizing realistic audio tracks that synchronize effectively with silent videos. Following the perspective of extracting essential signals from videos that can precisely control the mature text-to-audio generative diffusion models, this paper presents how to balance the representation of mel-spectrograms in terms of completeness and complexity through a new approach called Mel Quantization-Continuum Decomposition (Mel-QCD). We decompose the mel-spectrogram into three distinct types of signals, employing quantization or continuity to them, we can effectively predict them from video by a devised video-to-all (V2X) predictor. Then, the predicted signals are recomposed and fed into a ControlNet, along with a textual inversion design, to control the audio generation process. Our proposed Mel-QCD method demonstrates state-of-the-art performance across eight metrics, evaluating dimensions such as quality, synchronization, and semantic consistency. Our codes and demos will be released at \href{Website}{https://wjc2830.github.io/MelQCD/}.

cs.SD↗

arXiv2025

Debate-to-Write: A Persona-Driven Multi-Agent Framework for Diverse Argument Generation

Writing persuasive arguments is a challenging task for both humans and machines. It entails incorporating high-level beliefs from various perspectives on the topic, along with deliberate reasoning and planning to construct a coherent narrative. Current language models often generate surface tokens autoregressively, lacking explicit integration of these underlying controls, resulting in limited output diversity and coherence. In this work, we propose a persona-based multi-agent framework for argument writing. Inspired by the human debate, we first assign each agent a persona representing its high-level beliefs from a unique perspective, and then design an agent interaction process so that the agents can collaboratively debate and discuss the idea to form an overall plan for argument writing. Such debate process enables fluid and nonlinear development of ideas. We evaluate our framework on argumentative essay writing. The results show that our framework can generate more diverse and persuasive arguments through both automatic and human evaluations.

cs.CL↗

arXiv2024

VIVA: A Benchmark for Vision-Grounded Decision-Making with Human Values

Large vision language models (VLMs) have demonstrated significant potential for integration into daily life, making it crucial for them to incorporate human values when making decisions in real-world situations. This paper introduces VIVA, a benchmark for VIsion-grounded decision-making driven by human VAlues. While most large VLMs focus on physical-level skills, our work is the first to examine their multimodal capabilities in leveraging human values to make decisions under a vision-depicted situation. VIVA contains 1,240 images depicting diverse real-world situations and the manually annotated decisions grounded in them. Given an image there, the model should select the most appropriate action to address the situation and provide the relevant human values and reason underlying the decision. Extensive experiments based on VIVA show the limitation of VLMs in using human values to make multimodal decisions. Further analyses indicate the potential benefits of exploiting action consequences and predicted human values.

cs.CL↗

arXiv2024

AMERICANO: Argument Generation with Discourse-driven Decomposition and Agent Interaction

Argument generation is a challenging task in natural language processing, which requires rigorous reasoning and proper content organization. Inspired by recent chain-of-thought prompting that breaks down a complex task into intermediate steps, we propose Americano, a novel framework with agent interaction for argument generation. Our approach decomposes the generation process into sequential actions grounded on argumentation theory, which first executes actions sequentially to generate argumentative discourse components, and then produces a final argument conditioned on the components. To further mimic the human writing process and improve the left-to-right generation paradigm of current autoregressive language models, we introduce an argument refinement module which automatically evaluates and refines argument drafts based on feedback received. We evaluate our framework on the task of counterargument generation using a subset of Reddit/CMV dataset. The results show that our method outperforms both end-to-end and chain-of-thought prompting methods and can generate more coherent and persuasive arguments with diverse and rich contents.

cs.CL↗

arXiv2024

Language-Augmented Symbolic Planner for Open-World Task Planning

Enabling robotic agents to perform complex long-horizon tasks has been a long-standing goal in robotics and artificial intelligence (AI). Despite the potential shown by large language models (LLMs), their planning capabilities remain limited to short-horizon tasks and they are unable to replace the symbolic planning approach. Symbolic planners, on the other hand, may encounter execution errors due to their common assumption of complete domain knowledge which is hard to manually prepare for an open-world setting. In this paper, we introduce a Language-Augmented Symbolic Planner (LASP) that integrates pre-trained LLMs to enable conventional symbolic planners to operate in an open-world environment where only incomplete knowledge of action preconditions, objects, and properties is initially available. In case of execution errors, LASP can utilize the LLM to diagnose the cause of the error based on the observation and interact with the environment to incrementally build up its knowledge base necessary for accomplishing the given tasks. Experiments demonstrate that LASP is proficient in solving planning problems in the open-world setting, performing well even in situations where there are multiple gaps in the knowledge.

cs.RO↗

arXiv2024

SA-MDKIF: A Scalable and Adaptable Medical Domain Knowledge Injection Framework for Large Language Models

Recent advances in large language models (LLMs) have demonstrated exceptional performance in various natural language processing (NLP) tasks. However, their effective application in the medical domain is hampered by a lack of medical domain knowledge. In this study, we present SA-MDKIF, a scalable and adaptable framework that aims to inject medical knowledge into general-purpose LLMs through instruction tuning, thereby enabling adaptability for various downstream tasks. SA-MDKIF consists of two stages: skill training and skill adaptation. In the first stage, we define 12 basic medical skills and use AdaLoRA to train these skills based on uniformly formatted instructional datasets that we have constructed. In the next stage, we train the skill router using task-specific downstream data and use this router to integrate the acquired skills with LLMs during inference. Experimental results on 9 different medical tasks show that SA-MDKIF improves performance by 10-20% compared to the original LLMs. Notably, this improvement is particularly pronounced for unseen medical tasks, showing an improvement of up to 30%.

cs.CL↗

arXiv2023

Minimal-time Deadbeat Consensus and Individual Disagreement Degree Prediction for High-order Linear Multi-agent Systems

In this paper, a Hankel matrix-based fully distributed algorithm is proposed to address a minimal-time deadbeat consensus prediction problem for discrete-time high-order multi-agent systems (MASs). Therein, each agent can predict the consensus value with the minimum number of observable historical outputs of its own. Accordingly, compared to most existing algorithms only yielding asymptotic convergence, the present method can attain deadbeat consensus instead. Moreover, based on the consensus value prediction, instant individual disagreement degree value of MASs can be calculated in advance as well. Sufficient conditions are derived to guarantee both the minimal-time deadbeat consensus and the instant individual disagreement degree prediction. Finally, both the effectiveness and superiority of the proposed deadbeat consensus algorithm are substantiated by numerical simulations.

eess.SY↗