arXiv subjects

Wenhao Sun

Publications and source records attributed to Wenhao Sun.

At least 37 records · Page 2Linked to original sources

arXiv2025

SPAZER: Spatial-Semantic Progressive Reasoning Agent for Zero-shot 3D Visual Grounding

3D Visual Grounding (3DVG) aims to localize target objects within a 3D scene based on natural language queries. To alleviate the reliance on costly 3D training data, recent studies have explored zero-shot 3DVG by leveraging the extensive knowledge and powerful reasoning capabilities of pre-trained LLMs and VLMs. However, existing paradigms tend to emphasize either spatial (3D-based) or semantic (2D-based) understanding, limiting their effectiveness in complex real-world applications. In this work, we introduce SPAZER - a VLM-driven agent that combines both modalities in a progressive reasoning framework. It first holistically analyzes the scene and produces a 3D rendering from the optimal viewpoint. Based on this, anchor-guided candidate screening is conducted to perform a coarse-level localization of potential objects. Furthermore, leveraging retrieved relevant 2D camera images, 3D-2D joint decision-making is efficiently performed to determine the best-matching object. By bridging spatial and semantic reasoning neural streams, SPAZER achieves robust zero-shot grounding without training on 3D-labeled data. Extensive experiments on ScanRefer and Nr3D benchmarks demonstrate that SPAZER significantly outperforms previous state-of-the-art zero-shot methods, achieving notable gains of 9.0% and 10.9% in accuracy.

cs.CV↗

arXiv2025

Heavier chalcogenofenchones for fundamental gas-phase studies of molecular chirality

Monoterpene ketones are frequently studied compounds that enjoy great popularity both in chemistry and in physics due to comparatively high volatility, stability, conformational rigidity and commercial availability. Herein, we explore the heavier chalcogenoketone derivatives of fenchone as promising benchmark systems -- synthetically accessible in enantiomerically pure form -- for systematic studies of nuclear charge ($Z$) dependent properties in chiral compounds. Synthesis, structural characterization, thorough gas-phase rotational and vibrational spectroscopy as well as accompanying quantum chemical studies on the density-functional-theory level reported in this work foreshadow subsequent applications of this compound class for fundamental investigations of molecular chirality under well-defined conditions.

physics.chem-ph↗

arXiv2025

Multimodal Reasoning Agent for Zero-Shot Composed Image Retrieval

Zero-Shot Composed Image Retrieval (ZS-CIR) aims to retrieve target images given a compositional query, consisting of a reference image and a modifying text-without relying on annotated training data. Existing approaches often generate a synthetic target text using large language models (LLMs) to serve as an intermediate anchor between the compositional query and the target image. Models are then trained to align the compositional query with the generated text, and separately align images with their corresponding texts using contrastive learning. However, this reliance on intermediate text introduces error propagation, as inaccuracies in query-to-text and text-to-image mappings accumulate, ultimately degrading retrieval performance. To address these problems, we propose a novel framework by employing a Multimodal Reasoning Agent (MRA) for ZS-CIR. MRA eliminates the dependence on textual intermediaries by directly constructing triplets, , using only unlabeled image data. By training on these synthetic triplets, our model learns to capture the relationships between compositional queries and candidate images directly. Extensive experiments on three standard CIR benchmarks demonstrate the effectiveness of our approach. On the FashionIQ dataset, our method improves Average R@10 by at least 7.5\% over existing baselines; on CIRR, it boosts R@1 by 9.6\%; and on CIRCO, it increases mAP@5 by 9.5\%.

cs.CV↗

arXiv2025

AsymRnR: Video Diffusion Transformers Acceleration with Asymmetric Reduction and Restoration

Diffusion Transformers (DiTs) have proven effective in generating high-quality videos but are hindered by high computational costs. Existing video DiT sampling acceleration methods often rely on costly fine-tuning or exhibit limited generalization capabilities. We propose Asymmetric Reduction and Restoration (AsymRnR), a training-free and model-agnostic method to accelerate video DiTs. It builds on the observation that redundancies of feature tokens in DiTs vary significantly across different model blocks, denoising steps, and feature types. Our AsymRnR asymmetrically reduces redundant tokens in the attention operation, achieving acceleration with negligible degradation in output quality and, in some cases, even improving it. We also tailored a reduction schedule to distribute the reduction across components adaptively. To further accelerate this process, we introduce a matching cache for more efficient reduction. Backed by theoretical foundations and extensive experimental validation, AsymRnR integrates into state-of-the-art video DiTs and offers substantial speedup.

cs.CV↗

arXiv2025

Direct observation of time-dependent coherent chiral tunneling dynamics

Superpositions of handed molecular states give rise to achiral eigenstates, delocalized across a double-well potential via tunneling. A coherent superposition of these energy eigenstates could dynamically relocalize the molecules into chiral states, which has only been addressed theoretically. Here, we present a microwave six-wave mixing pump-probe study to create and probe coherent chiral tunneling dynamics in a rotational state. Through a time-resolved scheme, we uncover the periodic time evolution of the induced chiral wavepacket under field-free conditions. Moreover, we demonstrate precise phase control of this coherence via phase modulation during pump excitation.

quant-ph↗

arXiv2025

KARMA: Augmenting Embodied AI Agents with Long-and-short Term Memory Systems

Embodied AI agents responsible for executing interconnected, long-sequence household tasks often face difficulties with in-context memory, leading to inefficiencies and errors in task execution. To address this issue, we introduce KARMA, an innovative memory system that integrates long-term and short-term memory modules, enhancing large language models (LLMs) for planning in embodied agents through memory-augmented prompting. KARMA distinguishes between long-term and short-term memory, with long-term memory capturing comprehensive 3D scene graphs as representations of the environment, while short-term memory dynamically records changes in objects' positions and states. This dual-memory structure allows agents to retrieve relevant past scene experiences, thereby improving the accuracy and efficiency of task planning. Short-term memory employs strategies for effective and adaptive memory replacement, ensuring the retention of critical information while discarding less pertinent data. Compared to state-of-the-art embodied agents enhanced with memory, our memory-augmented embodied AI agent improves success rates by 1.3x and 2.3x in Composite Tasks and Complex Tasks within the AI2-THOR simulator, respectively, and enhances task execution efficiency by 3.4x and 62.7x. Furthermore, we demonstrate that KARMA's plug-and-play capability allows for seamless deployment on real-world robotic systems, such as mobile manipulation platforms.Through this plug-and-play memory system, KARMA significantly enhances the ability of embodied agents to generate coherent and contextually appropriate plans, making the execution of complex household tasks more efficient. The experimental videos from the work can be found at https://youtu.be/4BT7fnw9ehs. Our code is available at https://github.com/WZX0Swarm0Robotics/KARMA/tree/master.

cs.RO↗

arXiv2025

Paradigm-Based Automatic HDL Code Generation Using LLMs

While large language models (LLMs) have demonstrated the ability to generate hardware description language (HDL) code for digital circuits, they still face the hallucination problem, which can result in the generation of incorrect HDL code or misinterpretation of specifications. In this work, we introduce a human-expert-inspired method to mitigate the hallucination of LLMs and enhance their performance in HDL code generation. We begin by constructing specialized paradigm blocks that consist of several steps designed to divide and conquer generation tasks, mirroring the design methodology of human experts. These steps include information extraction, human-like design flows, and the integration of external tools. LLMs are then instructed to classify the type of circuit in order to match it with the appropriate paradigm block and execute the block to generate the HDL codes. Additionally, we propose a two-phase workflow for multi-round generation, aimed at effectively improving the testbench pass rate of the generated HDL codes within a limited number of generation and verification rounds. Experimental results demonstrate that our method significantly enhances the functional correctness of the generated Verilog code

cs.PL↗

arXiv2024

Why does silicon have an indirect band gap?

It is difficult to intuit how electronic structure features$-$such as band gap magnitude, location of band extrema, effective masses, etc.$-$arise from the underlying crystal chemistry of a material. Here we present a strategy to distill sparse and chemically-interpretable tight-binding models from density functional theory calculations, enabling us to interpret how multiple orbital interactions in a 3D crystal conspire to shape the overall band structure. Applying this process to silicon, we show that its indirect gap arises from a competition between first and second nearest-neighbor bonds$-$where second nearest-neighbor interactions pull the conduction band down from $Γ$ to $X$ in a cosine shape, but the first nearest-neighbor bonds push the band up near $X$, resulting in the characteristic dip of the silicon conduction band. By identifying the essential orbital interactions that shape the conduction band, we can further rationally tune bond strengths to morph the silicon band structure into the germanium band structure. Our computational approach serves as a general framework to extract the crystal chemistry origins of electronic structure features from density functional theory calculations, enabling a new paradigm of bonding-by-design.

cond-mat.mtrl-sci↗

arXiv2024

DaDu-E: Rethinking the Role of Large Language Model in Robotic Computing Pipeline

Performing complex tasks in open environments remains challenging for robots, even when using large language models (LLMs) as the core planner. Many LLM-based planners are inefficient due to their large number of parameters and prone to inaccuracies because they operate in open-loop systems. We think the reason is that only applying LLMs as planners is insufficient. In this work, we propose DaDu-E, a robust closed-loop planning framework for embodied AI robots. Specifically, DaDu-E is equipped with a relatively lightweight LLM, a set of encapsulated robot skill instructions, a robust feedback system, and memory augmentation. Together, these components enable DaDu-E to (i) actively perceive and adapt to dynamic environments, (ii) optimize computational costs while maintaining high performance, and (iii) recover from execution failures using its memory and feedback mechanisms. Extensive experiments on real-world and simulated tasks show that DaDu-E achieves task success rates comparable to embodied AI robots with larger models as planners like COME-Robot, while reducing computational requirements by $6.6 \times$. Users are encouraged to explore our system at: \url{https://rlc-lab.github.io/dadu-e/}.

cs.RO↗

arXiv2024

SPAgent: Adaptive Task Decomposition and Model Selection for General Video Generation and Editing

While open-source video generation and editing models have made significant progress, individual models are typically limited to specific tasks, failing to meet the diverse needs of users. Effectively coordinating these models can unlock a wide range of video generation and editing capabilities. However, manual coordination is complex and time-consuming, requiring users to deeply understand task requirements and possess comprehensive knowledge of each model's performance, applicability, and limitations, thereby increasing the barrier to entry. To address these challenges, we propose a novel video generation and editing system powered by our Semantic Planning Agent (SPAgent). SPAgent bridges the gap between diverse user intents and the effective utilization of existing generative models, enhancing the adaptability, efficiency, and overall quality of video generation and editing. Specifically, the SPAgent assembles a tool library integrating state-of-the-art open-source image and video generation and editing models as tools. After fine-tuning on our manually annotated dataset, SPAgent can automatically coordinate the tools for video generation and editing, through our novelly designed three-step framework: (1) decoupled intent recognition, (2) principle-guided route planning, and (3) capability-based execution model selection. Additionally, we enhance the SPAgent's video quality evaluation capability, enabling it to autonomously assess and incorporate new video generation and editing models into its tool library without human intervention. Experimental results demonstrate that the SPAgent effectively coordinates models to generate or edit videos, highlighting its versatility and adaptability across various video tasks.

cs.CV↗

arXiv2024

Classification-Based Automatic HDL Code Generation Using LLMs

While large language models (LLMs) have demonstrated the ability to generate hardware description language (HDL) code for digital circuits, they still suffer from the hallucination problem, which leads to the generation of incorrect HDL code or misunderstanding of specifications. In this work, we introduce a human-expert-inspired method to mitigate the hallucination of LLMs and improve the performance in HDL code generation. We first let LLMs classify the type of the circuit based on the specifications. Then, according to the type of the circuit, we split the tasks into several sub-procedures, including information extraction and human-like design flow using Electronic Design Automation (EDA) tools. Besides, we also use a search method to mitigate the variation in code generation. Experimental results show that our method can significantly improve the functional correctness of the generated Verilog and reduce the hallucination of LLMs.

cs.AR↗

arXiv2024

The geometry of high-dimensional phase diagrams: I. Generalized Gibbs Phase Rule

Phase diagrams are essential tools of the materials scientist, showing which phases are at equilibrium under a set of applied thermodynamic conditions. Essentially all phase diagrams today are two dimensional, typically constructed with axes of temperature-pressure or temperature-composition. For many modern materials, it would be valuable to construct phase diagrams that include additional forms of thermodynamic work--such as elastic, surface, electromagnetic or electrochemical work, etc.--which grows the free energy of materials into higher (>3) dimensions. Here, we extend Gibbs' original arguments on phase coexistence to derive a generalized Phase Rule, based in the combinatorial geometry of high-dimensional convex polytopes. The generalized Phase Rule offers a conceptual and geometric foundation to describe phase boundaries on high-dimensional phase diagrams, which are relevant for understanding the stability of modern materials in complex chemical environments. We revisit Gibbs arguments on the equilibrium of heterogeneous substances and show that phase coexistence regions in high-dimensional Internal Energy space, U(S,Xi,...,Xj), are simplicial convex polytopes--which are N-dimensional analogues of triangles and tetrahedra. In the first of this three-part series, we examine how the combinatorial relationships between the vertices and facets of simplicial polytopes leads to a generalized high-dimensional description of Gibbs' Phase Rule. Because Gibbs' Phase Rule describes the nature of phase boundaries on phase diagrams, this isomorphism between the physical principles of equilibrium thermodynamics and the geometry of simplicial polytopes provides the foundation to construct generalized phase diagrams, which can exist in any dimension, with any intensive or extensive thermodynamic variable on the axes.

math-ph↗

arXiv2024

The geometry of high-dimensional phase diagrams: II. The duality between closed and open chemical systems

Modern materials are often synthesized or operated in complex chemical environments, where there can be numerous elemental species, competing phases, and reaction pathways. When analyzing reactions using the Gibbs free energy, which has a natural variable of composition, it is often cumbersome to solve for the equilibrium states of a complicated heterogeneous mixture of phases. However, if one is interested only in the stability of a single target material, it may be easier to reframe the boundary conditions around only the target material-of-interest, with boundary conditions open to chemical exchange with an external reservoir. The corresponding phase diagram would thus have a chemical potential axis for the open volatile species, rather than a composition axis. Here we discuss how to derive, compute, and interpret phase diagrams with chemical potential axes, which are dual to the more common composition phase diagram. In our ambition to construct high-dimensional phase diagrams featuring any thermodynamic variable on its axes, here we examine the duality between extensive and intensive conjugate variables in equilibrium thermodynamics. This duality manifests from the distinction between closed and open boundary conditions of a thermodynamic system, to the relationship between the Internal Energy and its Legendre transformations, to the point-line duality in calculating convex hulls versus half-space intersections. Here we focus on the duality relationships of chemical work, with extensive composition variables, N, and intensive chemical potentials, μ. In particular, we explore mixed composition-chemical potential diagrams for oxynitride synthesis, lithium-ion cathode stability, and oxidation of high-entropy alloys. We further illustrate how chemical potential diagrams reveal the driving forces for non-equilibrium growth and dissolution kinetics.

cond-mat.mtrl-sci↗

arXiv2024

The geometry of high-dimensional phase diagrams: III. Engineering relative stability in four dimensions

Designing thermodynamic conditions to improve (or reduce) the stability of a target material is a key task in materials engineering. For example, during materials synthesis one aims to enhance the stability of a target phase relative to its precursors or competing byproduct phases. If an undesired phase forms in experiment, one aims to destabilize the undesired phase by dissolution or corrosion. When multiple thermodynamic knobs are available to engineer relative stability, it can be difficult to navigate the corresponding high-dimensional phase diagram to identify optimal pathways to promote or destabilize a target phase. We propose that instead of mapping the absolute phase boundaries of a target material, we can invoke a generalized Clausius-Clapeyron relation, which provides a 'compass' to point out which directions on a high-dimensional phase diagram are best to stabilize or destabilize a target phase. Phase boundaries on high-dimensional phase diagrams are also high-dimensional objects, and can represent phase coexistence between numerous phases simultaneously. On a 2D temperature-pressure phase diagram, phase boundaries are 1D lines separating two phases, with a slope given by the Clausius-Clapeyron relation, dP/dT = ΔS/ΔV. Here, we derive a parametric form of the Clausius-Clapeyron relation that readily scales to high-dimensional phase boundaries. The gradient of a phase boundary guides us on how to increase or decrease the relative stability of a target compound, meaning this generalized Clausius-Clapeyron relation enables us to engineer relative stability with respect to multiple thermodynamic conditions simultaneously. Using this approach, we analyze the acid stability of manganese oxide catalysts on a 4-dimensional Pourbaix diagram with axes of pH, redox potential, nanoparticle size, and aqueous [K+] ion concentration.

cond-mat.mtrl-sci↗

arXiv2024

Quasicrystal bulk and surface energies from density functional theory

Are quasicrystals stable or metastable? Density functional theory (DFT) is often used to evaluate thermodynamic stability, but quasicrystals are long-range aperiodic and their energies cannot be calculated using conventional ab initio methods. Here, we perform first-principles calculations on quasicrystal nanoparticles of increasing sizes, from which we can directly extrapolate their bulk and surface energies. Using this technique, we determine with high confidence that the icosahedral quasicrystals ScZn7.33 and YbCd5.7 are ground-state phases--revealing that translational symmetry is not a necessary condition for the T = 0 K stability of inorganic solids. Although we find the ScZn7.33 quasicrystal to be thermodynamically stable, we show on a mixed thermodynamic and kinetic phase diagram that its solidification from the melt is nucleation-limited, which illustrates why even stable materials may be kinetically challenging to grow. Our techniques here broadly open the door to first-principles investigations into the structure-bonding-stability relationships of aperiodic materials.

cond-mat.mtrl-sci↗

arXiv2023

IOPS: An Unified SpMM Accelerator Based on Inner-Outer-Hybrid Product

Sparse matrix multiplication (SpMM) is widely applied to numerous domains, such as graph processing, machine learning, and data analytics. However, inner product based SpMM induces redundant zero-element computing for mismatched nonzero operands, while outer product based approach lacks input reuse across Process Elements (PEs) and poor output locality for accumulating partial sum (psum) matrices. Besides, current works only focus on sparse-sparse matrix multiplication (SSMM) or sparse-dense matrix multiplication (SDMM), rarely performing efficiently for both. To address these problems, this paper proposes an unified SpMM accelerator, called IOPS, hybridizing inner with outer products. It reuses the input matrix among PEs with inner product dataflow, and removes zero-element calculations with outer product approach in each PE, which can efficiently process SSMM and SDMM. Moreover, an address mapping method is designed to accumulate the irregular sparse psum matrices, reducing the latency and DRAM access of psum accumulating. Furthermore, an adaptive partition strategy is proposed to tile the input matrices based on their sparsity ratios, effectively utilizing the storage of architecture and reducing DRAM access. Compared with the SSMM accelerator, SpArch, we achieve 1.7x~6.3x energy efficiency and 1.2x~4.4x resource efficiency, with 1.4x~2.1x DRAM access saving.

cs.AR↗

arXiv2023

Navigating phase diagram complexity to guide robotic inorganic materials synthesis

Efficient synthesis recipes are needed both to streamline the manufacturing of complex materials and to accelerate the realization of theoretically predicted materials. Oftentimes the solid-state synthesis of multicomponent oxides is impeded by undesired byproduct phases, which can kinetically trap reactions in an incomplete non-equilibrium state. We present a thermodynamic strategy to navigate high-dimensional phase diagrams in search of precursors that circumvent low-energy competing byproducts, while maximizing the reaction energy to drive fast phase transformation kinetics. Using a robotic inorganic materials synthesis laboratory, we perform a large-scale experimental validation of our precursor selection principles. For a set of 35 target quaternary oxides with chemistries representative of intercalation battery cathodes and solid-state electrolytes, we perform 224 reactions spanning 27 elements with 28 unique precursors. Our predicted precursors frequently yield target materials with higher phase purity than when starting from traditional precursors. Robotic laboratories offer an exciting new platform for data-driven experimental science, from which we can develop new insights into materials synthesis for both robot and human chemists.

cond-mat.mtrl-sci↗

arXiv2023

Bit-balance: Model-Hardware Co-design for Accelerating NNs by Exploiting Bit-level Sparsity

Bit-serial architectures can handle Neural Networks (NNs) with different weight precisions, achieving higher resource efficiency compared with bit-parallel architectures. Besides, the weights contain abundant zero bits owing to the fault tolerance of NNs, indicating that bit sparsity of NNs can be further exploited for performance improvement. However, the irregular proportion of zero bits in each weight causes imbalanced workloads in the Processing Element (PE) array, which degrades performance or induces overhead for sparse processing. Thus, this paper proposed a bit-sparsity quantization method to maintain the bit sparsity ratio of each weight to no more than a certain value for balancing workloads, with little accuracy loss. Then, we co-designed a sparse bit-serial architecture, called Bit-balance, to improve overall performance, supporting weight-bit sparsity and adaptive bitwidth computation. The whole design was implemented with 65nm technology at 1 GHz and performs at 326-, 30-, 56-, and 218-frame/s for AlexNet, VGG-16, ResNet-50, and GoogleNet respectively. Compared with sparse bit-serial accelerator, Bitlet, Bit-balance achieves 1.8x~2.7x energy efficiency (frame/J) and 2.1x~3.7x resource efficiency (frame/mm2).

cs.AR↗