arXiv subjects

Hang Li

Publications and source records attributed to Hang Li.

At least 127 records · Page 7Linked to original sources

arXiv2024

Learning Flow Fields in Attention for Controllable Person Image Generation

Controllable person image generation aims to generate a person image conditioned on reference images, allowing precise control over the person's appearance or pose. However, prior methods often distort fine-grained textural details from the reference image, despite achieving high overall image quality. We attribute these distortions to inadequate attention to corresponding regions in the reference image. To address this, we thereby propose learning flow fields in attention (Leffa), which explicitly guides the target query to attend to the correct reference key in the attention layer during training. Specifically, it is realized via a regularization loss on top of the attention map within a diffusion-based baseline. Our extensive experiments show that Leffa achieves state-of-the-art performance in controlling appearance (virtual try-on) and pose (pose transfer), significantly reducing fine-grained detail distortion while maintaining high image quality. Additionally, we show that our loss is model-agnostic and can be used to improve the performance of other diffusion models.

cs.CV↗

arXiv2024

TPRF: A Transformer-based Pseudo-Relevance Feedback Model for Efficient and Effective Retrieval

This paper considers Pseudo-Relevance Feedback (PRF) methods for dense retrievers in a resource constrained environment such as that of cheap cloud instances or embedded systems (e.g., smartphones and smartwatches), where memory and CPU are limited and GPUs are not present. For this, we propose a transformer-based PRF method (TPRF), which has a much smaller memory footprint and faster inference time compared to other deep language models that employ PRF mechanisms, with a marginal effectiveness loss. TPRF learns how to effectively combine the relevance feedback signals from dense passage representations. Specifically, TPRF provides a mechanism for modelling relationships and weights between the query and the relevance feedback signals. The method is agnostic to the specific dense representation used and thus can be generally applied to any dense retriever.

cs.IR↗

arXiv2024

Rethinking Machine Unlearning for Large Language Models

We explore machine unlearning (MU) in the domain of large language models (LLMs), referred to as LLM unlearning. This initiative aims to eliminate undesirable data influence (e.g., sensitive or illegal information) and the associated model capabilities, while maintaining the integrity of essential knowledge generation and not affecting causally unrelated information. We envision LLM unlearning becoming a pivotal element in the life-cycle management of LLMs, potentially standing as an essential foundation for developing generative AI that is not only safe, secure, and trustworthy, but also resource-efficient without the need of full retraining. We navigate the unlearning landscape in LLMs from conceptual formulation, methodologies, metrics, and applications. In particular, we highlight the often-overlooked aspects of existing LLM unlearning research, e.g., unlearning scope, data-model interaction, and multifaceted efficacy assessment. We also draw connections between LLM unlearning and related areas such as model editing, influence functions, model explanation, adversarial training, and reinforcement learning. Furthermore, we outline an effective assessment framework for LLM unlearning and explore its applications in copyright and privacy safeguards and sociotechnical harm reduction.

cs.LG↗

arXiv2024

DexGANGrasp: Dexterous Generative Adversarial Grasping Synthesis for Task-Oriented Manipulation

We introduce DexGanGrasp, a dexterous grasping synthesis method that generates and evaluates grasps with single view in real time. DexGanGrasp comprises a Conditional Generative Adversarial Networks (cGANs)-based DexGenerator to generate dexterous grasps and a discriminator-like DexEvalautor to assess the stability of these grasps. Extensive simulation and real-world expriments showcases the effectiveness of our proposed method, outperforming the baseline FFHNet with an 18.57% higher success rate in real-world evaluation. We further extend DexGanGrasp to DexAfford-Prompt, an open-vocabulary affordance grounding pipeline for dexterous grasping leveraging Multimodal Large Language Models (MLLMs) and Vision Language Models (VLMs), to achieve task-oriented grasping with successful real-world deployments.

cs.RO↗

arXiv2024

Optical Tweezers with AC Dielectric Levitation: A Powerful Approach to Microparticle Manipulation

Optical tweezers, with their high precision, dynamic control, and non-invasiveness, are increasingly important in scientific research and applications at the micro and nano scales. However, manipulation by optical tweezers is challenged by adsorption forces, including van der Waals forces, capillary forces, and electrostatic forces, which are present between micro- and nano-objects. Due to the inherent limitations of optical forces imposed by laser power, these adsorption forces are difficult to overcome. Inspired by maglev trains, we propose a multiphysics coupling method that combines dielectrophoretic and optical gradient forces to achieve broad applicability and low-damage micro-nanoscale particle manipulation. We developed a device that introduces electric fields to detach objects from hard substrates using alternating current (AC) dielectric levitation before manipulation with optical tweezers. We utilized micron-sized polystyrene (PS) microspheres as objects and elucidated the levitation mechanism through finite element simulation. For larger particles, such as a 100 μm PS microparticle and a 200 μm micro-gear, AC dielectric levitation enabled manipulation by optical tweezers. Also, the better viability of three kinds of cells displayed the low bio-damage of the proposed method. Given its broad applicability and biocompatibility, AC dielectric levitation technology significantly expands the capabilities of optical tweezers, allowing for the manipulation of larger particles and cells. This advancement addresses the limitations of optical tweezers in handling large-scale particles and enhances their versatility in various applications.

physics.optics↗

arXiv2024

Probing enhanced superconductivity in van der Waals polytypes of V$_x$TaS$_2$

Layered transition metal dichalcogenides (TMDs) stabilize in multiple structural forms with profoundly distinct and exotic electronic phases. Interfacing different layer types is a promising route to manipulate TMDs' properties, not only as a means to engineer quantum devices, but also as a route to explore fundamental physics in complex matter. Here we use angle-resolved photoemission (ARPES) to investigate a strong layering-dependent enhancement of superconductivity in TaS$_2$, in which the superconducting transition temperature, $T_c$, of its $2H$ structural phase is nearly tripled when insulating $1T$ layers are inserted into the system. The study is facilitated by a novel vanadium-intercalation approach to synthesizing various TaS$_2$ polytypes, which improves the quality of ARPES data while leaving key aspects of the electronic structure and properties intact. The spectra show the clear opening of the charge density wave gap in the pure $2H$ phase and its suppression when $1T$ layers are introduced to the system. Moreover, in the mixed-layer $4Hb$ system, we observe a strongly momentum-anisotropic increase in electron-phonon coupling near the Fermi level relative to the $2H$ phase. Both phenomena help to account for the increased $T_c$ in mixed $2H$/$1T$ layer structures.

cond-mat.supr-con↗

arXiv2024

AGILE: A Novel Reinforcement Learning Framework of LLM Agents

We introduce a novel reinforcement learning framework of LLM agents named AGILE (AGent that Interacts and Learns from Environments) designed to perform complex conversational tasks with users, leveraging LLMs, memory, tools, and interactions with experts. The agent possesses capabilities beyond conversation, including reflection, tool usage, and expert consultation. We formulate the construction of such an LLM agent as a reinforcement learning (RL) problem, in which the LLM serves as the policy model. We fine-tune the LLM using labeled data of actions and the PPO algorithm. We focus on question answering and release a dataset for agents called ProductQA, comprising challenging questions in online shopping. Our extensive experiments on ProductQA, MedMCQA and HotPotQA show that AGILE agents based on 7B and 13B LLMs trained with PPO can outperform GPT-4 agents. Our ablation study highlights the indispensability of memory, tools, consultation, reflection, and reinforcement learning in achieving the agent's strong performance. Datasets and code are available at https://github.com/bytarnish/AGILE.

cs.LG↗

arXiv2024

Chaotic dynamics of granules-beam coupled vibration: Route and threshold

Although granular materials are the second most processed in industry after water, the theoretical study of granules-structure interactions is not as advanced as that of fluid-structure interactions due to the lack of a unified view of the constitutive relation of granular materials. In the previous work, the theoretical model of granules-beam coupled vibration was developed and verified by experiments. However, it was also found that the system exhibits significant stiffness-softening Duffing characteristics even under micro-vibration, which implies that chaotic responses may be reached under certain conditions. To reveal the route and critical conditions for the system to enter chaos, in the present work, the chaotic dynamics of the system are studied. In qualitative analysis, Melnikov method is applied to analyze the instability behavior of the perturbed heteroclinic orbit of the system, thus the critical condition for the system to enter Smale horseshoe chaos, i.e., the Melnikov criterion, is obtained. The validity of the criterion is verified numerically. In experimental studies, the existence of chaotic responses and the route to chaos for granules-beam coupled vibrations is revealed. The experimental results suggest that the system response first experiences symmetry-breaking then undergoes a complete period-doubling cascade, and finally enters single-scroll chaos. In addition, although the Additional Dynamic Load (ADL) generated by granular media is highly complicated, a general and simple evolution pattern of the chaos threshold is found by parametric experiments, which is also supported by the Melnikov criterion. In short, the chaotic dynamics of granules-beam coupled vibration is revealed, which is a contribution to the engineering vibration aspect. On the theoretical side, an impressive result lies in that for the first time, the Melnikov criterion of such fractional-order...

nlin.CD↗

arXiv2024

GR-2: A Generative Video-Language-Action Model with Web-Scale Knowledge for Robot Manipulation

We present GR-2, a state-of-the-art generalist robot agent for versatile and generalizable robot manipulation. GR-2 is first pre-trained on a vast number of Internet videos to capture the dynamics of the world. This large-scale pre-training, involving 38 million video clips and over 50 billion tokens, equips GR-2 with the ability to generalize across a wide range of robotic tasks and environments during subsequent policy learning. Following this, GR-2 is fine-tuned for both video generation and action prediction using robot trajectories. It exhibits impressive multi-task learning capabilities, achieving an average success rate of 97.7% across more than 100 tasks. Moreover, GR-2 demonstrates exceptional generalization to new, previously unseen scenarios, including novel backgrounds, environments, objects, and tasks. Notably, GR-2 scales effectively with model size, underscoring its potential for continued growth and application. Project page: \url{https://gr2-manipulation.github.io}.

cs.RO↗

arXiv2024

On the Melnikov method for fractional-order systems

This paper is dedicated to clarifying and introducing the correct application of Melnikov method in fractional dynamics. Attention to the complex dynamics of hyperbolic orbits and to fractional calculus can be, respectively, traced back to Poincarés attack on the three-body problem a century ago and to the early days of calculus three centuries ago. Nowadays, fractional calculus has been widely applied in modeling dynamic problems across various fields due to its advantages in describing problems with non-locality. Some of these models have also been confirmed to exhibit hyperbolic orbit dynamics, and recently, they have been extensively studied based on Melnikov method, an analytical approach for homoclinic and heteroclinic orbit dynamics. Despite its decade-long application in fractional dynamics, there is a universal problem in these applications that remains to be clarified, i.e., defining fractional-order systems within finite memory boundaries leads to the neglect of perturbation calculation for parts of the stable and unstable manifolds in Melnikov analysis. After clarifying and redefining the problem, a rigorous analytical case is provided for reference. Unlike existing results, the Melnikov criterion here is derived in a globally closed form, which was previously considered unobtainable due to difficulties in the analysis of fractional-order perturbations characterized by convolution integrals with power-law type singular kernels. Finally, numerical methods are employed to verify the derived Melnikov criterion. Overall, the clarification for the problem and the presented case are expected to provide insights for future research in this topic.

nlin.CD↗

arXiv2024

Laboratorial radiative shocks with multiple parameters and first quantifying verifications to core-collapse supernovae

We present experiments to reproduce the characteristics of core-collapse supernovae with different stellar masses and initial explosion energies in the laboratory. In the experiments, shocks are driven in 1.2 atm and 1.9 atm xenon gas by laser with energy from 1600J to 2800J on the SGIII prototype laser facility. The average shock velocities and shocked densities are obtained from experiments. Experimental results reveal that higher laser energy and lower Xe gas density led to higher shock velocity, and lower Xe gas initial density has a higher compression. Modeling of the experiments using the 2D radiation hydrodynamic codes Icefire shows excellent agreement with the experimental results and gives the temperature. These results will contribute to time-domain astrophysical systems, such as gravitational supernovae, where a strong radiative shock propagates outward from the center of the star after the core collapses.

astro-ph.HE↗

arXiv2024

Sub-graph Based Diffusion Model for Link Prediction

Denoising Diffusion Probabilistic Models (DDPMs) represent a contemporary class of generative models with exceptional qualities in both synthesis and maximizing the data likelihood. These models work by traversing a forward Markov Chain where data is perturbed, followed by a reverse process where a neural network learns to undo the perturbations and recover the original data. There have been increasing efforts exploring the applications of DDPMs in the graph domain. However, most of them have focused on the generative perspective. In this paper, we aim to build a novel generative model for link prediction. In particular, we treat link prediction between a pair of nodes as a conditional likelihood estimation of its enclosing sub-graph. With a dedicated design to decompose the likelihood estimation process via the Bayesian formula, we are able to separate the estimation of sub-graph structure and its node features. Such designs allow our model to simultaneously enjoy the advantages of inductive learning and the strong generalization capability. Remarkably, comprehensive experiments across various datasets validate that our proposed method presents numerous advantages: (1) transferability across datasets without retraining, (2) promising generalization on limited training data, and (3) robustness against graph adversarial attacks.

cs.LG↗

arXiv2024

Towards Achieving Human Parity on End-to-end Simultaneous Speech Translation via LLM Agent

In this paper, we present Cross Language Agent -- Simultaneous Interpretation, CLASI, a high-quality and human-like Simultaneous Speech Translation (SiST) System. Inspired by professional human interpreters, we utilize a novel data-driven read-write strategy to balance the translation quality and latency. To address the challenge of translating in-domain terminologies, CLASI employs a multi-modal retrieving module to obtain relevant information to augment the translation. Supported by LLMs, our approach can generate error-tolerated translation by considering the input audio, historical context, and retrieved information. Experimental results show that our system outperforms other systems by significant margins. Aligned with professional human interpreters, we evaluate CLASI with a better human evaluation metric, valid information proportion (VIP), which measures the amount of information that can be successfully conveyed to the listeners. In the real-world scenarios, where the speeches are often disfluent, informal, and unclear, CLASI achieves VIP of 81.3% and 78.0% for Chinese-to-English and English-to-Chinese translation directions, respectively. In contrast, state-of-the-art commercial or open-source systems only achieve 35.4% and 41.6%. On the extremely hard dataset, where other systems achieve under 13% VIP, CLASI can still achieve 70% VIP.

cs.CL↗

arXiv2024

Revisiting the measurements and interpretations of DLVO forces

The DLVO theory and electrical double layer (EDL) theory are the foundation of colloid and interface science. With the invention and development of surface forces apparatus (SFA) and atomic force microscope (AFM), the measurements and interpretations of DLVO forces (i.e., mainly measuring the EDL force (electrostatic force) FEDL and van der Waals force FvdW, and interpreting the potential ψ, charge density σ, and Hamaker constant H) can be greatly facilitated by various surface force measurement techniques, and would have been very promising in advancing the DLVO theory, EDL theory, and colloid and interface science. However, although numerous studies have been conducted, pervasive anomalous results can be identified throughout the literature, main including: (1) the fitted ψ/σ is normally extremely small (ψ can be close to or (much) smaller than ψζ (zeta potential)) and varies greatly; (2) the fitted ψ/σ can exceed the allowable range of calculation; and (3) the measured FvdW and the fitted H vary greatly. Based on rigorous and comprehensive arguments, we have reasonably explained the pervasive anomalous results in the literature and further speculated that, the pervasive anomalous results are existing but not noticed and questioned owing to the two important aspects: (1) the pervasive unreasonable understandings of EDL theory and (2) the commonly neglected systematic errors. Consequently, we believe that the related studies have been seriously hampered. We therefore call for re-examination and re-analysis of related experimental results and theoretical understandings by careful consideration of the EDL theory and systematic errors. On these bases, we can interpret the experimental results properly and promote the development of EDL theory, colloid and interface science, and many related fields.

physics.chem-ph↗

arXiv2024

Filtered Lie-Trotter splitting for the "good" Boussinesq equation: low regularity error estimates

We investigate a filtered Lie-Trotter splitting scheme for the ``good" Boussinesq equation and derive an error estimate for initial data with very low regularity. Through the use of discrete Bourgain spaces, our analysis extends to initial data in $H^{s}$ for $0 1/2$ imposed by the bilinear estimate in smooth Sobolev spaces. We establish convergence rates of order $τ^{s/2}$ in $L^2$ for such levels of regularity. Our analytical findings are supported by numerical experiments.

math.NA↗

arXiv2024

Discovery of a metallic room-temperature d-wave altermagnet KV2Se2O

Beyond conventional ferromagnetism and antiferromagnetism, altermagnetism is a recently discovered unconventional magnetic phase characterized by time-reversal symmetry breaking and spin-split band structures in materials with zero net magnetization. This distinct magnetic phase not only enriches the understanding of fundamental physical concepts but also has profound impacts on condense-matter physics research and practical device applications. Spin-polarized band structures have been recently observed in semiconductors MnTe and MnTe2 with vanishing net magnetization, confirming the existence of this unconventional magnetic order. Metallic altermagnets have unique advantages for exploring novel physical phenomena related to low-energy quasiparticle excitations and for applications in spintronics as electrical conductivity in metals allows the direct manipulation of spin current through electric field. Here, through comprehensive characterization and analysis of the magnetic and electronic structures of KV2Se2O, we have unambiguously demonstrated a metallic room-temperature altermaget with d-wave spin-momentum locking. The highly anisotropic spin-polarized Fermi surfaces and the spin-density-wave order emerging in the altermagnetic phase make it an extraordinary platform for designing high-performance spintronic devices and studying many-body effects coupled with the unconventional magnetism.

cond-mat.mtrl-sci↗

arXiv2024

Driving Animatronic Robot Facial Expression From Speech

Animatronic robots hold the promise of enabling natural human-robot interaction through lifelike facial expressions. However, generating realistic, speech-synchronized robot expressions poses significant challenges due to the complexities of facial biomechanics and the need for responsive motion synthesis. This paper introduces a novel, skinning-centric approach to drive animatronic robot facial expressions from speech input. At its core, the proposed approach employs linear blend skinning (LBS) as a unifying representation, guiding innovations in both embodiment design and motion synthesis. LBS informs the actuation topology, facilitates human expression retargeting, and enables efficient speech-driven facial motion generation. This approach demonstrates the capability to produce highly realistic facial expressions on an animatronic face in real-time at over 4000 fps on a single Nvidia RTX 4090, significantly advancing robots' ability to replicate nuanced human expressions for natural interaction. To foster further research and development in this field, the code has been made publicly available at: \url{https://github.com/library87/OpenRoboExp}.

cs.RO↗

arXiv2024

Coexistence of large anomalous Hall effect and topological magnetic skyrmions in a Weyl nodal ring ferromagnet Mn5Ge3

Topological magnetic materials are expected to show multiple transport responses because of their unusual bulk electronic topology in momentum space and topological spin texture in real space. However, such multiple topological properties-hosting materials are rare in nature. In this work, we reveal the coexistence of a large tunable anomalous Hall effect and topological magnetic skyrmions in a Weyl nodal ring ferromagnet Mn5Ge3, by using electrical transport and Lorentz transmission electronic microscope (TEM) measurements. It was found that the intrinsic anomalous Hall conductivity (AHC) can reach up to 979.7 S/cm with current along [120] and magnetic field along [001] of the Mn5Ge3 single crystals. Our theoretical calculations reveal that the large AHC is closely related with two Weyl nodal rings in band structure near the Fermi level and is strongly modified by the content of Ge. Moreover, our Lorentz-TEM images and micromagnetic simulation results, together with the sizable topological Hall effect clearly point to the robust formation of magnetic skyrmions over a wide temperature-magnetic field region. These results prove Mn5Ge3 as a rare magnetic topological nodal-line semimetal with great significance to explore novel multiple topological phenomena, which facilitates the development of spintronics.

cond-mat.mtrl-sci↗