arXiv subjects

Wei Li

Publications and source records attributed to Wei Li.

At least 559 records · Page 31Linked to original sources

arXiv2024

Can a large language model be a gaslighter?

Large language models (LLMs) have gained human trust due to their capabilities and helpfulness. However, this in turn may allow LLMs to affect users' mindsets by manipulating language. It is termed as gaslighting, a psychological effect. In this work, we aim to investigate the vulnerability of LLMs under prompt-based and fine-tuning-based gaslighting attacks. Therefore, we propose a two-stage framework DeepCoG designed to: 1) elicit gaslighting plans from LLMs with the proposed DeepGaslighting prompting template, and 2) acquire gaslighting conversations from LLMs through our Chain-of-Gaslighting method. The gaslighting conversation dataset along with a corresponding safe dataset is applied to fine-tuning-based attacks on open-source LLMs and anti-gaslighting safety alignment on these LLMs. Experiments demonstrate that both prompt-based and fine-tuning-based attacks transform three open-source LLMs into gaslighters. In contrast, we advanced three safety alignment strategies to strengthen (by 12.05%) the safety guardrail of LLMs. Our safety alignment strategies have minimal impacts on the utility of LLMs. Empirical studies indicate that an LLM may be a potential gaslighter, even if it passed the harmfulness test on general dangerous queries.

cs.CR↗

arXiv2024

HybridBooth: Hybrid Prompt Inversion for Efficient Subject-Driven Generation

Recent advancements in text-to-image diffusion models have shown remarkable creative capabilities with textual prompts, but generating personalized instances based on specific subjects, known as subject-driven generation, remains challenging. To tackle this issue, we present a new hybrid framework called HybridBooth, which merges the benefits of optimization-based and direct-regression methods. HybridBooth operates in two stages: the Word Embedding Probe, which generates a robust initial word embedding using a fine-tuned encoder, and the Word Embedding Refinement, which further adapts the encoder to specific subject images by optimizing key parameters. This approach allows for effective and fast inversion of visual concepts into textual embedding, even from a single image, while maintaining the model's generalization capabilities.

cs.CV↗

arXiv2024

Neural Material Adaptor for Visual Grounding of Intrinsic Dynamics

While humans effortlessly discern intrinsic dynamics and adapt to new scenarios, modern AI systems often struggle. Current methods for visual grounding of dynamics either use pure neural-network-based simulators (black box), which may violate physical laws, or traditional physical simulators (white box), which rely on expert-defined equations that may not fully capture actual dynamics. We propose the Neural Material Adaptor (NeuMA), which integrates existing physical laws with learned corrections, facilitating accurate learning of actual dynamics while maintaining the generalizability and interpretability of physical priors. Additionally, we propose Particle-GS, a particle-driven 3D Gaussian Splatting variant that bridges simulation and observed images, allowing back-propagate image gradients to optimize the simulator. Comprehensive experiments on various dynamics in terms of grounded particle accuracy, dynamic rendering quality, and generalization ability demonstrate that NeuMA can accurately capture intrinsic dynamics.

cs.CV↗

arXiv2024

Unlocking the Power of Large Language Models for Entity Alignment

Entity Alignment (EA) is vital for integrating diverse knowledge graph (KG) data, playing a crucial role in data-driven AI applications. Traditional EA methods primarily rely on comparing entity embeddings, but their effectiveness is constrained by the limited input KG data and the capabilities of the representation learning techniques. Against this backdrop, we introduce ChatEA, an innovative framework that incorporates large language models (LLMs) to improve EA. To address the constraints of limited input KG data, ChatEA introduces a KG-code translation module that translates KG structures into a format understandable by LLMs, thereby allowing LLMs to utilize their extensive background knowledge to improve EA accuracy. To overcome the over-reliance on entity embedding comparisons, ChatEA implements a two-stage EA strategy that capitalizes on LLMs' capability for multi-step reasoning in a dialogue format, thereby enhancing accuracy while preserving efficiency. Our experimental results verify ChatEA's superior performance, highlighting LLMs' potential in facilitating EA tasks.

cs.CL↗

arXiv2024

Subspace Prototype Guidance for Mitigating Class Imbalance in Point Cloud Semantic Segmentation

Point cloud semantic segmentation can significantly enhance the perception of an intelligent agent. Nevertheless, the discriminative capability of the segmentation network is influenced by the quantity of samples available for different categories. To mitigate the cognitive bias induced by class imbalance, this paper introduces a novel method, namely subspace prototype guidance (\textbf{SPG}), to guide the training of segmentation network. Specifically, the point cloud is initially separated into independent point sets by category to provide initial conditions for the generation of feature subspaces. The auxiliary branch which consists of an encoder and a projection head maps these point sets into separate feature subspaces. Subsequently, the feature prototypes which are extracted from the current separate subspaces and then combined with prototypes of historical subspaces guide the feature space of main branch to enhance the discriminability of features of minority categories. The prototypes derived from the feature space of main branch are also employed to guide the training of the auxiliary branch, forming a supervisory loop to maintain consistent convergence of the entire network. The experiments conducted on the large public benchmarks (i.e. S3DIS, ScanNet v2, ScanNet200, Toronto-3D) and collected real-world data illustrate that the proposed method significantly improves the segmentation performance and surpasses the state-of-the-art method. The code is available at \url{https://github.com/Javion11/PointLiBR.git}.

cs.CV↗

arXiv2024

MosaicFusion: Diffusion Models as Data Augmenters for Large Vocabulary Instance Segmentation

We present MosaicFusion, a simple yet effective diffusion-based data augmentation approach for large vocabulary instance segmentation. Our method is training-free and does not rely on any label supervision. Two key designs enable us to employ an off-the-shelf text-to-image diffusion model as a useful dataset generator for object instances and mask annotations. First, we divide an image canvas into several regions and perform a single round of diffusion process to generate multiple instances simultaneously, conditioning on different text prompts. Second, we obtain corresponding instance masks by aggregating cross-attention maps associated with object prompts across layers and diffusion time steps, followed by simple thresholding and edge-aware refinement processing. Without bells and whistles, our MosaicFusion can produce a significant amount of synthetic labeled data for both rare and novel categories. Experimental results on the challenging LVIS long-tailed and open-vocabulary benchmarks demonstrate that MosaicFusion can significantly improve the performance of existing instance segmentation models, especially for rare and novel categories. Code: https://github.com/Jiahao000/MosaicFusion.

cs.CV↗

arXiv2024

OMG-Seg: Is One Model Good Enough For All Segmentation?

In this work, we address various segmentation tasks, each traditionally tackled by distinct or partially unified models. We propose OMG-Seg, One Model that is Good enough to efficiently and effectively handle all the segmentation tasks, including image semantic, instance, and panoptic segmentation, as well as their video counterparts, open vocabulary settings, prompt-driven, interactive segmentation like SAM, and video object segmentation. To our knowledge, this is the first model to handle all these tasks in one model and achieve satisfactory performance. We show that OMG-Seg, a transformer-based encoder-decoder architecture with task-specific queries and outputs, can support over ten distinct segmentation tasks and yet significantly reduce computational and parameter overhead across various tasks and datasets. We rigorously evaluate the inter-task influences and correlations during co-training. Code and models are available at https://github.com/lxtGH/OMG-Seg.

cs.CV↗

arXiv2024

Correlated interaction effects in three-dimensional semi-Dirac semimetal

Understanding the correlation effects in unconventional topological materials, in which the fermion excitations take unusual dispersion, is an important topic in recent condensed matter physics. We study the influence of short-range four-fermion interactions on three-dimensional semi-Dirac semimetal with an unusual fermion dispersion, that is linear along two directions and quadratic along the third one. Based on renormalization group theory, we find all of 11 unstable fixed points including 5 quantum critical points, 5 bicritical points, and one tricritical point. The physical essences of the quantum critical points are determined by analyzing the susceptibility exponents for all of the source terms in particle-hole and particle-particle channels. We also verify phase diagrams of the system in the parameter space through numerically studying the flows of the four-fermion coupling parameters and behaviors of the susceptibility exponents. These results are helpful for us to understand the physical properties of candidate materials for three-dimensional semi-Dirac semimetal such as ZrTe$_{5}$.

cond-mat.str-el↗

arXiv2024

MinerU: An Open-Source Solution for Precise Document Content Extraction

Document content analysis has been a crucial research area in computer vision. Despite significant advancements in methods such as OCR, layout detection, and formula recognition, existing open-source solutions struggle to consistently deliver high-quality content extraction due to the diversity in document types and content. To address these challenges, we present MinerU, an open-source solution for high-precision document content extraction. MinerU leverages the sophisticated PDF-Extract-Kit models to extract content from diverse documents effectively and employs finely-tuned preprocessing and postprocessing rules to ensure the accuracy of the final results. Experimental results demonstrate that MinerU consistently achieves high performance across various document types, significantly enhancing the quality and consistency of content extraction. The MinerU open-source project is available at https://github.com/opendatalab/MinerU.

cs.CV↗

arXiv2024

A Generalized Tensor Formulation for Hyperspectral Image Super-Resolution Under General Spatial Blurring

Hyperspectral super-resolution is commonly accomplished by the fusing of a hyperspectral imaging of low spatial resolution with a multispectral image of high spatial resolution, and many tensor-based approaches to this task have been recently proposed. Yet, it is assumed in such tensor-based methods that the spatial-blurring operation that creates the observed hyperspectral image from the desired super-resolved image is separable into independent horizontal and vertical blurring. Recent work has argued that such separable spatial degradation is ill-equipped to model the operation of real sensors which may exhibit, for example, anisotropic blurring. To accommodate this fact, a generalized tensor formulation based on a Kronecker decomposition is proposed to handle any general spatial-degradation matrix, including those that are not separable as previously assumed. Analysis of the generalized formulation reveals conditions under which exact recovery of the desired super-resolved image is guaranteed, and a practical algorithm for such recovery, driven by a blockwise-group-sparsity regularization, is proposed. Extensive experimental results demonstrate that the proposed generalized tensor approach outperforms not only traditional matrix-based techniques but also state-of-the-art tensor-based methods; the gains with respect to the latter are especially significant in cases of anisotropic spatial blurring.

eess.IV↗

arXiv2024

Deep Learning Enhanced Quantum Holography with Undetected Photons

Holography is an essential technique of generating three-dimensional images. Recently, quantum holography with undetected photons (QHUP) has emerged as a groundbreaking method capable of capturing complex amplitude images. Despite its potential, the practical application of QHUP has been limited by susceptibility to phase disturbances, low interference visibility, and limited spatial resolution. Deep learning, recognized for its ability in processing complex data, holds significant promise in addressing these challenges. In this report, we present an ample advancement in QHUP achieved by harnessing the power of deep learning to extract images from single-shot holograms, resulting in vastly reduced noise and distortion, alongside a notable enhancement in spatial resolution. The proposed and demonstrated deep learning QHUP (DL-QHUP) methodology offers a transformative solution by delivering high-speed imaging, improved spatial resolution, and superior noise resilience, making it suitable for diverse applications across an array of research fields stretching from biomedical imaging to remote sensing. DL-QHUP signifies a crucial leap forward in the realm of holography, demonstrating its immense potential to revolutionize imaging capabilities and pave the way for advancements in various scientific disciplines. The integration of DL-QHUP promises to unlock new possibilities in imaging applications, transcending existing limitations and offering unparalleled performance in challenging environments.

physics.optics↗

arXiv2024

Quiver algebras and their representations for arbitrary quivers

The quiver Yangians were originally defined for the quiver and superpotential from string theory on general toric Calabi-Yau threefolds, and serve as BPS algebras of these systems. Their characters reproduce the unrefined BPS indices, which correspond to classical Donaldson-Thomas (DT) invariants. We generalize this construction in two directions. First, we show that this definition extends to arbitrary quivers with potentials. Second, we explain how to define the characters to incorporate the refined BPS indices, which correspond to motivic DT invariants. We focus on two main classes of quivers: the BPS quivers of 4D $N=2$ theories and the quivers from the knot-quiver correspondence. The entire construction allows for straightforward generalizations to trigonometric, elliptic, and generalized cohomologies.

hep-th↗

arXiv2024

Contrasformer: A Brain Network Contrastive Transformer for Neurodegenerative Condition Identification

Understanding neurological disorder is a fundamental problem in neuroscience, which often requires the analysis of brain networks derived from functional magnetic resonance imaging (fMRI) data. Despite the prevalence of Graph Neural Networks (GNNs) and Graph Transformers in various domains, applying them to brain networks faces challenges. Specifically, the datasets are severely impacted by the noises caused by distribution shifts across sub-populations and the neglect of node identities, both obstruct the identification of disease-specific patterns. To tackle these challenges, we propose Contrasformer, a novel contrastive brain network Transformer. It generates a prior-knowledge-enhanced contrast graph to address the distribution shifts across sub-populations by a two-stream attention mechanism. A cross attention with identity embedding highlights the identity of nodes, and three auxiliary losses ensure group consistency. Evaluated on 4 functional brain network datasets over 4 different diseases, Contrasformer outperforms the state-of-the-art methods for brain networks by achieving up to 10.8\% improvement in accuracy, which demonstrates its efficacy in neurological disorder identification. Case studies illustrate its interpretability, especially in the context of neuroscience. This paper provides a solution for analyzing brain networks, offering valuable insights into neurological disorders. Our code is available at \url{https://github.com/AngusMonroe/Contrasformer}.

cs.LG↗

arXiv2024

Unique quantum metallic state in the titanium sesquioxide heterointerface superconductor

The emergence of quantum metallic state marked by a saturating finite electrical resistance in the zero-temperature limit in a variety of two-dimensional superconductors injects an exciting momentum to the realm of heterostructure superconductivity. Despite much research efforts over last few decades, there is not yet a general consensus on the nature of this unexpected quantum metal. Here, we report the observation of a unique quantum metallic state within the hallmark of Bose-metal in the titanium sesquioxide heterointerface superconductor Ti$_2$O$_3$/GaN. Remarkably, the quantum bosonic metallic state continuously tuned by a magnetic field in the vicinity of the two-dimensional superconductivity-metal transition persists in the normal phase, indicating the existence of composite bosons formed by electron Cooper pairs even in the normal phase. Our findings provide a distinct evidence for electron pairing in the normal phase of heterointerface superconductors, and shed fresh light on the pairing nature underlying heterointerface superconductivity.

cond-mat.supr-con↗

arXiv2024

Parameter Efficient Reinforcement Learning from Human Feedback

While Reinforcement Learning from Human Feedback (RLHF) effectively aligns pretrained Large Language and Vision-Language Models (LLMs, and VLMs) with human preferences, its computational cost and complexity hamper its wider adoption. To alleviate some of the computational burden of fine-tuning, parameter efficient methods, like LoRA were introduced. In this work, we empirically evaluate the setup of Parameter Efficient Reinforcement Learning from Human Feedback (PE-RLHF) that leverages LoRA fine-tuning for Reward Modeling, and Reinforcement Learning. We benchmark the PE-RLHF setup on six diverse datasets spanning summarization, harmless/helpful response generation, UI automation, and visual question answering in terms of effectiveness of the trained models, and the training resources required. Our findings show, for the first time, that PE-RLHF achieves comparable performance to RLHF, while significantly reducing training time (up to 90% faster for reward models, and 30% faster for RL), and memory footprint (up to 50% reduction for reward models, and 27% for RL). We provide comprehensive ablations across LoRA ranks, and model sizes for both reward modeling and reinforcement learning. By mitigating the computational burden associated with RLHF, we push for a broader adoption of PE-RLHF as an alignment technique for LLMs and VLMs.

cs.LG↗

arXiv2024

Multiscale simulation of rarefied gas flows in Divertor Tokamak Test facility

Simulating gas flow within the divertor, which is a crucial component in nuclear fusion reactors, is essential for assessing and enhancing its design and performance. Traditional methods, such as the direct simulation Monte Carlo and the discrete velocity method, often fall short in efficiency for these simulations. In this study, we utilize the general synthetic iterative scheme to simulate a simplified Tokamak divertor model, demonstrating its fast convergence and asymptotic-preserving properties in complex three-dimensional scenarios. A conservative estimate of speedup by three orders of magnitude is achieved by the general synthetic iterative scheme when compared to the direct simulation Monte Carlo method. We further investigate the relationship between pumping efficiency and factors like temperature, absorptivity, and the Knudsen number, providing valuable insights to guide the design and optimization of divertor structures.

physics.plasm-ph↗

arXiv2024

Nonparametric Estimation of Path-specific Effects in Presence of Nonignorable Missing Covariates

The path-specific effect (PSE) is of primary interest in mediation analysis when multiple intermediate variables between treatment and outcome are observed, as it can isolate the specific effect through each mediator, thus mitigating potential bias arising from other intermediate variables serving as mediator-outcome confounders. However, estimation and inference of PSE become challenging in the presence of nonignorable missing covariates, a situation particularly common in epidemiological research involving sensitive patient information. In this paper, we propose a fully nonparametric methodology to address this challenge. We establish identification for PSE by expressing it as a functional of observed data and demonstrate that the associated nuisance functions can be uniquely determined through sequential optimization problems by leveraging a shadow variable. Then we propose a sieve-based regression imputation approach for estimation. We establish the large-sample theory for the proposed estimator, and introduce a robust and efficient approach to make inference for PSE. The proposed method is applied to the NHANES dataset to investigate the mediation roles of dyslipidemia and obesity in the pathway from Type 2 diabetes mellitus to cardiovascular disease.

stat.ME↗

arXiv2024

Radiative transitions of $χ_{_{cJ}}\toψγ$ and $χ_{_{bJ}}\toΥγ$

In the framework of instantaneous Bethe-Salpeter equation, according to the $J ^ {PC}$ of quarkonia, we find that their wave functions all contain multiple partial waves, rather than pure waves. In the radiative electromagnetic transitions $χ_{_{cJ}}$$\rightarrow$$γψ$ and $χ_{_{bJ}}$$\rightarrow$$γΥ$ ($J=0,1,2$), the main wave of quarkonium gives the non-relativistic contribution, while other waves provide the relativistic corrections. Our results indicate that the relativistic effect of charmonium, especially highly excited states, is significant. Such as the relativistic effects of $χ_{_{cJ}}(2P)\toγψ(1S)$ ($J=0,1,2$) are $\{49.7\%,~30.9\%,~37.5\%\}$, much larger than the corresponding $\{17.8\%,~7.08\%,~12.9\%\}$ of $χ_{_{bJ}}(2P)\rightarrowγΥ(1S)$. The decay of $χ_{_{cJ}}(2P)\toγψ$ can be used to distinguish between $χ_{_{c0}}(3860)$ and $χ_{_{c0}}(3915)$, which particle is the charmonium $χ_{_{c0}}(2P)$. Although our result of $χ_{_{c1}}(3872)$$\rightarrow$$γψ(2S)$ is consistent with data, but the one of $χ_{_{c1}}(3872)$$\rightarrow$$γψ(1S)$ is much larger than data, so whether $χ_{_{c1}}(3872)$ is the conventional $χ_{_{c1}}(2P)$ remains an open question. The undiscovered $Υ(1D)$ and $Υ(2D)$ have large production rates in decays of $χ_{_{b0}}(2P)\rightarrowγΥ(1D)$ and $χ_{_{bJ}}(3P)\rightarrowγΥ(2D)$ ($J=0,1$), respectively. To search for $χ_{_{bJ}}(3P)$ $(J=0,1,2)$, the most competitive channels are the decays $χ_{_{bJ}}(3P)\rightarrowγΥ(3S)$. And the best way to find $χ_{_{b2}}(1F)$ is to search for the decay of $χ_{_{b2}}(1F)\rightarrowγΥ(1D)$.

hep-ph↗