arXiv subjects

Jing Li

Publications and source records attributed to Jing Li.

At least 379 records · Page 21Linked to original sources

arXiv2024

Open-domain Implicit Format Control for Large Language Model Generation

Controlling the format of outputs generated by large language models (LLMs) is a critical functionality in various applications. Current methods typically employ constrained decoding with rule-based automata or fine-tuning with manually crafted format instructions, both of which struggle with open-domain format requirements. To address this limitation, we introduce a novel framework for controlled generation in LLMs, leveraging user-provided, one-shot QA pairs. This study investigates LLMs' capabilities to follow open-domain, one-shot constraints and replicate the format of the example answers. We observe that this is a non-trivial problem for current LLMs. We also develop a dataset collection methodology for supervised fine-tuning that enhances the open-domain format control of LLMs without degrading output quality, as well as a benchmark on which we evaluate both the helpfulness and format correctness of LLM outputs. The resulting datasets, named OIFC-SFT, along with the related code, will be made publicly available at https://github.com/cofe-ai/OIFC.

cs.CL↗

arXiv2024

Highly Efficient No-reference 4K Video Quality Assessment with Full-Pixel Covering Sampling and Training Strategy

Deep Video Quality Assessment (VQA) methods have shown impressive high-performance capabilities. Notably, no-reference (NR) VQA methods play a vital role in situations where obtaining reference videos is restricted or not feasible. Nevertheless, as more streaming videos are being created in ultra-high definition (e.g., 4K) to enrich viewers' experiences, the current deep VQA methods face unacceptable computational costs. Furthermore, the resizing, cropping, and local sampling techniques employed in these methods can compromise the details and content of original 4K videos, thereby negatively impacting quality assessment. In this paper, we propose a highly efficient and novel NR 4K VQA technology. Specifically, first, a novel data sampling and training strategy is proposed to tackle the problem of excessive resolution. This strategy allows the VQA Swin Transformer-based model to effectively train and make inferences using the full data of 4K videos on standard consumer-grade GPUs without compromising content or details. Second, a weighting and scoring scheme is developed to mimic the human subjective perception mode, which is achieved by considering the distinct impact of each sub-region within a 4K frame on the overall perception. Third, we incorporate the frequency domain information of video frames to better capture the details that affect video quality, consequently further improving the model's generalizability. To our knowledge, this is the first technology for the NR 4K VQA task. Thorough empirical studies demonstrate it not only significantly outperforms existing methods on a specialized 4K VQA dataset but also achieves state-of-the-art performance across multiple open-source NR video quality datasets.

eess.IV↗

arXiv2024

MeshSegmenter: Zero-Shot Mesh Semantic Segmentation via Texture Synthesis

We present MeshSegmenter, a simple yet effective framework designed for zero-shot 3D semantic segmentation. This model successfully extends the powerful capabilities of 2D segmentation models to 3D meshes, delivering accurate 3D segmentation across diverse meshes and segment descriptions. Specifically, our model leverages the Segment Anything Model (SAM) model to segment the target regions from images rendered from the 3D shape. In light of the importance of the texture for segmentation, we also leverage the pretrained stable diffusion model to generate images with textures from 3D shape, and leverage SAM to segment the target regions from images with textures. Textures supplement the shape for segmentation and facilitate accurate 3D segmentation even in geometrically non-prominent areas, such as segmenting a car door within a car mesh. To achieve the 3D segments, we render 2D images from different views and conduct segmentation for both textured and untextured images. Lastly, we develop a multi-view revoting scheme that integrates 2D segmentation results and confidence scores from various views onto the 3D mesh, ensuring the 3D consistency of segmentation results and eliminating inaccuracies from specific perspectives. Through these innovations, MeshSegmenter offers stable and reliable 3D segmentation results both quantitatively and qualitatively, highlighting its potential as a transformative tool in the field of 3D zero-shot segmentation. The code is available at \url{https://github.com/zimingzhong/MeshSegmenter}.

cs.CV↗

arXiv2024

Nonreciprocal Single-Photon Band Structure in a Coupled-Spinning-Resonator chain

We analyze the single-photon band structure and the transport of a single photon in a one-dimensional coupled-spinning-resonator chain. The time-reversal symmetry of the resonators chain is broken by the spinning of the resonators, instead of external or synthetic magnetic field. Two nonreciprocal single-photon band gaps can be obtained in the coupled-spinning-resonator chain, whose width depends on the angular velocity of the spinning resonator. Based on the nonreciprocal band gaps, we can implement a single photon circulator at multiple frequency windows, and the direction of photon cycling is opposite for different band gaps. In addition, reciprocal single-photon band structures can also be realized in the coupled-spinning-resonator chain when all resonators rotate in the same direction with equal angular velocity. Our work open a new route to achieve, manipulate, and switch nonreciprocal or reciprocal single-photon band structures, and provides new opportunities to realize novel single-photon devices.

physics.optics↗

arXiv2024

HSFusion: A high-level vision task-driven infrared and visible image fusion network via semantic and geometric domain transformation

Infrared and visible image fusion has been developed from vision perception oriented fusion methods to strategies which both consider the vision perception and high-level vision task. However, the existing task-driven methods fail to address the domain gap between semantic and geometric representation. To overcome these issues, we propose a high-level vision task-driven infrared and visible image fusion network via semantic and geometric domain transformation, terms as HSFusion. Specifically, to minimize the gap between semantic and geometric representation, we design two separate domain transformation branches by CycleGAN framework, and each includes two processes: the forward segmentation process and the reverse reconstruction process. CycleGAN is capable of learning domain transformation patterns, and the reconstruction process of CycleGAN is conducted under the constraint of these patterns. Thus, our method can significantly facilitate the integration of semantic and geometric information and further reduces the domain gap. In fusion stage, we integrate the infrared and visible features that extracted from the reconstruction process of two seperate CycleGANs to obtain the fused result. These features, containing varying proportions of semantic and geometric information, can significantly enhance the high level vision tasks. Additionally, we generate masks based on segmentation results to guide the fusion task. These masks can provide semantic priors, and we design adaptive weights for two distinct areas in the masks to facilitate image fusion. Finally, we conducted comparative experiments between our method and eleven other state-of-the-art methods, demonstrating that our approach surpasses others in both visual appeal and semantic segmentation task.

cs.CV↗

arXiv2024

GRAPE: Generalizable and Robust Multi-view Facial Capture

Deep learning-based multi-view facial capture methods have shown impressive accuracy while being several orders of magnitude faster than a traditional mesh registration pipeline. However, the existing systems (e.g. TEMPEH) are strictly restricted to inference on the data captured by the same camera array used to capture their training data. In this study, we aim to improve the generalization ability so that a trained model can be readily used for inference (i.e. capture new data) on a different camera array. To this end, we propose a more generalizable initialization module to extract the camera array-agnostic 3D feature, including a visual hull-based head localization and a visibility-aware 3D feature aggregation module enabled by the visual hull. In addition, we propose an ``update-by-disagreement'' learning strategy to better handle data noise (e.g. inaccurate registration, scan noise) by discarding potentially inaccurate supervision signals during training. The resultant generalizable and robust topologically consistent multi-view facial capture system (GRAPE) can be readily used to capture data on a different camera array, reducing great effort on data collection and processing. Experiments on the FaMoS and FaceScape datasets demonstrate the effectiveness of the proposed method.

cs.CV↗

arXiv2024

Spectral-isolated photonic topological corner mode with a tunable mode area and stable frequency

Emergent collective modes in lattices give birth to many intriguing physical phenomena in condensed matter physics. Among these collective modes, large-area modes typically feature small-level spacings, while a mode with stable frequency tends to be spatially tightly confined. Here, we theoretically propose and experimentally demonstrate a spectral-isolated photonic topological corner mode with a tunable mode area and stable frequency in a two-dimensional photonic crystal. This mode emerges from hybridizing the large-area homogeneous mode and in-gap topological corner modes. Remarkably, this large-area homogeneous mode possesses unique chirality and has a tunable mode area under the change of the mass term of the inner topological non-trivial lattice. We experimentally observe such topological large-area corner modes(TLCM) in a 2D photonic system and demonstrate the robustness by introducing disorders in the structure. Our findings have propelled the forefront of higher-order topology research, transitioning it from single-lattice systems to multi-lattice systems. They may support promising potential applications, particularly in vertical-cavity surface-emitting lasers.

cond-mat.mes-hall↗

arXiv2024

Detailed Mapping of the Galactic Disk Structure in the Solar Neighborhood through LAMOST K Dwarfs

The Galactic disk is one of the main components of the Milky Way, which contributes most of the luminosity. Its structure is essential for understanding the formation and evolution of the Milky Way. Using 174,443 K-type dwarf stars observed by both LAMOST and Gaia DR3, we study the disk density profile in the local volume within 1,200 pc. In the azimuthal dimension, we find strong asymmetric signal of the thin disk. The surface density and the scale height of the southern disk significantly change versus the azimuthal angle at the same galactocentric distance $R$. Meanwhile, in the vertical dimension, the scale height of the northern disk has quite different trend than that of the southern one. The scale height of the southern disk shows a decreasing trend with $ϕ\sim-2.5^\circ$, and change to an increasing one with $ϕ\sim5.0^°$. Meanwhile, the scale height of the northern disk has a consistently smaller increase. Finally, we divide the entire sample into three subsamples based on metallicity and all three subsamples show significant non-axisymmetric and north-south asymmetric signals in the Galactic disk. Furthermore, we find that the scale height of the metal-poor ([Fe/H] $<$ -0.4 dex) subsample in the northern disk is greater than that of the metal-rich ([Fe/H] $>$ -0.1 dex) subsample. However, in the southern disk, the scale height exhibits varying relationships across different metallicity slices.

astro-ph.GA↗

arXiv2024

Knowledge Overshadowing Causes Amalgamated Hallucination in Large Language Models

Hallucination is often regarded as a major impediment for using large language models (LLMs), especially for knowledge-intensive tasks. Even when the training corpus consists solely of true statements, language models still generate hallucinations in the form of amalgamations of multiple facts. We coin this phenomenon as ``knowledge overshadowing'': when we query knowledge from a language model with multiple conditions, some conditions overshadow others, leading to hallucinated outputs. This phenomenon partially stems from training data imbalance, which we verify on both pretrained models and fine-tuned models, over a wide range of LM model families and sizes.From a theoretical point of view, knowledge overshadowing can be interpreted as over-generalization of the dominant conditions (patterns). We show that the hallucination rate grows with both the imbalance ratio (between the popular and unpopular condition) and the length of dominant condition description, consistent with our derived generalization bound. Finally, we propose to utilize overshadowing conditions as a signal to catch hallucination before it is produced, along with a training-free self-contrastive decoding method to alleviate hallucination during inference. Our proposed approach showcases up to 82% F1 for hallucination anticipation and 11.2% to 39.4% hallucination control, with different models and datasets.

cs.CL↗

arXiv2024

Diff-Oracle: Deciphering Oracle Bone Scripts with Controllable Diffusion Model

Deciphering oracle bone scripts plays an important role in Chinese archaeology and philology. However, a significant challenge remains due to the scarcity of oracle character images. To overcome this issue, we propose Diff-Oracle, a novel approach based on diffusion models to generate a diverse range of controllable oracle characters. Unlike traditional diffusion models that operate primarily on text prompts, Diff-Oracle incorporates a style encoder that utilizes style reference images to control the generation style. This encoder extracts style prompts from existing oracle character images, where style details are converted into a text embedding format via a pretrained language-vision model. On the other hand, a content encoder is integrated within Diff-Oracle to capture specific content details from content reference images, ensuring that the generated characters accurately represent the intended glyphs. To effectively train Diff-Oracle, we pre-generate pixel-level paired oracle character images (i.e., style and content images) by an image-to-image translation model. Extensive qualitative and quantitative experiments are conducted on datasets Oracle-241 and OBC306. While significantly surpassing present generative methods in terms of image generation, Diff-Oracle substantially benefits downstream oracle character recognition, outperforming all existing SOTAs by a large margin. In particular, on the challenging OBC306 dataset, Diff-Oracle leads to an accuracy gain of 7.70% in the zero-shot setting and is able to recognize unseen oracle character images with the accuracy of 84.62%, achieving a new benchmark for deciphering oracle bone scripts.

cs.CV↗

arXiv2024

Activity-Induced Stiffness, Entanglement Network and Dynamic Slowdown in Unentangled Semidilute Polymer Solutions

Active polymers possess numerous unique properties that are quite different from those observed in the system of small active molecule due to the intricate interplay between their activity and topological constraints. This study focuses on the conformational changes induced by activity, impacting effective stiffness and crucially influencing entanglement and dynamics. When the two terminals of a linear chain undergo active modification through coupling to a high-temperature thermal bath, there is a substantial increase in chain size, indicating a notable enhancement in effective stiffness. Unlike in passive semiflexible chains where stiffness predominantly affects local bond angles, activity-induced stiffness manifests at the scale of tens of monomers. While activity raises the ambient temperature, it significantly decreases diffusion by over an order of magnitude. The slowdown of dynamics observed can be attributed to increased entanglement due to chain elongation.

cond-mat.soft↗

arXiv2024

PROUD: PaRetO-gUided Diffusion Model for Multi-objective Generation

Recent advancements in the realm of deep generative models focus on generating samples that satisfy multiple desired properties. However, prevalent approaches optimize these property functions independently, thus omitting the trade-offs among them. In addition, the property optimization is often improperly integrated into the generative models, resulting in an unnecessary compromise on generation quality (i.e., the quality of generated samples). To address these issues, we formulate a constrained optimization problem. It seeks to optimize generation quality while ensuring that generated samples reside at the Pareto front of multiple property objectives. Such a formulation enables the generation of samples that cannot be further improved simultaneously on the conflicting property functions and preserves good quality of generated samples. Building upon this formulation, we introduce the PaRetO-gUided Diffusion model (PROUD), wherein the gradients in the denoising process are dynamically adjusted to enhance generation quality while the generated samples adhere to Pareto optimality. Experimental evaluations on image generation and protein generation tasks demonstrate that our PROUD consistently maintains superior generation quality while approaching Pareto optimality across multiple property functions compared to various baselines.

cs.LG↗

arXiv2024

The Compressible Navier-Stokes Equations on the Multi-Connected Domains

This paper investigates the isentropic compressible Navier-Stokes equations on k-connected domains under Navier-slip boundary conditions. We study the multi-solvability of the stationary systems on general domains, which is closely related with the Cauchy-Riemann systems and critical points of harmonic functions on the domain. Then based on the structure of Green's functions, the commutator estimates are obtained on the circular domains and extended to general domains with the help of conformal mappings. Moreover, we will utilize these assertions to discuss the global well-posedness and large time behaviours of the non-stationary systems on general domains with large initial values containing vacuum.

math.AP↗

arXiv2024

Unveiling Mass Transfer in Solar Flares: Insights from Elemental Abundance Evolutions Observed by Chang'E-2 Solar X-ray Monitor

Understanding how elemental abundances evolve during solar flares helps shed light on the mass and energy transfer between different solar atmospheric layers. However, prior studies have mostly concentrated on averaged abundances or specific flare phases, leaving a gap in exploring the comprehensive observations throughout the entire flare process. Consequently, investigations into this area are relatively scarce. Exploiting the Solar X-ray Monitor data obtained from the Chang'E-2 lunar orbiter, we present two comprehensive soft X-ray spectroscopic observations of flares in active regions, AR 11149 and 11158, demonstrating elemental abundance evolutions under different conditions. Our findings unveil the inverse first ionization potential (IFIP) effect during flares for Fe for the first time, and reaffirm its existence for Si. Additionally, we observed a rare depletion of elemental abundances, marking the second IFIP effect in flare decay phases. Our study offers a CSHKP model-based interpretation to elucidate the formation of both the FIP and IFIP effects in flare dynamics, with the inertia effect being incorporated into the ponderomotive force fractionation model.

astro-ph.SR↗

arXiv2024

Operating Single-Photon Circulator by Spinning Optical Resonators

A circulator is one of the crucial devices in quantum networks and simulations. We propose a four-port circulator that regulate the flow of single photons at muti-frequency points by studying the coherent transmission of a single photon in a coupled system of two resonators and two waveguides. When both resonators are static or rotate at the same angular velocity, single-photon transport demonstrates reciprocity; however, when the angular velocities differ, four distinct frequency points emerge where photon circulation can occur. In particular, when the angular velocities of the two resonators are equal and opposite, there are two different frequency points where photon circulation can be achieved, and there is a frequency point where a single photon input from any waveguide can be completely routed to the other waveguide. Interestingly, by rotating the two resonators, the single-photon circulation suppressed by the internal defect-induced backscattering can be restored.

quant-ph↗

arXiv2024

Cavity Modified Oscillating Bound States with a $Λ$-type giant emitter in a linear waveguide

We study a system composed by a three-level giant atom (3GA), a waveguide initially in the vacuum state, and a single-mode cavity. The 3GA-cavity system is in a strong-coupling regime, and the distance between the coupling points is comparable to the coherent length of a spontaneously emitted photon. The dynamics of the 3GA and its radiative field in the waveguide for long time are analyzed. Besides the steady value, we also found an oscillatory character of the excited state population, a signature of oscillating bound states which is generated by the superposition of two bound states in the continuum. The radiative field propagates in the cavity-like geometry formed by the coupling points. When one bound state is emergent, a sine-like interference pattern is visible for the emitted field intensity in spacetime. An oscillatory character in time and a beat in space for the emitted field intensity are observed when two bound states are emergent in a subspace. The wavelengths and the periods are controlled by the number of the photons in the cavity.

quant-ph↗

arXiv2024

CoSafe: Evaluating Large Language Model Safety in Multi-Turn Dialogue Coreference

As large language models (LLMs) constantly evolve, ensuring their safety remains a critical research problem. Previous red-teaming approaches for LLM safety have primarily focused on single prompt attacks or goal hijacking. To the best of our knowledge, we are the first to study LLM safety in multi-turn dialogue coreference. We created a dataset of 1,400 questions across 14 categories, each featuring multi-turn coreference safety attacks. We then conducted detailed evaluations on five widely used open-source LLMs. The results indicated that under multi-turn coreference safety attacks, the highest attack success rate was 56% with the LLaMA2-Chat-7b model, while the lowest was 13.9% with the Mistral-7B-Instruct model. These findings highlight the safety vulnerabilities in LLMs during dialogue coreference interactions.

cs.CL↗

arXiv2024

Gaussian Control with Hierarchical Semantic Graphs in 3D Human Recovery

Although 3D Gaussian Splatting (3DGS) has recently made progress in 3D human reconstruction, it primarily relies on 2D pixel-level supervision, overlooking the geometric complexity and topological relationships of different body parts. To address this gap, we introduce the Hierarchical Graph Human Gaussian Control (HUGS) framework for achieving high-fidelity 3D human reconstruction. Our approach involves leveraging explicitly semantic priors of body parts to ensure the consistency of geometric topology, thereby enabling the capture of the complex geometrical and topological associations among body parts. Additionally, we disentangle high-frequency features from global human features to refine surface details in body parts. Extensive experiments demonstrate that our method exhibits superior performance in human body reconstruction, particularly in enhancing surface details and accurately reconstructing body part junctions. Codes are available at https://wanghongsheng01.github.io/HUGS/.

cs.CV↗