arXiv subjects

Fei Li

Publications and source records attributed to Fei Li.

At least 91 records · Page 5Linked to original sources

arXiv2024

Baichuan Alignment Technical Report

We introduce Baichuan Alignment, a detailed analysis of the alignment techniques employed in the Baichuan series of models. This represents the industry's first comprehensive account of alignment methodologies, offering valuable insights for advancing AI research. We investigate the critical components that enhance model performance during the alignment process, including optimization methods, data strategies, capability enhancements, and evaluation processes. The process spans three key stages: Prompt Augmentation System(PAS), Supervised Fine-Tuning(SFT), and Preference Alignment. The problems encountered, the solutions applied, and the improvements made are thoroughly recorded. Through comparisons across well-established benchmarks, we highlight the technological advancements enabled by Baichuan Alignment. Baichuan-Instruct is an internal model, while Qwen2-Nova-72B and Llama3-PBM-Nova-70B are instruct versions of the Qwen2-72B and Llama-3-70B base models, optimized through Baichuan Alignment. Baichuan-Instruct demonstrates significant improvements in core capabilities, with user experience gains ranging from 17% to 28%, and performs exceptionally well on specialized benchmarks. In open-source benchmark evaluations, both Qwen2-Nova-72B and Llama3-PBM-Nova-70B consistently outperform their respective official instruct versions across nearly all datasets. This report aims to clarify the key technologies behind the alignment process, fostering a deeper understanding within the community. Llama3-PBM-Nova-70B model is available at https://huggingface.co/PKU-Baichuan-MLSystemLab/Llama3-PBM-Nova-70B.

cs.LG↗

arXiv2024

M$^{3}$D: A Multimodal, Multilingual and Multitask Dataset for Grounded Document-level Information Extraction

Multimodal information extraction (IE) tasks have attracted increasing attention because many studies have shown that multimodal information benefits text information extraction. However, existing multimodal IE datasets mainly focus on sentence-level image-facilitated IE in English text, and pay little attention to video-based multimodal IE and fine-grained visual grounding. Therefore, in order to promote the development of multimodal IE, we constructed a multimodal multilingual multitask dataset, named M$^{3}$D, which has the following features: (1) It contains paired document-level text and video to enrich multimodal information; (2) It supports two widely-used languages, namely English and Chinese; (3) It includes more multimodal IE tasks such as entity recognition, entity chain extraction, relation extraction and visual grounding. In addition, our dataset introduces an unexplored theme, i.e., biography, enriching the domains of multimodal IE resources. To establish a benchmark for our dataset, we propose an innovative hierarchical multimodal IE model. This model effectively leverages and integrates multimodal information through a Denoised Feature Fusion Module (DFFM). Furthermore, in non-ideal scenarios, modal information is often incomplete. Thus, we designed a Missing Modality Construction Module (MMCM) to alleviate the issues caused by missing modalities. Our model achieved an average performance of 53.80% and 53.77% on four tasks in English and Chinese datasets, respectively, which set a reasonable standard for subsequent research. In addition, we conducted more analytical experiments to verify the effectiveness of our proposed module. We believe that our work can promote the development of the field of multimodal IE.

cs.CL↗

arXiv2024

3 mm Spectroscopic Observations of Massive Star-Forming Regions with IRAM 30-m

Broadband spectroscopic observations with high sensitivity provide an unbiased way to detect emissions of molecules in space. We present deep observations from ~ 105.8 GHz to 113.6 GHz toward 50 Galactic massive star-forming regions using IRAM 30-m millimeter telescope, with noise levels ranging from 6 to 29 at frequency channel spacing of 195 kHz, which corresponds to ~ 0.54 km/s at 110 GHz. Totally, 27 molecular species have been identified, of which 16 are complex organic molecules. The related parameters, such as peak temperature, integrated intensity, and line width of the identified molecular lines were obtained. The line widths of the chemically related molecules show strong positive correlations, suggesting they likely originate from similar gases within star-forming regions. This work highlights the fundamental properties of the detected molecular lines and offers a valuable dataset for further studies on the astrochemical evolution of molecules in massive star-forming cores.

astro-ph.SR↗

arXiv2024

High-Quality Passive Acoustic Mapping with the Cross-Correlated Angular Spectrum Method

While passive acoustic mapping (PAM) has been advanced for monitoring acoustic cavitation activity in focused ultrasound (FUS) therapy, achieving both real-time and high-quality imaging capabilities is still challenging. The angular spectrum (AS) method presents the most efficient algorithm for PAM, but it suffers from artifacts and low resolution due to the diffraction pattern of the imaging array. Data-adaptive beamformers suppress artifacts well, but their overwhelming computational complexity, more than two orders of magnitude higher than the classical time exposure acoustic (TEA) method, hinders their application in real-time. In this work, we introduce the cross-correlated AS method to address the challenge. This method is based on cross-correlating the AS back-propagated wave fields, in the frequency domain, measured by different apodized sub-apertures of the transducer array to provide the normalized correlation coefficient (NCC) matrix for artifacts suppression. We observed that the spatial pattern of NCC matrix is variable which can be utilized by the triple apodization with cross-correlation (TAX) with AS scheme, namely the AS-TAX method, for optimal artifacts suppression outcomes. Both the phantom and mouse tumor experiments showed that: 1) the AS-TAX method has comparable image quality as the data-adaptive beamformers, reducing the energy spread area by 34.8-66.6% and improving image signal-to-noise ratio by 10.7-14.5 dB compared to TEA; 2) it reduces the computational complexity by two orders of magnitude compared to TEA allowing millisecond-level image reconstruction speed with a parallel implementation; 3) it can well map microbubble cavitation activity of different status (stable or inertial). The AS-TAX method represents a real-time approach to monitor cavitation-based FUS therapy with high image quality.

physics.med-ph↗

arXiv2024

Mr.TPL: A Method for Multi-Pin Net Router in Triple Patterning Lithography

Triple patterning lithography (TPL) has been recognized as one of the most promising solutions to print critical features in advanced technology nodes. A critical challenge within TPL is the effective assignment of the layout to masks. Recently, various layout decomposition methods and TPL-aware routing methods have been proposed to consider TPL. However, these methods typically result in numerous conflicts and stitches, and are mainly designed for 2-pin nets. This paper proposes a multi-pin net routing method in triple patterning lithography, called Mr.TPL. Experimental results demonstrate that Mr.TPL reduces color conflicts by 81.17%, decreases stitches by 76.89%, and achieves up to 5.4X speed improvement compared to the state-of-the-art TPL-aware routing method.

cs.OH↗

arXiv2024

The Effect of Galaxy Interactions on Starbursts in Milky Way-Mass Galaxies in FIRE Simulations

Simulations and observations suggest that galaxy interactions may enhance the star formation rate (SFR) in merging galaxies. One proposed mechanism is the torque exerted on the gas and stars in the larger galaxy by the smaller galaxy. We analyze the interaction torques and star formation activity on six galaxies from the FIRE-2 simulation suite with masses comparable to the Milky Way galaxy at redshift $z=0$. We trace the halos from $z = 3.6$ to $z=0$, calculating the torque exerted by the nearby galaxies on the gas in the central galaxy. We calculate the correlation between the torque and the SFR across the simulations for various mass ratios. For near-equal-stellar-mass-ratio interactions in the galaxy sample, occurring between $z=1.2-3.6$, there is a positive and statistically significant correlation between the torque from nearby galaxies on the gas of the central galaxies and the SFR. For all other samples, no statistically significant correlation is found between the torque and the SFR. Our analysis shows that some, but not all, major interactions cause starbursts in the simulated Milky Way-mass galaxies, and that most starbursts are not caused by galaxy interactions. The transition from `bursty' at high redshift ($z\gtrsim1$) to `steady' star-formation state at later times is independent of the interaction history of the galaxies, and most of the interactions do not leave significant imprints on the overall trend of the star formation history of the galaxies.

astro-ph.GA↗

arXiv2024

Photon acceleration of high-intensity vector vortex beams into the extreme ultraviolet

Extreme ultraviolet (XUV) light sources allow for the probing of bound electron dynamics on attosecond scales, interrogation of high-energy-density matter, and access to novel regimes of strong-field quantum electrodynamics. Despite the importance of these applications, coherent XUV sources remain relatively rare, and those that do exist are limited in their peak intensity and spatio-polarization structure. Here, we demonstrate that photon acceleration of an optical vector vortex pulse in the moving density gradient of an electron beam-driven plasma wave can produce a high-intensity, tunable-wavelength XUV pulse with the same vector vortex structure as the original pulse. Quasi-3D, boosted-frame particle-in-cell simulations show the transition of optical vector vortex pulses with 800-nm wavelengths and intensities below $10^{18}$ W/cm$^2$ to XUV vector vortex pulses with 36-nm wavelengths and intensities exceeding $10^{20}$ W/cm$^2$ over a distance of 1.2 cm. The XUV pulses have sub-femtosecond durations and nearly flat phase fronts. The production of such high-quality, high-intensity XUV vector vortex pulses could expand the utility of XUV light as a diagnostic and driver of novel light-matter interactions.

physics.optics↗

arXiv2024

Multiple kernel concept factorization algorithm based on global fusion

Non-negative Matrix Factorization(NMF) algorithm can only be used to find low rank approximation of original non-negative data while Concept Factorization(CF) algorithm extends matrix factorization to single non-linear kernel space, improving learning ability and adaptability of matrix factorization. In unsupervised environment, to design or select proper kernel function for specific dataset, a new algorithm called Globalized Multiple Kernel CF(GMKCF)was proposed. Multiple candidate kernel functions were input in the same time and learned in the CF framework based on global linear fusion, obtaining a clustering result with high quality and stability and solving the problem of kernel function selection that the CF faced. The convergence of the proposed algorithm was verified by solving the model with alternate iteration. The experimental results on several real databases show that the proposed algorithm outperforms comparison algorithms in data clustering, such as Kernel K-Means(KKM), Spectral Clustering(SC), Kernel CF(KCF), Co-regularized multi-view spectral clustering(Coreg), and Robust Multiple KKM(RMKKM).

cs.LG↗

arXiv2024

Some three-weight linear codes and their complete weight enumerators and weight hierarchies

Linear codes with a few weights can be applied to secrete sharing, authentication codes, association schemes and strongly regular graphs. For an odd prime power $q$, we construct a class of three-weight $\F_q$-linear codes from quadratic functions via a bivariate construction and then determine the complete weight enumerators and weight hierarchies of these linear codes completely. This paper generalizes some results in Li et al. (2022) and Hu et al. (2024).

cs.IT↗

arXiv2024

Autocorrelation Measurement of Attosecond Pulses Based on Two-Photon Double Ionization

Autocorrelation measurement is theoretically demonstrated to characterize attosecond pulses by studying the two-photon double ionization (TPDI) process. An interferometric autocorrelation curve is presented in the change of TPDI probability with the time delay between two identical attosecond pulses, and its full width at half maximum (FWHM) $τ_{e}$ has a relationship $τ_{e}=1.77τ+15$ with the FWHM $τ$ of the attosecond pulse. The curve is also decoded to obtain the center frequency and FWHM of the attosecond pulse by fitting. In addition, the required peak intensity of the attosecond pulse is estimated to be on the order of $10^{16}\,\rm{Wcm^{-2}}$ in autocorrelation experiments. The findings pave the way for autocorrelation measurement of intense isolated attosecond pulses.

physics.atom-ph↗

arXiv2024

Simulating the Schwinger Model with a Regularized Variational Quantum Imaginary Time Evolution

The Schwinger model serves as a benchmark for testing non-perturbative algorithms in quantum chromodynamics (QCD), emphasizing its similarities to QCD in strong coupling regimes, primarily due to the phenomena such as confinement and charge screening. However, classical algorithms encounter challenges when simulating the Schwinger model, such as the "sign problem" and the difficulty in handling large-scale systems. These limitations motivate the exploration of alternative simulation approaches, including quantum computing techniques, to overcome the obstacles. While existing variational quantum algorithms (VQAs) methods for simulating the Schwinger model primarily rely on mathematical gradient-based optimization, which sometimes fail to provide intuitive and physically-guided optimization pathways. In contrast, the Variational Quantum Imaginary Time Evolution (VQITE) method offers a physically-inspired optimization approach. Therefore, we introduce that VQITE holds promise as a potent tool for simulating the Schwinger model. However, the standard VQITE method is not sufficiently stable, as it encounters difficulties with the non-invertible matrix problem. To address this issue, we have proposed a regularized version of the VQITE, which we have named the Regularized-VQITE (rVQITE) method, as it incorporates a truncation-based approach. Through numerical simulations, we demonstrate that our proposed rVQITE approach achieves better performance and exhibits faster convergence compared to other related techniques. We employ the rVQITE method to simulate the phase diagrams of various physical observables in the Schwinger model, and the resulting phase boundaries are in agreement with those obtained from an exact computational approach.

quant-ph↗

arXiv2024

Design and Implementation of TAO DAQ System

Purpose: The Taishan Antineutrino Observatory (TAO) is a satellite experiment of the Jiangmen Underground Neutrino Observatory (JUNO), also known as JUNO-TAO. Located close to one of the reactors of the Taishan Nuclear Power Plant, TAO will measure the antineutrino energy spectrum precisely as a reference spectrum for JUNO. The data acquisition (DAQ) system is designed to acquire data from the TAO readout electronics and process it with software trigger and data compression algorithms. The data storage bandwidth is limited by the onsite network to be less than 100 Mb/s. Methods: The system is designed based on a distributed architecture, with fully decoupled modules to facilitate customized design and implementation. It is divided into two main components: the data flow system and the online software. The online software serves as the foundation, providing the electronics configuration, the process management, the run control, and the information sharing. The data flow system facilitates continuous data acquisition from various electronic boards or trigger systems, assembles and processes raw data, and ultimately stores it on the disk. Results: The core functionality of the system has been designed and developed. The usability of the data flow system interface and the software trigger results have been verified during the pre-installation testing phase. Conclusion: The DAQ system has been deployed for the TAO experiment. It has also successfully been applied to the integration test of the detector and electronics prototypes.

physics.ins-det↗

arXiv2024

Dark Matter Physics in General NMSSM

In the General Next-to-Minimal Supersymmetric Standard Model (GNMSSM), singlet particles may form a secluded sector of dark matter (DM), in which Singlino-like DM could achieve the observed relic abundance through various channels such as $\tildeχ_1^0 \tildeχ_1^0 \to h_s h_s, A_s A_s, h_s A_s$, where $h_s$ and $A_s$ represent singlet-dominated CP-even and CP-odd Higgs bosons. We provide analytical formulas for both the spin-independent and spin-dependent cross sections of Singlino DM scattering with nucleons, illustrating their dependence on the model's parameters in a clear manner. We also present analytic expressions for the annihilation cross sections of these three important channels. Based on these preparations, we conducted Bayesian analyses of the GNMSSM and concluded that the theory significantly favored Singlino-dominated DM over Bino-like DM across a much broader range of parameters. The combined results from our numerical analyses and the formulas distinctly highlight crucial aspects of DM physics within the GNMSSM.

hep-ph↗

arXiv2024

Benchmarks and Challenges in Pose Estimation for Egocentric Hand Interactions with Objects

We interact with the world with our hands and see it through our own (egocentric) perspective. A holistic 3Dunderstanding of such interactions from egocentric views is important for tasks in robotics, AR/VR, action recognition and motion generation. Accurately reconstructing such interactions in 3D is challenging due to heavy occlusion, viewpoint bias, camera distortion, and motion blur from the head movement. To this end, we designed the HANDS23 challenge based on the AssemblyHands and ARCTIC datasets with carefully designed training and testing splits. Based on the results of the top submitted methods and more recent baselines on the leaderboards, we perform a thorough analysis on 3D hand(-object) reconstruction tasks. Our analysis demonstrates the effectiveness of addressing distortion specific to egocentric cameras, adopting high-capacity transformers to learn complex hand-object interactions, and fusing predictions from different views. Our study further reveals challenging scenarios intractable with state-of-the-art methods, such as fast hand motion, object reconstruction from narrow egocentric views, and close contact between two hands and objects. Our efforts will enrich the community's knowledge foundation and facilitate future hand studies on egocentric hand-object interactions.

cs.CV↗

arXiv2024

Noise Suppression for CRP Gathers Based on Self2Self with Dropout

Noise suppression in seismic data processing is a crucial research focus for enhancing subsequent imaging and reservoir prediction. Deep learning has shown promise in computer vision and holds significant potential for seismic data processing. However, supervised learning, which relies on clean labels to train network prediction models, faces challenges due to the unavailability of clean labels for seismic exploration data. In contrast, self-supervised learning substitutes traditional supervised learning with surrogate tasks by different auxiliary means, exploiting internal input data information. Inspired by Self2Self with Dropout, this paper presents a self-supervised learning-based noise suppression method called Self-Supervised Deep Convolutional Networks (SSDCN), specifically designed for Common Reflection Point (CRP) gathers. We utilize pairs of Bernoulli-sampled instances of the input noisy image as surrogate tasks to leverage its inherent structure. Furthermore, SSDCN incorporates geological knowledge through the normal moveout correction technique, which capitalizes on the approximately horizontal behavior and strong self-similarity observed in useful signal events within CRP gathers. By exploiting the discrepancy in self-similarity between the useful signals and noise in CRP gathers, SSDCN effectively extracts self-similarity features during training iterations, prioritizing the extraction of useful signals to achieve noise suppression. Experimental results on synthetic and actual CRP gathers demonstrate that SSDCN achieves high-fidelity noise suppression.

physics.geo-ph↗

arXiv2024

AVESFormer: Efficient Transformer Design for Real-Time Audio-Visual Segmentation

Recently, transformer-based models have demonstrated remarkable performance on audio-visual segmentation (AVS) tasks. However, their expensive computational cost makes real-time inference impractical. By characterizing attention maps of the network, we identify two key obstacles in AVS models: 1) attention dissipation, corresponding to the over-concentrated attention weights by Softmax within restricted frames, and 2) inefficient, burdensome transformer decoder, caused by narrow focus patterns in early stages. In this paper, we introduce AVESFormer, the first real-time Audio-Visual Efficient Segmentation transformer that achieves fast, efficient and light-weight simultaneously. Our model leverages an efficient prompt query generator to correct the behaviour of cross-attention. Additionally, we propose ELF decoder to bring greater efficiency by facilitating convolutions suitable for local features to reduce computational burdens. Extensive experiments demonstrate that our AVESFormer significantly enhances model performance, achieving 79.9% on S4, 57.9% on MS3 and 31.2% on AVSS, outperforming previous state-of-the-art and achieving an excellent trade-off between performance and speed. Code can be found at https://github.com/MarkXCloud/AVESFormer.git.

cs.CV↗

arXiv2024

Generative Sentiment Analysis via Latent Category Distribution and Constrained Decoding

Fine-grained sentiment analysis involves extracting and organizing sentiment elements from textual data. However, existing approaches often overlook issues of category semantic inclusion and overlap, as well as inherent structural patterns within the target sequence. This study introduces a generative sentiment analysis model. To address the challenges related to category semantic inclusion and overlap, a latent category distribution variable is introduced. By reconstructing the input of a variational autoencoder, the model learns the intensity of the relationship between categories and text, thereby improving sequence generation. Additionally, a trie data structure and constrained decoding strategy are utilized to exploit structural patterns, which in turn reduces the search space and regularizes the generation process. Experimental results on the Restaurant-ACOS and Laptop-ACOS datasets demonstrate a significant performance improvement compared to baseline models. Ablation experiments further confirm the effectiveness of latent category distribution and constrained decoding strategy.

cs.CL↗

arXiv2024

Lomics: Generation of Pathways and Gene Sets using Large Language Models for Transcriptomic Analysis

Interrogation of biological pathways is an integral part of omics data analysis. Large language models (LLMs) enable the generation of custom pathways and gene sets tailored to specific scientific questions. These targeted sets are significantly smaller than traditional pathway enrichment analysis libraries, reducing multiple hypothesis testing and potentially enhancing statistical power. Lomics (Large Language Models for Omics Studies) v1.0 is a python-based bioinformatics toolkit that streamlines the generation of pathways and gene sets for transcriptomic analysis. It operates in three steps: 1) deriving relevant pathways based on the researcher's scientific question, 2) generating valid gene sets for each pathway, and 3) outputting the results as .GMX files. Lomics also provides explanations for pathway selections. Consistency and accuracy are ensured through iterative processes, JSON format validation, and HUGO Gene Nomenclature Committee (HGNC) gene symbol verification. Lomics serves as a foundation for integrating LLMs into omics research, potentially improving the specificity and efficiency of pathway analysis.

q-bio.MN↗