arXiv subjects

Meng Liu

Publications and source records attributed to Meng Liu.

At least 127 records · Page 7Linked to original sources

arXiv2023

Efficient ion acceleration driven by a Laguerre Gaussian laser in near-critical-density plasma

Laser-driven ion accelerators have the advantages of compact size, high density, and short bunch duration over conventional accelerators. Nevertheless, it is still challenging to generate ion beams with quasi-monoenergetic peak and low divergence in the experiments with the current ultra-high intensity laser and thin target technologies. Here we propose a scheme that a Laguerre Gaussian laser irradiates a near-critical-density (NCD) plasma to generate a quasi-monoenergetic and low-divergence proton beam. The Laguerre Gaussian laser pulse in NCD plasma excites a moving longitudinal electrostatic field with a large amplitude, and it maintains the inward bowl-shape for dozens of laser durations. This special distribution of the longitudinal electrostatic field can accelerate and converge the protons simultaneously. Our particle-in-cell (PIC) simulation shows that the efficient proton acceleration can be realized with the Laguerre Gaussian laser intensity ranging from 3.9-10^21 Wcm^-2 -1.6-10^22 Wcm^-2 available in the near future, e.g., a quasi-monoenergetic proton beam with peak energy ~115 MeV and divergence angles less than 50 can be generated by a 5.3-10^21 Wcm^-2 pulse. This work could provide a reference for the high-quality ion beam generation with PW-class laser systems available recently.

physics.plasm-ph↗

arXiv2023

3D Molecular Geometry Analysis with 2D Graphs

Ground-state 3D geometries of molecules are essential for many molecular analysis tasks. Modern quantum mechanical methods can compute accurate 3D geometries but are computationally prohibitive. Currently, an efficient alternative to computing ground-state 3D molecular geometries from 2D graphs is lacking. Here, we propose a novel deep learning framework to predict 3D geometries from molecular graphs. To this end, we develop an equilibrium message passing neural network (EMPNN) to better capture ground-state geometries from molecular graphs. To provide a testbed for 3D molecular geometry analysis, we develop a benchmark that includes a large-scale molecular geometry dataset, data splits, and evaluation protocols. Experimental results show that EMPNN can efficiently predict more accurate ground-state 3D geometries than RDKit and other deep learning methods. Results also show that the proposed framework outperforms self-supervised learning methods on property prediction tasks.

physics.chem-ph↗

arXiv2023

Learnable Pillar-based Re-ranking for Image-Text Retrieval

Image-text retrieval aims to bridge the modality gap and retrieve cross-modal content based on semantic similarities. Prior work usually focuses on the pairwise relations (i.e., whether a data sample matches another) but ignores the higher-order neighbor relations (i.e., a matching structure among multiple data samples). Re-ranking, a popular post-processing practice, has revealed the superiority of capturing neighbor relations in single-modality retrieval tasks. However, it is ineffective to directly extend existing re-ranking algorithms to image-text retrieval. In this paper, we analyze the reason from four perspectives, i.e., generalization, flexibility, sparsity, and asymmetry, and propose a novel learnable pillar-based re-ranking paradigm. Concretely, we first select top-ranked intra- and inter-modal neighbors as pillars, and then reconstruct data samples with the neighbor relations between them and the pillars. In this way, each sample can be mapped into a multimodal pillar space only using similarities, ensuring generalization. After that, we design a neighbor-aware graph reasoning module to flexibly exploit the relations and excavate the sparse positive items within a neighborhood. We also present a structure alignment constraint to promote cross-modal collaboration and align the asymmetric modalities. On top of various base backbones, we carry out extensive experiments on two benchmark datasets, i.e., Flickr30K and MS-COCO, demonstrating the effectiveness, superiority, generalization, and transferability of our proposed re-ranking paradigm.

cs.CV↗

arXiv2023

SARF: Aliasing Relation Assisted Self-Supervised Learning for Few-shot Relation Reasoning

Few-shot relation reasoning on knowledge graphs (FS-KGR) aims to infer long-tail data-poor relations, which has drawn increasing attention these years due to its practicalities. The pre-training of previous methods needs to manually construct the meta-relation set, leading to numerous labor costs. Self-supervised learning (SSL) is treated as a solution to tackle the issue, but still at an early stage for FS-KGR task. Moreover, most of the existing methods ignore leveraging the beneficial information from aliasing relations (AR), i.e., data-rich relations with similar contextual semantics to the target data-poor relation. Therefore, we proposed a novel Self-Supervised Learning model by leveraging Aliasing Relations to assist FS-KGR, termed SARF. Concretely, four main components are designed in our model, i.e., SSL reasoning module, AR-assisted mechanism, fusion module, and scoring function. We first generate the representation of the co-occurrence patterns in a generative manner. Meanwhile, the representations of aliasing relations are learned to enhance reasoning in the AR-assist mechanism. Besides, multiple strategies, i.e., simple summation and learnable fusion, are offered for representation fusion. Finally, the generated representation is used for scoring. Extensive experiments on three few-shot benchmarks demonstrate that SARF achieves state-of-the-art performance compared with other methods in most cases.

cs.LG↗

arXiv2023

Gradient-Guided Importance Sampling for Learning Binary Energy-Based Models

Learning energy-based models (EBMs) is known to be difficult especially on discrete data where gradient-based learning strategies cannot be applied directly. Although ratio matching is a sound method to learn discrete EBMs, it suffers from expensive computation and excessive memory requirements, thereby resulting in difficulties in learning EBMs on high-dimensional data. Motivated by these limitations, in this study, we propose ratio matching with gradient-guided importance sampling (RMwGGIS). Particularly, we use the gradient of the energy function w.r.t. the discrete data space to approximately construct the provably optimal proposal distribution, which is subsequently used by importance sampling to efficiently estimate the original ratio matching objective. We perform experiments on density modeling over synthetic discrete data, graph generation, and training Ising models to evaluate our proposed method. The experimental results demonstrate that our method can significantly alleviate the limitations of ratio matching, perform more effectively in practice, and scale to high-dimensional problems. Our implementation is available at https://github.com/divelab/RMwGGIS.

cs.LG↗

arXiv2023

Estimating Stellar Parameters and Identifying Very Metal-poor Stars Using Convolutional Neural Networks for Low-resolution Spectra (R~200)

Very metal-poor (VMP, [Fe/H]<-2.0) stars offer a wealth of information on the nature and evolution of elemental production in the early galaxy and universe. The upcoming China Space Station Telescope (CSST) will provide us with a large amount of spectroscopic data that may contain plenty of VMP stars, and thus it is crucial to determine the stellar atmospheric parameters ($T_{eff}$, $\log g$, and [Fe/H]) for low-resolution spectra similar to the CSST spectra (R~200). In this paper, a two-dimensional Convolutional Neural Network (CNN) model with three convolutional layers and two fully connected layers is constructed. The principal aim of this work is to measure the ability of this model to estimate stellar parameters on low-resolution (R~200) spectra and to identify VMP stars so that we can better search for VMP stars in the spectra observed by CSST.We mainly use 10,008 observed spectra of VMP stars from LAMOST DR3, and 16,638 spectra of common stars ([Fe/H]>-2.0) from LAMOST DR8 for the experiment and make comparisons. All spectra are reduced to R~200 to match the resolution of the CSST and are preprocessed and collapsed into two-dimensional spectra for input to the CNN model. The results show that the MAE values are 99.40 K for $T_{eff}$, 0.22 dex for $\log g$, 0.14 dex for [Fe/H], and 0.26 dex for [C/Fe], respectively. Besides, the CNN model efficiently identifies VMP stars with a precision of 94.77%. The validation and practicality of this model are also tested on the MARCS synthetic spectra. This paper powerfully demonstrates the effectiveness of the proposed CNN model in estimating stellar parameters for low-resolution spectra (R~200) and recognizing VMP stars that are of interest for stellar population and galactic evolution work.

astro-ph.SR↗

arXiv2023

Cross-modal Audio-visual Co-learning for Text-independent Speaker Verification

Visual speech (i.e., lip motion) is highly related to auditory speech due to the co-occurrence and synchronization in speech production. This paper investigates this correlation and proposes a cross-modal speech co-learning paradigm. The primary motivation of our cross-modal co-learning method is modeling one modality aided by exploiting knowledge from another modality. Specifically, two cross-modal boosters are introduced based on an audio-visual pseudo-siamese structure to learn the modality-transformed correlation. Inside each booster, a max-feature-map embedded Transformer variant is proposed for modality alignment and enhanced feature generation. The network is co-learned both from scratch and with pretrained models. Experimental results on the LRSLip3, GridLip, LomGridLip, and VoxLip datasets demonstrate that our proposed method achieves 60% and 20% average relative performance improvement over independently trained audio-only/visual-only and baseline fusion systems, respectively.

cs.SD↗

arXiv2022

Identifying hot subdwarf stars from photometric data using Gaussian mixture model and graph neural network

Hot subdwarf stars are very important for understanding stellar evolution, stellar astrophysics, and binary star systems. Identifying more such stars can help us better understand their statistical distribution, properties, and evolution. In this paper, we present a new method to search for hot subdwarf stars in photometric data (b, y, g, r, i, z) using a machine learning algorithm, graph neural network, and Gaussian mixture model. We use a Gaussian mixture model and Markov distance to build the graph structure, and on the graph structure, we use a graph neural network to identify hot subdwarf stars from 86 084 stars, when the recall, precision, and f1 score are maximized on the original, weight and synthetic minority oversampling technique datasets. Finally, from 21 885 candidates, we selected approximately 6 000 stars that were the most similar to the hot subdwarf star.

astro-ph.SR↗

arXiv2022

Coordinating Cross-modal Distillation for Molecular Property Prediction

In recent years, molecular graph representation learning (GRL) has drawn much more attention in molecular property prediction (MPP) problems. The existing graph methods have demonstrated that 3D geometric information is significant for better performance in MPP. However, accurate 3D structures are often costly and time-consuming to obtain, limiting the large-scale application of GRL. It is an intuitive solution to train with 3D to 2D knowledge distillation and predict with only 2D inputs. But some challenging problems remain open for 3D to 2D distillation. One is that the 3D view is quite distinct from the 2D view, and the other is that the gradient magnitudes of atoms in distillation are discrepant and unstable due to the variable molecular size. To address these challenging problems, we exclusively propose a distillation framework that contains global molecular distillation and local atom distillation. We also provide a theoretical insight to justify how to coordinate atom and molecular information, which tackles the drawback of variable molecular size for atom information distillation. Experimental results on two popular molecular datasets demonstrate that our proposed model achieves superior performance over other methods. Specifically, on the largest MPP dataset PCQM4Mv2 served as an "ImageNet Large Scale Visual Recognition Challenge" in the field of graph ML, the proposed method achieved a 6.9% improvement compared with the best works. And we obtained fourth place with the MAE of 0.0734 on the test-challenge set for OGB-LSC 2022 Graph Regression Task. We will release the code soon.

cs.LG↗

arXiv2022

Spherical Message Passing for 3D Graph Networks

We consider representation learning of 3D molecular graphs in which each atom is associated with a spatial position in 3D. This is an under-explored area of research, and a principled message passing framework is currently lacking. In this work, we conduct analyses in the spherical coordinate system (SCS) for the complete identification of 3D graph structures. Based on such observations, we propose the spherical message passing (SMP) as a novel and powerful scheme for 3D molecular learning. SMP dramatically reduces training complexity, enabling it to perform efficiently on large-scale molecules. In addition, SMP is capable of distinguishing almost all molecular structures, and the uncovered cases may not exist in practice. Based on meaningful physically-based representations of 3D information, we further propose the SphereNet for 3D molecular learning. Experimental results demonstrate that the use of meaningful 3D information in SphereNet leads to significant performance improvements in prediction tasks. Our results also demonstrate the advantages of SphereNet in terms of capability, efficiency, and scalability. Our code is publicly available as part of the DIG library (https://github.com/divelab/DIG).

cs.LG↗

arXiv2022

I4U System Description for NIST SRE'20 CTS Challenge

This manuscript describes the I4U submission to the 2020 NIST Speaker Recognition Evaluation (SRE'20) Conversational Telephone Speech (CTS) Challenge. The I4U's submission was resulted from active collaboration among researchers across eight research teams - I$^2$R (Singapore), UEF (Finland), VALPT (Italy, Spain), NEC (Japan), THUEE (China), LIA (France), NUS (Singapore), INRIA (France) and TJU (China). The submission was based on the fusion of top performing sub-systems and sub-fusion systems contributed by individual teams. Efforts have been spent on the use of common development and validation sets, submission schedule and milestone, minimizing inconsistency in trial list and score file format across sites.

eess.AS↗

arXiv2022

FedVMR: A New Federated Learning method for Video Moment Retrieval

Despite the great success achieved, existing video moment retrieval (VMR) methods are developed under the assumption that data are centralizedly stored. However, in real-world applications, due to the inherent nature of data generation and privacy concerns, data are often distributed on different silos, bringing huge challenges to effective large-scale training. In this work, we try to overcome above limitation by leveraging the recent success of federated learning. As the first that is explored in VMR field, the new task is defined as video moment retrieval with distributed data. Then, a novel federated learning method named FedVMR is proposed to facilitate large-scale and secure training of VMR models in decentralized environment. Experiments on benchmark datasets demonstrate its effectiveness. This work is the very first attempt to enable safe and efficient VMR training in decentralized scene, which is hoped to pave the way for further study in the related research field.

cs.CV↗

arXiv2022

Topological structure of complex predictions

Complex prediction models such as deep learning are the output from fitting machine learning, neural networks, or AI models to a set of training data. These are now standard tools in science. A key challenge with the current generation of models is that they are highly parameterized, which makes describing and interpreting the prediction strategies difficult. We use topological data analysis to transform these complex prediction models into pictures representing a topological view. The result is a map of the predictions that enables inspection. The methods scale up to large datasets across different domains and enable us to detect labeling errors in training data, understand generalization in image classification, and inspect predictions of likely pathogenic mutations in the BRCA1 gene.

cs.LG↗

arXiv2022

Deep Spectro-temporal Artifacts for Detecting Synthesized Speech

The Audio Deep Synthesis Detection (ADD) Challenge has been held to detect generated human-like speech. With our submitted system, this paper provides an overall assessment of track 1 (Low-quality Fake Audio Detection) and track 2 (Partially Fake Audio Detection). In this paper, spectro-temporal artifacts were detected using raw temporal signals, spectral features, as well as deep embedding features. To address track 1, low-quality data augmentation, domain adaptation via finetuning, and various complementary feature information fusion were aggregated in our system. Furthermore, we analyzed the clustering characteristics of subsystems with different features by visualization method and explained the effectiveness of our proposed greedy fusion strategy. As for track 2, frame transition and smoothing were detected using self-supervised learning structure to capture the manipulation of PF attacks in the time domain. We ranked 4th and 5th in track 1 and track 2, respectively.

cs.SD↗

arXiv2022

Spoofing-Aware Attention based ASV Back-end with Multiple Enrollment Utterances and a Sampling Strategy for the SASV Challenge 2022

Current state-of-the-art automatic speaker verification (ASV) systems are vulnerable to presentation attacks, and several countermeasures (CMs), which distinguish bona fide trials from spoofing ones, have been explored to protect ASV. However, ASV systems and CMs are generally developed and optimized independently without considering their inter-relationship. In this paper, we propose a new spoofing-aware ASV back-end module that efficiently computes a combined ASV score based on speaker similarity and CM score. In addition to the learnable fusion function of the two scores, the proposed back-end module has two types of attention components, scaled-dot and feed-forward self-attention, so that intra-relationship information of multiple enrollment utterances can also be learned at the same time. Moreover, a new effective trials-sampling strategy is designed for simulating new spoofing-aware verification scenarios introduced in the Spoof-Aware Speaker Verification (SASV) challenge 2022.

eess.AS↗

arXiv2022

Fingering instabilities in binary granular systems

Fingering instabilities akin to the Rayleigh-Taylor (RT) instability in fluids have been observed in a binary granular system consisting of dense and small particles layered on top of lighter and larger particles, when the system is subjected to vertical vibration and fluidizing gas flow. Using observations from experiments and numerical modelling we explore whether the theory developed to describe the Rayleigh-Taylor (RT) instability in fluids is also applicable to binary granular systems. Our results confirm the applicability of the classic RT instability theory for binary granular systems demonstrating that several key features are observed in both types of systems, viz: (i) The characteristic wavenumber of the instability is constant with time, (ii) the amplitude of the characteristic wavenumber initially grows exponentially and (iii) the dispersion relation between the wavenumbers k of the interface instability and the growth rates n(k) of their amplitudes holds in both fluid-fluid and binary granular systems. Our results also demonstrate that inter-particle friction is essential for the RT instability to occur in granular media. For zero particle friction the interface instability bears a greater resembles to the Richtmyer-Meshkov instability. We further define a yield criterion Y for the interface by treating the granular medium as a viscoplastic material; only for Y > 15 fingering occurs. Interestingly, previous work has shown that instabilities in the Earth's lower mantle, another viscoplastic material, also occur for similar values of Y.

cond-mat.soft↗

arXiv2022

GraphFM: Improving Large-Scale GNN Training via Feature Momentum

Training of graph neural networks (GNNs) for large-scale node classification is challenging. A key difficulty lies in obtaining accurate hidden node representations while avoiding the neighborhood explosion problem. Here, we propose a new technique, named feature momentum (FM), that uses a momentum step to incorporate historical embeddings when updating feature representations. We develop two specific algorithms, known as GraphFM-IB and GraphFM-OB, that consider in-batch and out-of-batch data, respectively. GraphFM-IB applies FM to in-batch sampled data, while GraphFM-OB applies FM to out-of-batch data that are 1-hop neighborhood of in-batch data. We provide a convergence analysis for GraphFM-IB and some theoretical insight for GraphFM-OB. Empirically, we observe that GraphFM-IB can effectively alleviate the neighborhood explosion problem of existing methods. In addition, GraphFM-OB achieves promising performance on multiple large-scale graph datasets.

cs.LG↗

arXiv2022

Generating 3D Molecules for Target Protein Binding

A fundamental problem in drug discovery is to design molecules that bind to specific proteins. To tackle this problem using machine learning methods, here we propose a novel and effective framework, known as GraphBP, to generate 3D molecules that bind to given proteins by placing atoms of specific types and locations to the given binding site one by one. In particular, at each step, we first employ a 3D graph neural network to obtain geometry-aware and chemically informative representations from the intermediate contextual information. Such context includes the given binding site and atoms placed in the previous steps. Second, to preserve the desirable equivariance property, we select a local reference atom according to the designed auxiliary classifiers and then construct a local spherical coordinate system. Finally, to place a new atom, we generate its atom type and relative location w.r.t. the constructed local coordinate system via a flow model. We also consider generating the variables of interest sequentially to capture the underlying dependencies among them. Experiments demonstrate that our GraphBP is effective to generate 3D molecules with binding ability to target protein binding sites. Our implementation is available at https://github.com/divelab/GraphBP.

q-bio.BM↗