arXiv subjects

Lei Li

Publications and source records attributed to Lei Li.

At least 523 records · Page 29Linked to original sources

arXiv2022

Calibrating Factual Knowledge in Pretrained Language Models

Previous literature has proved that Pretrained Language Models (PLMs) can store factual knowledge. However, we find that facts stored in the PLMs are not always correct. It motivates us to explore a fundamental question: How do we calibrate factual knowledge in PLMs without re-training from scratch? In this work, we propose a simple and lightweight method CaliNet to achieve this goal. To be specific, we first detect whether PLMs can learn the right facts via a contrastive score between right and fake facts. If not, we then use a lightweight method to add and adapt new parameters to specific factual texts. Experiments on the knowledge probing task show the calibration effectiveness and efficiency. In addition, through closed-book question answering, we find that the calibrated PLM possesses knowledge generalization ability after fine-tuning. Beyond the calibration performance, we further investigate and visualize the knowledge calibration mechanism.

cs.CL↗

arXiv2022

Learning Multi-resolution Functional Maps with Spectral Attention for Robust Shape Matching

In this work, we present a novel non-rigid shape matching framework based on multi-resolution functional maps with spectral attention. Existing functional map learning methods all rely on the critical choice of the spectral resolution hyperparameter, which can severely affect the overall accuracy or lead to overfitting, if not chosen carefully. In this paper, we show that spectral resolution tuning can be alleviated by introducing spectral attention. Our framework is applicable in both supervised and unsupervised settings, and we show that it is possible to train the network so that it can adapt the spectral resolution, depending on the given shape input. More specifically, we propose to compute multi-resolution functional maps that characterize correspondence across a range of spectral resolutions, and introduce a spectral attention network that helps to combine this representation into a single coherent final correspondence. Our approach is not only accurate with near-isometric input, for which a high spectral resolution is typically preferred, but also robust and able to produce reasonable matching even in the presence of significant non-isometric distortion, which poses great challenges to existing methods. We demonstrate the superior performance of our approach through experiments on a suite of challenging near-isometric and non-isometric shape matching benchmarks.

cs.CV↗

arXiv2022

From Mimicking to Integrating: Knowledge Integration for Pre-Trained Language Models

Investigating better ways to reuse the released pre-trained language models (PLMs) can significantly reduce the computational cost and the potential environmental side-effects. This paper explores a novel PLM reuse paradigm, Knowledge Integration (KI). Without human annotations available, KI aims to merge the knowledge from different teacher-PLMs, each of which specializes in a different classification problem, into a versatile student model. To achieve this, we first derive the correlation between virtual golden supervision and teacher predictions. We then design a Model Uncertainty--aware Knowledge Integration (MUKI) framework to recover the golden supervision for the student. Specifically, MUKI adopts Monte-Carlo Dropout to estimate model uncertainty for the supervision integration. An instance-wise re-weighting mechanism based on the margin of uncertainty scores is further incorporated, to deal with the potential conflicting supervision from teachers. Experimental results demonstrate that MUKI achieves substantial improvements over baselines on benchmark datasets. Further analysis shows that MUKI can generalize well for merging teacher models with heterogeneous architectures, and even teachers major in cross-lingual datasets.

cs.CL↗

arXiv2022

PARAGEN : A Parallel Generation Toolkit

PARAGEN is a PyTorch-based NLP toolkit for further development on parallel generation. PARAGEN provides thirteen types of customizable plugins, helping users to experiment quickly with novel ideas across model architectures, optimization, and learning strategies. We implement various features, such as unlimited data loading and automatic model selection, to enhance its industrial usage. ParaGen is now deployed to support various research and industry applications at ByteDance. PARAGEN is available at https://github.com/bytedance/ParaGen.

cs.CL↗

arXiv2022

Safety-based Speed Control of a Wheelchair using Robust Adaptive Model Predictive Control

Electric-powered wheelchair plays an important role in providing accessibility for people with mobility impairment. Ensuring the safety of wheelchair operation in different application scenarios and for diverse users is crucial when the designing controller for tracking tasks. In this work, we propose a safety-based speed tracking control algorithm for wheelchair systems with external disturbances and uncertain parameters at the dynamic level. The set-membership approach is applied to estimate the sets of uncertain parameters online and a designed model predictive control scheme with online model and control parameter adaptation is presented to guarantee safety-related constraints during the tracking process. The proposed controller can drive the wheelchair speed to a desired reference within safety constraints. For the inadmissible reference that violates the constraints, the proposed controller can steer the system to the neighbourhood of the closest admissible reference. The effectiveness of the proposed control scheme is validated based on the high-fidelity speed tracking results of two tasks that involve feasible and infeasible references.

eess.SY↗

arXiv2022

Block-Structured Optimization for Subgraph Detection in Interdependent Networks

We propose a generalized framework for block-structured nonconvex optimization, which can be applied to structured subgraph detection in interdependent networks, such as multi-layer networks, temporal networks, networks of networks, and many others. Specifically, we design an effective, efficient, and parallelizable projection algorithm, namely Graph Block-structured Gradient Projection (GBGP), to optimize a general non-linear function subject to graph-structured constraints. We prove that our algorithm: 1) runs in nearly-linear time on the network size; 2) enjoys a theoretical approximation guarantee. Moreover, we demonstrate how our framework can be applied to two very practical applications and conduct comprehensive experiments to show the effectiveness and efficiency of our proposed algorithm.

cs.LG↗

arXiv2022

Numerical investigation of azimuthal thermoacoustic instability in a gas turbine model combustor

Self-excited spinning mode azimuthal instability in an annular combustor with non-swirling flow is investigated using large eddy simulation (LES). Compressible Navier-Stoke equations are solved with a flamelet combustion model to describe the subgrid chemistry$-$turbulence interactions. Two flamelet models, with and without heat loss effects, are compared to elucidate the non-adiabatic wall effects on the thermoacoustic instability. The azimuthal modes are captured well by both models with only marginal differences in the computed frequencies and amplitudes. By comparing with the experimental measurements, the frequencies given by the LES are approximately 10\% higher and the amplitudes are well predicted. Further analysis of the experimental and LES data shows a similar dominant anti-clockwise spinning mode, under which a good agreement is observed for the phase-averaged heat release rate fluctuations. Dynamic mode decomposition (DMD) is applied to shed more light on this spinning mode. The LES and experimental DMD modes reconstructed for their azimuthal mode frequencies agree very well for the heat release fluctuations. The DMD mode structure for the acoustic pressure from the LES shows a considerable non-zero profile at the combustor outlet, which could be essential for azimuthal modes to establish in this annular combustor. Finally, a low-order modelling study was conducted using an acoustic network combined with the flame transfer function extracted from LES. The results show that the dominant mode is associated with the plenum showing a first longitudinal and azimuthal mixed mode structure. By tuning the plenum length to match the effective volume, the predicted frequency becomes very close to the measured value.

physics.flu-dyn↗

arXiv2022

Hybrid nano-domain structures of organic-inorganic perovskites from molecule-cage coupling effects

In hybrid perovskites, the organic molecules and inorganic frameworks exhibit distinct static and dynamic characteristics. Their coupling will lead to unprecedented phenomena, which have attracted wide research interests. In this paper, we employed Deep Potential molecular dynamics (DPMD), a large-scale MD simulation scheme with DFT accuracy, to study $\mathrm{FA/MAPbI_3}$ hybrid perovskites. A spontaneous hybrid nano-domain behavior, namely multiple molecular rotation nano-domains embedded into a single $\mathrm{[PbI_6]^{4-}}$ octahedra rotation domain, was firstly discovered at low temperatures. The behavior originates from the interplay between the long range order of molecular rotation and local lattice deformation, and clarifies the puzzling diffraction patterns of $\mathrm{FAPbI_3}$ at low temperatures. Our work provides new insights into the structural characteristics and stability of hybrid perovskite, as well as new ideas for the structural characterization of organic-inorganic coupled systems.

cond-mat.mtrl-sci↗

arXiv2022

SRFeat: Learning Locally Accurate and Globally Consistent Non-Rigid Shape Correspondence

In this work, we present a novel learning-based framework that combines the local accuracy of contrastive learning with the global consistency of geometric approaches, for robust non-rigid matching. We first observe that while contrastive learning can lead to powerful point-wise features, the learned correspondences commonly lack smoothness and consistency, owing to the purely combinatorial nature of the standard contrastive losses. To overcome this limitation we propose to boost contrastive feature learning with two types of smoothness regularization that inject geometric information into correspondence learning. With this novel combination in hand, the resulting features are both highly discriminative across individual points, and, at the same time, lead to robust and consistent correspondences, through simple proximity queries. Our framework is general and is applicable to local feature learning in both the 3D and 2D domains. We demonstrate the superiority of our approach through extensive experiments on a wide range of challenging matching benchmarks, including 3D non-rigid shape correspondence and 2D image keypoint matching.

cs.CV↗

arXiv2022

LightNER: A Lightweight Tuning Paradigm for Low-resource NER via Pluggable Prompting

Most NER methods rely on extensive labeled data for model training, which struggles in the low-resource scenarios with limited training data. Existing dominant approaches usually suffer from the challenge that the target domain has different label sets compared with a resource-rich source domain, which can be concluded as class transfer and domain transfer. In this paper, we propose a lightweight tuning paradigm for low-resource NER via pluggable prompting (LightNER). Specifically, we construct the unified learnable verbalizer of entity categories to generate the entity span sequence and entity categories without any label-specific classifiers, thus addressing the class transfer issue. We further propose a pluggable guidance module by incorporating learnable parameters into the self-attention layer as guidance, which can re-modulate the attention and adapt pre-trained weights. Note that we only tune those inserted module with the whole parameter of the pre-trained language model fixed, thus, making our approach lightweight and flexible for low-resource scenarios and can better transfer knowledge across domains. Experimental results show that LightNER can obtain comparable performance in the standard supervised setting and outperform strong baselines in low-resource settings. Code is in https://github.com/zjunlp/DeepKE/tree/main/example/ner/few-shot.

cs.CL↗

arXiv2022

3D Textured Shape Recovery with Learned Geometric Priors

3D textured shape recovery from partial scans is crucial for many real-world applications. Existing approaches have demonstrated the efficacy of implicit function representation, but they suffer from partial inputs with severe occlusions and varying object types, which greatly hinders their application value in the real world. This technical report presents our approach to address these limitations by incorporating learned geometric priors. To this end, we generate a SMPL model from learned pose prediction and fuse it into the partial input to add prior knowledge of human bodies. We also propose a novel completeness-aware bounding box adaptation for handling different levels of scales and partialness of partial scans.

cs.CV↗

arXiv2022

A Proof for the Collatz Conjecture

It is well known that the following Collatz Conjecture is one of the unsolved problems in mathematics. Collatz Conjecture: For any positive integer $n>1$, the following recursive algorithm will convergent to 1 by a finite number of steps. A) If $n$ is an even number then $n/2 \to n$, B) If $n$ is an odd number then $3n+1 \to n$. This paper proposes a proof for the Collatz Conjecture by the elementary mathematical induction.

math.GM↗

arXiv2022

Topology of black hole thermodynamics in Lovelock gravity

In this work, we present a convenient method to perform the topological analysis of black hole thermodynamics. Utilizing the spinodal curve, thermodynamic critical points of a black hole are endowed with a topological quantity, Brouwer degree, which reflects intrinsic properties of the system under smooth deformations. Specially, in our setup, it can be easily calculated without exact solution of critical points. This enables us to conveniently investigate the topological transition between different thermodynamic systems, and give a topological classification for them. In this framework, topology of Lovelock AdS black holes with spherical horizon geometry is explored. Results show that charged black holes in arbitrary dimensions can be classified into the same topology class, whereas the $d=7$ and $d \geq 8$ uncharged black holes are in different topology classes. Moreover, we revisit the relation between different phase structures of these black holes from the viewpoint of topology. Some general topological properties of critical points are also discussed.

gr-qc↗

arXiv2022

Spin Josephson effects of spin-orbit-coupled Bose-Einstein condensates in a non-Hermitian double well

In this paper, we investigate the spin and tunneling dynamics of a spin-orbit-coupled noninteracting Bose-Einstein condensate in a periodically driven non-Hermitian double-well potential. Under high-frequency driving, we obtain the effective time-averaged Hamiltonian by using the standard time-averaging method, and analytically calculate the Floquet quasienergies, revealing that the parity-time (PT)-breaking phase transition appears even for arbitrarily small non-Hermitian parameters when the spin-orbit coupling strength takes half-integer value, irrespective of the values of other parameters used. When the system is PT-symmetric with balanced gain and loss, we find numerically and analytically that in the broken PT-symmetric regions, there will exist the net spin current together with a vanishing atomic current, if we drop the contribution of the exponential growth of the norm to the current behaviors. When the system is non-PT-symmetric, though the quasienergies are partial complex, a stable net spin current can be generated by controlling the periodic driving field, which is accompanied by a spatial localization of the condensate in the well with gain. The results deepen the understanding of non-Hermitian physics and could be useful for engineering a variety of devices for spintronics.

cond-mat.quant-gas↗

arXiv2022

Analytical results for the superflow of spin-orbit-coupled Bose-Einstein condensates in optical lattices

In this paper, we show that for sufficiently strong atomic interactions, there exist analytical solutions of current-carrying nonlinear Bloch states at the Brillouin zone edge to the model of spin-orbit-coupled Bose-Einstein condensates (BECs) with symmetric spin interaction loaded into optical lattices. These simple but generic exact solutions provide an analytical demonstration of some intriguing properties which have neither an analog in the regular BEC lattice systems nor in the uniform spin-orbit-coupled BEC systems. It is an analytical example for understanding the superfluid and other related properties of the spin-orbit-coupled BEC lattice systems.

cond-mat.quant-gas↗

arXiv2022

Multi-Modality Cardiac Image Computing: A Survey

Multi-modality cardiac imaging plays a key role in the management of patients with cardiovascular diseases. It allows a combination of complementary anatomical, morphological and functional information, increases diagnosis accuracy, and improves the efficacy of cardiovascular interventions and clinical outcomes. Fully-automated processing and quantitative analysis of multi-modality cardiac images could have a direct impact on clinical research and evidence-based patient management. However, these require overcoming significant challenges including inter-modality misalignment and finding optimal methods to integrate information from different modalities. This paper aims to provide a comprehensive review of multi-modality imaging in cardiology, the computing methods, the validation strategies, the related clinical workflows and future perspectives. For the computing methodologies, we have a favored focus on the three tasks, i.e., registration, fusion and segmentation, which generally involve multi-modality imaging data, \textit{either combining information from different modalities or transferring information across modalities}. The review highlights that multi-modality cardiac imaging data has the potential of wide applicability in the clinic, such as trans-aortic valve implantation guidance, myocardial viability assessment, and catheter ablation therapy and its patient selection. Nevertheless, many challenges remain unsolved, such as missing modality, combination of imaging and non-imaging data, and uniform analysis and representation of different modalities. There is also work to do in defining how the well-developed techniques fit in clinical workflows and how much additional and relevant information they introduce. These problems are likely to continue to be an active field of research and the questions to be answered in the future.

eess.IV↗

arXiv2022

AIM 2022 Challenge on Super-Resolution of Compressed Image and Video: Dataset, Methods and Results

This paper reviews the Challenge on Super-Resolution of Compressed Image and Video at AIM 2022. This challenge includes two tracks. Track 1 aims at the super-resolution of compressed image, and Track~2 targets the super-resolution of compressed video. In Track 1, we use the popular dataset DIV2K as the training, validation and test sets. In Track 2, we propose the LDV 3.0 dataset, which contains 365 videos, including the LDV 2.0 dataset (335 videos) and 30 additional videos. In this challenge, there are 12 teams and 2 teams that submitted the final results to Track 1 and Track 2, respectively. The proposed methods and solutions gauge the state-of-the-art of super-resolution on compressed image and video. The proposed LDV 3.0 dataset is available at https://github.com/RenYang-home/LDV_dataset. The homepage of this challenge is at https://github.com/RenYang-home/AIM22_CompressSR.

eess.IV↗

arXiv2022

A deep learning framework for geodesics under spherical Wasserstein-Fisher-Rao metric and its application for weighted sample generation

Wasserstein-Fisher-Rao (WFR) distance is a family of metrics to gauge the discrepancy of two Radon measures, which takes into account both transportation and weight change. Spherical WFR distance is a projected version of WFR distance for probability measures so that the space of Radon measures equipped with WFR can be viewed as metric cone over the space of probability measures with spherical WFR. Compared to the case for Wasserstein distance, the understanding of geodesics under the spherical WFR is less clear and still an ongoing research focus. In this paper, we develop a deep learning framework to compute the geodesics under the spherical WFR metric, and the learned geodesics can be adopted to generate weighted samples. Our approach is based on a Benamou-Brenier type dynamic formulation for spherical WFR. To overcome the difficulty in enforcing the boundary constraint brought by the weight change, a Kullback-Leibler (KL) divergence term based on the inverse map is introduced into the cost function. Moreover, a new regularization term using the particle velocity is introduced as a substitute for the Hamilton-Jacobi equation for the potential in dynamic formula. When used for sample generation, our framework can be beneficial for applications with given weighted samples, especially in the Bayesian inference, compared to sample generation with previous flow models.

cs.LG↗