arXiv subjects

Rui Li

Publications and source records attributed to Rui Li.

At least 361 records · Page 20Linked to original sources

arXiv2023

NMR study of Ni50+xTi50-x Strain-Glass

We studied Ni50+xTi50-x with compositions up to x = 2, performing 47Ti and 49Ti nuclear magnetic resonance (NMR) measurements from 4 K to 400 K. For large x in this system, a strain-glass appears in which frozen ferroelastic nano-domains replace the displacive martensite structural transition. Here we demonstrate that NMR can provide an extremely effective probe of the strain glass freezing process, with large changes in NMR line-shape due to the effects of random strains which become motionally narrowed at high temperatures. At the same time with high-resolution x-ray diffraction we confirm the lack of structural changes in x >= 1.2 samples, while we show that there is little change in the electronic behavior across the strain glass freezing temperature. NMR spin-lattice relaxation time (T1) measurements provide a further measure of the dynamics of the freezing process, and indicate a predominantly thermally activated behavior both above and below the strain-glass freezing temperature. We show that the strain glass results are consistent with a very small density of critically divergent domains undergoing a Vogel-Fulcher-type freezing process, coexisting with domains exhibiting faster dynamics and stronger pinning.

cond-mat.mtrl-sci↗

arXiv2023

SFusion: Self-attention based N-to-One Multimodal Fusion Block

People perceive the world with different senses, such as sight, hearing, smell, and touch. Processing and fusing information from multiple modalities enables Artificial Intelligence to understand the world around us more easily. However, when there are missing modalities, the number of available modalities is different in diverse situations, which leads to an N-to-One fusion problem. To solve this problem, we propose a self-attention based fusion block called SFusion. Different from preset formulations or convolution based methods, the proposed block automatically learns to fuse available modalities without synthesizing or zero-padding missing ones. Specifically, the feature representations extracted from upstream processing model are projected as tokens and fed into self-attention module to generate latent multimodal correlations. Then, a modal attention mechanism is introduced to build a shared representation, which can be applied by the downstream decision model. The proposed SFusion can be easily integrated into existing multimodal analysis networks. In this work, we apply SFusion to different backbone networks for human activity recognition and brain tumor segmentation tasks. Extensive experimental results show that the SFusion block achieves better performance than the competing fusion strategies. Our code is available at https://github.com/scut-cszcl/SFusion.

cs.CV↗

arXiv2023

Towards Generalizable Medical Image Segmentation with Pixel-wise Uncertainty Estimation

Deep neural networks (DNNs) achieve promising performance in visual recognition under the independent and identically distributed (IID) hypothesis. In contrast, the IID hypothesis is not universally guaranteed in numerous real-world applications, especially in medical image analysis. Medical image segmentation is typically formulated as a pixel-wise classification task in which each pixel is classified into a category. However, this formulation ignores the hard-to-classified pixels, e.g., some pixels near the boundary area, as they usually confuse DNNs. In this paper, we first explore that hard-to-classified pixels are associated with high uncertainty. Based on this, we propose a novel framework that utilizes uncertainty estimation to highlight hard-to-classified pixels for DNNs, thereby improving its generalization. We evaluate our method on two popular benchmarks: prostate and fundus datasets. The results of the experiment demonstrate that our method outperforms state-of-the-art methods.

eess.IV↗

arXiv2023

Spatial-then-Temporal Self-Supervised Learning for Video Correspondence

In low-level video analyses, effective representations are important to derive the correspondences between video frames. These representations have been learned in a self-supervised fashion from unlabeled images or videos, using carefully designed pretext tasks in some recent studies. However, the previous work concentrates on either spatial-discriminative features or temporal-repetitive features, with little attention to the synergy between spatial and temporal cues. To address this issue, we propose a spatial-then-temporal self-supervised learning method. Specifically, we firstly extract spatial features from unlabeled images via contrastive learning, and secondly enhance the features by exploiting the temporal cues in unlabeled videos via reconstructive learning. In the second step, we design a global correlation distillation loss to ensure the learning not to forget the spatial cues, and a local correlation distillation loss to combat the temporal discontinuity that harms the reconstruction. The proposed method outperforms the state-of-the-art self-supervised methods, as established by the experimental results on a series of correspondence-based video analysis tasks. Also, we performed ablation studies to verify the effectiveness of the two-step design as well as the distillation losses.

cs.CV↗

arXiv2023

Microscopy image reconstruction with physics-informed denoising diffusion probabilistic model

Light microscopy is a widespread and inexpensive imaging technique facilitating biomedical discovery and diagnostics. However, light diffraction barrier and imperfections in optics limit the level of detail of the acquired images. The details lost can be reconstructed among others by deep learning models. Yet, deep learning models are prone to introduce artefacts and hallucinations into the reconstruction. Recent state-of-the-art image synthesis models like the denoising diffusion probabilistic models (DDPMs) are no exception to this. We propose to address this by incorporating the physical problem of microscopy image formation into the model's loss function. To overcome the lack of microscopy data, we train this model with synthetic data. We simulate the effects of the microscope optics through the theoretical point spread function and varying the noise levels to obtain synthetic data. Furthermore, we incorporate the physical model of a light microscope into the reverse process of a conditioned DDPM proposing a physics-informed DDPM (PI-DDPM). We show consistent improvement and artefact reductions when compared to model-based methods, deep-learning regression methods and regular conditioned DDPMs.

q-bio.QM↗

arXiv2023

Proposal for practical Rydberg quantum gates using a native two-photon excitation

Rydberg quantum gate serving as an indispensable computing unit for neutral-atom quantum computation, has attracted intense research efforts for the last decade. However the state-of-the-art experiments have not reached the high gate fidelity as predicted by most theories due to the unexpected large loss remaining in Rydberg and intermediate states. In this paper we report our findings in constructing a native two-qubit controlled-NOT gate based on pulse optimization. We focus on the method of commonly-used two-photon Rydberg excitation with smooth Gaussian-shaped pulses which is straightforward for experimental demonstration. By utilizing optimized pulse shapes the scheme reveals a remarkable reduction in the decays from Rydberg and intermediate states, as well as a high-tolerance to the residual thermal motion of atoms. We extract a conservative lower bound for the gate fidelity > 0.992 after taking into account the experimental imperfections. Our results not only reduce the gap between experimental and theoretical prediction because of the optimal control, but also facilitate the connectivity of distant atomic qubits in a larger atom array by reducing the requirement of strong blockade, which is promising for developing multiqubit quantum computation in large-scale atomic arrays.

quant-ph↗

arXiv2023

Improving Hyperparameter Learning under Approximate Inference in Gaussian Process Models

Approximate inference in Gaussian process (GP) models with non-conjugate likelihoods gets entangled with the learning of the model hyperparameters. We improve hyperparameter learning in GP models and focus on the interplay between variational inference (VI) and the learning target. While VI's lower bound to the marginal likelihood is a suitable objective for inferring the approximate posterior, we show that a direct approximation of the marginal likelihood as in Expectation Propagation (EP) is a better learning objective for hyperparameter optimization. We design a hybrid training procedure to bring the best of both worlds: it leverages conjugate-computation VI for inference and uses an EP-like marginal likelihood approximation for hyperparameter learning. We compare VI, EP, Laplace approximation, and our proposed training procedure and empirically demonstrate the effectiveness of our proposal across a wide range of data sets.

cs.LG↗

arXiv2023

Research on Multilingual News Clustering Based on Cross-Language Word Embeddings

Classifying the same event reported by different countries is of significant importance for public opinion control and intelligence gathering. Due to the diverse types of news, relying solely on transla-tors would be costly and inefficient, while depending solely on translation systems would incur considerable performance overheads in invoking translation interfaces and storing translated texts. To address this issue, we mainly focus on the clustering problem of cross-lingual news. To be specific, we use a combination of sentence vector representations of news headlines in a mixed semantic space and the topic probability distributions of news content to represent a news article. In the training of cross-lingual models, we employ knowledge distillation techniques to fit two semantic spaces into a mixed semantic space. We abandon traditional static clustering methods like K-Means and AGNES in favor of the incremental clustering algorithm Single-Pass, which we further modify to better suit cross-lingual news clustering scenarios. Our main contributions are as follows: (1) We adopt the English standard BERT as the teacher model and XLM-Roberta as the student model, training a cross-lingual model through knowledge distillation that can represent sentence-level bilingual texts in both Chinese and English. (2) We use the LDA topic model to represent news as a combina-tion of cross-lingual vectors for headlines and topic probability distributions for con-tent, introducing concepts such as topic similarity to address the cross-lingual issue in news content representation. (3) We adapt the Single-Pass clustering algorithm for the news context to make it more applicable. Our optimizations of Single-Pass include ad-justing the distance algorithm between samples and clusters, adding cluster merging operations, and incorporating a news time parameter.

cs.CL↗

arXiv2023

The News Delivery Channel Recommendation Based on Granular Neural Network

With the continuous maturation and expansion of neural network technology, deep neural networks have been widely utilized as the fundamental building blocks of deep learning in a variety of applications, including speech recognition, machine translation, image processing, and the creation of recommendation systems. Therefore, many real-world complex problems can be solved by the deep learning techniques. As is known, traditional news recommendation systems mostly employ techniques based on collaborative filtering and deep learning, but the performance of these algorithms is constrained by the sparsity of the data and the scalability of the approaches. In this paper, we propose a recommendation model using granular neural network model to recommend news to appropriate channels by analyzing the properties of news. Specifically, a specified neural network serves as the foundation for the granular neural network that the model is considered to be build. Different information granularities are attributed to various types of news material, and different information granularities are released between networks in various ways. When processing data, granular output is created, which is compared to the interval values pre-set on various platforms and used to quantify the analysis's effectiveness. The analysis results could help the media to match the proper news in depth, maximize the public attention of the news and the utilization of media resources.

cs.IR↗

arXiv2023

Low-energy hole subband dispersions in a cylindrical Ge nanowire: the effects of the nanowire growth direction

We examine the validity of the spherical approximation $γ_{s}=(2γ_{2}+3γ_{3})/5$ in the Luttinger-Kohn Hamiltonian in calculating the subband dispersions of the hole gas. We calculate the realistic hole subband dispersions (without the spherical approximation) in a cylindrical Ge nanowire by using quasi-degenerate perturbation theory. The realistic low-energy hole subband dispersions have a double-well anticrossing structure, that consists with the spherical approximation prediction. However, the realistic subband dispersions are also nanowire growth direction dependent. When the nanowire growth direction is restricted in the (100) crystal plane, the detailed growth direction dependences of the subband parameters are given. We find the spherical approximation is good approximation, it can nicely reproduce the real result in some special growth directions.

cond-mat.mes-hall↗

arXiv2023

Dual-modality Smart Shoes for Quantitative Assessment of Hemiplegic Patients' Lower Limbs' Muscle Strength

Stroke can lead to the impaired motor ability of the patient's lower limbs and hemiplegia. Accurate assessment of the lower limbs' motor ability is important for diagnosis and rehabilitation. To digitalize such assessment so that each test can be traced back any time and subjectivity can be avoided, we test how dual-modality smart shoes equipped with pressure-sensitive insoles and inertial measurement units can be used for this purpose. A 5m walking test protocol, including the left and right turns, is designed. Data are collected from 23 patients and 17 healthy subjects. For the lower limbs' motor ability, the tests are observed by two physicians and assessed using the five graded Medical Research Council scale for muscle examination. The average of two physicians' scores for the same patient is used as the ground truth. Using the feature set we developed, 100\% accuracy is achieved in classifying the patients and healthy subjects. For patients' muscle strength, a mean absolute error of 0.143 and a maximum error of 0.395 is achieved using our feature set and the regression method, closer to the ground truth than the scores from each physician (mean absolute error: 0.217, maximum error: 0.5). We thus validate the possibility of using such smart shoes to objectively and accurately evaluate the lower limbs' muscle strength of the stroke patients.

cs.HC↗

arXiv2023

To Copy Rather Than Memorize: A Vertical Learning Paradigm for Knowledge Graph Completion

Embedding models have shown great power in knowledge graph completion (KGC) task. By learning structural constraints for each training triple, these methods implicitly memorize intrinsic relation rules to infer missing links. However, this paper points out that the multi-hop relation rules are hard to be reliably memorized due to the inherent deficiencies of such implicit memorization strategy, making embedding models underperform in predicting links between distant entity pairs. To alleviate this problem, we present Vertical Learning Paradigm (VLP), which extends embedding models by allowing to explicitly copy target information from related factual triples for more accurate prediction. Rather than solely relying on the implicit memory, VLP directly provides additional cues to improve the generalization ability of embedding models, especially making the distant link prediction significantly easier. Moreover, we also propose a novel relative distance based negative sampling technique (ReD) for more effective optimization. Experiments demonstrate the validity and generality of our proposals on two standard benchmarks. Our code is available at https://github.com/rui9812/VLP.

cs.CL↗

arXiv2023

Feature Alignment and Uniformity for Test Time Adaptation

Test time adaptation (TTA) aims to adapt deep neural networks when receiving out of distribution test domain samples. In this setting, the model can only access online unlabeled test samples and pre-trained models on the training domains. We first address TTA as a feature revision problem due to the domain gap between source domains and target domains. After that, we follow the two measurements alignment and uniformity to discuss the test time feature revision. For test time feature uniformity, we propose a test time self-distillation strategy to guarantee the consistency of uniformity between representations of the current batch and all the previous batches. For test time feature alignment, we propose a memorized spatial local clustering strategy to align the representations among the neighborhood samples for the upcoming batch. To deal with the common noisy label problem, we propound the entropy and consistency filters to select and drop the possible noisy labels. To prove the scalability and efficacy of our method, we conduct experiments on four domain generalization benchmarks and four medical image segmentation tasks with various backbones. Experiment results show that our method not only improves baseline stably but also outperforms existing state-of-the-art test time adaptation methods. Code is available at \href{https://github.com/SakurajimaMaiii/TSD}{https://github.com/SakurajimaMaiii/TSD}.

cs.CV↗

arXiv2023

Simulation Software of the JUNO Experiment

The Jiangmen Underground Neutrino Observatory (JUNO) is a multi-purpose experiment, under construction in southeast China, that is designed to determine the neutrino mass ordering and precisely measure neutrino oscillation parameters. Monte Carlo simulation plays an important role for JUNO detector design, detector commissioning, offline data processing, and physics processing. The JUNO experiment has the world's largest liquid scintillator detector instrumented with many thousands of PMTs. The broad energy range of interest, long lifetime, and the large scale present data processing challenges across all areas. This paper describes the JUNO simulation software, highlighting the challenges of JUNO simulation and solutions to meet these challenges, including such issues as support for time-correlated analysis, event mixing, event correlation and handling the simulation of many millions of optical photons.

hep-ex↗

arXiv2023

Learning to Fuse Monocular and Multi-view Cues for Multi-frame Depth Estimation in Dynamic Scenes

Multi-frame depth estimation generally achieves high accuracy relying on the multi-view geometric consistency. When applied in dynamic scenes, e.g., autonomous driving, this consistency is usually violated in the dynamic areas, leading to corrupted estimations. Many multi-frame methods handle dynamic areas by identifying them with explicit masks and compensating the multi-view cues with monocular cues represented as local monocular depth or features. The improvements are limited due to the uncontrolled quality of the masks and the underutilized benefits of the fusion of the two types of cues. In this paper, we propose a novel method to learn to fuse the multi-view and monocular cues encoded as volumes without needing the heuristically crafted masks. As unveiled in our analyses, the multi-view cues capture more accurate geometric information in static areas, and the monocular cues capture more useful contexts in dynamic areas. To let the geometric perception learned from multi-view cues in static areas propagate to the monocular representation in dynamic areas and let monocular cues enhance the representation of multi-view cost volume, we propose a cross-cue fusion (CCF) module, which includes the cross-cue attention (CCA) to encode the spatially non-local relative intra-relations from each source to enhance the representation of the other. Experiments on real-world datasets prove the significant effectiveness and generalization ability of the proposed method.

cs.CV↗

arXiv2023

Prototype Knowledge Distillation for Medical Segmentation with Missing Modality

Multi-modality medical imaging is crucial in clinical treatment as it can provide complementary information for medical image segmentation. However, collecting multi-modal data in clinical is difficult due to the limitation of the scan time and other clinical situations. As such, it is clinically meaningful to develop an image segmentation paradigm to handle this missing modality problem. In this paper, we propose a prototype knowledge distillation (ProtoKD) method to tackle the challenging problem, especially for the toughest scenario when only single modal data can be accessed. Specifically, our ProtoKD can not only distillate the pixel-wise knowledge of multi-modality data to single-modality data but also transfer intra-class and inter-class feature variations, such that the student model could learn more robust feature representation from the teacher model and inference with only one single modality data. Our method achieves state-of-the-art performance on BraTS benchmark. The code is available at \url{https://github.com/SakurajimaMaiii/ProtoKD}.

cs.CV↗

arXiv2023

Bootstrap The Original Latent: Learning a Private Model from a Black-box Model

In this paper, considering the balance of data/model privacy of model owners and user needs, we propose a new setting called Back-Propagated Black-Box Adaptation (BPBA) for users to better train their private models via the guidance of the back-propagated results of a Black-box foundation/source model. Our setting can ease the usage of foundation/source models as well as prevent the leakage and misuse of foundation/source models. Moreover, we also propose a new training strategy called Bootstrap The Original Latent (BTOL) to fully utilize the foundation/source models. Our strategy consists of a domain adapter and a freeze-and-thaw strategy. We apply our BTOL under BPBA and Black-box UDA settings on three different datasets. Experiments show that our strategy is efficient and robust in various settings without manual augmentations.

cs.CV↗

arXiv2023

Accelerated partial separable model using dimension-reduced optimization technique for ultra-fast cardiac MRI

Objective. Imaging dynamic object with high temporal resolution is challenging in magnetic resonance imaging (MRI). Partial separable (PS) model was proposed to improve the imaging quality by reducing the degrees of freedom of the inverse problem. However, PS model still suffers from long acquisition time and even longer reconstruction time. The main objective of this study is to accelerate the PS model, shorten the time required for acquisition and reconstruction, and maintain good image quality simultaneously. Approach. We proposed to fully exploit the dimension reduction property of the PS model, which means implementing the optimization algorithm in subspace. We optimized the data consistency term, and used a Tikhonov regularization term based on the Frobenius norm of temporal difference. The proposed dimension-reduced optimization technique was validated in free-running cardiac MRI. We have performed both retrospective experiments on public dataset and prospective experiments on in-vivo data. The proposed method was compared with four competing algorithms based on PS model, and two non-PS model methods. Main results. The proposed method has robust performance against shortened acquisition time or suboptimal hyper-parameter settings, and achieves superior image quality over all other competing algorithms. The proposed method is 20-fold faster than the widely accepted PS+Sparse method, enabling image reconstruction to be finished in just a few seconds. Significance. Accelerated PS model has the potential to save much time for clinical dynamic MRI examination, and is promising for real-time MRI applications.

eess.IV↗