Searcharxiv⌕ Search

arXiv subjects

Cheng Li

Publications and source records attributed to Cheng Li.

At least 271 records · Page 15Linked to original sources

Self-Supervised Federated Learning for Fast MR Imaging

Federated learning (FL) based magnetic resonance (MR) image reconstruction can facilitate learning valuable priors from multi-site institutions without violating patient's privacy for accelerating MR imaging. However, existing methods rely on fully sampled data for collaborative training of the model. The client that only possesses undersampled data can neither participate in FL nor benefit from other clients. Furthermore, heterogeneous data distributions hinder FL from training an effective deep learning reconstruction model and thus cause performance degradation. To address these issues, we propose a Self-Supervised Federated Learning method (SSFedMRI). SSFedMRI explores the physics-based contrastive reconstruction networks in each client to realize cross-site collaborative training in the absence of fully sampled data. Furthermore, a personalized soft update scheme is designed to simultaneously capture the global shared representations among different centers and maintain the specific data distribution of each client. The proposed method is evaluated on four datasets and compared to the latest state-of-the-art approaches. Experimental results demonstrate that SSFedMRI possesses strong capability in reconstructing accurate MR images both visually and quantitatively on both in-distribution and out-of-distribution datasets.

eess.IV↗

A study on the Weibull and Pareto distributions motivated by Chvátal's theorem

Let $B(n,p)$ denote a binomial random variable with parameters $n$ and $p$. Chvátal's theorem says that for any fixed $n\geq 2$, as $m$ ranges over $\{0,\ldots,n\}$, the probability $q_m:=P(B(n,m/n)\leq m)$ is the smallest when $m$ is closest to $\frac{2n}{3}$. Motivated by this theorem, we consider the minimum value problem on the probability that a random variable is at most its expectation, when its distribution is the Weibull distribution or the Pareto distribution in this note.

math.PR↗

Understand Data Preprocessing for Effective End-to-End Training of Deep Neural Networks

In this paper, we primarily focus on understanding the data preprocessing pipeline for DNN Training in the public cloud. First, we run experiments to test the performance implications of the two major data preprocessing methods using either raw data or record files. The preliminary results show that data preprocessing is a clear bottleneck, even with the most efficient software and hardware configuration enabled by NVIDIA DALI, a high-optimized data preprocessing library. Second, we identify the potential causes, exercise a variety of optimization methods, and present their pros and cons. We hope this work will shed light on the new co-design of ``data storage, loading pipeline'' and ``training framework'' and flexible resource configurations between them so that the resources can be fully exploited and performance can be maximized.

cs.LG↗

Deep insight into charge equilibration and the effects on producing neutron-rich isotopes around N = 126 in the multinucleon transfer reactions

The dynamics of the charge equilibration (CE) and the effects on the production of the neutron-rich isotopes around $N = 126$ in multinucleon transfer reactions are still not well understood. In this Letter, we investigate the mechanism of the CE from different viewpoints by using the extended version of the dinuclear system model (DNS-sysu) and the improved quantum molecular dynamics (ImQMD) model. From the macroscopic and microscopic dynamical viewpoints, we find incomplete CE for the mass asymmetry reaction systems even in the very deep collisions, and the behavior of "inverse CE" that the tendency of the fragments is away from the $N/Z$ value of the compound system in the reaction $^{140}$Xe + $^{198}$Pt. Unlike the slow process presented in the ImQMD model, the behavior of fast equilibration with the characteristic time $\sim$ 0.52 zs is obtained based on the DNS-sysu model, which is consistent with the experimental data. By performing the systematic calculation, the correlation between the CE and the mass asymmetry of the reaction systems is clarified, which not only accounts for the observed intriguing phenomena of the CE but also provides essential information for producing the neutron-rich isotopes around $N = 126$.

nucl-th↗

Model-based Federated Learning for Accurate MR Image Reconstruction from Undersampled k-space Data

Deep learning-based methods have achieved encouraging performances in the field of magnetic resonance (MR) image reconstruction. Nevertheless, to properly learn a powerful and robust model, these methods generally require large quantities of data, the collection of which from multiple centers may cause ethical and data privacy violation issues. Lately, federated learning has served as a promising solution to exploit multi-center data while getting rid of the data transfer between institutions. However, high heterogeneity exists in the data from different centers, and existing federated learning methods tend to use average aggregation methods to combine the client's information, which limits the performance and generalization capability of the trained models. In this paper, we propose a Model-based Federated learning framework (ModFed). ModFed has three major contributions: 1) Different from the existing data-driven federated learning methods, model-driven neural networks are designed to relieve each client's dependency on large data; 2) An adaptive dynamic aggregation scheme is proposed to address the data heterogeneity issue and improve the generalization capability and robustness the trained neural network models; 3) A spatial Laplacian attention mechanism and a personalized client-side loss regularization are introduced to capture the detailed information for accurate image reconstruction. ModFed is evaluated on three in-vivo datasets. Experimental results show that ModFed has strong capability in improving image reconstruction quality and enforcing model generalization capability when compared to the other five state-of-the-art federated learning approaches. Codes will be made available at https://github.com/ternencewu123/ModFed.

eess.IV↗

The optimal detection angles for producing N=126 neutron-rich isotones in the multinucleon transfer reactions

The challenge of isotopic identification over a wide angular distribution has limited the measurement of neutron-rich nuclei produced via the multinucleon transfer (MNT) process. To investigate the optimal detection angles for the N=126 isotones, we propose a method to construct the reasonable scattering angles of the MNT products in the dinuclear system (DNS-sysu) model. The reactions $^{136,144}$Xe + $^{208}$Pb are investigated. The calculated results are in rather good agreement with the available experimental data in the reaction $^{136}$Xe + $^{208}$Pb. The entrance channel effects on the scattering angle are investigated. It is found that the scattering angular distribution strongly depends on the isospin and the impact parameter of the collision system. The optimal angle ranges for detecting $N=126$ neutron-rich nuclides $^{204}$Pt, $^{203}$Ir, $^{202}$Os, and $^{201}$Re in the $^{136}$Xe + $^{208}$Pb reaction at the incident energy $E_{c.m.} = 526MeV$ are predicted. Our results suggest that the angle range $45^{\circ} \leqslant θ_{\mathrm{lab}} \leqslant 50^{\circ}$ is most favorable for detecting unknown N=126 isotones. Given the current difficulties in separating and identifying experimental MNT fragments, the results of this work could provide significant contributions to future experiments.

nucl-th↗

Few-shot Class-incremental Learning for Cross-domain Disease Classification

The ability to incrementally learn new classes from limited samples is crucial to the development of artificial intelligence systems for real clinical application. Although existing incremental learning techniques have attempted to address this issue, they still struggle with only few labeled data, particularly when the samples are from varied domains. In this paper, we explore the cross-domain few-shot incremental learning (CDFSCIL) problem. CDFSCIL requires models to learn new classes from very few labeled samples incrementally, and the new classes may be vastly different from the target space. To counteract this difficulty, we propose a cross-domain enhancement constraint and cross-domain data augmentation method. Experiments on MedMNIST show that the classification performance of this method is better than other similar incremental learning methods.

cs.CV↗

Efficient Multi-Stage Video Denoising with Recurrent Spatio-Temporal Fusion

In recent years, denoising methods based on deep learning have achieved unparalleled performance at the cost of large computational complexity. In this work, we propose an Efficient Multi-stage Video Denoising algorithm, called EMVD, to drastically reduce the complexity while maintaining or even improving the performance. First, a fusion stage reduces the noise through a recursive combination of all past frames in the video. Then, a denoising stage removes the noise in the fused frame. Finally, a refinement stage restores the missing high frequency in the denoised frame. All stages operate on a transform-domain representation obtained by learnable and invertible linear operators which simultaneously increase accuracy and decrease complexity of the model. A single loss on the final output is sufficient for successful convergence, hence making EMVD easy to train. Experiments on real raw data demonstrate that EMVD outperforms the state of the art when complexity is constrained, and even remains competitive against methods whose complexities are several orders of magnitude higher. Further, the low complexity and memory requirements of EMVD enable real-time video denoising on commercial SoC in mobile devices.

eess.IV↗

MGA: Medical generalist agent through text-guided knowledge transformation

Multi-modal representation methods have achieved advanced performance in medical applications by extracting more robust features from multi-domain data. However, existing methods usually need to train additional branches for downstream tasks, which may increase the model complexities in clinical applications as well as introduce additional human inductive bias. Besides, very few studies exploit the rich clinical knowledge embedded in clinical daily reports. To this end, we propose a novel medical generalist agent, MGA, that can address three kinds of common clinical tasks via clinical reports knowledge transformation. Unlike the existing methods, MGA can easily adapt to different tasks without specific downstream branches when their corresponding annotations are missing. More importantly, we are the first attempt to use medical professional language guidance as a transmission medium to guide the agent's behavior. The proposed method is implemented on four well-known X-ray open-source datasets, MIMIC-CXR, CheXpert, MIMIC-CXR-JPG, and MIMIC-CXR-MS. Promising results are obtained, which validate the effectiveness of our proposed MGA. Code is available at: https://github.com/SZUHvern/MGA

cs.CV↗

Fixed-domain Posterior Contraction Rates for Spatial Gaussian Process Model with Nugget

Spatial Gaussian process regression models typically contain finite dimensional covariance parameters that need to be estimated from the data. We study the Bayesian estimation of covariance parameters including the nugget parameter in a general class of stationary covariance functions under fixed-domain asymptotics, which is theoretically challenging due to the increasingly strong dependence among spatial observations. We propose a novel adaptation of the Schwartz's consistency theorem for showing posterior contraction rates of the covariance parameters including the nugget. We derive a new polynomial evidence lower bound, and propose consistent higher-order quadratic variation estimators that satisfy concentration inequalities with exponentially small tails. Our Bayesian fixed-domain asymptotics theory leads to explicit posterior contraction rates for the microergodic and nugget parameters in the isotropic Matern covariance function under a general stratified sampling design. We verify our theory and the Bayesian predictive performance in simulation studies and an application to sea surface temperature data.

math.ST↗

BiFeat: Supercharge GNN Training via Graph Feature Quantization

Graph Neural Networks (GNNs) is a promising approach for applications with nonEuclidean data. However, training GNNs on large scale graphs with hundreds of millions nodes is both resource and time consuming. Different from DNNs, GNNs usually have larger memory footprints, and thus the GPU memory capacity and PCIe bandwidth are the main resource bottlenecks in GNN training. To address this problem, we present BiFeat: a graph feature quantization methodology to accelerate GNN training by significantly reducing the memory footprint and PCIe bandwidth requirement so that GNNs can take full advantage of GPU computing capabilities. Our key insight is that unlike DNN, GNN is less prone to the information loss of input features caused by quantization. We identify the main accuracy impact factors in graph feature quantization and theoretically prove that BiFeat training converges to a network where the loss is within $ε$ of the optimal loss of uncompressed network. We perform extensive evaluation of BiFeat using several popular GNN models and datasets, including GraphSAGE on MAG240M, the largest public graph dataset. The results demonstrate that BiFeat achieves a compression ratio of more than 30 and improves GNN training speed by 200%-320% with marginal accuracy loss. In particular, BiFeat achieves a record by training GraphSAGE on MAG240M within one hour using only four GPUs.

cs.LG↗

The need for multicomponent dust attenuation in modeling nebular emission: Constraints from SDSS-IV MaNGA

A fundamental assumption adopted in nearly every extragalactic emission-line study is that the attenuation of different emission lines can be described by a single attenuation curve. Here we show this assumption fails in many cases with important implications for derived results. We developed a new method to measure the differential nebular attenuation among three kinds of transitions: the Balmer lines of hydrogen, high-ionization transitions, and low-ionization transitions. This method bins the observed data in a multidimensional space spanned by attenuation-insensitive line ratios. Within each small bin, the variations in line ratios are mainly driven by the variations in the nebular attenuation. This allows us to measure the nebular attenuation using both forbidden lines and Balmer lines. We applied this method to a sample of 2.4 million star-forming spaxels from SDSS-IV MaNGA. We found that the attenuation of high ionization lines and Balmer lines can be well described by a single Fitzpatrick (1999) extinction curve with $R_V=3.1$. However, no single attenuation curve can simultaneously account for all three transitions. This strongly suggests that different lines have different effective attenuations, likely because spectroscopy at kiloparsec resolutions mixes multiple regions with different intrinsic line ratios and different levels of attenuation. As a result, the assumption that different lines follow the same attenuation curve breaks down. Using a single attenuation curve determined by Balmer lines to correct attenuation-sensitive forbidden line ratios could bias the nebular parameters derived by 0.06--0.25 dex at $A_V = 1$, depending on the details of the dust attenuation model. Observations of a statistically large sample of H II regions with high spatial resolutions and large spectral coverage are vital for improved modeling and deriving accurate corrections for this effect.

astro-ph.GA↗

Experimental generation of polarization entanglement from spontaneous parametric down-conversion pumped by spatiotemporally highly incoherent light

The influence of pump coherence on the entanglement produced in spontaneous parametric down-conversion (SPDC) is important to understand, both from a fundamental perspective, and from a practical standpoint for controlled generation of entangled states. In this context, it is known that in the absence of postselection, the pump coherence in a given degree of freedom (DOF) imposes an upper limit on the generated entanglement in the same DOF. However, the cross-influence of the pump coherence on the generated entanglement in a different DOF is not well-understood. Here, we experimentally investigate the effect of a spatiotemporally highly-incoherent (STHI) light-emitting diode (LED) pump on the polarization entanglement generated in SPDC. Our quantum state tomography measurements using multimode collection fibers to reduce the influence of postselection yield a two-qubit state with a concurrence of 0.531+/-0.006 and a purity of 0.647+/-0.005, in excellent agreement with our theoretically predicted concurrence of 0.552 and purity of 0.652. Therefore, while the use of an STHI pump causes reduction in the entanglement and purity of the output polarization two-qubit state, the viability of SPDC with STHI pumps is nevertheless important for two reasons: (i) STHI sources are ubiquitous and available at a wider range of wavelengths than lasers, and (ii) the generated STHI polarization-entangled two-photon states could potentially be useful in long-distance quantum communication schemes due to their robustness to scattering.

quant-ph↗

Relating galaxies across different redshift to study galaxy evolution

We propose a general framework leveraging the halo-galaxy connection to link galaxies observed at different redshift in a statistical way, and use the link to infer the redshift evolution of the galaxy population. Our tests based on hydrodynamic simulations show that our method can accurately recover the stellar mass assembly histories up to $z\sim 3$ for present star-forming and quiescent galaxies down to $10^{10}h^{-1}M_{\odot}$. Applying the method to observational data shows that the stellar mass evolution of the main progenitors of galaxies depends strongly on the properties of descendants, such as stellar mass, halo mass, and star formation states. Galaxies hosted by low-mass groups/halos at the present time have since $z\sim 1.8$ grown their stellar mass $\sim 2.5$ times as fast as those hosted by massive clusters. This dependence on host halo mass becomes much weaker for descendant galaxies with similar star formation states. Star-forming galaxies grow about 2-4 times faster than their quiescent counterparts since $z\sim 1.8$. Both TNG and EAGLE simulations over-predict the progenitor stellar mass at $z>1$, particularly for low-mass descendants.

astro-ph.GA↗

Wave Statistics and Energy Dissipation of Breaking Waves Generated by a Wave Plate

The present study focuses on direct numerical simulations of breaking waves generated by a wave plate at constant water depths. The aim is to quantify the dynamics and kinematics in the breaking process, together with the air entrainment and energy dissipation due to breaking. Good agreement is achieved between numerical and experimental results in terms of free-surface profiles, energy budget, and bubble statistics during wave breaking. A parametric study was conducted to examine the effects of wave properties and initial conditions on breaking features. According to research on the Bond number ($Bo$, the ratio of gravitational to surface tension forces), a larger surface tension produces more significant parasitic capillaries at the forward face of the wave profile and a thicker plunging jet, which causes a delayed breaking time and is tightly correlated with the main cavity size. A close relationship between wave statistics and the initial conditions of the wave plate is discovered, showing that breaker types can be classified according to the ratio of wave height to water depth, $H/d$. Moreover, the energy dissipation rate due to breaking can be related to the initial conditions of the wave plate by applying the conventional dissipation scaling of turbulence theory, and further correlated with the local breaking crest geometry using inertial-scaling arguments. The proposed scaling of the dissipation rate during the active breaking period is found to be in good agreement with our numerical results.

physics.flu-dyn↗

SuperScaler: Supporting Flexible DNN Parallelization via a Unified Abstraction

With the growing model size, deep neural networks (DNN) are increasingly trained over massive GPU accelerators, which demands a proper parallelization plan that transforms a DNN model into fine-grained tasks and then schedules them to GPUs for execution. Due to the large search space, the contemporary parallelization plan generators often rely on empirical rules that couple transformation and scheduling, and fall short in exploring more flexible schedules that yield better memory usage and compute efficiency. This tension can be exacerbated by the emerging models with increasing complexity in their structure and model size. SuperScaler is a system that facilitates the design and generation of highly flexible parallelization plans. It formulates the plan design and generation into three sequential phases explicitly: model transformation, space-time scheduling, and data dependency preserving. Such a principled approach decouples multiple seemingly intertwined factors and enables the composition of highly flexible parallelization plans. As a result, SuperScaler can not only generate empirical parallelization plans, but also construct new plans that achieve up to 3.5X speedup compared to state-of-the-art solutions like DeepSpeed, Megatron and Alpa, for emerging DNN models like Swin-Transformer and AlphaFold2, as well as well-optimized models like GPT-3.

cs.DC↗

Dark Matter Phenomenology in 2HDMS

The constituents of dark matter are still an unresolved puzzle. Several Beyond Standard Model (BSM) Physics offer suitable candidates. In this study here we consider the Two Higgs Doublet model augmented with a complex scalar singlet (2HDMS) and focus on the dark matter phenomenology of 2HDMS with the complex scalar singlet as the dark matter candidate. The parameter space allowed from existing experimental constraints from dark matter, flavour physics and collider searches has been studied. The discovery potential for such a 2HDMS at HL-LHC and at future $e^+e^-$ colliders has been worked out.

hep-ph↗

Collective excitations of the Chern-insulator states in commensurate double moiré superlattices of twisted bilayer graphene on hexagonal boron nitride

We study the collective excitation modes of the Chern insulator states in magic-angle twisted bilayer graphene aligned with hexagonal boron nitride (TBG/BN) at odd integer fillings ($ν$) of the flat bands. For the $1 \times 1$ commensurate double moiré superlattices in TBG/BN at three twist angles ($θ'$) between BN and graphene, self-consistent Hartree-Fock calculations show that the electron-electron interaction and the broken $C_{2z}$ symmetry lead to the Chern-insulator ground states with valley-spin flavor polarized HF bands at odd $ν$. In the active-band approximation, the HF bands in the same flavor of TBG/BN are much more separated than those of the pristine TBG with TBG/BN having a larger intra-flavor band gap so that the energies of the lowest intra-flavor exciton modes of TBG/BN computed within the time-dependent HF method are much higher than those of TBG and reach about 20 meV, and the exciton wavefunctions of TBG/BN become less localized than those of TBG. The inter-flavor valley-wave modes in TBG/BN have excitation energies higher than 2.5 meV which is also much larger than that of TBG, while the spin-wave modes all have zero excitation gap. In contrast to TBG with particle-hole symmetric excitation modes for positive and negative $ν$, the excitation spectrums and gaps of TBG/BN at positive $ν$ are rather different from those at negative $ν$. The quantitative behavior of the excitation spectrum of TBG/BN also varies with $θ'$. Full HF calculations demonstrate that more HF bands besides the two central bands can have rather large contributions from the single-particle flat-band states, then the lowest exciton modes that determine the optical properties of the Chern insulator states in TBG/BN are generally the ones between the remote and flat-like bands, while the valley-wave modes have similar energies as those in the active-band approximation.

cond-mat.mes-hall↗