arXiv subjects

Di Wang

Publications and source records attributed to Di Wang.

At least 505 records · Page 28Linked to original sources

arXiv2022

Unsupervised Misaligned Infrared and Visible Image Fusion via Cross-Modality Image Generation and Registration

Recent learning-based image fusion methods have marked numerous progress in pre-registered multi-modality data, but suffered serious ghosts dealing with misaligned multi-modality data, due to the spatial deformation and the difficulty narrowing cross-modality discrepancy. To overcome the obstacles, in this paper, we present a robust cross-modality generation-registration paradigm for unsupervised misaligned infrared and visible image fusion (IVIF). Specifically, we propose a Cross-modality Perceptual Style Transfer Network (CPSTN) to generate a pseudo infrared image taking a visible image as input. Benefiting from the favorable geometry preservation ability of the CPSTN, the generated pseudo infrared image embraces a sharp structure, which is more conducive to transforming cross-modality image alignment into mono-modality registration coupled with the structure-sensitive of the infrared image. In this case, we introduce a Multi-level Refinement Registration Network (MRRN) to predict the displacement vector field between distorted and pseudo infrared images and reconstruct registered infrared image under the mono-modality setting. Moreover, to better fuse the registered infrared images and visible images, we present a feature Interaction Fusion Module (IFM) to adaptively select more meaningful features for fusion in the Dual-path Interaction Fusion Network (DIFN). Extensive experimental results suggest that the proposed method performs superior capability on misaligned cross-modality image fusion.

cs.CV↗

arXiv2022

Optimal Rates of (Locally) Differentially Private Heavy-tailed Multi-Armed Bandits

In this paper we investigate the problem of stochastic multi-armed bandits (MAB) in the (local) differential privacy (DP/LDP) model. Unlike previous results that assume bounded/sub-Gaussian reward distributions, we focus on the setting where each arm's reward distribution only has $(1+v)$-th moment with some $v\in (0, 1]$. In the first part, we study the problem in the central $ε$-DP model. We first provide a near-optimal result by developing a private and robust Upper Confidence Bound (UCB) algorithm. Then, we improve the result via a private and robust version of the Successive Elimination (SE) algorithm. Finally, we establish the lower bound to show that the instance-dependent regret of our improved algorithm is optimal. In the second part, we study the problem in the $ε$-LDP model. We propose an algorithm that can be seen as locally private and robust version of SE algorithm, which provably achieves (near) optimal rates for both instance-dependent and instance-independent regret. Our results reveal differences between the problem of private MAB with bounded/sub-Gaussian rewards and heavy-tailed rewards. To achieve these (near) optimal rates, we develop several new hard instances and private robust estimators as byproducts, which might be used to other related problems. Finally, experiments also support our theoretical findings and show the effectiveness of our algorithms.

cs.LG↗

arXiv2022

From topological amplitude to rescattering dynamics

We proposed a theoretical framework to correlate the topological diagram at quark level and rescattering dynamics at hadron level. In the framework, both the hadronic triangle diagram, and the topological-scattering diagram, which is the intermediate structure between topological diagram and triangle diagram, are expressed in the tensor form. The completeness of topological-scattering diagram is confirmed by the quark substructure of meson-meson scattering. The coefficient of each triangle diagram can be derived from the topological-scattering diagram and the total rescattering amplitudes are consistent with the ones derived from the chiral Lagrangian. If only the short-distance $T$ diagram is considered as the weak vertex in triangle diagram, the rescattering contributions in the $C$, $E$ and $P$ diagrams have definite proportional relation of $L(C):L(E):L(P)=-2:1:1$ under the $SU(3)_F$ symmetry, and the rescattering contributions in the $T$ and $A$ diagrams only arise from the $SU(3)_F$ breaking effects. Taking $D\to Kπ$ and $D\to ππ$ modes as examples, we present our framework in detail. We find the Isospin relations in these decays are still valid in terms of triangle diagrams. Besides, the conclusions in the $D$ meson decays under the $SU(3)_F$ symmetry can be generalized to the $B$ meson decays under the $SU(4)_F$ symmetry.

hep-ph↗

arXiv2022

Heterogeneous Federated Learning

Federated learning learns from scattered data by fusing collaborative models from local nodes. However, due to chaotic information distribution, the model fusion may suffer from structural misalignment with regard to unmatched parameters. In this work, we propose a novel federated learning framework to resolve this issue by establishing a firm structure-information alignment across collaborative models. Specifically, we design a feature-oriented regulation method ({$Ψ$-Net}) to ensure explicit feature information allocation in different neural network structures. Applying this regulating method to collaborative models, matchable structures with similar feature information can be initialized at the very early training stage. During the federated learning process under either IID or non-IID scenarios, dedicated collaboration schemes further guarantee ordered information distribution with definite structure matching, so as the comprehensive model alignment. Eventually, this framework effectively enhances the federated learning applicability to extensive heterogeneous settings, while providing excellent convergence speed, accuracy, and computation/communication efficiency.

cs.LG↗

arXiv2022

C-P Map: A Novel Evaluation Toolkit for Speaker Verification

Evaluation trials are used to probe performance of automatic speaker verification (ASV) systems. In spite of the clear importance and impact, evaluation trials have not been seriously treated in research and engineering practice. This paper firstly presents a theoretical analysis on evaluation trials and highlights potential bias with the most popular cross-pairing approach used in trials design. To interpret and settle this problem, we define the concept of trial config and C-P map derived from it. The C-P map measures the performance of an ASV system on various trial configs in a 2-dimensional map. On the map, each location represents a particular trial config and its corresponding color represents the system performance. Experiments conducted on representative ASV systems show that the proposed C-P map offers a powerful evaluation toolkit for ASV performance analysis and comparison. The source code for C-P map has been release at https://gitlab.com/csltstu/sunine.

cs.SD↗

arXiv2022

From topological amplitude to rescattering dynamics in doubly charmed baryon decays

The doubly charmed baryon $Ξ^{++}_{cc}$ was observed by LHCb cooperation in 2017. The branching fractions of two-body doubly charmed baryon decays were predicted in the framework of rescattering mechanism, and some $SU(3)_F$ relations were investigated in the topological amplitudes. In this work, we study the correlation between topological diagram at quark level and rescattering triangle diagram at hadron level in the doubly charmed baryon decay. The completeness of our framework is confirmed from the fact that all the twelve possible structures of meson-baryon scattering appear once each in the the intermediate form between topological diagram and triangle diagram, topological-scattering diagram. It is found the triangle diagrams derived from the topological diagrams are consistent with the ones derived directly from the chiral Lagrangian. The relative magnitudes of rescattering contributions in the $C$, $C^\prime$, $E$, $E^\prime$, $P$ and $P^\prime$ diagrams extracted from $SU(3)_F$ symmetry are consistent with the numerical analysis in literature. Taking $Ξ^{++}_{cc}\to Ξ^+_cπ^+$, $Ξ^{+}_{cc}\to Ξ^0_cπ^+$ and $Ξ^{+}_{cc}\to Ξ^+_cπ^0$ modes as examples, we show the isospin relation is satisfied in terms of triangle diagrams.

hep-ph↗

arXiv2022

A Survey of Large-Scale Deep Learning Serving System Optimization: Challenges and Opportunities

Deep Learning (DL) models have achieved superior performance in many application domains, including vision, language, medical, commercial ads, entertainment, etc. With the fast development, both DL applications and the underlying serving hardware have demonstrated strong scaling trends, i.e., Model Scaling and Compute Scaling, for example, the recent pre-trained model with hundreds of billions of parameters with ~TB level memory consumption, as well as the newest GPU accelerators providing hundreds of TFLOPS. With both scaling trends, new problems and challenges emerge in DL inference serving systems, which gradually trends towards Large-scale Deep learning Serving systems (LDS). This survey aims to summarize and categorize the emerging challenges and optimization opportunities for large-scale deep learning serving systems. By providing a novel taxonomy, summarizing the computing paradigms, and elaborating the recent technique advances, we hope that this survey could shed light on new optimization perspectives and motivate novel works in large-scale deep learning system optimization.

cs.LG↗

arXiv2022

Direct observation of strong anomalous Hall effect and proximity-induced ferromagnetic state in SrIrO3

The 5d iridium-based transition metal oxides have gained broad interest because of their strong spin-orbit coupling which favors new or exotic quantum electronic states. On the other hand, they rarely exhibit more mainstream orders like ferromagnetism due to generally weak electron-electron correlation strength. Here, we show a proximity-induced ferromagnetic (FM) state with TC = 100 K and strong magnetocrystalline anisotropy in a SrIrO3 (SIO) heterostructure via interfacial charge transfer by using a ferromagnetic insulator in contact with SIO. Electrical transport allows to selectively probe the FM state of the SIO layer and the direct observation of a strong, intrinsic and positive anomalous Hall effect (AHE). For T < 20 K, the AHE displays unusually large coercive and saturation field, a fingerprint of a strong pseudospin-lattice coupling. A Hall angle, sxyAHE/sxx, larger by an order of magnitude than in typical 3d metals and a FM net moment of about 0.1 mB/Ir, is reported. This emphasizes how efficiently the nontrivial topological band properties of SIO can be manipulated by structural modifications and the exchange interaction with 3d TMOs.

cond-mat.str-el↗

arXiv2022

Accelerated numerical algorithms for steady states of Gross-Pitaevskii equations coupled with microwaves

We present two accelerated numerical algorithms for single-component and binary Gross-Pitaevskii (GP) equations coupled with microwaves (electromagnetic fields) in steady state. One is based on a normalized gradient flow formulation, called the ASGF method, while the other on a perturbed, projected conjugate gradient approach for the nonlinear constrained optimization, called the PPNCG method. The coupled GP equations are nonlocal in space, describing pseudo-spinor Bose-Einstein condensates (BECs) interacting with an electromagnetic field. Our interest in this study is to develop efficient, iterative numerical methods for steady symmetric and central vortex states of the nonlocal GP equation systems. In the algorithms, the GP equations are discretized by a Legendre-Galerkin spectral method in a polar coordinate in two-dimensional (2D) space. The new algorithms are shown to outperform the existing ones through a host of benchmark examples, among which the PPNCG method performs the best. Additional numerical simulations of the central vortex states are provided to demonstrate the usefulness and efficiency of the new algorithms.

math.NA↗

arXiv2022

Differentially Private (Gradient) Expectation Maximization Algorithm with Statistical Guarantees

(Gradient) Expectation Maximization (EM) is a widely used algorithm for estimating the maximum likelihood of mixture models or incomplete data problems. A major challenge facing this popular technique is how to effectively preserve the privacy of sensitive data. Previous research on this problem has already lead to the discovery of some Differentially Private (DP) algorithms for (Gradient) EM. However, unlike in the non-private case, existing techniques are not yet able to provide finite sample statistical guarantees. To address this issue, we propose in this paper the first DP version of (Gradient) EM algorithm with statistical guarantees. Moreover, we apply our general framework to three canonical models: Gaussian Mixture Model (GMM), Mixture of Regressions Model (MRM) and Linear Regression with Missing Covariates (RMC). Specifically, for GMM in the DP model, our estimation error is near optimal in some cases. For the other two models, we provide the first finite sample statistical guarantees. Our theory is supported by thorough numerical experiments.

cs.LG↗

arXiv2022

Faster Rates of Private Stochastic Convex Optimization

In this paper, we revisit the problem of Differentially Private Stochastic Convex Optimization (DP-SCO) and provide excess population risks for some special classes of functions that are faster than the previous results of general convex and strongly convex functions. In the first part of the paper, we study the case where the population risk function satisfies the Tysbakov Noise Condition (TNC) with some parameter $θ>1$. Specifically, we first show that under some mild assumptions on the loss functions, there is an algorithm whose output could achieve an upper bound of $\tilde{O}((\frac{1}{\sqrt{n}}+\frac{\sqrt{d\log \frac{1}δ}}{nε})^\fracθ{θ-1})$ for $(ε, δ)$-DP when $θ\geq 2$, here $n$ is the sample size and $d$ is the dimension of the space. Then we address the inefficiency issue, improve the upper bounds by $\text{Poly}(\log n)$ factors and extend to the case where $θ\geq \barθ>1$ for some known $\barθ$. Next we show that the excess population risk of population functions satisfying TNC with parameter $θ\geq 2$ is always lower bounded by $Ω((\frac{d}{nε})^\fracθ{θ-1}) $ and $Ω((\frac{\sqrt{d\log \frac{1}δ}}{nε})^\fracθ{θ-1})$ for $ε$-DP and $(ε, δ)$-DP, respectively. In the second part, we focus on a special case where the population risk function is strongly convex. Unlike the previous studies, here we assume the loss function is {\em non-negative} and {\em the optimal value of population risk is sufficiently small}. With these additional assumptions, we propose a new method whose output could achieve an upper bound of $O(\frac{d\log\frac{1}δ}{n^2ε^2}+\frac{1}{n^τ})$ for any $τ\geq 1$ in $(ε,δ)$-DP model if the sample size $n$ is sufficiently large.

cs.LG↗

arXiv2022

Differentially Private $\ell_1$-norm Linear Regression with Heavy-tailed Data

We study the problem of Differentially Private Stochastic Convex Optimization (DP-SCO) with heavy-tailed data. Specifically, we focus on the $\ell_1$-norm linear regression in the $ε$-DP model. While most of the previous work focuses on the case where the loss function is Lipschitz, here we only need to assume the variates has bounded moments. Firstly, we study the case where the $\ell_2$ norm of data has bounded second order moment. We propose an algorithm which is based on the exponential mechanism and show that it is possible to achieve an upper bound of $\tilde{O}(\sqrt{\frac{d}{nε}})$ (with high probability). Next, we relax the assumption to bounded $θ$-th order moment with some $θ\in (1, 2)$ and show that it is possible to achieve an upper bound of $\tilde{O}(({\frac{d}{nε}})^\frac{θ-1}θ)$. Our algorithms can also be extended to more relaxed cases where only each coordinate of the data has bounded moments, and we can get an upper bound of $\tilde{O}({\frac{d}{\sqrt{nε}}})$ and $\tilde{O}({\frac{d}{({nε})^\frac{θ-1}θ}})$ in the second and $θ$-th moment case respectively.

cs.LG↗

arXiv2022

The trigger and data acquisition system of the FASER experiment

The FASER experiment is a new small and inexpensive experiment that is placed 480 meters downstream of the ATLAS experiment at the CERN LHC. FASER is designed to capture decays of new long-lived particles, produced outside of the ATLAS detector acceptance. These rare particles can decay in the FASER detector together with about 500-1000 Hz of other particles originating from the ATLAS interaction point. A very high efficiency trigger and data acquisition system is required to ensure that the physics events of interest will be recorded. This paper describes the trigger and data acquisition system of the FASER experiment and presents performance results of the system acquired during initial commissioning.

physics.ins-det↗

arXiv2021

VDPC: Variational Density Peak Clustering Algorithm

The widely applied density peak clustering (DPC) algorithm makes an intuitive cluster formation assumption that cluster centers are often surrounded by data points with lower local density and far away from other data points with higher local density. However, this assumption suffers from one limitation that it is often problematic when identifying clusters with lower density because they might be easily merged into other clusters with higher density. As a result, DPC may not be able to identify clusters with variational density. To address this issue, we propose a variational density peak clustering (VDPC) algorithm, which is designed to systematically and autonomously perform the clustering task on datasets with various types of density distributions. Specifically, we first propose a novel method to identify the representatives among all data points and construct initial clusters based on the identified representatives for further analysis of the clusters' property. Furthermore, we divide all data points into different levels according to their local density and propose a unified clustering framework by combining the advantages of both DPC and DBSCAN. Thus, all the identified initial clusters spreading across different density levels are systematically processed to form the final clusters. To evaluate the effectiveness of the proposed VDPC algorithm, we conduct extensive experiments using 20 datasets including eight synthetic, six real-world and six image datasets. The experimental results show that VDPC outperforms two classical algorithms (i.e., DPC and DBSCAN) and four state-of-the-art extended DPC algorithms.

cs.LG↗

arXiv2021

Expander Decomposition and Pruning: Faster, Stronger, and Simpler

We study the problem of graph clustering where the goal is to partition a graph into clusters, i.e. disjoint subsets of vertices, such that each cluster is well connected internally while sparsely connected to the rest of the graph. In particular, we use a natural bicriteria notion motivated by Kannan, Vempala, and Vetta which we refer to as {\em expander decomposition}. Expander decomposition has become one of the building blocks in the design of fast graph algorithms, most notably in the nearly linear time Laplacian solver by Spielman and Teng, and it also has wide applications in practice. We design algorithm for the parametrized version of expander decomposition, where given a graph $G$ of $m$ edges and a parameter $ϕ$, our algorithm finds a partition of the vertices into clusters such that each cluster induces a subgraph of conductance at least $ϕ$ (i.e. a $ϕ$ expander), and only a $\widetilde{O}(ϕ)$ fraction of the edges in $G$ have endpoints across different clusters. Our algorithm runs in $\widetilde{O}(m/ϕ)$ time, and is the first nearly linear time algorithm when $ϕ$ is at least $1/\log^{O(1)} m$, which is the case in most practical settings and theoretical applications. Previous results either take $Ω(m^{1+o(1)})$ time, or attain nearly linear time but with a weaker expansion guarantee where each output cluster is guaranteed to be contained inside some unknown $ϕ$ expander. Our result achieve both nearly linear running time and the strong expander guarantee for clusters. Moreover, a main technique we develop for our result can be applied to obtain a much better \emph{expander pruning} algorithm, which is the key tool for maintaining an expander decomposition on dynamic graphs. Finally, we note that our algorithm is developed from first principles based on relatively simple and basic techniques, thus making it very likely to be practical.

cs.DS↗

arXiv2021

The factorization-assisted topological-amplitude approach and its applications

Heavy meson decays provide an important platform for studies of both QCD and electroweak dynamics, which may contain some portals to understanding of nonperturbative QCD and physics beyond the Standard Model. The factorization-assisted topological-amplitude approach was proposed to study two-body non-leptonic $D$ meson decays, where a promising QCD inspired approach from first principles is still missing. It was also applied to $B$ meson decays whose subleading power contributions are difficult to calculate. By factorizing topological amplitudes into short distance Wilson coefficients and long distance hadronic matrix elements either to be calculated or to be parameterized, it provides an effective framework to extract information of nonperturbative dynamics involved. With important flavor SU(3) breaking effects taken into account, the data of the decay branching ratios (and also CP asymmetries in $B$ decays) can be fitted well. The extracted amplitudes were further applied to make predictions for other observables, such as CP asymmetries in $D$ decays, mixing parameters in the $D^0-\bar{D}^0$ system, and so on. By this review, we will describe the formulation of the factorization-assisted topological-amplitude approach and summarize its applications in $D$ and $B$ meson decays and highlight some of its achievements.

hep-ph↗

arXiv2021

Fed2: Feature-Aligned Federated Learning

Federated learning learns from scattered data by fusing collaborative models from local nodes. However, the conventional coordinate-based model averaging by FedAvg ignored the random information encoded per parameter and may suffer from structural feature misalignment. In this work, we propose Fed2, a feature-aligned federated learning framework to resolve this issue by establishing a firm structure-feature alignment across the collaborative models. Fed2 is composed of two major designs: First, we design a feature-oriented model structure adaptation method to ensure explicit feature allocation in different neural network structures. Applying the structure adaptation to collaborative models, matchable structures with similar feature information can be initialized at the very early training stage. During the federated learning process, we then propose a feature paired averaging scheme to guarantee aligned feature distribution and maintain no feature fusion conflicts under either IID or non-IID scenarios. Eventually, Fed2 could effectively enhance the federated learning convergence performance under extensive homo- and heterogeneous settings, providing excellent convergence speed, accuracy, and computation/communication efficiency.

cs.LG↗

arXiv2021

Automated Runtime-Aware Scheduling for Multi-Tenant DNN Inference on GPU

With the fast development of deep neural networks (DNNs), many real-world applications are adopting multiple models to conduct compound tasks, such as co-running classification, detection, and segmentation models on autonomous vehicles. Such multi-tenant DNN inference cases greatly exacerbate the computational complexity and call for comprehensive collaboration for graph-level operator scheduling, runtime-level resource awareness, as well as hardware scheduler support. However, the current scheduling support for such multi-tenant inference is still relatively backward. In this work, we propose a resource-aware scheduling framework for efficient multi-tenant DNN inference on GPU, which automatically coordinates DNN computing in different execution levels. Leveraging the unified scheduling intermediate representation and the automated ML-based searching algorithm, optimal schedules could be generated to wisely adjust model concurrency and interleave DNN model operators, maintaining a continuously balanced resource utilization across the entire inference process, and eventually improving the runtime efficiency. Experiments show that we could consistently achieve 1.3-1.7x speed-up, compared to regular DNN runtime libraries (e.g., CuDNN, TVM) and particular concurrent scheduling methods (e.g., NVIDIA Multi-Stream).

cs.DC↗