arXiv subjects

Di Wang

Publications and source records attributed to Di Wang.

At least 487 records · Page 27Linked to original sources

arXiv2022

SEAT: Stable and Explainable Attention

Currently, attention mechanism becomes a standard fixture in most state-of-the-art natural language processing (NLP) models, not only due to outstanding performance it could gain, but also due to plausible innate explanation for the behaviors of neural architectures it provides, which is notoriously difficult to analyze. However, recent studies show that attention is unstable against randomness and perturbations during training or testing, such as random seeds and slight perturbation of embedding vectors, which impedes it from becoming a faithful explanation tool. Thus, a natural question is whether we can find some substitute of the current attention which is more stable and could keep the most important characteristics on explanation and prediction of attention. In this paper, to resolve the problem, we provide a first rigorous definition of such alternate namely SEAT (Stable and Explainable Attention). Specifically, a SEAT should has the following three properties: (1) Its prediction distribution is enforced to be close to the distribution based on the vanilla attention; (2) Its top-k indices have large overlaps with those of the vanilla attention; (3) It is robust w.r.t perturbations, i.e., any slight perturbation on SEAT will not change the prediction distribution too much, which implicitly indicates that it is stable to randomness and perturbations. Finally, through intensive experiments on various datasets, we compare our SEAT with other baseline methods using RNN, BiLSTM and BERT architectures via six different evaluation metrics for model interpretation, stability and accuracy. Results show that SEAT is more stable against different perturbations and randomness while also keeps the explainability of attention, which indicates it is a more faithful explanation. Moreover, compared with vanilla attention, there is almost no utility (accuracy) degradation for SEAT.

cs.CL↗

arXiv2022

Semantic-aware Texture-Structure Feature Collaboration for Underwater Image Enhancement

Underwater image enhancement has become an attractive topic as a significant technology in marine engineering and aquatic robotics. However, the limited number of datasets and imperfect hand-crafted ground truth weaken its robustness to unseen scenarios, and hamper the application to high-level vision tasks. To address the above limitations, we develop an efficient and compact enhancement network in collaboration with a high-level semantic-aware pretrained model, aiming to exploit its hierarchical feature representation as an auxiliary for the low-level underwater image enhancement. Specifically, we tend to characterize the shallow layer features as textures while the deep layer features as structures in the semantic-aware model, and propose a multi-path Contextual Feature Refinement Module (CFRM) to refine features in multiple scales and model the correlation between different features. In addition, a feature dominative network is devised to perform channel-wise modulation on the aggregated texture and structure features for the adaptation to different feature patterns of the enhancement network. Extensive experiments on benchmarks demonstrate that the proposed algorithm achieves more appealing results and outperforms state-of-the-art methods by large margins. We also apply the proposed algorithm to the underwater salient object detection task to reveal the favorable semantic-aware ability for high-level vision tasks. The code is available at STSC.

cs.CV↗

arXiv2022

1st ICLR International Workshop on Privacy, Accountability, Interpretability, Robustness, Reasoning on Structured Data (PAIR^2Struct)

Recent years have seen advances on principles and guidance relating to accountable and ethical use of artificial intelligence (AI) spring up around the globe. Specifically, Data Privacy, Accountability, Interpretability, Robustness, and Reasoning have been broadly recognized as fundamental principles of using machine learning (ML) technologies on decision-critical and/or privacy-sensitive applications. On the other hand, in tremendous real-world applications, data itself can be well represented as various structured formalisms, such as graph-structured data (e.g., networks), grid-structured data (e.g., images), sequential data (e.g., text), etc. By exploiting the inherently structured knowledge, one can design plausible approaches to identify and use more relevant variables to make reliable decisions, thereby facilitating real-world deployments.

stat.ML↗

arXiv2022

On PAC Learning Halfspaces in Non-interactive Local Privacy Model with Public Unlabeled Data

In this paper, we study the problem of PAC learning halfspaces in the non-interactive local differential privacy model (NLDP). To breach the barrier of exponential sample complexity, previous results studied a relaxed setting where the server has access to some additional public but unlabeled data. We continue in this direction. Specifically, we consider the problem under the standard setting instead of the large margin setting studied before. Under different mild assumptions on the underlying data distribution, we propose two approaches that are based on the Massart noise model and self-supervised learning and show that it is possible to achieve sample complexities that are only linear in the dimension and polynomial in other terms for both private and public data, which significantly improve the previous results. Our methods could also be used for other private PAC learning problems.

cs.LG↗

arXiv2022

Truthful Generalized Linear Models

In this paper we study estimating Generalized Linear Models (GLMs) in the case where the agents (individuals) are strategic or self-interested and they concern about their privacy when reporting data. Compared with the classical setting, here we aim to design mechanisms that can both incentivize most agents to truthfully report their data and preserve the privacy of individuals' reports, while their outputs should also close to the underlying parameter. In the first part of the paper, we consider the case where the covariates are sub-Gaussian and the responses are heavy-tailed where they only have the finite fourth moments. First, motivated by the stationary condition of the maximizer of the likelihood function, we derive a novel private and closed form estimator. Based on the estimator, we propose a mechanism which has the following properties via some appropriate design of the computation and payment scheme for several canonical models such as linear regression, logistic regression and Poisson regression: (1) the mechanism is $o(1)$-jointly differentially private (with probability at least $1-o(1)$); (2) it is an $o(\frac{1}{n})$-approximate Bayes Nash equilibrium for a $(1-o(1))$-fraction of agents to truthfully report their data, where $n$ is the number of agents; (3) the output could achieve an error of $o(1)$ to the underlying parameter; (4) it is individually rational for a $(1-o(1))$ fraction of agents in the mechanism ; (5) the payment budget required from the analyst to run the mechanism is $o(1)$. In the second part, we consider the linear regression model under more general setting where both covariates and responses are heavy-tailed and only have finite fourth moments. By using an $\ell_4$-norm shrinkage operator, we propose a private estimator and payment scheme which have similar properties as in the sub-Gaussian case.

cs.LG↗

arXiv2022

Observation of robust zero-energy state and enhanced superconducting gap in a tri-layer heterostructure of MnTe/Bi2Te3/Fe(Te, Se)

The interface between magnetic material and superconductors has long been predicted to host unconventional superconductivity, such as spin-triplet pairing and topological nontrivial pairing state, particularly when spin-orbital coupling (SOC) is incorporated. To identify these novel pairing states, fabricating homogenous heterostructures which contain such various properties are preferred, but often challenging. Here we synthesized a tri-layer type van-der Waals heterostructure of MnTe/Bi2Te3/Fe(Te, Se), which combined s-wave superconductivity, thickness dependent magnetism and strong SOC. Via low-temperature scanning tunneling microscopy (STM), we observed robust zero-energy states with notably nontrivial properties and an enhanced superconducting gap size on single unit-cell (UC) MnTe surface. In contrast, no zero-energy state was observed on 2UC MnTe. First-principle calculations further suggest the 1UC MnTe has large interfacial Dzyaloshinskii-Moriya interaction (DMI) and a frustrated AFM state, which could promote non-collinear spin textures. It thus provides a promising platform for exploring topological nontrivial superconductivity.

cond-mat.supr-con↗

arXiv2022

Pay Attention to Hard Trials

Performance of speaker recognition systems is evaluated on test trials. Although as crucial as rulers for tailors, trials have not been carefully treated so far, and most existing benchmarks compose trials by naive cross-pairing. In this paper, we argue that the cross-pairing approach produces overwhelming easy trials, which in turn leads to potential bias in system and technique comparison. To solve the problem, we advocate more attention to hard trials. We present an SVM-based approach to identifying hard trials and use it to construct new evaluation sets for VoxCeleb1 and SITW. With the new sets, we can re-evaluate the contribution of some recent technologies. The code and the identified hard trials will be published online at http://project.cslt.org.

cs.SD↗

arXiv2022

The SAMI Galaxy Survey: Using concentrated star-formation and stellar population ages to understand environmental quenching

We study environmental quenching using the spatial distribution of current star-formation and stellar population ages with the full SAMI Galaxy Survey. By using a star-formation concentration index [C-index, defined as log10(r_{50,Halpha}/r_{50,cont})], we separate our sample into regular galaxies (C-index>-0.2) and galaxies with centrally concentrated star-formation (SF-concentrated; C-index<-0.2). Concentrated star-formation is a potential indicator of galaxies currently undergoing `outside-in' quenching. Our environments cover ungrouped galaxies, low-mass groups (M_200<10^12.5 M_sun), high-mass groups (M_200 in the range 10^{12.5-14} M_sun) and clusters (M_200>10^14 M_sun). We find the fraction of SF-concentrated galaxies increases as halo mass increases with 9\pm2 per cent, 8\pm3 per cent, 19\pm4 per cent and 29\pm4 per cent for ungrouped galaxies, low-mass groups, high-mass groups and clusters, respectively. We interpret these results as evidence for `outside-in' quenching in groups and clusters. To investigate the quenching time-scale in SF-concentrated galaxies, we calculate light-weighted age (Age_L) and mass-weighted age (Age_M) using full spectral fitting, as well as the Dn4000 and Hdelta_A indices. We assume that the average galaxy age radial profile before entering a group or cluster is similar to ungrouped regular galaxies. At large radius (1-2 R_e), SF-concentrated galaxies in high-mass groups have older ages than ungrouped regular galaxies with an age difference of 1.83\pm0.38 Gyr for Age_L and 1.34\pm0.56 Gyr for Age_M. This suggests that while `outside-in' quenching can be effective in groups, the process will not quickly quench the entire galaxy. In contrast, the ages at 1-2 R_e of cluster SF-concentrated galaxies and ungrouped regular galaxies are consistent (0.19\pm0.21 Gyr for Age_L, 0.40\pm0.61 Gyr for Age_M), suggesting the quenching process must be rapid.

astro-ph.GA↗

arXiv2022

Nonparametric Quantile Regression for Homogeneity Pursuit in Panel Data Models

Many panel data have the latent subgroup effect on individuals, and it is important to correctly identify these groups since the efficiency of resulting estimators can be improved significantly by pooling the information of individuals within each group. However, the currently assumed parametric and semiparametric relationship between the response and predictors may be misspecified, which leads to a wrong grouping result, and the nonparametric approach hence can be considered to avoid such mistakes. Moreover, the response may depend on predictors in different ways at various quantile levels, and the corresponding grouping structure may also vary. To tackle these problems, this article proposes a nonparametric quantile regression method for homogeneity pursuit in panel data models with individual effects, and a pairwise fused penalty is used to automatically select the number of groups. The asymptotic properties are established, and an ADMM algorithm is also developed. The finite sample performance is evaluated by simulation experiments, and the usefulness of the proposed methodology is further illustrated by an empirical example.

stat.ME↗

arXiv2022

Estimating Smooth GLM in Non-interactive Local Differential Privacy Model with Public Unlabeled Data

In this paper, we study the problem of estimating smooth Generalized Linear Models (GLMs) in the Non-interactive Local Differential Privacy (NLDP) model. Different from its classical setting, our model allows the server to access some additional public but unlabeled data. In the first part of the paper we focus on GLMs. Specifically, we first consider the case where each data record is i.i.d. sampled from a zero-mean multivariate Gaussian distribution. Motivated by the Stein's lemma, we present an $(ε, δ)$-NLDP algorithm for GLMs. Moreover, the sample complexity of public and private data for the algorithm to achieve an $\ell_2$-norm estimation error of $α$ (with high probability) is ${O}(p α^{-2})$ and $\tilde{O}(p^3α^{-2}ε^{-2})$ respectively, where $p$ is the dimension of the feature vector. This is a significant improvement over the previously known exponential or quasi-polynomial in $α^{-1}$, or exponential in $p$ sample complexities of GLMs with no public data. Then we consider a more general setting where each data record is i.i.d. sampled from some sub-Gaussian distribution with bounded $\ell_1$-norm. Based on a variant of Stein's lemma, we propose an $(ε, δ)$-NLDP algorithm for GLMs whose sample complexity of public and private data to achieve an $\ell_\infty$-norm estimation error of $α$ is ${O}(p^2α^{-2})$ and $\tilde{O}(p^2α^{-2}ε^{-2})$ respectively, under some mild assumptions and if $α$ is not too small ({\em i.e.,} $α\geq Ω(\frac{1}{\sqrt{p}})$). In the second part of the paper, we extend our idea to the problem of estimating non-linear regressions and show similar results as in GLMs for both multivariate Gaussian and sub-Gaussian cases. Finally, we demonstrate the effectiveness of our algorithms through experiments on both synthetic and real-world datasets.

cs.LG↗

arXiv2022

Kullback-Leibler-Based Discrete Failure Time Models for Integration of Published Prediction Models with New Time-To-Event Dataset

Prediction of time-to-event data often suffers from rare event rates, small sample sizes, high dimensionality and low signal-to-noise ratios. Incorporating published prediction models from large-scale studies is expected to improve the performance of prognosis prediction on internal individual-level time-to-event data. However, existing integration approaches typically assume that underlying distributions from the external and internal data sources are similar, which is often invalid. To account for challenges including heterogeneity, data sharing, and privacy constraints, we propose a discrete failure time modeling procedure, which utilizes a discrete hazard-based Kullback-Leibler discriminatory information measuring the discrepancy between the published models and the internal dataset. Simulations show the advantage of the proposed method compared with those solely based on the internal data or published models. We apply the proposed method to improve prediction performance on a kidney transplant dataset from a local hospital by integrating this small-scale dataset with published survival models obtained from the national transplant registry.

stat.ME↗

arXiv2022

The FASER Detector

FASER, the ForwArd Search ExpeRiment, is an experiment dedicated to searching for light, extremely weakly-interacting particles at CERN's Large Hadron Collider (LHC). Such particles may be produced in the very forward direction of the LHC's high-energy collisions and then decay to visible particles inside the FASER detector, which is placed 480 m downstream of the ATLAS interaction point, aligned with the beam collisions axis. FASER also includes a sub-detector, FASER$ν$, designed to detect neutrinos produced in the LHC collisions and to study their properties. In this paper, each component of the FASER detector is described in detail, as well as the installation of the experiment system and its commissioning using cosmic-rays collected in September 2021 and during the LHC pilot beam test carried out in October 2021. FASER will start taking LHC collision data in 2022, and will run throughout LHC Run 3.

physics.ins-det↗

arXiv2022

Deep neural network heatmaps capture Alzheimer's disease patterns reported in a large meta-analysis of neuroimaging studies

Deep neural networks currently provide the most advanced and accurate machine learning models to distinguish between structural MRI scans of subjects with Alzheimer's disease and healthy controls. Unfortunately, the subtle brain alterations captured by these models are difficult to interpret because of the complexity of these multi-layer and non-linear models. Several heatmap methods have been proposed to address this issue and analyze the imaging patterns extracted from the deep neural networks, but no quantitative comparison between these methods has been carried out so far. In this work, we explore these questions by deriving heatmaps from Convolutional Neural Networks (CNN) trained using T1 MRI scans of the ADNI data set, and by comparing these heatmaps with brain maps corresponding to Support Vector Machines (SVM) coefficients. Three prominent heatmap methods are studied: Layer-wise Relevance Propagation (LRP), Integrated Gradients (IG), and Guided Grad-CAM (GGC). Contrary to prior studies where the quality of heatmaps was visually or qualitatively assessed, we obtained precise quantitative measures by computing overlap with a ground-truth map from a large meta-analysis that combined 77 voxel-based morphometry (VBM) studies independently from ADNI. Our results indicate that all three heatmap methods were able to capture brain regions covering the meta-analysis map and achieved better results than SVM coefficients. Among them, IG produced the heatmaps with the best overlap with the independent meta-analysis.

eess.IV↗

arXiv2022

Achievement of 200,000 hours of operation at KEK 7-GeV electron 4-GeV positron injector linac

KEK electron positron injector LINAC initiated the injection operation into Photon Factory (PF) light source in 1982. Since then for 39 years, it has served for multiple projects, namely, TRISTAN, PF-AR, KEKB, and SuperKEKB. Its total operation time has accumulated 200 thousand hours on May 7, 2020. We are extremely proud of the achievement following continuous efforts by our seniors. The construction of the injector LINAC started in 1978, and it was commissioned for PF with 2.5 GeV electron in 1982. In parallel, the positron generator linac was constructed for the TRISTAN collider project. The slow positron facility was also commissioned in 1992. After the KEKB asymmetric-energy collider project was commissioned in 1998 with direct energy injections, the techniques such as two-bunch acceleration and simultaneous injection were developed. As the soft structure design of the LINAC was too weak against the great east Japan earthquake, it took three years to recover. Then the construction and commissioning for the SuperKEKB project went on, and the simultaneous top-up injection into four storage rings contributes to the both elementary particle physics and photon science.

physics.acc-ph↗

arXiv2022

Levi-type Schur-Sergeev duality for general linear super groups

In this note, we investigate a kind of double centralizer property for general linear supergroups. For the super space $V=\mathbb{K}^{m\mid n}$ over an algebraically closed field $\mathbb{K}$ whose characteristic is not equal to $2$, we consider its $\mathbb{Z}_2$-homogeneous one-dimensional extension $\underline V=V\oplus\mathbb{K}v$, and the natural action of the supergroup $\tilde G:=\text{GL}(V)\times \textbf{G}_m$ on $\underline V$. Then we have the tensor product supermodule ($\underline{V}^{\otimes r}$, $ρ_r$) of $\tilde G$. We present a kind of generalized Schur-Sergeev duality which is said that the Schur superalgebras $S'(m|n,r)$ of $\tilde G$ and a so-called weak degenerate double Hecke algebra $\underline{\mathcal{H}}_r$ are double centralizers. The weak degenerate double Hecke algebra is an infinite dimensional algebra, which has a natural representation on the tensor product space. This notion comes from \cite{B-Y-Y2020}, with a little modification.

math.RT↗

arXiv2022

Rate-Optimal Robust Estimation of High-Dimensional Vector Autoregressive Models

High-dimensional time series data appear in many scientific areas in the current data-rich environment. Analysis of such data poses new challenges to data analysts because of not only the complicated dynamic dependence between the series, but also the existence of aberrant observations, such as missing values, contaminated observations, and heavy-tailed distributions. For high-dimensional vector autoregressive (VAR) models, we introduce a unified estimation procedure that is robust to model misspecification, heavy-tailed noise contamination, and conditional heteroscedasticity. The proposed methodology enjoys both statistical optimality and computational efficiency, and can handle many popular high-dimensional models, such as sparse, reduced-rank, banded, and network-structured VAR models. With proper regularization and data truncation, the estimation convergence rates are shown to be almost optimal in the minimax sense under a bounded $(2+2ε)$-th moment condition. When $ε\geq1$, the rates of convergence match those obtained under the sub-Gaussian assumption. Consistency of the proposed estimators is also established for some $ε\in(0,1)$, with minimax optimal convergence rates associated with $ε$. The efficacy of the proposed estimation methods is demonstrated by simulation and a U.S. macroeconomic example.

stat.ME↗

arXiv2022

The tracking detector of the FASER experiment

FASER is a new experiment designed to search for new light weakly-interacting long-lived particles (LLPs) and study high-energy neutrino interactions in the very forward region of the LHC collisions at CERN. The experimental apparatus is situated 480 m downstream of the ATLAS interaction-point aligned with the beam collision axis. The FASER detector includes four identical tracker stations constructed from silicon microstrip detectors. Three of the tracker stations form a tracking spectrometer, and enable FASER to detect the decay products of LLPs decaying inside the apparatus, whereas the fourth station is used for the neutrino analysis. The spectrometer has been installed in the LHC complex since March 2021, while the fourth station is not yet installed. FASER will start physics data taking when the LHC resumes operation in early 2022. This paper describes the design, construction and testing of the tracking spectrometer, including the associated components such as the mechanics, readout electronics, power supplies and cooling system.

physics.ins-det↗

arXiv2022

A Survey of Multi-Tenant Deep Learning Inference on GPU

Deep Learning (DL) models have achieved superior performance. Meanwhile, computing hardware like NVIDIA GPUs also demonstrated strong computing scaling trends with 2x throughput and memory bandwidth for each generation. With such strong computing scaling of GPUs, multi-tenant deep learning inference by co-locating multiple DL models onto the same GPU becomes widely deployed to improve resource utilization, enhance serving throughput, reduce energy cost, etc. However, achieving efficient multi-tenant DL inference is challenging which requires thorough full-stack system optimization. This survey aims to summarize and categorize the emerging challenges and optimization opportunities for multi-tenant DL inference on GPU. By overviewing the entire optimization stack, summarizing the multi-tenant computing innovations, and elaborating the recent technological advances, we hope that this survey could shed light on new optimization perspectives and motivate novel works in future large-scale DL system optimization.

cs.DC↗