arXiv subjects

Di Wang

Publications and source records attributed to Di Wang.

At least 559 records · Page 31Linked to original sources

arXiv2020

AntiDote: Attention-based Dynamic Optimization for Neural Network Runtime Efficiency

Convolutional Neural Networks (CNNs) achieved great cognitive performance at the expense of considerable computation load. To relieve the computation load, many optimization works are developed to reduce the model redundancy by identifying and removing insignificant model components, such as weight sparsity and filter pruning. However, these works only evaluate model components' static significance with internal parameter information, ignoring their dynamic interaction with external inputs. With per-input feature activation, the model component significance can dynamically change, and thus the static methods can only achieve sub-optimal results. Therefore, we propose a dynamic CNN optimization framework in this work. Based on the neural network attention mechanism, we propose a comprehensive dynamic optimization framework including (1) testing-phase channel and column feature map pruning, as well as (2) training-phase optimization by targeted dropout. Such a dynamic optimization framework has several benefits: (1) First, it can accurately identify and aggressively remove per-input feature redundancy with considering the model-input interaction; (2) Meanwhile, it can maximally remove the feature map redundancy in various dimensions thanks to the multi-dimension flexibility; (3) The training-testing co-optimization favors the dynamic pruning and helps maintain the model accuracy even with very high feature pruning ratio. Extensive experiments show that our method could bring 37.4% to 54.5% FLOPs reduction with negligible accuracy drop on various of test networks.

cs.CV↗

arXiv2020

Biominerals with Texture Gradients are Functionally Graded Bioceramics Toughened by Stress Delocalization

Biomineralizing organisms are widely noted and extensively studied due to their ability to generate structures exhibiting exceptional crystallographic control. Primarily, it is the organisms, such as sea-urchins or bivalves, that generate nearly single-crystalline biocrystals that have attracted attention. In contrast, biomineralizing organisms with seemingly disordered polycrystalline bio-armor have been left relatively unstudied. However, the crystalline ordering in the black-lipped pearl oyster, Pinctada margaritifera, reveals that biominerals with varying crystal textures are an unrecognized class of functionally graded materials. Changing crystal textures inevitably cause a variation in Young modulus due to the orientation-dependent mechanical properties of crystals. The case of Pinctada margaritifera demonstrates that bioceramics with such crystallographical gradients are toughened by stress delocalization and reduced stress intensity factors, outperforming non-graded counterparts. These findings suggest that a multitude of biominerals, which are perceived as poorly ordered because of their polycrystallinity and changing crystal textures, may be considered as graded materials with hitherto unidentified emergent mechanical properties. The underlying design principle is remarkably simple and applicable to a wide range of crystalline material classes and may thus serve as a blueprint for future bioinspired functional materials.

cond-mat.mtrl-sci↗

arXiv2020

Noninteractive Locally Private Learning of Linear Models via Polynomial Approximations

Minimizing a convex risk function is the main step in many basic learning algorithms. We study protocols for convex optimization which provably leak very little about the individual data points that constitute the loss function. Specifically, we consider differentially private algorithms that operate in the local model, where each data record is stored on a separate user device and randomization is performed locally by those devices. We give new protocols for \emph{noninteractive} LDP convex optimization---i.e., protocols that require only a single randomized report from each user to an untrusted aggregator. We study our algorithms' performance with respect to expected loss---either over the data set at hand (empirical risk) or a larger population from which our data set is assumed to be drawn. Our error bounds depend on the form of individuals' contribution to the expected loss. For the case of \emph{generalized linear losses} (such as hinge and logistic losses), we give an LDP algorithm whose sample complexity is only linear in the dimensionality $p$ and quasipolynomial in other terms (the privacy parameters $ε$ and $δ$, and the desired excess risk $α$). This is the first algorithm for nonsmooth losses with sub-exponential dependence on $p$. For the Euclidean median problem, where the loss is given by the Euclidean distance to a given data point, we give a protocol whose sample complexity grows quasipolynomially in $p$. This is the first protocol with sub-exponential dependence on $p$ for a loss that is not a generalized linear loss . Our result for the hinge loss is based on a technique, dubbed polynomial of inner product approximation, which may be applicable to other problems. Our results for generalized linear losses and the Euclidean median are based on new reductions to the case of hinge loss.

cs.LG↗

arXiv2020

$p$-Norm Flow Diffusion for Local Graph Clustering

Local graph clustering and the closely related seed set expansion problem are primitives on graphs that are central to a wide range of analytic and learning tasks such as local clustering, community detection, nodes ranking and feature inference. Prior work on local graph clustering mostly falls into two categories with numerical and combinatorial roots respectively. In this work, we draw inspiration from both fields and propose a family of convex optimization formulations based on the idea of diffusion with p-norm network flow for $p\in (1,\infty)$. In the context of local clustering, we characterize the optimal solutions for these optimization problems and show their usefulness in finding low conductance cuts around input seed set. In particular, we achieve quadratic approximation of conductance in the case of $p=2$ similar to the Cheeger-type bounds of spectral methods, constant factor approximation when $p\rightarrow\infty$ similar to max-flow based methods, and a smooth transition for general $p$ values in between. Thus, our optimization formulation can be viewed as bridging the numerical and combinatorial approaches, and we can achieve the best of both worlds in terms of speed and noise robustness. We show that the proposed problem can be solved in strongly local running time for $p\ge 2$ and conduct empirical evaluations on both synthetic and real-world graphs to illustrate our approach compares favorably with existing methods.

cs.LG↗

arXiv2020

Geno: A Developer Tool for Authoring Multimodal Interaction on Existing Web Applications

Supporting voice commands in applications presents significant benefits to users. However, adding such support to existing GUI-based web apps is effort-consuming with a high learning barrier, as shown in our formative study, due to the lack of unified support for creating multimodal interfaces. We present Geno---a developer tool for adding the voice input modality to existing web apps without requiring significant NLP expertise. Geno provides a high-level workflow for developers to specify functionalities to be supported by voice (intents), create language models for detecting intents and the relevant information (parameters) from user utterances, and fulfill the intents by either programmatically invoking the corresponding functions or replaying GUI actions on the web app. Geno further supports multimodal references to GUI context in voice commands (e.g. "move this [event] to next week" while pointing at an event with the cursor). In a study, developers with little NLP expertise were able to add multimodal voice command support for two existing web apps using Geno.

cs.HC↗

arXiv2020

Liquid Resource Types

This article presents liquid resource types, a technique for automatically verifying the resource consumption of functional programs. Existing resource analysis techniques trade automation for flexibility -- automated techniques are restricted to relatively constrained families of resource bounds, while more expressive proof techniques admitting value-dependent bounds rely on handwritten proofs. Liquid resource types combine the best of these approaches, using logical refinements to automatically prove precise bounds on a program's resource consumption. The type system augments refinement types with potential annotations to conduct an amortized resource analysis. Importantly, users can annotate data structure declarations to indicate how potential is allocated within the type, allowing the system to express bounds with polynomials and exponentials, as well as more precise expressions depending on program values. We prove the soundness of the type system, provide a library of flexible and reusable data structures for conducting resource analysis, and use our prototype implementation to automatically verify resource bounds that previously required a manual proof.

cs.PL↗

arXiv2020

Hyperbolic spoof plasmons in layered equivalent graphene metasurfaces

Advances in graphene plasmonics offer numerous opportunities for enabling the design and manufacture of a variety of nanoelectronics and other exciting optical devices. However, due to the limitation of material properties, its operating frequency cannot drop to the microwave range. In this work, a new concept of microwave equivalent graphene based on the ultrathin monolayer plasmonic metasurface is proposed and demonstrated. Based on this concept, elliptical and hyperbolic dispersion can be theoretically obtained by stacking the equivalent graphene metasurfaces periodically. As proofs of the concept and method, an elliptical and an all-metal hyperbolic metamaterial are designed and numerically demonstrated. As a specified realization of the method, a practical hyperbolic metamaterial is fabricated and experimentally investigated with its validity verified by the directional propagation and photonic spin Hall effect. Furthermore, to investigate the validity of the method under extreme parameter conditions, a proof-of-concept hyperlens is designed and fabricated, with its near-field resolution of 0.05$λ$ experimentally verified. Based on the proposed concept, diverse optical graphene metamaterials such as focusing lens, dispersion-dependent directional couplers, and epsilon-near-zero materials can also be realized in the microwave regime.

physics.app-ph↗

arXiv2020

Towards Assessment of Randomized Smoothing Mechanisms for Certifying Adversarial Robustness

As a certified defensive technique, randomized smoothing has received considerable attention due to its scalability to large datasets and neural networks. However, several important questions remain unanswered, such as (i) whether the Gaussian mechanism is an appropriate option for certifying $\ell_2$-norm robustness, and (ii) whether there is an appropriate randomized (smoothing) mechanism to certify $\ell_\infty$-norm robustness. To shed light on these questions, we argue that the main difficulty is how to assess the appropriateness of each randomized mechanism. In this paper, we propose a generic framework that connects the existing frameworks in \cite{lecuyer2018certified, li2019certified}, to assess randomized mechanisms. Under our framework, for a randomized mechanism that can certify a certain extent of robustness, we define the magnitude of its required additive noise as the metric for assessing its appropriateness. We also prove lower bounds on this metric for the $\ell_2$-norm and $\ell_\infty$-norm cases as the criteria for assessment. Based on our framework, we assess the Gaussian and Exponential mechanisms by comparing the magnitude of additive noise required by these mechanisms and the lower bounds (criteria). We first conclude that the Gaussian mechanism is indeed an appropriate option to certify $\ell_2$-norm robustness. Surprisingly, we show that the Gaussian mechanism is also an appropriate option for certifying $\ell_\infty$-norm robustness, instead of the Exponential mechanism. Finally, we generalize our framework to $\ell_p$-norm for any $p\geq2$. Our theoretical findings are verified by evaluations on CIFAR10 and ImageNet.

cs.LG↗

arXiv2020

Distributed Equivalent Substitution Training for Large-Scale Recommender Systems

We present Distributed Equivalent Substitution (DES) training, a novel distributed training framework for large-scale recommender systems with dynamic sparse features. DES introduces fully synchronous training to large-scale recommendation system for the first time by reducing communication, thus making the training of commercial recommender systems converge faster and reach better CTR. DES requires much less communication by substituting the weights-rich operators with the computationally equivalent sub-operators and aggregating partial results instead of transmitting the huge sparse weights directly through the network. Due to the use of synchronous training on large-scale Deep Learning Recommendation Models (DLRMs), DES achieves higher AUC(Area Under ROC). We successfully apply DES training on multiple popular DLRMs of industrial scenarios. Experiments show that our implementation outperforms the state-of-the-art PS-based training framework, achieving up to 68.7% communication savings and higher throughput compared to other PS-based recommender systems.

cs.LG↗

arXiv2020

Electronic and magnetic properties of the Jahn-Teller active fluoride $\mathrm{NaCrF_{3}}$ from first-principles calculations

In perovskite-type compounds, the interplay of cooperative Jahn-Teller effect, electronic correlations and orbital degree of freedom leads to intriguing properties. $\mathrm{NaCrF_{3}}$ is a newly synthesized Jahn-Teller active fluoroperovskite where the $\mathrm{CrF_{6}^{4-}}$ octahedrons are considerably distorted. Based on the first-principles calculation, we analyze its electronic structure and magnetic properties. Our numerical results show that the $\mathrm{Cr^{2+}}$ ions adopt the high-spin $t_{2g\uparrow}^{3}e_{g\uparrow}^{1}$ configuration with $G$-type orbital ordering. We also estimate the magnetic exchange couplings and find that the in-plane and interplanar nearest-neighbor interactions are ferromagnetic and antiferromagnetic, respectively. The ground state of this material is $A$-type antiferromagnetic, in agreement with the experiments. Reasonable Curie-Weiss and $\mathrm{N\acute{e}el}$ temperatures compared to the experiments are given by mean-field approximation theory. Our results give a complete explanation of its electronic structure, magnetic and orbital order, and help to further comprehend the behaviors of Jahn-Teller active perovskite-type fluoride.

cond-mat.mtrl-sci↗

arXiv2020

Charmed baryon decays in $SU(3)_F$ symmetry

In the recent years, fruitful results on charmed baryons are obtained by BESIII, Belle and LHCb. We investigate the two-body non-leptonic decays of charmed baryons in the flavor $SU(3)$ symmetry. Hundreds of amplitude relations are clearly provided, and are classified according to the $I$-, $U$- and $V$-spin symmetries. Among them, some amplitude relations are tested by the experimental data, or used to predict the branching fractions based on the exact flavor symmetry without any other approximation. Some relations of $K^0_S-K^0_L$ asymmetries and $CP$ asymmetries are obtained under the $U$-spin symmetry in the modes of charmed baryon decaying into neutral kaons. Besides, the $U$-spin breaking effect is explored in the $Λ_c^+\to Σ^+K^{*0}$ and $Ξ_c^+\to p\bar{K}^{*0}$ modes.

hep-ph↗

arXiv2020

EEG-Based Emotion Recognition Using Regularized Graph Neural Networks

Electroencephalography (EEG) measures the neuronal activities in different brain regions via electrodes. Many existing studies on EEG-based emotion recognition do not fully exploit the topology of EEG channels. In this paper, we propose a regularized graph neural network (RGNN) for EEG-based emotion recognition. RGNN considers the biological topology among different brain regions to capture both local and global relations among different EEG channels. Specifically, we model the inter-channel relations in EEG signals via an adjacency matrix in a graph neural network where the connection and sparseness of the adjacency matrix are inspired by neuroscience theories of human brain organization. In addition, we propose two regularizers, namely node-wise domain adversarial training (NodeDAT) and emotion-aware distribution learning (EmotionDL), to better handle cross-subject EEG variations and noisy labels, respectively. Extensive experiments on two public datasets, SEED and SEED-IV, demonstrate the superior performance of our model than state-of-the-art models in most experimental settings. Moreover, ablation studies show that the proposed adjacency matrix and two regularizers contribute consistent and significant gain to the performance of our RGNN model. Finally, investigations on the neuronal activities reveal important brain regions and inter-channel relations for EEG-based emotion recognition.

cs.CV↗

arXiv2020

Distributed Kernel Ridge Regression with Communications

This paper focuses on generalization performance analysis for distributed algorithms in the framework of learning theory. Taking distributed kernel ridge regression (DKRR) for example, we succeed in deriving its optimal learning rates in expectation and providing theoretically optimal ranges of the number of local processors. Due to the gap between theory and experiments, we also deduce optimal learning rates for DKRR in probability to essentially reflect the generalization performance and limitations of DKRR. Furthermore, we propose a communication strategy to improve the learning performance of DKRR and demonstrate the power of communications in DKRR via both theoretical assessments and numerical experiments.

cs.LG↗

arXiv2020

Heterogeneous-Temporal Graph Convolutional Networks: Make the Community Detection Much Better

Community detection has long been an important yet challenging task to analyze complex networks with a focus on detecting topological structures of graph data. Essentially, real-world graph data contains various features, node and edge types which dynamically vary over time, and this invalidates most existing community detection approaches. To cope with these issues, this paper proposes the heterogeneous-temporal graph convolutional networks (HTGCN) to detect communities from hetergeneous and temporal graphs. Particularly, we first design a heterogeneous GCN component to acquire feature representations for each heterogeneous graph at each time step. Then, a residual compressed aggregation component is proposed to represent "dynamic" features for "varying" communities, which are then aggregated with "static" features extracted from current graph. Extensive experiments are evaluated on two real-world datasets, i.e., DBLP and IMDB. The promising results demonstrate that the proposed HTGCN is superior to both benchmark and the state-of-the-art approaches, e.g., GCN, GAT, GNN, LGNN, HAN and STAR, with respect to a number of evaluation criteria.

cs.LG↗

arXiv2020

A chip-scale oscillation-mode optomechanical inertial sensor near the thermodynamical limits

High-precision inertial sensing and gravity sensing are key in navigation, oil exploration, and earthquake prediction. In contrast to prior accelerometers using piezoelectric or electronic capacitance readout techniques, optical readout provides narrow-linewidth high-sensitivity laser detection along with low-noise resonant optomechanical transduction near the thermodynamical limits. Here an optomechanical inertial sensor with 8.2micro-g/Hz^1/2 velocity random walk (VRW) at acquisition rate of 100 Hz and 50.9 micro-g bias instability is demonstrated, suitable for consumer and industrial grade applications, e.g., inertial navigation, inclination sensing, platform stabilization, and/or wearable device motion detection. Driven into optomechanical sustained-oscillation, the slot photonic crystal cavity provides radio-frequency readout of the optically-driven transduction with enhanced 625 microg/Hz sensitivity. Measuring the optomechanically-stiffened oscillation shift, instead of the optical transmission shift, provides a 220x VRW enhancement over pre-oscillation mode detection due to the strong optomechanical transduction. Supported by theory, this inertial sensor operates 2.56x above the thermodynamical limit at small integration times, with 43-dB dynamic range, in a solid-state room-temperature readout architecture.

physics.ins-det↗

arXiv2020

Strange Axial-vector Mesons in D Meson Decays

The nature of strange axial-vector mesons are not well understood and can be investigated in $D$ meson decays. In this work, it is found that the experimental data of $D^0\rightarrow K^\pm K^\mp_1(1270)(\to ρK \,\,\text{or}\,\,K^{*}π)$ in the $D^0\to K^+K^-π^+π^-$ mode, disagree with the equality relation under the narrow width approximation and $CP$ conservation of strong decays. Considering more other results of $K_1(1270)$ decays, the data of $\mathcal{B}(D^0\rightarrow K^-K^+_1(1270)(\to K^{*0}π^+))$ is probably overestimated by one order of magnitude. We then calculate the branching fractions of the corresponding processes with $K_1(1400)$ in the factorization approach, and find $\mathcal{B}(D^0\rightarrow K^-K^+_1(1400)(\to K^{*0}π^+))$ is comparable to the predicted $\mathcal{B}(D^0\rightarrow K^-K^+_1(1270)(\to K^{*0}π^+))$ using the equality relation. Besides, we suggest to measure the ratios between $K_1(1270)\to ρK$ and $K^*π$ or to test the equality relations in other $D$ meson decay modes.

hep-ph↗

arXiv2019

Robust Feature-Based Point Registration Using Directional Mixture Model

This paper presents a robust probabilistic point registration method for estimating the rigid transformation (i.e. rotation matrix and translation vector) between two pointcloud dataset. The method improves the robustness of point registration and consequently the robot localization in the presence of outliers in the pointclouds which always occurs due to occlusion, dynamic objects, and sensor errors. The framework models the point registration task based on directional statistics on a unit sphere. In particular, a Kent distribution mixture model is adopted and the process of point registration has been carried out in the two phases of Expectation-Maximization algorithm. The proposed method has been evaluated on the pointcloud dataset from LiDAR sensors in an indoor environment.

cs.CV↗

arXiv2019

Local Flow Partitioning for Faster Edge Connectivity

We study the problem of computing a minimum cut in a simple, undirected graph and give a deterministic $O(m \log^2 n \log\log^2 n)$ time algorithm. This improves both on the best previously known deterministic running time of $O(m \log^{12} n)$ (Kawarabayashi and Thorup, STOC 2015) and the best previously known randomized running time of $O(m \log^{3} n)$ (Karger, J.ACM 2000) for this problem, though Karger's algorithm can be further applied to weighted graphs. Moreover, our result extends to balanced directed graphs, where the balance of a directed graph captures how close the graph is to being Eulerian. Our approach is using the Kawarabayashi and Thorup graph compression technique, which repeatedly finds low-conductance cuts. To find these cuts they use a diffusion-based local algorithm. We use instead a flow-based local algorithm and suitably adjust their framework to work with our flow-based subroutine. Both flow and diffusion based methods have a long history of being applied to finding low conductance cuts. Diffusion algorithms have several variants that are naturally local while it is more complicated to make flow methods local. Some prior work has proven nice properties for local flow based algorithms with respect to improving or cleaning up low conductance cuts. Our flow subroutine, however, is the first that is both local and produces low conductance cuts. Thus, it may be of independent interest.

cs.DS↗