arXiv subjects

Di Wang

Publications and source records attributed to Di Wang.

At least 541 records · Page 30Linked to original sources

arXiv2021

CARE: Commonsense-Aware Emotional Response Generation with Latent Concepts

Rationality and emotion are two fundamental elements of humans. Endowing agents with rationality and emotion has been one of the major milestones in AI. However, in the field of conversational AI, most existing models only specialize in one aspect and neglect the other, which often leads to dull or unrelated responses. In this paper, we hypothesize that combining rationality and emotion into conversational agents can improve response quality. To test the hypothesis, we focus on one fundamental aspect of rationality, i.e., commonsense, and propose CARE, a novel model for commonsense-aware emotional response generation. Specifically, we first propose a framework to learn and construct commonsense-aware emotional latent concepts of the response given an input message and a desired emotion. We then propose three methods to collaboratively incorporate the latent concepts into response generation. Experimental results on two large-scale datasets support our hypothesis and show that our model can produce more accurate and commonsense-aware emotional responses and achieve better human ratings than state-of-the-art models that only specialize in one aspect.

cs.CL↗

arXiv2021

Magneto-electric Tuning of Pinning-Type Permanent Magnets through Atomic-Scale Engineering of Grain Boundaries

Pinning-type magnets maintaining high coercivity, i.e. the ability to sustain magnetization, at high temperature are at the core of thriving clean-energy technologies. Among these, Sm2Co17-based magnets are excellent candidates owing to their high-temperature stability. However, despite decades of efforts to optimize the intragranular microstructure, the coercivity currently only reaches 20~30% of the theoretical limits. Here, the roles of the grain-interior nanostructure and the grain boundaries in controlling coercivity are disentangled by an emerging magneto-electric approach. Through hydrogen charging/discharging by applying voltages of only ~ 1 V, the coercivity is reversibly tuned by an unprecedented value of ~ 1.3 T. In situ magneto-structural measurements and atomic-scale tracking of hydrogen atoms reveal that the segregation of hydrogen atoms at the grain boundaries, rather than the change of the crystal structure, dominates the reversible and substantial change of coercivity. Hydrogen lowers the local magnetocrystalline anisotropy and facilitates the magnetization reversal starting from the grain boundaries. Our study reveals the previously neglected critical role of grain boundaries in the conventional magnetisation-switching paradigm, suggesting a critical reconsideration of strategies to overcome the coercivity limits in permanent magnets, via for instance atomic-scale grain boundary engineering.

cond-mat.mtrl-sci↗

arXiv2020

Towards Latency-aware DNN Optimization with GPU Runtime Analysis and Tail Effect Elimination

Despite the superb performance of State-Of-The-Art (SOTA) DNNs, the increasing computational cost makes them very challenging to meet real-time latency and accuracy requirements. Although DNN runtime latency is dictated by model property (e.g., architecture, operations), hardware property (e.g., utilization, throughput), and more importantly, the effective mapping between these two, many existing approaches focus only on optimizing model property such as FLOPS reduction and overlook the mismatch between DNN model and hardware properties. In this work, we show that the mismatch between the varied DNN computation workloads and GPU capacity can cause the idle GPU tail effect, leading to GPU under-utilization and low throughput. As a result, the FLOPs reduction cannot bring effective latency reduction, which causes sub-optimal accuracy versus latency trade-offs. Motivated by this, we propose a GPU runtime-aware DNN optimization methodology to eliminate such GPU tail effect adaptively on GPU platforms. Our methodology can be applied on top of existing SOTA DNN optimization approaches to achieve better latency and accuracy trade-offs. Experiments show 11%-27% latency reduction and 2.5%-4.0% accuracy improvement over several SOTA DNN pruning and NAS methods, respectively

cs.AR↗

arXiv2020

Deep Learning Analysis and Age Prediction from Shoeprints

Human walking and gaits involve several complex body parts and are influenced by personality, mood, social and cultural traits, and aging. These factors are reflected in shoeprints, which in turn can be used to predict age, a problem not systematically addressed using any computational approach. We collected 100,000 shoeprints of subjects ranging from 7 to 80 years old and used the data to develop a deep learning end-to-end model ShoeNet to analyze age-related patterns and predict age. The model integrates various convolutional neural network models together using a skip mechanism to extract age-related features, especially in pressure and abrasion regions from pair-wise shoeprints. The results show that 40.23% of the subjects had prediction errors within 5-years of age and the prediction accuracy for gender classification reached 86.07%. Interestingly, the age-related features mostly reside in the asymmetric differences between left and right shoeprints. The analysis also reveals interesting age-related and gender-related patterns in the pressure distributions on shoeprints; in particular, the pressure forces spread from the middle of the toe toward outside regions over age with gender-specific variations on heel regions. Such statistics provide insight into new methods for forensic investigations, medical studies of gait-pattern disorders, biometrics, and sport studies.

cs.CV↗

arXiv2020

Third ArchEdge Workshop: Exploring the Design Space of Efficient Deep Neural Networks

This paper gives an overview of our ongoing work on the design space exploration of efficient deep neural networks (DNNs). Specifically, we cover two aspects: (1) static architecture design efficiency and (2) dynamic model execution efficiency. For static architecture design, different from existing end-to-end hardware modeling assumptions, we conduct full-stack profiling at the GPU core level to identify better accuracy-latency trade-offs for DNN designs. For dynamic model execution, different from prior work that tackles model redundancy at the DNN-channels level, we explore a new dimension of DNN feature map redundancy to be dynamically traversed at runtime. Last, we highlight several open questions that are poised to draw research attention in the next few years.

cs.AR↗

arXiv2020

Probabilistic Resource-Aware Session Types

Session types guarantee that message-passing processes adhere to predefined communication protocols. Prior work on session types has focused on deterministic languages but many message-passing systems, such as Markov chains and randomized distributed algorithms, are probabilistic. To model and analyze such systems, this article introduces probabilistic session types and explores their application in automatic expected resource analysis. Probabilistic session types describe probability distributions over messages and are a conservative extension of intuitionistic (binary) session types. To send on a probabilistic channel, processes have to utilize internal randomness from a probabilistic branching expression or external randomness from receiving on a probabilistic channel. The analysis for expected resource bounds is integrated with the type system and is a variant of automatic amortized resource analysis. It can automatically derive symbolic bounds for different cost metrics by reducing type inference to linear constraint solving. The technical contributions include the meta theory that is based on a novel nested multiverse semantics and a type-reconstruction algorithm that allows flexible mixing of different sources of randomness without burdening the programmer with type annotations. The type system has been implemented in the language PRast. Experiments demonstrate that PRast is applicable in different domains such as resource analysis of randomized distributed algorithms, verification of limiting distributions in Markov chains, and analysis of probabilistic digital contracts.

cs.PL↗

arXiv2020

Constructing Hyperbolic Metamaterials with Arbitrary Medium

Recent advances in hyperbolic metamaterials have spurred many breakthroughs in the field of manipulating light propagation. However, the unusual electromagnetic properties also put extremely high demands on its compositional materials. Limited by the finite relative permittivity of the natural materials, the effective permittivity of the constructed hyperbolic metamaterials is also confined to a narrow range. Here, based on the proposed concept of structure-induced spoof surface plasmon, we prove that arbitrary materials can be selected to construct the hyperbolic metamaterials with independent relative effective permittivity components. Besides, the theoretical achievable ranges of the relative effective permittivity components are unlimited. As proofs of the method, three novel hyperbolic metamaterials are designed with their functionalities validated numerically and experimentally by specified directional propagation. To further illustrate the superiority of the method, an all-metal low-loss hyperbolic metamaterial filled with air is proposed and demonstrated. The proposed methodology effectively reduces the design requirement for hyperbolic metamaterials and provides new ideas for the scenarios where large permittivity coverage is needed such as microwave and terahertz focus, super-resolution imaging, electromagnetic cloaking, and so on.

physics.app-ph↗

arXiv2020

Empirical Risk Minimization in the Non-interactive Local Model of Differential Privacy

In this paper, we study the Empirical Risk Minimization (ERM) problem in the non-interactive Local Differential Privacy (LDP) model. Previous research on this problem \citep{smith2017interaction} indicates that the sample complexity, to achieve error $α$, needs to be exponentially depending on the dimensionality $p$ for general loss functions. In this paper, we make two attempts to resolve this issue by investigating conditions on the loss functions that allow us to remove such a limit. In our first attempt, we show that if the loss function is $(\infty, T)$-smooth, by using the Bernstein polynomial approximation we can avoid the exponential dependency in the term of $α$. We then propose player-efficient algorithms with $1$-bit communication complexity and $O(1)$ computation cost for each player. The error bound of these algorithms is asymptotically the same as the original one. With some additional assumptions, we also give an algorithm which is more efficient for the server. In our second attempt, we show that for any $1$-Lipschitz generalized linear convex loss function, there is an $(ε, δ)$-LDP algorithm whose sample complexity for achieving error $α$ is only linear in the dimensionality $p$. Our results use a polynomial of inner product approximation technique. Finally, motivated by the idea of using polynomial approximation and based on different types of polynomial approximations, we propose (efficient) non-interactive locally differentially private algorithms for learning the set of k-way marginal queries and the set of smooth queries.

cs.LG↗

arXiv2020

High-dimensional vector autoregressive time series modeling via tensor decomposition

The classical vector autoregressive model is a fundamental tool for multivariate time series analysis. However, it involves too many parameters when the number of time series and lag order are even moderately large. This paper proposes to rearrange the transition matrices of the model into a tensor form such that the parameter space can be restricted along three directions simultaneously via tensor decomposition. In contrast, the reduced-rank regression method can restrict the parameter space in only one direction. Besides achieving substantial dimension reduction, the proposed model is interpretable from the factor modeling perspective. Moreover, to handle high-dimensional time series, this paper considers imposing sparsity on factor matrices to improve the model interpretability and estimation efficiency, which leads to a sparsity-inducing estimator. For the low-dimensional case, we derive asymptotic properties of the proposed least squares estimator and introduce an alternating least squares algorithm. For the high-dimensional case, we establish non-asymptotic properties of the sparsity-inducing estimator and propose an ADMM algorithm for regularized estimation. Simulation experiments and a real data example demonstrate the advantages of the proposed approach over various existing methods.

stat.ME↗

arXiv2020

On Differentially Private Stochastic Convex Optimization with Heavy-tailed Data

In this paper, we consider the problem of designing Differentially Private (DP) algorithms for Stochastic Convex Optimization (SCO) on heavy-tailed data. The irregularity of such data violates some key assumptions used in almost all existing DP-SCO and DP-ERM methods, resulting in failure to provide the DP guarantees. To better understand this type of challenges, we provide in this paper a comprehensive study of DP-SCO under various settings. First, we consider the case where the loss function is strongly convex and smooth. For this case, we propose a method based on the sample-and-aggregate framework, which has an excess population risk of $\tilde{O}(\frac{d^3}{nε^4})$ (after omitting other factors), where $n$ is the sample size and $d$ is the dimensionality of the data. Then, we show that with some additional assumptions on the loss functions, it is possible to reduce the \textit{expected} excess population risk to $\tilde{O}(\frac{ d^2}{ nε^2 })$. To lift these additional conditions, we also provide a gradient smoothing and trimming based scheme to achieve excess population risks of $\tilde{O}(\frac{ d^2}{nε^2})$ and $\tilde{O}(\frac{d^\frac{2}{3}}{(nε^2)^\frac{1}{3}})$ for strongly convex and general convex loss functions, respectively, \textit{with high probability}. Experiments suggest that our algorithms can effectively deal with the challenges caused by data irregularity.

cs.LG↗

arXiv2020

Estimating Stochastic Linear Combination of Non-linear Regressions Efficiently and Scalably

Recently, many machine learning and statistical models such as non-linear regressions, the Single Index, Multi-index, Varying Coefficient Index Models and Two-layer Neural Networks can be reduced to or be seen as a special case of a new model which is called the \textit{Stochastic Linear Combination of Non-linear Regressions} model. However, due to the high non-convexity of the problem, there is no previous work study how to estimate the model. In this paper, we provide the first study on how to estimate the model efficiently and scalably. Specifically, we first show that with some mild assumptions, if the variate vector $x$ is multivariate Gaussian, then there is an algorithm whose output vectors have $\ell_2$-norm estimation errors of $O(\sqrt{\frac{p}{n}})$ with high probability, where $p$ is the dimension of $x$ and $n$ is the number of samples. The key idea of the proof is based on an observation motived by the Stein's lemma. Then we extend our result to the case where $x$ is bounded and sub-Gaussian using the zero-bias transformation, which could be seen as a generalization of the classic Stein's lemma. We also show that with some additional assumptions there is an algorithm whose output vectors have $\ell_\infty$-norm estimation errors of $O(\frac{1}{\sqrt{p}}+\sqrt{\frac{p}{n}})$ with high probability. We also provide a concrete example to show that there exists some link function which satisfies the previous assumptions. Finally, for both Gaussian and sub-Gaussian cases we propose a faster sub-sampling based algorithm and show that when the sub-sample sizes are large enough then the estimation errors will not be sacrificed by too much. Experiments for both cases support our theoretical results. To the best of our knowledge, this is the first work that studies and provides theoretical guarantees for the stochastic linear combination of non-linear regressions model.

cs.LG↗

arXiv2020

M Theory from F Theory

We write down a $GL(D+1)$ (D for the dimension of string theory) manifest fundamental brane worldvolume current algebra description of M theory, which consists of a pair of vector field $X^m$ and dual 2-form field $X_{mn}$, compositing together to parametrize the spacetime, with a selfduality condition for sectioning. The worldvolume of the brane itself is a (D+2) dimensional object, and the background spacetime after sectioning has dimension (D+1). We summarize the features of the algebra. The field contents of the corresponding background geometry, the usual vielbein $e_a{}^m$ and the 3-form $A_{mnp}$, could be identified as different blocks of the composite spacetime vielbein, by solving the orthogonality condition. Their behaviour under gauge transformation are also determined by the corresponding rules of the composite vielbein. Then by solving F theory $\mathcal{V}$ constraints, we reduce the number of worldvolume and show how to recover M theory from F theory.

hep-th↗

arXiv2020

Robust High Dimensional Expectation Maximization Algorithm via Trimmed Hard Thresholding

In this paper, we study the problem of estimating latent variable models with arbitrarily corrupted samples in high dimensional space ({\em i.e.,} $d\gg n$) where the underlying parameter is assumed to be sparse. Specifically, we propose a method called Trimmed (Gradient) Expectation Maximization which adds a trimming gradients step and a hard thresholding step to the Expectation step (E-step) and the Maximization step (M-step), respectively. We show that under some mild assumptions and with an appropriate initialization, the algorithm is corruption-proofing and converges to the (near) optimal statistical rate geometrically when the fraction of the corrupted samples $ε$ is bounded by $ \tilde{O}(\frac{1}{\sqrt{n}})$. Moreover, we apply our general framework to three canonical models: mixture of Gaussians, mixture of regressions and linear regression with missing covariates. Our theory is supported by thorough numerical results.

stat.ML↗

arXiv2020

Learning Robust Algorithms for Online Allocation Problems Using Adversarial Training

We address the challenge of finding algorithms for online allocation (i.e. bipartite matching) using a machine learning approach. In this paper, we focus on the AdWords problem, which is a classical online budgeted matching problem of both theoretical and practical significance. In contrast to existing work, our goal is to accomplish algorithm design {\em tabula rasa}, i.e., without any human-provided insights or expert-tuned training data beyond specifying the objective and constraints of the optimization problem. We construct a framework based on insights and ideas from game theory, adversarial training and GANs Key to our approach is to generate adversarial examples that expose the weakness of any given algorithm. A unique challenge in our context is to generate complete examples from scratch rather than perturbing given examples and we demonstrate this can be accomplished for the Adwords problem. We use this framework to co-train an algorithm network and an adversarial network against each other until they converge to an equilibrium. This approach finds algorithms and adversarial examples that are consistent with known optimal results. Secondly, we address the question of robustness of the algorithm, namely can we design algorithms that are both strong under practical distributions, as well as exhibit robust performance against adversarial instances. To accomplish this, we train algorithm networks using a mixture of adversarial and practical distributions like power-laws; the resulting networks exhibit a smooth trade-off between the two input regimes.

cs.LG↗

arXiv2020

Raising Expectations: Automating Expected Cost Analysis with Types

This article presents a type-based analysis for deriving upper bounds on the expected execution cost of probabilistic programs. The analysis is naturally compositional, parametric in the cost model, and supports higher order functions and inductive data types. The derived bounds are multivariate polynomials that are functions of data structures. Bound inference is enabled by local type rules that reduce type inference to linear constraint solving. The type system is based on the potential method of amortized analysis and extends automatic amortized resource analysis (AARA) for deterministic programs. A main innovation is that bounds can contain symbolic probabilities, which may appear in data structures and function arguments. Another contribution is a novel soundness proof that establishes the correctness of the derived bounds with respect to a distribution-based operational cost semantics that also includes nontrivial diverging behavior. For cost models like time, derived bounds imply termination with probability one. To highlight the novel ideas, the presentation focuses on linear potential and a core language. However, the analysis is implemented as an extension of Resource Aware ML and supports polynomial bounds and user defined data structures. The effectiveness of the technique is evaluated by analyzing the sample complexity of discrete distributions and with a novel average-case estimation for deterministic programs that combines expected cost analysis with statistical methods.

cs.PL↗

arXiv2020

Cox-nnet v2.0: improved neural-network based survival prediction extended to large-scale EMR dataset

Cox-nnet is a neural-network based prognosis prediction method, originally applied to genomics data. Here we propose the version 2 of Cox-nnet, with significant improvement on efficiency and interpretability, making it suitable to predict prognosis based on large-scale electronic medical records (EMR) datasets. We also add permutation-based feature importance scores and the direction of feature coefficients. Applying on an EMR dataset of OPTN kidney transplantation, Cox-nnet v2.0 reduces the training time of Cox-nnet up to 32 folds (n=10,000) and achieves better prediction accuracy than Cox-PH (p<0.05). Availability and implementation: Cox-nnet v2.0 is freely available to the public at https://github.com/lanagarmire/Cox-nnet-v2.0

q-bio.QM↗

arXiv2020

Enabling optical steganography, data storage, and encryption with plasmonic colors

Plasmonic color generation utilizing ultra-thin metasurfaces as well as metallic nanoparticles hold a great promise for a wide range of applications, including color displays, data storage, and information encryption due to its high spatial resolution and mechanical/chemical stability. Most of the recently demonstrated systems generate static colors; however, more advanced applications such as data storage require fast and flexible means to tune the plasmonic colors, while keeping them vibrant and stable. Here, a surface-relief aluminum metasurface that reflects polarization-tunable plasmonic colors is designed and experimentally demonstrated. Excitation of localized surface plasmons encodes discrete combinations of the incident and reflected polarized light into diverse colors. A single storage unit - a nanopixel - stores a multiple-bit piece of information in the orientation of its constituent nanoantennae. This information is then reliably retrieved by inspecting the reflected color sequence with two linear polarizers. It is the broad color variability and high spatial resolution of the proposed encoding approach that supports a strong promise for rapid parallel read-out and encryption of high-density optical data. Our method also enables the robust generation of dynamic kaleidoscopic images with no detrimental "cross-talk" effect. The approach opens up a new route for advanced dynamic steganography, high-density parallel-access optical data storage, and optical information encryption.

physics.optics↗

arXiv2020

ECG Beats Fast Classification Base on Sparse Dictionaries

Feature extraction plays an important role in Electrocardiogram (ECG) Beats classification system. Compared to other popular methods, VQ method performs well in feature extraction from ECG with advantages of dimensionality reduction. In VQ method, a set of dictionaries corresponding to segments of ECG beats is trained, and VQ codes are used to represent each heartbeat. However, in practice, VQ codes optimized by k-means or k-means++ exist large quantization errors, which results in VQ codes for two heartbeats of the same type being very different. So the essential differences between different types of heartbeats cannot be representative well. On the other hand, VQ uses too much data during codebook construction, which limits the speed of dictionary learning. In this paper, we propose a new method to improve the speed and accuracy of VQ method. To reduce the computation of codebook construction, a set of sparse dictionaries corresponding to wave segments of ECG beats is constructed. After initialized, sparse dictionaries are updated efficiently by Feature-sign and Lagrange dual algorithm. Based on those dictionaries, a set of codes can be computed to represent original ECG beats.Experimental results show that features extracted from ECG by our method are more efficient and separable. The accuracy of our method is higher than other methods with less time consumption of feature extraction

eess.SP↗