arXiv subjects

Jie Ding

Publications and source records attributed to Jie Ding.

At least 127 records · Page 7Linked to original sources

arXiv2020

Structure-Preserving and Efficient Numerical Methods for Ion Transport

Ion transport, often described by the Poisson--Nernst--Planck (PNP) equations, is ubiquitous in electrochemical devices and many biological processes of significance. In this work, we develop conservative, positivity-preserving, energy dissipating, and implicit finite difference schemes for solving the multi-dimensional PNP equations with multiple ionic species. A central-differencing discretization based on harmonic-mean approximations is employed for the Nernst--Planck (NP) equations. The backward Euler discretization in time is employed to derive a fully implicit nonlinear system, which is efficiently solved by a newly proposed Newton's method. The improved computational efficiency of the Newton's method originates from the usage of the electrostatic potential as the iteration variable, rather than the unknowns of the nonlinear system that involves both the potential and concentration of multiple ionic species. Numerical analysis proves that the numerical schemes respect three desired analytical properties (conservation, positivity preserving, and energy dissipation) fully discretely. Based on advantages brought by the harmonic-mean approximations, we are able to establish estimate on the upper bound of condition numbers of coefficient matrices in linear systems that are solved iteratively. The solvability and stability of the linearized problem in the Newton's method are rigorously established as well. Numerical tests are performed to confirm the anticipated numerical accuracy, computational efficiency, and structure-preserving properties of the developed schemes. Adaptive time stepping is implemented for further efficiency improvement. Finally, the proposed numerical approaches are applied to characterize ion transport subject to a sinusoidal applied potential.

math.NA↗

arXiv2020

Speech Emotion Recognition with Dual-Sequence LSTM Architecture

Speech Emotion Recognition (SER) has emerged as a critical component of the next generation human-machine interfacing technologies. In this work, we propose a new dual-level model that predicts emotions based on both MFCC features and mel-spectrograms produced from raw audio signals. Each utterance is preprocessed into MFCC features and two mel-spectrograms at different time-frequency resolutions. A standard LSTM processes the MFCC features, while a novel LSTM architecture, denoted as Dual-Sequence LSTM (DS-LSTM), processes the two mel-spectrograms simultaneously. The outputs are later averaged to produce a final classification of the utterance. Our proposed model achieves, on average, a weighted accuracy of 72.7% and an unweighted accuracy of 73.3%---a 6% improvement over current state-of-the-art unimodal models---and is comparable with multimodal models that leverage textual information as well as audio signals.

eess.AS↗

arXiv2019

DRASIC: Distributed Recurrent Autoencoder for Scalable Image Compression

We propose a new architecture for distributed image compression from a group of distributed data sources. The work is motivated by practical needs of data-driven codec design, low power consumption, robustness, and data privacy. The proposed architecture, which we refer to as Distributed Recurrent Autoencoder for Scalable Image Compression (DRASIC), is able to train distributed encoders and one joint decoder on correlated data sources. Its compression capability is much better than the method of training codecs separately. Meanwhile, the performance of our distributed system with 10 distributed sources is only within 2 dB peak signal-to-noise ratio (PSNR) of the performance of a single codec trained with all data sources. We experiment distributed sources with different correlations and show how our data-driven methodology well matches the Slepian-Wolf Theorem in Distributed Source Coding (DSC). To the best of our knowledge, this is the first data-driven DSC framework for general distributed code design with deep learning.

cs.CV↗

arXiv2019

On the iterations and the argument distribution of meromorphic functions

This paper consists of tow parts. One is to study the existence of a point $a$ in the intersection of Julia set and escaping set such that $\arg z=θ$ is a singular direction if $θ$ is a limit point of $\{\arg f^n(a)\}$ under some growth condition of a meromorphic function. The other is to study the connection between the Fatou set and singular direction. We prove that the absent of singular direction deduces the non-existence of annuli in the Fatou set.

math.DS↗

arXiv2019

Restricted Recurrent Neural Networks

Recurrent Neural Network (RNN) and its variations such as Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU), have become standard building blocks for learning online data of sequential nature in many research areas, including natural language processing and speech data analysis. In this paper, we present a new methodology to significantly reduce the number of parameters in RNNs while maintaining performance that is comparable or even better than classical RNNs. The new proposal, referred to as Restricted Recurrent Neural Network (RRNN), restricts the weight matrices corresponding to the input data and hidden states at each time step to share a large proportion of parameters. The new architecture can be regarded as a compression of its classical counterpart, but it does not require pre-training or sophisticated parameter fine-tuning, both of which are major issues in most existing compression techniques. Experiments on natural language modeling show that compared with its classical counterpart, the restricted recurrent architecture generally produces comparable results at about 50\% compression rate. In particular, the Restricted LSTM can outperform classical RNN with even less number of parameters.

cs.CL↗

arXiv2019

Variable Grouping Based Bayesian Additive Regression Tree

Using ensemble methods for regression has been a large success in obtaining high-accuracy prediction. Examples are Bagging, Random forest, Boosting, BART (Bayesian additive regression tree), and their variants. In this paper, we propose a new perspective named variable grouping to enhance the predictive performance. The main idea is to seek for potential grouping of variables in such way that there is no nonlinear interaction term between variables of different groups. Given a sum-of-learner model, each learner will only be responsible for one group of variables, which would be more efficient in modeling nonlinear interactions. We propose a two-stage method named variable grouping based Bayesian additive regression tree (GBART) with a well-developed python package gbart available. The first stage is to search for potential interactions and an appropriate grouping of variables. The second stage is to build a final model based on the discovered groups. Experiments on synthetic and real data show that the proposed method can perform significantly better than classical approaches.

cs.LG↗

arXiv2019

On Harmonic Approximations of Inharmonic Signals

In this work, we present the misspecified Gaussian Cramér-Rao lower bound for the parameters of a harmonic signal, or pitch, when signal measurements are collected from an almost, but not quite, harmonic model. For the asymptotic case of large sample sizes, we present a closed-form expression for the bound corresponding to the pseduo-true fundamental frequency. Using simulation studies, it is shown that the bound is sharp and is attained by maximum likelihood estimators derived under the misspecified harmonic assumption. It is shown that misspecified harmonic models achieve a lower mean squared error than correctly specified unstructured models for moderately inharmonic signals. Examining voices from a speech database, we conclude that human speech belongs to this class of signals, verifying that the use of a harmonic model for voiced speech is preferable.

eess.SP↗

arXiv2019

Deep Clustering of Compressed Variational Embeddings

Motivated by the ever-increasing demands for limited communication bandwidth and low-power consumption, we propose a new methodology, named joint Variational Autoencoders with Bernoulli mixture models (VAB), for performing clustering in the compressed data domain. The idea is to reduce the data dimension by Variational Autoencoders (VAEs) and group data representations by Bernoulli mixture models (BMMs). Once jointly trained for compression and clustering, the model can be decomposed into two parts: a data vendor that encodes the raw data into compressed data, and a data consumer that classifies the received (compressed) data. In this way, the data vendor benefits from data security and communication bandwidth, while the data consumer benefits from low computational complexity. To enable training using the gradient descent algorithm, we propose to use the Gumbel-Softmax distribution to resolve the infeasibility of the back-propagation algorithm when assessing categorical samples.

eess.IV↗

arXiv2019

Perception-Distortion Trade-off with Restricted Boltzmann Machines

In this work, we introduce a new procedure for applying Restricted Boltzmann Machines (RBMs) to missing data inference tasks, based on linearization of the effective energy function governing the distribution of observations. We compare the performance of our proposed procedure with those obtained using existing reconstruction procedures trained on incomplete data. We place these performance comparisons within the context of the perception-distortion trade-off observed in other data reconstruction tasks, which has, until now, remained unexplored in tasks relying on incomplete training data.

cs.LG↗

arXiv2019

Supervised Encoding for Discrete Representation Learning

Classical supervised classification tasks search for a nonlinear mapping that maps each encoded feature directly to a probability mass over the labels. Such a learning framework typically lacks the intuition that encoded features from the same class tend to be similar and thus has little interpretability for the learned features. In this paper, we propose a novel supervised learning model named Supervised-Encoding Quantizer (SEQ). The SEQ applies a quantizer to cluster and classify the encoded features. We found that the quantizer provides an interpretable graph where each cluster in the graph represents a class of data samples that have a particular style. We also trained a decoder that can decode convex combinations of the encoded features from similar and different clusters and provide guidance on style transfer between sub-classes.

cs.CV↗

arXiv2019

A continuous transition from $\mathcal{E}$-sets to $R$-sets and beyond

The well-known $\mathcal{E}$-sets introduced by Hayman in 1960 are collections of Euclidean discs in the complex plane with the following property: The set of angles $θ$ for which the ray $\arg(z)=θ$ meets infinitely many discs in a given $\mathcal{E}$-set has linear measure zero. An important special case of an $\mathcal{E}$-set is known as the $R$-set. These sets appear in numerous papers in the theories of complex differential and functional equations. This paper offers a continuous transition from $\mathcal{E}$-sets to $R$-sets, and then to much thinner sets. In addition to rays, plane curves that originate from the zero distribution theory of exponential polynomials will be considered. It turns out that almost every such curve meets at most finitely many discs in the collection in question. Analogous discussions are provided in the case of the unit disc $\mathbb{D}$, where the curves tend to the boundary $\partial\mathbb{D}$ tangentially or non-tangentially. Finally, these findings will be used for improving well-known estimates for logarithmic derivatives, logarithmic differences and logarithmic $q$-differences of meromorphic functions, as well as for improving standard results on exceptional sets.

math.CV↗

arXiv2019

Optimal Preamble Length for Spectral Efficiency in Grant-Free RA with Massive MIMO

Grant-free random access (RA) with massive MIMO is a promising RA technique for massive access with low signaling overhead. In the grant-free RA with massive MIMO, preamble length has a critical impact on the performance of the system. In this paper, the optimal preamble length is investigated to maximize spectral efficiency (SE) of the grant-free RA with massive MIMO, where effects of the preamble length on the preamble collision and preamble overhead as well as channel estimation accuracy are taken into account. Simulation results agree well with our analyses and confirm the existence of optimal preamble length for SE maximization in the grant-free RA with massive MIMO. Moreover, properties of the optimal preamble length with respect to system parameters are revealed. Compared to the granted access, it is shown that longer preamble length is required for SE maximization in the grant-free RA.

eess.SP↗

arXiv2019

Estimation of the Evolutionary Spectra with Application to Stationarity Test

In this work, we propose a new inference procedure for understanding non-stationary processes, under the framework of evolutionary spectra developed by Priestley. Among various frameworks of modeling non-stationary processes, the distinguishing feature of the evolutionary spectra is its focus on the physical meaning of frequency. The classical estimate of the evolutionary spectral density is based on a double-window technique consisting of a short-time Fourier transform and a smoothing. However, smoothing is known to suffer from the so-called bias leakage problem. By incorporating Thomson's multitaper method that was originally designed for stationary processes, we propose an improved estimate of the evolutionary spectral density, and analyze its bias/variance/resolution tradeoff. As an application of the new estimate, we further propose a non-parametric rank-based stationarity test, and provide various experimental studies.

stat.ME↗

arXiv2018

Model Selection Techniques -- An Overview

In the era of big data, analysts usually explore various statistical models or machine learning methods for observed data in order to facilitate scientific discoveries or gain predictive power. Whatever data and fitting procedures are employed, a crucial step is to select the most appropriate model or method from a set of candidates. Model selection is a key ingredient in data analysis for reliable and reproducible statistical inference or prediction, and thus central to scientific studies in fields such as ecology, economics, engineering, finance, political science, biology, and epidemiology. There has been a long history of model selection techniques that arise from researches in statistics, information theory, and signal processing. A considerable number of methods have been proposed, following different philosophies and exhibiting varying performances. The purpose of this article is to bring a comprehensive overview of them, in terms of their motivation, large sample performance, and applicability. We provide integrated and practically relevant discussions on theoretical properties of state-of- the-art model selection approaches. We also share our thoughts on some controversial views on the practice of model selection.

stat.ML↗

arXiv2018

Multiple Preambles for High Success Rate of Grant-Free Random Access with Massive MIMO

Grant-free random access (RA) with massive MIMO is a promising RA technique with low signaling overhead that provides significant benefits in increasing the channel reuse efficiency. Since user equipment (UE) detection and channel estimation in grant-free RA rely solely on the received preambles, preamble designs that enable high success rate of UE detection and channel estimation are very much in need to ensure the performance gain of grant-free RA with massive MIMO. In this paper, a super preamble consisting of multiple consecutive preambles is proposed for high success rate of grant-free RA with massive MIMO. With the proposed approach, the success of UE detection and channel estimation for a RA UE depends on two conditions: 1) it is a solvable UE; 2) its super preamble is detected. Accordingly, we theoretically analyze the solvable rate of RA UEs with multiple preambles and propose a reliable UE detection algorithm to obtain the super preambles of RA UEs by exploiting the quasi-orthogonality characteristic of massive MIMO. Theoretical analysis and simulation results show that turning a preamble into a super preamble consisting of two or three shorter preambles, the success rate of UE detection and channel estimation could be significantly increased using the proposed approach.

cs.IT↗

arXiv2018

Bayesian model comparison with the Hyvärinen score: computation and consistency

The Bayes factor is a widely used criterion in model comparison and its logarithm is a difference of out-of-sample predictive scores under the logarithmic scoring rule. However, when some of the candidate models involve vague priors on their parameters, the log-Bayes factor features an arbitrary additive constant that hinders its interpretation. As an alternative, we consider model comparison using the Hyvärinen score. We propose a method to consistently estimate this score for parametric models, using sequential Monte Carlo methods. We show that this score can be estimated for models with tractable likelihoods as well as nonlinear non-Gaussian state-space models with intractable likelihoods. We prove the asymptotic consistency of this new model selection criterion under strong regularity assumptions in the case of non-nested models, and we provide qualitative insights for the nested case. We also use existing characterizations of proper scoring rules on discrete spaces to extend the Hyvärinen score to discrete observations. Our numerical illustrations include Lévy-driven stochastic volatility models and diffusion models for population dynamics.

stat.ME↗

arXiv2018

Virtual Carrier Sensing Based Random Access in Massive MIMO Systems

The 5th generation mobile communication systems aim to support massive access for future wireless applications. Unfortunately, wireless resource scarcity in random access (RA) is a fundamental bottleneck for enabling massive access. To address this problem, we propose a virtual carrier sensing (VCS) based RA scheme in massive MIMO systems. The essence of the proposed scheme lies in exploiting wireless spatial resources of uplink channels occupied by assigned user equipments (UEs) to increase channel resources for RA. With the proposed scheme, RA UEs are able to exploit the spatial resources that are approximately orthogonal to those of assigned UEs, thus sharing the uplink channel resource with assigned UEs without causing significant interference to them. Specifically, to ensure RA UEs avoid serious interference with assigned UEs, base station (BS) sends tailored virtual carriers to RA UEs on behalf of assigned UEs. RA UEs then conduct VCS to determine whether or not the uplink channel resource is available for RA. Closed-form approximations for probability of channel availability and uplink achievable rate with the proposed scheme are derived. Theoretical analysis and simulation results show that the proposed scheme is able to significantly increase channel resources of RA for massive access.

cs.IT↗

arXiv2018

Success Probability of Grant-Free Random Access with Massive MIMO

Massive MIMO opens up new avenues for enabling highly efficient random access (RA) by offering abundance of spatial degrees of freedom. In this paper, we investigate the grant-free RA with massive MIMO and derive the analytic expressions of success probability of the grant-free RA for conjugate beamforming and zero-forcing beamforming techniques.With the derived analytic expressions, we further shed light on the impact of system parameters on the success probability. Simulation results verify the accuracy of the analyses. It is confirmed that the grant-free RA with massive MIMO is an attractive RA technique with low signaling overhead that could simultaneously accommodate a number of RA users, which is multiple times the number of RA channels, with close-to-one success probability. In addition, when the number of antennas in massive MIMO is sufficiently large, we show that the number of orthogonal preambles would dominate the success probability.

cs.IT↗