arXiv subjects

Linwei Wang

Publications and source records attributed to Linwei Wang.

At least 37 records · Page 2Linked to original sources

arXiv2022

Neural State-Space Modeling with Latent Causal-Effect Disentanglement

Despite substantial progress in deep learning approaches to time-series reconstruction, no existing methods are designed to uncover local activities with minute signal strength due to their negligible contribution to the optimization loss. Such local activities however can signify important abnormal events in physiological systems, such as an extra foci triggering an abnormal propagation of electrical waves in the heart. We discuss a novel technique for reconstructing such local activity that, while small in signal strength, is the cause of subsequent global activities that have larger signal strength. Our central innovation is to approach this by explicitly modeling and disentangling how the latent state of a system is influenced by potential hidden internal interventions. In a novel neural formulation of state-space models (SSMs), we first introduce causal-effect modeling of the latent dynamics via a system of interacting neural ODEs that separately describes 1) the continuous-time dynamics of the internal intervention, and 2) its effect on the trajectory of the system's native state. Because the intervention can not be directly observed but have to be disentangled from the observed subsequent effect, we integrate knowledge of the native intervention-free dynamics of a system, and infer the hidden intervention by assuming it to be responsible for differences observed between the actual and hypothetical intervention-free dynamics. We demonstrated a proof-of-concept of the presented framework on reconstructing ectopic foci disrupting the course of normal cardiac electrical propagation from remote observations.

cs.LG↗

arXiv2021

Fast Posterior Estimation of Cardiac Electrophysiological Model Parameters via Bayesian Active Learning

Probabilistic estimation of cardiac electrophysiological model parameters serves an important step towards model personalization and uncertain quantification. The expensive computation associated with these model simulations, however, makes direct Markov Chain Monte Carlo (MCMC) sampling of the posterior probability density function (pdf) of model parameters computationally intensive. Approximated posterior pdfs resulting from replacing the simulation model with a computationally efficient surrogate, on the other hand, have seen limited accuracy. In this paper, we present a Bayesian active learning method to directly approximate the posterior pdf function of cardiac model parameters, in which we intelligently select training points to query the simulation model in order to learn the posterior pdf using a small number of samples. We integrate a generative model into Bayesian active learning to allow approximating posterior pdf of high-dimensional model parameters at the resolution of the cardiac mesh. We further introduce new acquisition functions to focus the selection of training points on better approximating the shape rather than the modes of the posterior pdf of interest. We evaluated the presented method in estimating tissue excitability in a 3D cardiac electrophysiological model in a range of synthetic and real-data experiments. We demonstrated its improved accuracy in approximating the posterior pdf compared to Bayesian active learning using regular acquisition functions, and substantially reduced computational cost in comparison to existing standard or accelerated MCMC sampling.

cs.LG↗

arXiv2021

Analysis of Discriminator in RKHS Function Space for Kullback-Leibler Divergence Estimation

Several scalable sample-based methods to compute the Kullback Leibler (KL) divergence between two distributions have been proposed and applied in large-scale machine learning models. While they have been found to be unstable, the theoretical root cause of the problem is not clear. In this paper, we study a generative adversarial network based approach that uses a neural network discriminator to estimate KL divergence. We argue that, in such case, high fluctuations in the estimates are a consequence of not controlling the complexity of the discriminator function space. We provide a theoretical underpinning and remedy for this problem by first constructing a discriminator in the Reproducing Kernel Hilbert Space (RKHS). This enables us to leverage sample complexity and mean embedding to theoretically relate the error probability bound of the KL estimates to the complexity of the discriminator in RKHS. Based on this theory, we then present a scalable way to control the complexity of the discriminator for a reliable estimation of KL divergence. We support both our proposed theory and method to control the complexity of the RKHS discriminator through controlled experiments.

cs.LG↗

arXiv2021

Vela Pulsar: Single Pulses Analysis with Machine Learning Techniques

We study individual pulses of Vela (PSR\ B0833-45\,/\,J0835-4510) from daily observations of over three hours (around 120,000 pulses per observation), performed simultaneously with the two radio telescopes at the Argentine Institute of Radioastronomy. We select 4 days of observations in January-March 2021 and study their statistical properties with machine learning techniques. We first use density based DBSCAN clustering techniques, associating pulses mainly by amplitudes, and find a correlation between higher amplitudes and earlier arrival times. We also find a weaker (polarization dependent) correlation with the mean width of the pulses. We identify clusters of the so-called mini-giant pulses, with $\sim10\times$ the average pulse amplitude. We then perform an independent study, with Self-Organizing Maps (SOM) clustering techniques. We use Variational AutoEncoder (VAE) reconstruction of the pulses to separate them clearly from the noise and select one of the days of observation to train VAE and apply it to thre rest of the observations. We use SOM to determine 4 clusters of pulses per day per radio telescope and conclude that our main results are robust and self-consistent. These results support models for emitting regions at different heights (separated each by roughly a hundred km) in the pulsar magnetosphere. We also model the pulses amplitude distribution with interstellar scintillation patterns at the inter-pulses time-scale finding a characterizing exponent $n_{\mathrm{ISS}}\sim7-10$. In the appendices we discuss independent checks of hardware systematics with the simultaneous use of the two radio telescopes in different one-polarization / two-polarizations configurations. We also provide a detailed analysis of the processes of radio-interferences cleaning and individual pulse folding.

astro-ph.HE↗

arXiv2021

Semi-Supervised Learning for Eye Image Segmentation

Recent advances in appearance-based models have shown improved eye tracking performance in difficult scenarios like occlusion due to eyelashes, eyelids or camera placement, and environmental reflections on the cornea and glasses. The key reason for the improvement is the accurate and robust identification of eye parts (pupil, iris, and sclera regions). The improved accuracy often comes at the cost of labeling an enormous dataset, which is complex and time-consuming. This work presents two semi-supervised learning frameworks to identify eye-parts by taking advantage of unlabeled images where labeled datasets are scarce. With these frameworks, leveraging the domain-specific augmentation and novel spatially varying transformations for image segmentation, we show improved performance on various test cases. For instance, for a model trained on just 48 labeled images, these frameworks achieved an improvement of 0.38% and 0.65% in segmentation performance over the baseline model, which is trained only with the labeled dataset.

cs.CV↗

arXiv2020

Enhancing Mixup-based Semi-Supervised Learning with Explicit Lipschitz Regularization

The success of deep learning relies on the availability of large-scale annotated data sets, the acquisition of which can be costly, requiring expert domain knowledge. Semi-supervised learning (SSL) mitigates this challenge by exploiting the behavior of the neural function on large unlabeled data. The smoothness of the neural function is a commonly used assumption exploited in SSL. A successful example is the adoption of mixup strategy in SSL that enforces the global smoothness of the neural function by encouraging it to behave linearly when interpolating between training examples. Despite its empirical success, however, the theoretical underpinning of how mixup regularizes the neural function has not been fully understood. In this paper, we offer a theoretically substantiated proposition that mixup improves the smoothness of the neural function by bounding the Lipschitz constant of the gradient function of the neural networks. We then propose that this can be strengthened by simultaneously constraining the Lipschitz constant of the neural function itself through adversarial Lipschitz regularization, encouraging the neural function to behave linearly while also constraining the slope of this linear function. On three benchmark data sets and one real-world biomedical data set, we demonstrate that this combined regularization results in improved generalization performance of SSL when learning from a small amount of labeled data. We further demonstrate the robustness of the presented method against single-step adversarial attacks. Our code is available at https://github.com/Prasanna1991/Mixup-LR.

cs.LG↗

arXiv2020

Learning Geometry-Dependent and Physics-Based Inverse Image Reconstruction

Deep neural networks have shown great potential in image reconstruction problems in Euclidean space. However, many reconstruction problems involve imaging physics that are dependent on the underlying non-Euclidean geometry. In this paper, we present a new approach to learn inverse imaging that exploit the underlying geometry and physics. We first introduce a non-Euclidean encoding-decoding network that allows us to describe the unknown and measurement variables over their respective geometrical domains. We then learn the geometry-dependent physics in between the two domains by explicitly modeling it via a bipartite graph over the graphical embedding of the two geometry. We applied the presented network to reconstructing electrical activity on the heart surface from body-surface potential. In a series of generalization tasks with increasing difficulty, we demonstrated the improved ability of the presented network to generalize across geometrical changes underlying the data in comparison to its Euclidean alternatives.

eess.IV↗

arXiv2020

Quantifying the Uncertainty in Model Parameters Using Gaussian Process-Based Markov Chain Monte Carlo: An Application to Cardiac Electrophysiological Models

Estimation of patient-specific model parameters is important for personalized modeling, although sparse and noisy clinical data can introduce significant uncertainty in the estimated parameter values. This importance source of uncertainty, if left unquantified, will lead to unknown variability in model outputs that hinder their reliable adoptions. Probabilistic estimation model parameters, however, remains an unresolved challenge because standard Markov Chain Monte Carlo sampling requires repeated model simulations that are computationally infeasible. A common solution is to replace the simulation model with a computationally-efficient surrogate for a faster sampling. However, by sampling from an approximation of the exact posterior probability density function (pdf) of the parameters, the efficiency is gained at the expense of sampling accuracy. In this paper, we address this issue by integrating surrogate modeling into Metropolis Hasting (MH) sampling of the exact posterior pdfs to improve its acceptance rate. It is done by first quickly constructing a Gaussian process (GP) surrogate of the exact posterior pdfs using deterministic optimization. This efficient surrogate is then used to modify commonly-used proposal distributions in MH sampling such that only proposals accepted by the surrogate will be tested by the exact posterior pdf for acceptance/rejection, reducing unnecessary model simulations at unlikely candidates. Synthetic and real-data experiments using the presented method show a significant gain in computational efficiency without compromising the accuracy. In addition, insights into the non-identifiability and heterogeneity of tissue properties can be gained from the obtained posterior distributions.

stat.ML↗

arXiv2020

Semi-supervised Medical Image Classification with Global Latent Mixing

Computer-aided diagnosis via deep learning relies on large-scale annotated data sets, which can be costly when involving expert knowledge. Semi-supervised learning (SSL) mitigates this challenge by leveraging unlabeled data. One effective SSL approach is to regularize the local smoothness of neural functions via perturbations around single data points. In this work, we argue that regularizing the global smoothness of neural functions by filling the void in between data points can further improve SSL. We present a novel SSL approach that trains the neural network on linear mixing of labeled and unlabeled data, at both the input and latent space in order to regularize different portions of the network. We evaluated the presented model on two distinct medical image data sets for semi-supervised classification of thoracic disease and skin lesion, demonstrating its improved performance over SSL with local perturbations and SSL with global mixing but at the input space only. Our code is available at https://github.com/Prasanna1991/LatentMixing.

cs.LG↗

arXiv2020

Bayesian Optimization on Large Graphs via a Graph Convolutional Generative Model: Application in Cardiac Model Personalization

Personalization of cardiac models involves the optimization of organ tissue properties that vary spatially over the non-Euclidean geometry model of the heart. To represent the high-dimensional (HD) unknown of tissue properties, most existing works rely on a low-dimensional (LD) partitioning of the geometrical model. While this exploits the geometry of the heart, it is of limited expressiveness to allow partitioning that is small enough for effective optimization. Recently, a variational auto-encoder (VAE) was utilized as a more expressive generative model to embed the HD optimization into the LD latent space. Its Euclidean nature, however, neglects the rich geometrical information in the heart. In this paper, we present a novel graph convolutional VAE to allow generative modeling of non-Euclidean data, and utilize it to embed Bayesian optimization of large graphs into a small latent space. This approach bridges the gap of previous works by introducing an expressive generative model that is able to incorporate the knowledge of spatial proximity and hierarchical compositionality of the underlying geometry. It further allows transferring of the learned features across different geometries, which was not possible with a regular VAE. We demonstrate these benefits of the presented method in synthetic and real data experiments of estimating tissue excitability in a cardiac electrophysiological model.

eess.IV↗

arXiv2020

High-dimensional Bayesian Optimization of Personalized Cardiac Model Parameters via an Embedded Generative Model

The estimation of patient-specific tissue properties in the form of model parameters is important for personalized physiological models. However, these tissue properties are spatially varying across the underlying anatomical model, presenting a significance challenge of high-dimensional (HD) optimization at the presence of limited measurement data. A common solution to reduce the dimension of the parameter space is to explicitly partition the anatomical mesh, either into a fixed small number of segments or a multi-scale hierarchy. This anatomy-based reduction of parameter space presents a fundamental bottleneck to parameter estimation, resulting in solutions that are either too low in resolution to reflect tissue heterogeneity, or too high in dimension to be reliably estimated within feasible computation. In this paper, we present a novel concept that embeds a generative variational auto-encoder (VAE) into the objective function of Bayesian optimization, providing an implicit low-dimensional (LD) search space that represents the generative code of the HD spatially-varying tissue properties. In addition, the VAE-encoded knowledge about the generative code is further used to guide the exploration of the search space. The presented method is applied to estimating tissue excitability in a cardiac electrophysiological model. Synthetic and real-data experiments demonstrate its ability to improve the accuracy of parameter estimation with more than 10x gain in efficiency.

stat.ML↗

arXiv2020

Progressive Learning and Disentanglement of Hierarchical Representations

Learning rich representation from data is an important task for deep generative models such as variational auto-encoder (VAE). However, by extracting high-level abstractions in the bottom-up inference process, the goal of preserving all factors of variations for top-down generation is compromised. Motivated by the concept of "starting small", we present a strategy to progressively learn independent hierarchical representations from high- to low-levels of abstractions. The model starts with learning the most abstract representation, and then progressively grow the network architecture to introduce new representations at different levels of abstraction. We quantitatively demonstrate the ability of the presented model to improve disentanglement in comparison to existing works on two benchmark data sets using three disentanglement metrics, including a new metric we proposed to complement the previously-presented metric of mutual information gap. We further present both qualitative and quantitative evidence on how the progression of learning improves disentangling of hierarchical representations. By drawing on the respective advantage of hierarchical representation learning and progressive learning, this is to our knowledge the first attempt to improve disentanglement by progressively growing the capacity of VAE to learn hierarchical representations.

cs.LG↗

arXiv2020

Contribution of high risk groups' unmet needs may be underestimated in epidemic models without risk turnover: a mechanistic modelling analysis

BACKGROUND. Epidemic models of STIs are often used to characterize the contribution of risk groups to overall transmission by projecting the transmission population attributable fraction (tPAF) of unmet prevention and treatment needs within risk groups. However, evidence suggests that STI risk is dynamic over an individual's sexual life course, which manifests as turnover between risk groups. We sought to examine the mechanisms by which turnover influences modelled projections of the tPAF of high risk groups. METHODS. We developed a unifying, data-guided framework to simulate risk group turnover in deterministic, compartmental transmission models. We applied the framework to an illustrative model of an STI and examined the mechanisms by which risk group turnover influenced equilibrium prevalence across risk groups. We then fit a model with and without turnover to the same risk-stratified STI prevalence targets and compared the inferred level of risk heterogeneity and tPAF of the highest risk group projected by the two models. RESULTS. The influence of turnover on group-specific prevalence was mediated by three main phenomena: movement of infectious individuals between risk groups; changes to herd immunity; and changes in discordant partnerships. Faster turnover led to a smaller ratio of STI prevalence between the highest and lowest risk groups. Compared to the fitted model without turnover, the fitted model with turnover inferred greater risk heterogeneity and consistently projected a larger tPAF of the highest risk group over time. IMPLICATIONS. If turnover is not captured in epidemic models, the projected contribution of high risk groups, and thus, the potential impact of prioritizing interventions to address their needs, could be underestimated. To aid the next generation of tPAF models, data collection efforts to parameterize risk group turnover should be prioritized.

q-bio.PE↗

arXiv2019

Venue-based HIV testing at sex work hotspots to reach adolescent girls and young women living with HIV: a cross-sectional study in Mombasa, Kenya

Background: We estimated the potential number of newly diagnosed HIV infections among adolescent girls and young women (AGYW) using a venue-based approach to HIV testing at sex work hotspots. Methods: We used hotspot enumeration and cross-sectional bio-behavioural survey data from the 2015 Transitions Study of AGYW aged 14-24 years who frequented hotspots in Mombasa, Kenya. We compared the HIV cascade among AGYW who sell sex (YSW, N=408) versus those who do not (NSW, N=891); and triangulated the potential (100% test acceptance and accuracy) and feasible (accounting for test acceptance and sensitivity) number of AGYW that could be newly diagnosed via hotspot-based HIV rapid testing in Mombasa. We identified the profile of AGYW recently tested for HIV (in the past year) using multivariable logistic regression. Results: N=37/365 (10.1%) YSW and N=30/828 (3.6%) NSW were living with HIV, of whom 27.0% (N=10/37) and 30.0% (N=9/30) were diagnosed and aware (p=0.79). Rapid test acceptance was 89.3% and sensitivity was 80.4%. Hotspot enumeration estimated 15,635 (range: 12,172-19,097) AGYW in hotspots in Mombasa. The potential and feasible number of new diagnosis were 627 (310-1,081), and 450 (223-776), respectively. Thus, hotspot-based testing could feasibly reduce the undiagnosed fraction from 71.6% to 20.2%. The profile of AGYW who recently tested was similar among YSW and NSW. YSW were two-fold more likely to report a recent HIV test after adjusting for other determinants [odds ratio (95% CI): 2.1 (1.6-3.1)]. Conclusion: Reaching AGYW via hotspot-based HIV testing could fill gaps left by traditional, clinic-based HIV prevention and testing services.

q-bio.OT↗

arXiv2019

Wavelets to the Rescue: Improving Sample Quality of Latent Variable Deep Generative Models

Variational Autoencoders (VAE) are probabilistic deep generative models underpinned by elegant theory, stable training processes, and meaningful manifold representations. However, they produce blurry images due to a lack of explicit emphasis over high-frequency textural details of the images, and the difficulty to directly model the complex joint probability distribution over the high-dimensional image space. In this work, we approach these two challenges with a novel wavelet space VAE that uses the decoder to model the images in the wavelet coefficient space. This enables the VAE to emphasize over high-frequency components within an image obtained via wavelet decomposition. Additionally, by decomposing the complex function of generating high-dimensional images into inverse wavelet transformation and generation of wavelet coefficients, the latter becomes simpler to model by the VAE. We empirically validate that deep generative models operating in the wavelet space can generate images of higher quality than the image (RGB) space counterparts. Quantitatively, on benchmark natural image datasets, we achieve consistently better FID scores than VAE based architectures and competitive FID scores with a variety of GAN models for the same architectural and experimental setup. Furthermore, the proposed wavelet-based generative model retains desirable attributes like disentangled and informative latent representation without losing the quality in the generated samples.

cs.CV↗

arXiv2019

Improving Disentangled Representation Learning with the Beta Bernoulli Process

To improve the ability of VAE to disentangle in the latent space, existing works mostly focus on enforcing independence among the learned latent factors. However, the ability of these models to disentangle often decreases as the complexity of the generative factors increases. In this paper, we investigate the little-explored effect of the modeling capacity of a posterior density on the disentangling ability of the VAE. We note that the independence within and the complexity of the latent density are two different properties we constrain when regularizing the posterior density: while the former promotes the disentangling ability of VAE, the latter -- if overly limited -- creates an unnecessary competition with the data reconstruction objective in VAE. Therefore, if we preserve the independence but allow richer modeling capacity in the posterior density, we will lift this competition and thereby allow improved independence and data reconstruction at the same time. We investigate this theoretical intuition with a VAE that utilizes a non-parametric latent factor model, the Indian Buffet Process (IBP), as a latent density that is able to grow with the complexity of the data. Across three widely-used benchmark data sets and two clinical data sets little explored for disentangled learning, we qualitatively and quantitatively demonstrated the improved disentangling performance of IBP-VAE over the state of the art. In the latter two clinical data sets riddled with complex factors of variations, we further demonstrated that unsupervised disentangling of nuisance factors via IBP-VAE -- when combined with a supervised objective -- can not only improve task accuracy in comparison to relevant supervised deep architectures but also facilitate knowledge discovery related to task decision-making. A shorter version of this work will appear in the ICDM 2019 conference proceedings.

cs.LG↗

arXiv2019

Semi-Supervised Learning by Disentangling and Self-Ensembling Over Stochastic Latent Space

The success of deep learning in medical imaging is mostly achieved at the cost of a large labeled data set. Semi-supervised learning (SSL) provides a promising solution by leveraging the structure of unlabeled data to improve learning from a small set of labeled data. Self-ensembling is a simple approach used in SSL to encourage consensus among ensemble predictions of unknown labels, improving generalization of the model by making it more insensitive to the latent space. Currently, such an ensemble is obtained by randomization such as dropout regularization and random data augmentation. In this work, we hypothesize -- from the generalization perspective -- that self-ensembling can be improved by exploiting the stochasticity of a disentangled latent space. To this end, we present a stacked SSL model that utilizes unsupervised disentangled representation learning as the stochastic embedding for self-ensembling. We evaluate the presented model for multi-label classification using chest X-ray images, demonstrating its improved performance over related SSL models as well as the interpretability of its disentangled representations.

cs.LG↗

arXiv2019

A Variational Approach to Sparse Model Error Estimation in Cardiac Electrophysiological Imaging

Noninvasive reconstruction of cardiac electrical activity from surface electrocardiograms (ECG) involves solving an ill-posed inverse problem. Cardiac electrophysiological (EP) models have been used as important a priori knowledge to constrain this inverse problem. However, the reconstruction suffer from inaccuracy and uncertainty of the prior model itself which could be mitigated by estimating a priori model error. Unfortunately, due to the need to handle an additional large number of unknowns in a problem that already suffers from ill-posedness, model error estimation remains an unresolved challenge. In this paper, we address this issue by modeling and estimating the a priori model error in a low dimensional space using a novel sparse prior based on the variational approximation of L0 norm. This prior is used in a posterior regularized Bayesian formulation to quantify the error in a priori EP model during the reconstruction of transmural action potential from ECG data. Through synthetic and real-data experiments, we demonstrate the ability of the presented method to timely capture a priori model error and thus to improve reconstruction accuracy compared to approaches without model error correction.

eess.IV↗