arXiv subjects

Xiang Ji

Publications and source records attributed to Xiang Ji.

At least 73 records · Page 4Linked to original sources

arXiv2023

Automatic differentiation is no panacea for phylogenetic gradient computation

Gradients of probabilistic model likelihoods with respect to their parameters are essential for modern computational statistics and machine learning. These calculations are readily available for arbitrary models via automatic differentiation implemented in general-purpose machine-learning libraries such as TensorFlow and PyTorch. Although these libraries are highly optimized, it is not clear if their general-purpose nature will limit their algorithmic complexity or implementation speed for the phylogenetic case compared to phylogenetics-specific code. In this paper, we compare six gradient implementations of the phylogenetic likelihood functions, in isolation and also as part of a variational inference procedure. We find that although automatic differentiation can scale approximately linearly in tree size, it is much slower than the carefully-implemented gradient calculation for tree likelihood and ratio transformation operations. We conclude that a mixed approach combining phylogenetic libraries with machine learning libraries will provide the optimal combination of speed and model flexibility moving forward.

q-bio.PE↗

arXiv2023

Massive Parallelization of Massive Sample-size Survival Analysis

Large-scale observational health databases are increasingly popular for conducting comparative effectiveness and safety studies of medical products. However, increasing number of patients poses computational challenges when fitting survival regression models in such studies. In this paper, we use graphics processing units (GPUs) to parallelize the computational bottlenecks of massive sample-size survival analyses. Specifically, we develop and apply time- and memory-efficient single-pass parallel scan algorithms for Cox proportional hazards models and forward-backward parallel scan algorithms for Fine-Gray models for analysis with and without a competing risk using a cyclic coordinate descent optimization approach. We demonstrate that GPUs accelerate the computation of fitting these complex models in large databases by orders of magnitude as compared to traditional multi-core CPU parallelism. Our implementation enables efficient large-scale observational studies involving millions of patients and thousands of patient characteristics. The above implementation is available in the open-source R package Cyclops (Suchard et al., 2013).

stat.CO↗

arXiv2023

Many-core algorithms for high-dimensional gradients on phylogenetic trees

The rapid growth in genomic pathogen data spurs the need for efficient inference techniques, such as Hamiltonian Monte Carlo (HMC) in a Bayesian framework, to estimate parameters of these phylogenetic models where the dimensions of the parameters increase with the number of sequences $N$. HMC requires repeated calculation of the gradient of the data log-likelihood with respect to (wrt) all branch-length-specific (BLS) parameters that traditionally takes $\mathcal{O}(N^2)$ operations using the standard pruning algorithm. A recent study proposes an approach to calculate this gradient in $\mathcal{O}(N)$, enabling researchers to take advantage of gradient-based samplers such as HMC. The CPU implementation of this approach makes the calculation of the gradient computationally tractable for nucleotide-based models but falls short in performance for larger state-space size models, such as codon models. Here, we describe novel massively parallel algorithms to calculate the gradient of the log-likelihood wrt all BLS parameters that take advantage of graphics processing units (GPUs) and result in many fold higher speedups over previous CPU implementations. We benchmark these GPU algorithms on three computing systems using three evolutionary inference examples: carnivores, dengue and yeast, and observe a greater than 128-fold speedup over the CPU implementation for codon-based models and greater than 8-fold speedup for nucleotide-based models. As a practical demonstration, we also estimate the timing of the first introduction of West Nile virus into the continental Unites States under a codon model with a relaxed molecular clock from 104 full viral genomes, an inference task previously intractable. We provide an implementation of our GPU algorithms in BEAGLE v4.0.0, an open source library for statistical phylogenetics that enables parallel calculations on multi-core CPUs and GPUs.

stat.CO↗

arXiv2023

Blur Interpolation Transformer for Real-World Motion from Blur

This paper studies the challenging problem of recovering motion from blur, also known as joint deblurring and interpolation or blur temporal super-resolution. The challenges are twofold: 1) the current methods still leave considerable room for improvement in terms of visual quality even on the synthetic dataset, and 2) poor generalization to real-world data. To this end, we propose a blur interpolation transformer (BiT) to effectively unravel the underlying temporal correlation encoded in blur. Based on multi-scale residual Swin transformer blocks, we introduce dual-end temporal supervision and temporally symmetric ensembling strategies to generate effective features for time-varying motion rendering. In addition, we design a hybrid camera system to collect the first real-world dataset of one-to-many blur-sharp video pairs. Experimental results show that BiT has a significant gain over the state-of-the-art methods on the public dataset Adobe240. Besides, the proposed real-world dataset effectively helps the model generalize well to real blurry scenarios. Code and data are available at https://github.com/zzh-tech/BiT.

cs.CV↗

arXiv2022

Speeding Up Question Answering Task of Language Models via Inverted Index

Natural language processing applications, such as conversational agents and their question-answering capabilities, are widely used in the real world. Despite the wide popularity of large language models (LLMs), few real-world conversational agents take advantage of LLMs. Extensive resources consumed by LLMs disable developers from integrating them into end-user applications. In this study, we leverage an inverted indexing mechanism combined with LLMs to improve the efficiency of question-answering models for closed-domain questions. Our experiments show that using the index improves the average response time by 97.44%. In addition, due to the reduced search scope, the average BLEU score improved by 0.23 while using the inverted index.

cs.CL↗

arXiv2022

Iterative Methods at Lower Precision

Since numbers in the computer are represented with a fixed number of bits, loss of accuracy during calculation is unavoidable. At high precision where more bits (e.g. 64) are allocated to each number, round-off errors are typically small. On the other hand, calculating at lower precision, such as half (16 bits), has the advantage of being much faster. This research focuses on experimenting with arithmetic at different precision levels for large-scale inverse problems, which are represented by linear systems with ill-conditioned matrices. We modified the Conjugate Gradient Method for Least Squares (CGLS) and the Chebyshev Semi-Iterative Method (CS) with Tikhonov regularization to do arithmetic at lower precision using the MATLAB chop function, and we ran experiments on applications from image processing and compared their performance at different precision levels. We concluded that CGLS is a more stable algorithm, but overflows easily due to the computation of inner products, while CS is less likely to overflow but it has more erratic convergence behavior. When the noise level is high, CS outperforms CGLS by being able to run more iterations before overflow occurs; when the noise level is close to zero, CS appears to be more susceptible to accumulation of round-off errors.

math.NA↗

arXiv2022

Sample Complexity of Nonparametric Off-Policy Evaluation on Low-Dimensional Manifolds using Deep Networks

We consider the off-policy evaluation problem of reinforcement learning using deep convolutional neural networks. We analyze the deep fitted Q-evaluation method for estimating the expected cumulative reward of a target policy, when the data are generated from an unknown behavior policy. We show that, by choosing network size appropriately, one can leverage any low-dimensional manifold structure in the Markov decision process and obtain a sample-efficient estimator without suffering from the curse of high data ambient dimensionality. Specifically, we establish a sharp error bound for fitted Q-evaluation, which depends on the intrinsic dimension of the state-action space, the smoothness of Bellman operator, and a function class-restricted $χ^2$-divergence. It is noteworthy that the restricted $χ^2$-divergence measures the behavior and target policies' {\it mismatch in the function space}, which can be small even if the two policies are not close to each other in their tabular forms. We also develop a novel approximation result for convolutional neural networks in Q-function estimation. Numerical experiments are provided to support our theoretical analysis.

cs.LG↗

arXiv2022

Accelerating Bayesian inference of dependency between complex biological traits

Inferring dependencies between complex biological traits while accounting for evolutionary relationships between specimens is of great scientific interest yet remains infeasible when trait and specimen counts grow large. The state-of-the-art approach uses a phylogenetic multivariate probit model to accommodate binary and continuous traits via a latent variable framework, and utilizes an efficient bouncy particle sampler (BPS) to tackle the computational bottleneck -- integrating many latent variables from a high-dimensional truncated normal distribution. This approach breaks down as the number of specimens grows and fails to reliably characterize conditional dependencies between traits. Here, we propose an inference pipeline for phylogenetic probit models that greatly outperforms BPS. The novelty lies in 1) a combination of the recent Zigzag Hamiltonian Monte Carlo (Zigzag-HMC) with linear-time gradient evaluations and 2) a joint sampling scheme for highly correlated latent variables and correlation matrix elements. In an application exploring HIV-1 evolution from 535 viruses, the inference requires joint sampling from an 11,235-dimensional truncated normal and a 24-dimensional covariance matrix. Our method yields a 5-fold speedup compared to BPS and makes it possible to learn partial correlations between candidate viral mutations and virulence. Computational speedup now enables us to tackle even larger problems: we study the evolution of influenza H1N1 glycosylations on around 900 viruses. For broader applicability, we extend the phylogenetic probit model to incorporate categorical traits, and demonstrate its use to study Aquilegia flower and pollinator co-evolution.

stat.ME↗

arXiv2022

Energetic Variational Approach for Prediction of Thermal Electrokinetics in Charging and Discharging Processes of Electrical Double Layer Capacitors

This work proposes a new variational, thermodynamically consistent model to predict thermal electrokinetics in electric double layer capacitors (EDLCs) by using an energetic variational approach. The least action principle and maximum dissipation principle from the non-equilibrium thermodynamics are employed to develop modified Nernst-Planck equations for non-isothermal ion transport with temperature inhomogeneity. Laws of thermodynamics are employed to derive a temperature evolution equation with heat sources due to thermal pressure and electrostatic interactions. Numerical simulations successfully predict temperature oscillation in the charging-discharging processes of EDLCs, indicating that the developed model is able to capture reversible and irreversible heat generations. The impact of ionic sizes and scan rate of surface potential on ion transport, heat generation, and charge current is systematically assessed in cyclic voltammetry simulations. It is found that the thermal electrokinetics in EDLCs cannot follow the surface potential with fast scan rates, showing delayed dynamics with hysteresis diagrams. Our work thus provides a useful tool for physics-based prediction of thermal electrokinetics in EDLCs.

physics.chem-ph↗

arXiv2022

Neural Global Shutter: Learn to Restore Video from a Rolling Shutter Camera with Global Reset Feature

Most computer vision systems assume distortion-free images as inputs. The widely used rolling-shutter (RS) image sensors, however, suffer from geometric distortion when the camera and object undergo motion during capture. Extensive researches have been conducted on correcting RS distortions. However, most of the existing work relies heavily on the prior assumptions of scenes or motions. Besides, the motion estimation steps are either oversimplified or computationally inefficient due to the heavy flow warping, limiting their applicability. In this paper, we investigate using rolling shutter with a global reset feature (RSGR) to restore clean global shutter (GS) videos. This feature enables us to turn the rectification problem into a deblur-like one, getting rid of inaccurate and costly explicit motion estimation. First, we build an optic system that captures paired RSGR/GS videos. Second, we develop a novel algorithm incorporating spatial and temporal designs to correct the spatial-varying RSGR distortion. Third, we demonstrate that existing image-to-image translation algorithms can recover clean GS videos from distorted RSGR inputs, yet our algorithm achieves the best performance with the specific designs. Our rendered results are not only visually appealing but also beneficial to downstream tasks. Compared to the state-of-the-art RS solution, our RSGR solution is superior in both effectiveness and efficiency. Considering it is easy to realize without changing the hardware, we believe our RSGR solution can potentially replace the RS solution in taking distortion-free videos with low noise and low budget.

cs.CV↗

arXiv2022

Optimal Estimation of Off-Policy Policy Gradient via Double Fitted Iteration

Policy gradient (PG) estimation becomes a challenge when we are not allowed to sample with the target policy but only have access to a dataset generated by some unknown behavior policy. Conventional methods for off-policy PG estimation often suffer from either significant bias or exponentially large variance. In this paper, we propose the double Fitted PG estimation (FPG) algorithm. FPG can work with an arbitrary policy parameterization, assuming access to a Bellman-complete value function class. In the case of linear value function approximation, we provide a tight finite-sample upper bound on policy gradient estimation error, that is governed by the amount of distribution mismatch measured in feature space. We also establish the asymptotic normality of FPG estimation error with a precise covariance characterization, which is further shown to be statistically optimal with a matching Cramer-Rao lower bound. Empirically, we evaluate the performance of FPG on both policy gradient estimation and policy optimization, using either softmax tabular or ReLU policy networks. Under various metrics, our results show that FPG significantly outperforms existing off-policy PG estimation methods based on importance sampling and variance reduction techniques.

stat.ML↗

arXiv2022

Topological energy braiding of the non-Bloch bands

The non-Hermitian skin effect, as a unique feature of non-Hermitian systems, will break the topological energy braiding of the Bloch bands in open boundary systems. Going beyond the Bloch band theory, we unveil the energy braiding of the non-Bloch bands by introducing a one-dimensional non-Hermitian tight-binding model. We find an entirely new generic class of topological non-Bloch bands such as Hopf link, which is generally generated by the non-Hermitian skin effect. The energy braiding is topologically robust against any perturbations without gap closing. Furthermore, non-Bloch topological invariants are proposed based on the generalized Brillouin zone to characterize the topology of these non-Bloch bands. The topological phase transition between the distinct phases occurs with the non-Bloch bands touching at exceptional points. We hope that our work can shed light on the topological energy braiding of the non-Bloch bands for non-Hermitian systems.

cond-mat.mes-hall↗

arXiv2022

Bootstrapping Fitted Q-Evaluation for Off-Policy Inference

Bootstrapping provides a flexible and effective approach for assessing the quality of batch reinforcement learning, yet its theoretical property is less understood. In this paper, we study the use of bootstrapping in off-policy evaluation (OPE), and in particular, we focus on the fitted Q-evaluation (FQE) that is known to be minimax-optimal in the tabular and linear-model cases. We propose a bootstrapping FQE method for inferring the distribution of the policy evaluation error and show that this method is asymptotically efficient and distributionally consistent for off-policy statistical inference. To overcome the computation limit of bootstrapping, we further adapt a subsampling procedure that improves the runtime by an order of magnitude. We numerically evaluate the bootrapping method in classical RL environments for confidence interval estimation, estimating the variance of off-policy evaluator, and estimating the correlation between multiple off-policy evaluators.

stat.ML↗

arXiv2022

Computational Statistics and Data Science in the Twenty-first Century

Data science has arrived, and computational statistics is its engine. As the scale and complexity of scientific and industrial data grow, the discipline of computational statistics assumes an increasingly central role among the statistical sciences. An explosion in the range of real-world applications means the development of more and more specialized computational methods, but five Core Challenges remain. We provide a high-level introduction to computational statistics by focusing on its central challenges, present recent model-specific advances and preach the ever-increasing role of non-sequential computational paradigms such as multi-core, many-core and quantum computing. Data science is bringing major changes to computational statistics, and these changes will shape the trajectory of the discipline in the 21st century.

stat.OT↗

arXiv2022

SQUAB I: The first release of Strange QUasar candidates with ABnormal astrometric characteristics from Gaia EDR3 and SDSS

Given their extremely large distances and small apparent sizes, quasars are generally considered as objects with near-zero parallax and proper motion. However, some special quasars may have abnormal astrometric characteristics, such as quasar pairs, lensed quasars, AGNs with bright parsec-scale optical jets, which are scientifically interesting objects, such as binary black holes. These quasars may come with astrometric jitter detectable with Gaia data, or significant changes in the position at different wavelengths. In this work, we aim to find these quasar candidates from Gaia EDR3 astrometric data combining with Sloan Digital Sky Survey (SDSS) spectroscopic data to provide a candidate catalog to the science community. We propose a series of criteria for selecting abnormal quasars based on Gaia astrometric data. We obtain two catalogs containing 155 sources and 44 sources, respectively. They are potential candidates of quasar pairs.

astro-ph.GA↗

arXiv2021

Scalable Bayesian divergence time estimation with ratio transformations

Divergence time estimation is crucial to provide temporal signals for dating biologically important events, from species divergence to viral transmissions in space and time. With the advent of high-throughput sequencing, recent Bayesian phylogenetic studies have analyzed hundreds to thousands of sequences. Such large-scale analyses challenge divergence time reconstruction by requiring inference on highly-correlated internal node heights that often become computationally infeasible. To overcome this limitation, we explore a ratio transformation that maps the original N - 1 internal node heights into a space of one height parameter and N - 2 ratio parameters. To make analyses scalable, we develop a collection of linear-time algorithms to compute the gradient and Jacobian-associated terms of the log-likelihood with respect to these ratios. We then apply Hamiltonian Monte Carlo sampling with the ratio transform in a Bayesian framework to learn the divergence times in four pathogenic virus phylogenies: West Nile virus, rabies virus, Lassa virus and Ebola virus. Our method both resolves a mixing issue in the West Nile virus example and improves inference efficiency by at least 5-fold for the Lassa and rabies virus examples. Our method also makes it now computationally feasible to incorporate mixed-effects molecular clock models for the Ebola virus example, confirms the findings from the original study and reveals clearer multimodal distributions of the divergence times of some clades of interest.

q-bio.PE↗

arXiv2021

Variations of Broad Emission Lines from periodicity QSOs under the interpretation of supermassive binary black holes with misaligned circumbinary broad line regions

Quasars with periodic light curves are considered as candidates of supermassive binary black hole (BBH) systems. One way for further confirmations may be searching for other characteristic signatures, such as those in their broad emission lines (BELs), if any, which require a thorough understanding on the response of BELs to the BBH systems. In Ji et al. (2021), we have investigated the response of circumbinary broad line region (BLR) to the central active secondary black hole under the relativistic Doppler boosting (BBH-DB) and intrinsic variation (BBH-IntDB) dominant mechanisms for continuum variation by assuming the middle plane of the BLR aligned with the BBH orbital plane. In this paper, we explore how the BEL profiles vary when the BLR is misaligned from the BBH orbital plane with different offset angles under both the BBH-DB and BBH-IntDB scenarios. Given a fixed inclination angle of the BBH orbital plane viewed in edge-on and similar continuum light curves produced by the two scenarios, increasing offset angles make the initial opening angle of the circumbinary BLR enlarged due to orbital precession caused by the BBH system, especially for clouds in the inner region, which result in Lorentz-like BEL profiles for the BBH-DB model but still Gaussian-like profiles for the BBH-IntDB model at the vertical BLR case. The amplitude of profile variations decrease with increasing offset angles for the BBH-DB scenario, while keep nearly constant for the BBH-IntDB scenario, since the Doppler boosting effect is motion direction preferred but the intrinsic variation is radiated isotropically. If the circumbinary BLR is composed of a coplanar and a vertical components with their number of clouds following the mass ratio of the BBHs, then the bi-BLR features are more significant for the BBH-IntDB model that require larger mass ratio to generate similar continuum variation than the BBH-DB model.

astro-ph.GA↗

arXiv2021

From viral evolution to spatial contagion: a biologically modulated Hawkes model

Mutations sometimes increase contagiousness for evolving pathogens. During an epidemic, scientists use viral genome data to infer a shared evolutionary history and connect this history to geographic spread. We propose a model that directly relates a pathogen's evolution to its spatial contagion dynamics -- effectively combining the two epidemiological paradigms of phylogenetic inference and self-exciting process modeling -- and apply this \emph{phylogenetic Hawkes process} to a Bayesian analysis of 23,422 viral cases from the 2014-2016 Ebola outbreak in West Africa. The proposed model is able to detect individual viruses with significantly elevated rates of spatiotemporal propagation for a subset of 1,610 samples that provide genome data. Finally, to facilitate model application in big data settings, we develop massively parallel implementations for the gradient and Hessian of the log-likelihood and apply our high performance computing framework within an adaptively preconditioned Hamiltonian Monte Carlo routine.

q-bio.PE↗