arXiv subjects

Xin Liu

Publications and source records attributed to Xin Liu.

At least 685 records · Page 38Linked to original sources

arXiv2022

NOEMA Detection of Circumnuclear Molecular Gas in X-ray Weak Dual Active Galactic Nuclei: No Evidence for Heavy Obscuration

Dual active galactic nuclei (AGN), which are the manifestation of two actively accreting supermassive black holes (SMBHs) hosted by a pair of merging galaxies, are a unique laboratory for studying the physics of SMBH feeding and feedback during an indispensable stage of galaxy evolution. In this work, we present NOEMA CO(2-1) observations of seven kpc-scale dual-AGN candidates drawn from a recent Chandra survey of low-redshift, optically classified AGN pairs. These systems are selected because they show unexpectedly low 2-10 keV X-ray luminosities for their small physical separations signifying an intermediate-to-late stage of merger. Circumnuclear molecular gas traced by the CO(2-1) emission is significantly detected in 6 of the 7 pairs and 10 of the 14 nuclei, with an estimated mass ranging between $(0.2 - 21) \times10^9\rm~M_{\odot}$. The primary nuclei, i.e., the ones with the higher stellar velocity dispersion, tend to have a higher molecular gas mass than the secondary. Most CO-detected nuclei show a compact morphology, with a velocity field consistent with a kpc-scale rotating structure. The inferred hydrogen column densities range between $5\times10^{21} - 2\times10^{23}\rm~cm^{-2}$, but mostly at a few times $10^{22}\rm~cm^{-2}$, in broad agreement with those derived from X-ray spectral analysis. Together with the relatively weak mid-infrared emission, the moderate column density argues against the prevalence of heavily obscured, intrinsically luminous AGNs in these seven systems, but favors a feedback scenario in which AGN activity triggered by a recent pericentric passage of the galaxy pair can expel circumnuclear gas and suppress further SMBH accretion.

astro-ph.GA↗

arXiv2022

Accuracy-Privacy Trade-off in Deep Ensemble: A Membership Inference Perspective

Deep ensemble learning has been shown to improve accuracy by training multiple neural networks and averaging their outputs. Ensemble learning has also been suggested to defend against membership inference attacks that undermine privacy. In this paper, we empirically demonstrate a trade-off between these two goals, namely accuracy and privacy (in terms of membership inference attacks), in deep ensembles. Using a wide range of datasets and model architectures, we show that the effectiveness of membership inference attacks increases when ensembling improves accuracy. We analyze the impact of various factors in deep ensembles and demonstrate the root cause of the trade-off. Then, we evaluate common defenses against membership inference attacks based on regularization and differential privacy. We show that while these defenses can mitigate the effectiveness of membership inference attacks, they simultaneously degrade ensemble accuracy. We illustrate similar trade-off in more advanced and state-of-the-art ensembling techniques, such as snapshot ensembles and diversified ensemble networks. Finally, we propose a simple yet effective defense for deep ensembles to break the trade-off and, consequently, improve the accuracy and privacy, simultaneously.

cs.LG↗

arXiv2022

Dwarf AGNs from Variability for the Origins of Seeds (DAVOS): Optical Variability of Broad-line Dwarf AGNs from the Zwicky Transient Facility

We study the optical variability of a sample of candidate low-mass (dwarf ang Seyfert) active galactic nuclei (AGNs) using Zwicky Transient Facility g-band light curves. Our sample is compiled from broad-line AGNs in dwarf galaxies reported in the literature with single-epoch virial black hole (BH) masses in the range $M_{\rm{BH}} \sim 10^{4}$--$10^{8}\ M_{\odot}$. We measure the characteristic ``damping'' timescale of the optical variability $τ_{\rm{DRW}}$, beyond which the power spectral density flattens, of a final sample of 79 candidate low-mass AGNs with high-quality light curves. Our results provide further confirmation of the $M_{\rm{BH}} - τ_{\rm{DRW}}$ relation from Burke et al. 2022 within $1σ$ agreement, adding 78 new low-mass AGNs to the relation. The agreement suggests that the virial BH mass estimates for these AGNs are generally reasonable. We expect that the optical light curve of an accreting intermediate-mass black hole (IMBH) to vary with a rest-frame damping timescale of $\sim$ tens of hours, which could enable detection and direct mass estimation of accreting IMBHs in wide-field time-domain imaging surveys with sufficient cadence like with the Vera C. Rubin Observatory.

astro-ph.GA↗

arXiv2022

A Communication-Efficient and Privacy-Aware Distributed Algorithm for Sparse PCA

Sparse principal component analysis (PCA) improves interpretability of the classic PCA by introducing sparsity into the dimension-reduction process. Optimization models for sparse PCA, however, are generally non-convex, non-smooth and more difficult to solve, especially on large-scale datasets requiring distributed computation over a wide network. In this paper, we develop a distributed and centralized algorithm called DSSAL1 for sparse PCA that aims to achieve low communication overheads by adapting a newly proposed subspace-splitting strategy to accelerate convergence. Theoretically, convergence to stationary points is established for DSSAL1. Extensive numerical results show that DSSAL1 requires far fewer rounds of communication than state-of-the-art peer methods. In addition, we make the case that since messages exchanged in DSSAL1 are well-masked, the possibility of private-data leakage in DSSAL1 is much lower than in some other distributed algorithms.

math.OC↗

arXiv2022

Seeking Consensus on Subspaces in Federated Principal Component Analysis

In this paper, we develop an algorithm for federated principal component analysis (PCA) with emphases on both communication efficiency and data privacy. Generally speaking, federated PCA algorithms based on direct adaptations of classic iterative methods, such as simultaneous subspace iterations (SSI), are unable to preserve data privacy, while algorithms based on variable-splitting and consensus-seeking, such as alternating direction methods of multipliers (ADMM), lack in communication-efficiency. In this work, we propose a novel consensus-seeking formulation by equalizing subspaces spanned by splitting variables instead of equalizing variables themselves, thus greatly relaxing feasibility restrictions and allowing much faster convergence. Then we develop an ADMM-like algorithm with several special features to make it practically efficient, including a low-rank multiplier formula and techniques for treating subproblems. We establish that the proposed algorithm can better protect data privacy than classic methods adapted to the federated PCA setting. We derive convergence results, including a worst-case complexity estimate, for the proposed ADMM-like algorithm in the presence of the nonlinear equality constraints. Extensive empirical results are presented to show that the new algorithm, while enhancing data privacy, requires far fewer rounds of communication than existing peer algorithms for federated PCA.

math.OC↗

arXiv2022

Rectified Pessimistic-Optimistic Learning for Stochastic Continuum-armed Bandit with Constraints

This paper studies the problem of stochastic continuum-armed bandit with constraints (SCBwC), where we optimize a black-box reward function $f(x)$ subject to a black-box constraint function $g(x)\leq 0$ over a continuous space $\mathcal X$. We model reward and constraint functions via Gaussian processes (GPs) and propose a Rectified Pessimistic-Optimistic Learning framework (RPOL), a penalty-based method incorporating optimistic and pessimistic GP bandit learning for reward and constraint functions, respectively. We consider the metric of cumulative constraint violation $\sum_{t=1}^T(g(x_t))^{+},$ which is strictly stronger than the traditional long-term constraint violation $\sum_{t=1}^Tg(x_t).$ The rectified design for the penalty update and the pessimistic learning for the constraint function in RPOL guarantee the cumulative constraint violation is minimal. RPOL can achieve sublinear regret and cumulative constraint violation for SCBwC and its variants (e.g., under delayed feedback and non-stationary environment). These theoretical results match their unconstrained counterparts. Our experiments justify RPOL outperforms several existing baseline algorithms.

cs.LG↗

arXiv2022

Federated Learning Hyper-Parameter Tuning from a System Perspective

Federated learning (FL) is a distributed model training paradigm that preserves clients' data privacy. It has gained tremendous attention from both academia and industry. FL hyper-parameters (e.g., the number of selected clients and the number of training passes) significantly affect the training overhead in terms of computation time, transmission time, computation load, and transmission load. However, the current practice of manually selecting FL hyper-parameters imposes a heavy burden on FL practitioners because applications have different training preferences. In this paper, we propose FedTune, an automatic FL hyper-parameter tuning algorithm tailored to applications' diverse system requirements in FL training. FedTune iteratively adjusts FL hyper-parameters during FL training and can be easily integrated into existing FL systems. Through extensive evaluations of FedTune for diverse applications and FL aggregation algorithms, we show that FedTune is lightweight and effective, achieving 8.48%-26.75% system overhead reduction compared to using fixed FL hyper-parameters. This paper assists FL practitioners in designing high-performance FL training solutions. The source code of FedTune is available at https://github.com/DataSysTech/FedTune.

cs.LG↗

arXiv2022

AGNet: Weighing Black Holes with Deep Learning

Supermassive black holes (SMBHs) are ubiquitously found at the centers of most massive galaxies. Measuring SMBH mass is important for understanding the origin and evolution of SMBHs. However, traditional methods require spectroscopic data which is expensive to gather. We present an algorithm that weighs SMBHs using quasar light time series, circumventing the need for expensive spectra. We train, validate, and test neural networks that directly learn from the Sloan Digital Sky Survey (SDSS) Stripe 82 light curves for a sample of $38,939$ spectroscopically confirmed quasars to map out the nonlinear encoding between SMBH mass and multi-color optical light curves. We find a 1$σ$ scatter of 0.37 dex between the predicted SMBH mass and the fiducial virial mass estimate based on SDSS single-epoch spectra, which is comparable to the systematic uncertainty in the virial mass estimate. Our results have direct implications for more efficient applications with future observations from the Vera C. Rubin Observatory. Our code, \textsf{AGNet}, is publicly available at \url{https://github.com/snehjp2/AGNet}.

astro-ph.GA↗

arXiv2022

Balanced softmax cross-entropy for incremental learning with and without memory

When incrementally trained on new classes, deep neural networks are subject to catastrophic forgetting which leads to an extreme deterioration of their performance on the old classes while learning the new ones. Using a small memory containing few samples from past classes has shown to be an effective method to mitigate catastrophic forgetting. However, due to the limited size of the replay memory, there is a large imbalance between the number of samples for the new and the old classes in the training dataset resulting in bias in the final model. To address this issue, we propose to use the Balanced Softmax Cross-Entropy and show that it can be seamlessly combined with state-of-the-art approaches for class-incremental learning in order to improve their accuracy while also potentially decreasing the computational cost of the training procedure. We further extend this approach to the more demanding class-incremental learning without memory setting and achieve competitive results with memory-based approaches. Experiments on the challenging ImageNet, ImageNet-Subset and CIFAR100 benchmarks with various settings demonstrate the benefits of our approach.

cs.LG↗

arXiv2022

Getting the Most out of Simile Recognition

Simile recognition involves two subtasks: simile sentence classification that discriminates whether a sentence contains simile, and simile component extraction that locates the corresponding objects (i.e., tenors and vehicles). Recent work ignores features other than surface strings. In this paper, we explore expressive features for this task to achieve more effective data utilization. Particularly, we study two types of features: 1) input-side features that include POS tags, dependency trees and word definitions, and 2) decoding features that capture the interdependence among various decoding decisions. We further construct a model named HGSR, which merges the input-side features as a heterogeneous graph and leverages decoding features via distillation. Experiments show that HGSR significantly outperforms the current state-of-the-art systems and carefully designed baselines, verifying the effectiveness of introduced features. Our code is available at https://github.com/DeepLearnXMU/HGSR.

cs.CL↗

arXiv2022

Complex Hyperbolic Knowledge Graph Embeddings with Fast Fourier Transform

The choice of geometric space for knowledge graph (KG) embeddings can have significant effects on the performance of KG completion tasks. The hyperbolic geometry has been shown to capture the hierarchical patterns due to its tree-like metrics, which addressed the limitations of the Euclidean embedding models. Recent explorations of the complex hyperbolic geometry further improved the hyperbolic embeddings for capturing a variety of hierarchical structures. However, the performance of the hyperbolic KG embedding models for non-transitive relations is still unpromising, while the complex hyperbolic embeddings do not deal with multi-relations. This paper aims to utilize the representation capacity of the complex hyperbolic geometry in multi-relational KG embeddings. To apply the geometric transformations which account for different relations and the attention mechanism in the complex hyperbolic space, we propose to use the fast Fourier transform (FFT) as the conversion between the real and complex hyperbolic space. Constructing the attention-based transformations in the complex space is very challenging, while the proposed Fourier transform-based complex hyperbolic approaches provide a simple and effective solution. Experimental results show that our methods outperform the baselines, including the Euclidean and the real hyperbolic embedding models.

cs.LG↗

arXiv2022

A Transient "Changing-look'' Active Galactic Nucleus Resolved on Month Timescales from First-year Sloan Digital Sky Survey V Data

We report the discovery of a new ``changing-look'' active galactic nucleus (CLAGN) event, in the quasar SDSS J162829.17+432948.5 at z=0.2603, identified through repeat spectroscopy from the fifth Sloan Digital Sky Survey (SDSS-V). Optical photometry taken during 2020--2021 shows a dramatic dimming of $Δ$g${\approx}$1 mag, followed by a rapid recovery on a timescale of several months, with the ${\lesssim}$2 month period of rebrightening captured in new SDSS-V and Las Cumbres Observatory spectroscopy. This is one of the fastest CLAGN transitions observed to date. Archival observations suggest that the object experienced a much more gradual dimming over the period of 2011--2013. Our spectroscopy shows that the photometric changes were accompanied by dramatic variations in the quasar-like continuum and broad-line emission. The excellent agreement between the pre- and postdip photometric and spectroscopic appearances of the source, as well as the fact that the dimmest spectra can be reproduced by applying a single extinction law to the brighter spectral states, favor a variable line-of-sight obscuration as the driver of the observed transitions. Such an interpretation faces several theoretical challenges, and thus an alternative accretion-driven scenario cannot be excluded. The recent events observed in this quasar highlight the importance of spectroscopic monitoring of large active galactic nucleus samples on weeks-to-months timescales, which the SDSS-V is designed to achieve.

astro-ph.GA↗

arXiv2022

Stochastic Gauss-Newton Algorithms for Online PCA

In this paper, we propose a stochastic Gauss-Newton (SGN) algorithm to study the online principal component analysis (OPCA) problem, which is formulated by using the symmetric low-rank product (SLRP) model for dominant eigenspace calculation. Compared with existing OPCA solvers, SGN is of improved robustness with respect to the varying input data and algorithm parameters. In addition, turning to an evaluation of data stream based on approximated objective functions, we develop a new adaptive stepsize strategy for SGN (AdaSGN) which requires no priori knowledge of the input data, and numerically illustrate its comparable performance with SGN adopting the manaully-tuned diminishing stepsize. Without assuming the eigengap to be positive, we also establish the global and optimal convergence rate of SGN with the specified stepsize using the diffusion approximation theory.

math.OC↗

arXiv2022

Opportunistic Episodic Reinforcement Learning

In this paper, we propose and study opportunistic reinforcement learning - a new variant of reinforcement learning problems where the regret of selecting a suboptimal action varies under an external environmental condition known as the variation factor. When the variation factor is low, so is the regret of selecting a suboptimal action and vice versa. Our intuition is to exploit more when the variation factor is high, and explore more when the variation factor is low. We demonstrate the benefit of this novel framework for finite-horizon episodic MDPs by designing and evaluating OppUCRL2 and OppPSRL algorithms. Our algorithms dynamically balance the exploration-exploitation trade-off for reinforcement learning by introducing variation factor-dependent optimism to guide exploration. We establish an $\tilde{O}(HS \sqrt{AT})$ regret bound for the OppUCRL2 algorithm and show through simulations that both OppUCRL2 and OppPSRL algorithm outperform their original corresponding algorithms.

cs.LG↗

arXiv2022

Immediate blowup of entropy-bounded classical solutions to the vacuum free boundary problem of non-isentropic compressible Navier--Stokes equations

This paper considers the immediate blowup of entropy-bounded classical solutions to the vacuum free boundary problem of non-isentropic compressible Navier-Stokes equations. The viscosities and the heat conductivity could be constants, or more physically, the degenerate, temperature-dependent functions which vanish on the vacuum boundary (i.e., $μ=\barμ θ^α, ~ λ=\barλ θ^α,~ κ=\barκ θ^α$, for constants $0\leq α\leq 1/(γ-1)$, $\barμ>0,~2\barμ+n\barλ\geq0,~\barκ\geq 0$, and adiabatic exponent $γ>1$). With prescribed decaying rate of the initial density across the vacuum boundary, we prove that: (1) for three-dimensional spherically symmetric flows with non-vanishing bulk viscosity and zero heat conductivity, entropy-bounded classical solutions do not exist for any small time, provided the initial velocity is expanding near the boundary; (2) for three-dimensional spherically symmetric flows with non-vanishing heat conductivity, the normal derivative of the temperature of the classical solution across the free boundary does not degenerate, and therefore the entropy immediately blowups if the decaying rate of the initial density is not of $1/(γ-1)$ power of the distance function to the boundary; (3) for one-dimensional flow with zero heat conductivity, the non-existence result is similar but need more restrictions on the decaying rate. Together with our previous results on local or global entropy-bounded classical solutions (Liu and Yuan, SIAM J. Math. Anal. (2) 51, 2019; Liu and Yuan, Math. Models Methods Appl. Sci. (9) 12, 2019), this paper shows the necessity of proper degenerate conditions on the density and temperature across the boundary for the well-posedness of the entropy-bounded classical solutions to the vacuum boundary problem of the viscous gas.

math.AP↗

arXiv2022

Self-Supervised Learning via Maximum Entropy Coding

A mainstream type of current self-supervised learning methods pursues a general-purpose representation that can be well transferred to downstream tasks, typically by optimizing on a given pretext task such as instance discrimination. In this work, we argue that existing pretext tasks inevitably introduce biases into the learned representation, which in turn leads to biased transfer performance on various downstream tasks. To cope with this issue, we propose Maximum Entropy Coding (MEC), a more principled objective that explicitly optimizes on the structure of the representation, so that the learned representation is less biased and thus generalizes better to unseen downstream tasks. Inspired by the principle of maximum entropy in information theory, we hypothesize that a generalizable representation should be the one that admits the maximum entropy among all plausible representations. To make the objective end-to-end trainable, we propose to leverage the minimal coding length in lossy data coding as a computationally tractable surrogate for the entropy, and further derive a scalable reformulation of the objective that allows fast computation. Extensive experiments demonstrate that MEC learns a more generalizable representation than previous methods based on specific pretext tasks. It achieves state-of-the-art performance consistently on various downstream tasks, including not only ImageNet linear probe, but also semi-supervised classification, object detection, instance segmentation, and object tracking. Interestingly, we show that existing batch-wise and feature-wise self-supervised objectives could be seen equivalent to low-order approximations of MEC. Code and pre-trained models are available at https://github.com/xinliu20/MEC.

cs.CV↗

arXiv2022

GraphCSPN: Geometry-Aware Depth Completion via Dynamic GCNs

Image guided depth completion aims to recover per-pixel dense depth maps from sparse depth measurements with the help of aligned color images, which has a wide range of applications from robotics to autonomous driving. However, the 3D nature of sparse-to-dense depth completion has not been fully explored by previous methods. In this work, we propose a Graph Convolution based Spatial Propagation Network (GraphCSPN) as a general approach for depth completion. First, unlike previous methods, we leverage convolution neural networks as well as graph neural networks in a complementary way for geometric representation learning. In addition, the proposed networks explicitly incorporate learnable geometric constraints to regularize the propagation process performed in three-dimensional space rather than in two-dimensional plane. Furthermore, we construct the graph utilizing sequences of feature patches, and update it dynamically with an edge attention module during propagation, so as to better capture both the local neighboring features and global relationships over long distance. Extensive experiments on both indoor NYU-Depth-v2 and outdoor KITTI datasets demonstrate that our method achieves the state-of-the-art performance, especially when compared in the case of using only a few propagation steps. Code and models are available at the project page.

cs.CV↗

arXiv2022

MASTAF: A Model-Agnostic Spatio-Temporal Attention Fusion Network for Few-shot Video Classification

We propose MASTAF, a Model-Agnostic Spatio-Temporal Attention Fusion network for few-shot video classification. MASTAF takes input from a general video spatial and temporal representation,e.g., using 2D CNN, 3D CNN, and Video Transformer. Then, to make the most of such representations, we use self- and cross-attention models to highlight the critical spatio-temporal region to increase the inter-class variations and decrease the intra-class variations. Last, MASTAF applies a lightweight fusion network and a nearest neighbor classifier to classify each query video. We demonstrate that MASTAF improves the state-of-the-art performance on three few-shot video classification benchmarks(UCF101, HMDB51, and Something-Something-V2), e.g., by up to 91.6%, 69.5%, and 60.7% for five-way one-shot video classification, respectively.

cs.CV↗