arXiv subjects

Yifan Chen

Publications and source records attributed to Yifan Chen.

At least 199 records · Page 11Linked to original sources

arXiv2022

Stringent axion constraints with Event Horizon Telescope polarimetric measurements of M87$^\star$

The hitherto unprecedented angular resolution of the Event Horizon Telescope (EHT) has created exciting opportunities in the search for new physics. Recently, the linear polarization of radiation emitted near the supermassive black hole M87$^\star$ was measured on four separate days, precisely enabling tests of the existence of a dense axion cloud produced by a spinning black hole. The presence of an axion cloud leads to a frequency-independent oscillation in the electric vector position angle (EVPA) of this linear polarization. For a nearly face-on M87$^\star$, this oscillation in the EVPA appears as a propagating wave along the photon ring. In this paper, we leverage the azimuthal distribution of EVPA measured by the EHT to study the axion-photon coupling. We propose a novel differential analysis procedure to reduce the astrophysical background, and derive stringent constraints on the existence of axions in the previously unexplored mass window $\sim (10^{-21}-10^{-20})$~eV.

hep-ph↗

arXiv2022

A FAIR and AI-ready Higgs boson decay dataset

To enable the reusability of massive scientific datasets by humans and machines, researchers aim to adhere to the principles of findability, accessibility, interoperability, and reusability (FAIR) for data and artificial intelligence (AI) models. This article provides a domain-agnostic, step-by-step assessment guide to evaluate whether or not a given dataset meets these principles. We demonstrate how to use this guide to evaluate the FAIRness of an open simulated dataset produced by the CMS Collaboration at the CERN Large Hadron Collider. This dataset consists of Higgs boson decays and quark and gluon background, and is available through the CERN Open Data Portal. We use additional available tools to assess the FAIRness of this dataset, and incorporate feedback from members of the FAIR community to validate our results. This article is accompanied by a Jupyter notebook to visualize and explore this dataset. This study marks the first in a planned series of articles that will guide scientists in the creation of FAIR AI models and datasets in high energy particle physics.

hep-ex↗

arXiv2021

Sketching as a Tool for Understanding and Accelerating Self-attention for Long Sequences

Transformer-based models are not efficient in processing long sequences due to the quadratic space and time complexity of the self-attention modules. To address this limitation, Linformer and Informer are proposed to reduce the quadratic complexity to linear (modulo logarithmic factors) via low-dimensional projection and row selection respectively. These two models are intrinsically connected, and to understand their connection, we introduce a theoretical framework of matrix sketching. Based on the theoretical analysis, we propose Skeinformer to accelerate self-attention and further improve the accuracy of matrix approximation to self-attention with three carefully designed components: column sampling, adaptive row normalization and pilot sampling reutilization. Experiments on the Long Range Arena (LRA) benchmark demonstrate that our methods outperform alternatives with a consistently smaller time/space footprint.

cs.LG↗

arXiv2021

Skyformer: Remodel Self-Attention with Gaussian Kernel and Nyström Method

Transformers are expensive to train due to the quadratic time and space complexity in the self-attention mechanism. On the other hand, although kernel machines suffer from the same computation bottleneck in pairwise dot products, several approximation schemes have been successfully incorporated to considerably reduce their computational cost without sacrificing too much accuracy. In this work, we leverage the computation methods for kernel machines to alleviate the high computational cost and introduce Skyformer, which replaces the softmax structure with a Gaussian kernel to stabilize the model training and adapts the Nyström method to a non-positive semidefinite matrix to accelerate the computation. We further conduct theoretical analysis by showing that the matrix approximation error of our proposed method is small in the spectral norm. Experiments on Long Range Arena benchmark show that the proposed method is sufficient in getting comparable or even better performance than the full self-attention while requiring fewer computation resources.

cs.LG↗

arXiv2021

Multiscale Elliptic PDEs Upscaling and Function Approximation via Subsampled Data

There is an intimate connection between numerical upscaling of multiscale PDEs and scattered data approximation of heterogeneous functions: the coarse variables selected for deriving an upscaled equation (in the former) correspond to the sampled information used for approximation (in the latter). As such, both problems can be thought of as recovering a target function based on some coarse data that are either artificially chosen by an upscaling algorithm, or determined by some physical measurement process. The purpose of this paper is then to study that, under such a setup and for a specific elliptic problem, how the lengthscale of the coarse data, which we refer to as the subsampled lengthscale, influences the accuracy of recovery, given limited computational budgets. Our analysis and experiments identify that, reducing the subsampling lengthscale may improve the accuracy, implying a guiding criterion for coarse-graining or data acquisition in this computationally constrained scenario, especially leading to direct insights for the implementation of the Gamblets method in the numerical homogenization literature. Moreover, reducing the lengthscale to zero may lead to a blow-up of approximation error if the target function does not have enough regularity, suggesting the need for a stronger prior assumption on the target function to be approximated. We introduce a singular weight function to deal with it, both theoretically and numerically. This work sheds light on the interplay of the lengthscale of coarse data, the computational costs, the regularity of the target function, and the accuracy of approximations and numerical simulations.

math.NA↗

arXiv2021

Solving and Learning Nonlinear PDEs with Gaussian Processes

We introduce a simple, rigorous, and unified framework for solving nonlinear partial differential equations (PDEs), and for solving inverse problems (IPs) involving the identification of parameters in PDEs, using the framework of Gaussian processes. The proposed approach: (1) provides a natural generalization of collocation kernel methods to nonlinear PDEs and IPs; (2) has guaranteed convergence for a very general class of PDEs, and comes equipped with a path to compute error bounds for specific PDE approximations; (3) inherits the state-of-the-art computational complexity of linear solvers for dense kernel matrices. The main idea of our method is to approximate the solution of a given PDE as the maximum a posteriori (MAP) estimator of a Gaussian process conditioned on solving the PDE at a finite number of collocation points. Although this optimization problem is infinite-dimensional, it can be reduced to a finite-dimensional one by introducing additional variables corresponding to the values of the derivatives of the solution at collocation points; this generalizes the representer theorem arising in Gaussian process regression. The reduced optimization problem has the form of a quadratic objective function subject to nonlinear constraints; it is solved with a variant of the Gauss--Newton method. The resulting algorithm (a) can be interpreted as solving successive linearizations of the nonlinear PDE, and (b) in practice is found to converge in a small number of iterations (2 to 10), for a wide range of PDEs. Most traditional approaches to IPs interleave parameter updates with numerical solution of the PDE; our algorithm solves for both parameter and PDE solution simultaneously. Experiments on nonlinear elliptic PDEs, Burgers' equation, a regularized Eikonal equation, and an IP for permeability identification in Darcy flow illustrate the efficacy and scope of our framework.

math.NA↗

arXiv2021

NTIRE 2021 Challenge on Perceptual Image Quality Assessment

This paper reports on the NTIRE 2021 challenge on perceptual image quality assessment (IQA), held in conjunction with the New Trends in Image Restoration and Enhancement workshop (NTIRE) workshop at CVPR 2021. As a new type of image processing technology, perceptual image processing algorithms based on Generative Adversarial Networks (GAN) have produced images with more realistic textures. These output images have completely different characteristics from traditional distortions, thus pose a new challenge for IQA methods to evaluate their visual quality. In comparison with previous IQA challenges, the training and testing datasets in this challenge include the outputs of perceptual image processing algorithms and the corresponding subjective scores. Thus they can be used to develop and evaluate IQA methods on GAN-based distortions. The challenge has 270 registered participants in total. In the final testing stage, 13 participating teams submitted their models and fact sheets. Almost all of them have achieved much better results than existing IQA methods, while the winning method can demonstrate state-of-the-art performance.

eess.IV↗

arXiv2021

Pre-Trained Models for Heterogeneous Information Networks

In network representation learning we learn how to represent heterogeneous information networks in a low-dimensional space so as to facilitate effective search, classification, and prediction solutions. Previous network representation learning methods typically require sufficient task-specific labeled data to address domain-specific problems. The trained model usually cannot be transferred to out-of-domain datasets. We propose a self-supervised pre-training and fine-tuning framework, PF-HIN, to capture the features of a heterogeneous information network. Unlike traditional network representation learning models that have to train the entire model all over again for every downstream task and dataset, PF-HIN only needs to fine-tune the model and a small number of extra task-specific parameters, thus improving model efficiency and effectiveness. During pre-training, we first transform the neighborhood of a given node into a sequence. PF-HIN is pre-trained based on two self-supervised tasks, masked node modeling and adjacent node prediction. We adopt deep bi-directional transformer encoders to train the model, and leverage factorized embedding parameterization and cross-layer parameter sharing to reduce the parameters. In the fine-tuning stage, we choose four benchmark downstream tasks, i.e., link prediction, similarity search, node classification, and node clustering. PF-HIN consistently and significantly outperforms state-of-the-art alternatives on each of these tasks, on four datasets.

cs.AI↗

arXiv2021

Sun Heated MeV-scale Dark Matter and the XENON1T Electron Recoil Excess

The XENON1T collaboration reported an excess of the low-energy electron recoil events between 1 and 7 keV. We explore the possibility to explain such an anomaly by the MeV-scale dark matter (DM) heated by the interior of the Sun due to the same DM-electron interaction as in the detector. The kinetic energies of heated DM particles can reach a few keV, and can potentially account for the excess signals detected by XENON1T. We study different form factors of the DM-electron interactions, $F(q)\propto q^i$ with $i=0,1,2$ and $q$ being the momentum exchange, and find that for all these cases the inclusion of the Sun-heated DM component improves the fit to the XENON1T data. The inferred DM-electron scattering cross section (at $q=αm_e$ where $α$ is the fine structure constant and $m_e$ is electron mass) is from $\sim 10^{-38}$~cm$^2$ (for $i=0$) to $\sim 10^{-42}$~cm$^2$ (for $i=2$). We also derive constraints on the DM-electron cross sections for different form factors, which are stronger than previous results with similar assumptions. We emphasize that the Sun-heated DM scenario relies on the minimum assumption on DM models, which serves as a general explanation of the XENON1T anomaly via DM-electron interaction. The spectrum of the Sun-heated DM is typically soft comparing to other boosted DM, so the small recoil events are expected to be abundant in this scenario. More sensitive direct detection experiments with lower thresholds can possibly distinguish this scenario with other boosted DM models or solar axion models.

hep-ph↗

arXiv2021

Exponential Convergence for Multiscale Linear Elliptic PDEs via Adaptive Edge Basis Functions

In this paper, we introduce a multiscale framework based on adaptive edge basis functions to solve second-order linear elliptic PDEs with rough coefficients. One of the main results is that we prove the proposed multiscale method achieves nearly exponential convergence in the approximation error with respect to the computational degrees of freedom. Our strategy is to perform an energy orthogonal decomposition of the solution space into a coarse scale component comprising $a$-harmonic functions in each element of the mesh, and a fine scale component named the bubble part that can be computed locally and efficiently. The coarse scale component depends entirely on function values on edges. Our approximation on each edge is made in the Lions-Magenes space $H_{00}^{1/2}(e)$, which we will demonstrate to be a natural and powerful choice. We construct edge basis functions using local oversampling and singular value decomposition. When local information of the right-hand side is adaptively incorporated into the edge basis functions, we prove a nearly exponential convergence rate of the approximation error. Numerical experiments validate and extend our theoretical analysis; in particular, we observe no obvious degradation in accuracy for high-contrast media problems.

math.NA↗

arXiv2021

Region-Adaptive Deformable Network for Image Quality Assessment

Image quality assessment (IQA) aims to assess the perceptual quality of images. The outputs of the IQA algorithms are expected to be consistent with human subjective perception. In image restoration and enhancement tasks, images generated by generative adversarial networks (GAN) can achieve better visual performance than traditional CNN-generated images, although they have spatial shift and texture noise. Unfortunately, the existing IQA methods have unsatisfactory performance on the GAN-based distortion partially because of their low tolerance to spatial misalignment. To this end, we propose the reference-oriented deformable convolution, which can improve the performance of an IQA network on GAN-based distortion by adaptively considering this misalignment. We further propose a patch-level attention module to enhance the interaction among different patch regions, which are processed independently in previous patch-based methods. The modified residual block is also proposed by applying modifications to the classic residual block to construct a patch-region-based baseline called WResNet. Equipping this baseline with the two proposed modules, we further propose Region-Adaptive Deformable Network (RADN). The experiment results on the NTIRE 2021 Perceptual Image Quality Assessment Challenge dataset show the superior performance of RADN, and the ensemble approach won fourth place in the final testing phase of the challenge. Code is available at https://github.com/IIGROUP/RADN.

cs.CV↗

arXiv2021

Consistency of Empirical Bayes And Kernel Flow For Hierarchical Parameter Estimation

Gaussian process regression has proven very powerful in statistics, machine learning and inverse problems. A crucial aspect of the success of this methodology, in a wide range of applications to complex and real-world problems, is hierarchical modeling and learning of hyperparameters. The purpose of this paper is to study two paradigms of learning hierarchical parameters: one is from the probabilistic Bayesian perspective, in particular, the empirical Bayes approach that has been largely used in Bayesian statistics; the other is from the deterministic and approximation theoretic view, and in particular the kernel flow algorithm that was proposed recently in the machine learning literature. Analysis of their consistency in the large data limit, as well as explicit identification of their implicit bias in parameter learning, are established in this paper for a Matérn-like model on the torus. A particular technical challenge we overcome is the learning of the regularity parameter in the Matérn-like field, for which consistency results have been very scarce in the spatial statistics literature. Moreover, we conduct extensive numerical experiments beyond the Matérn-like model, comparing the two algorithms further. These experiments demonstrate learning of other hierarchical parameters, such as amplitude and lengthscale; they also illustrate the setting of model misspecification in which the kernel flow approach could show superior performance to the more traditional empirical Bayes approach.

math.ST↗

arXiv2021

Fast Statistical Leverage Score Approximation in Kernel Ridge Regression

Nyström approximation is a fast randomized method that rapidly solves kernel ridge regression (KRR) problems through sub-sampling the n-by-n empirical kernel matrix appearing in the objective function. However, the performance of such a sub-sampling method heavily relies on correctly estimating the statistical leverage scores for forming the sampling distribution, which can be as costly as solving the original KRR. In this work, we propose a linear time (modulo poly-log terms) algorithm to accurately approximate the statistical leverage scores in the stationary-kernel-based KRR with theoretical guarantees. Particularly, by analyzing the first-order condition of the KRR objective, we derive an analytic formula, which depends on both the input distribution and the spectral density of stationary kernels, for capturing the non-uniformity of the statistical leverage scores. Numerical experiments demonstrate that with the same prediction accuracy our method is orders of magnitude more efficient than existing methods in selecting the representative sub-samples in the Nyström approximation.

stat.ML↗

arXiv2021

Testing the ALP-photon coupling with polarization measurements of Sagittarius A*

Ultra-light bosons such as axions or axion-like particles (ALPs), are promising candidates to solve the dark matter problem. A unique way to detect such ALPs is to search for the periodic oscillation feature of the position angles of linearly polarized photons emitted from the regions where ALP dark matter forms a solitonic core. In this work, we use the high-resolution polarimetric measurements of the radiation near the super-massive black hole (SMBH) in the center of the Milky Way, i.e., Sagittarius A$^\star$ (Sgr A$^\star$), by a sub-array of the Event Horizon Telescope to search for the ultra-light ALPs. We derive upper limits on the ALP-photon coupling of $\sim 10^{-12}{\rm GeV^{-1}}$ for ALP masses of $m\sim (10^{-19}-10^{-18})$eV. Our results are stronger than that derived from the observations of SN1987A and a population of supernovae in the mass window of ($10^{-19}-10^{-17}$)eV. Improved polarimetric measurements with the full Event Horizon Telescope can further strengthen the constraints.

astro-ph.HE↗

arXiv2020

A universal method for depositing patterned materials in-situ

Current techniques of patterned material deposition require separate steps for patterning and material deposition. The complexity and harsh working conditions post serious limitations for fabrication. Here, we introduce a novel single-step and easy-to-adapt method that can deposit materials in-situ. Its unique methodology is based on the semiconductor nanoparticle assisted photon-induced chemical reduction and optical trapping. This universal mechanism can be used for depositing a large selection of materials including metals, insulators and magnets, with quality on par with current technologies. Patterning with several materials together with optical-diffraction-limited resolution accuracy can be achieved from macroscopic to microscopic scale. Furthermore, the setup is naturally compatible with optical microscopy based measurements, thus sample characterisation and material deposition can be realised in-situ. Various devices fabricated with this method in 2D or 3D show it is ready for deployment in practical applications. This revolutionary method will provide a distinct tool in material technology.

cond-mat.mes-hall↗

arXiv2020

Function Approximation via The Subsampled Poincar\' e Inequality

Function approximation and recovery via some sampled data have long been studied in a wide array of applied mathematics and statistics fields. Analytic tools, such as the Poincaré inequality, have been handy for estimating the approximation errors in different scales. The purpose of this paper is to study a generalized Poincar\' e inequality, where the measurement function is of subsampled type, with a small but non-zero lengthscale that will be made precise. Our analysis identifies this inequality as a basic tool for function recovery problems. We discuss and demonstrate the optimality of the inequality concerning the subsampled lengthscale, connecting it to existing results in the literature. In application to function approximation problems, the approximation accuracy using different basis functions and under different regularity assumptions is established by using the subsampled Poincaré inequality. We observe that the error bound blows up as the subsampled lengthscale approaches zero, due to the fact that the underlying function is not regular enough to have well-defined pointwise values. A weighted version of the Poincar\' e inequality is proposed to address this problem; its optimality is also discussed.

math.NA↗

arXiv2020

Towards Question-based Recommender Systems

Conversational and question-based recommender systems have gained increasing attention in recent years, with users enabled to converse with the system and better control recommendations. Nevertheless, research in the field is still limited, compared to traditional recommender systems. In this work, we propose a novel Question-based recommendation method, Qrec, to assist users to find items interactively, by answering automatically constructed and algorithmically chosen questions. Previous conversational recommender systems ask users to express their preferences over items or item facets. Our model, instead, asks users to express their preferences over descriptive item features. The model is first trained offline by a novel matrix factorization algorithm, and then iteratively updates the user and item latent factors online by a closed-form solution based on the user answers. Meanwhile, our model infers the underlying user belief and preferences over items to learn an optimal question-asking strategy by using Generalized Binary Search, so as to ask a sequence of questions to the user. Our experimental results demonstrate that our proposed matrix factorization model outperforms the traditional Probabilistic Matrix Factorization model. Further, our proposed Qrec model can greatly improve the performance of state-of-the-art baselines, and it is also effective in the case of cold-start user and item recommendations.

cs.IR↗

arXiv2020

Optimal transport natural gradient for statistical manifolds with continuous sample space

We study the Wasserstein natural gradient in parametric statistical models with continuous sample spaces. Our approach is to pull back the $L^2$-Wasserstein metric tensor in the probability density space to a parameter space, equipping the latter with a positive definite metric tensor, under which it becomes a Riemannian manifold, named the Wasserstein statistical manifold. In general, it is not a totally geodesic sub-manifold of the density space, and therefore its geodesics will differ from the Wasserstein geodesics, except for the well-known Gaussian distribution case, a fact which can also be validated under our framework. We use the sub-manifold geometry to derive a gradient flow and natural gradient descent method in the parameter space. When parametrized densities lie in $\bR$, the induced metric tensor establishes an explicit formula. In optimization problems, we observe that the natural gradient descent outperforms the standard gradient descent when the Wasserstein distance is the objective function. In such a case, we prove that the resulting algorithm behaves similarly to the Newton method in the asymptotic regime. The proof calculates the exact Hessian formula for the Wasserstein distance, which further motivates another preconditioner for the optimization process. To the end, we present examples to illustrate the effectiveness of the natural gradient in several parametric statistical models, including the Gaussian measure, Gaussian mixture, Gamma distribution, and Laplace distribution.

math.OC↗