arXiv subjects

Lin Xiao

Publications and source records attributed to Lin Xiao.

At least 55 records · Page 3Linked to original sources

arXiv2023

JWST Discovery of Dust Reservoirs in Nearby Type IIP Supernovae 2004et and 2017eaw

Supernova (SN) explosions have been sought for decades as a possible source of dust in the Universe, providing the seeds of galaxies, stars, and planetary systems. SN 1987A offers one of the most promising examples of significant SN dust formation, but until the James Webb Space Telescope (JWST), instruments have traditionally lacked the sensitivity at both late times (>1 yr post-explosion) and longer wavelengths (i.e., >10 um) to detect analogous dust reservoirs. Here we present JWST/MIRI observations of two historic Type IIP SNe, 2004et and SN 2017eaw, at nearly 18 and 5 yr post-explosion, respectively. We fit the spectral energy distributions as functions of dust mass and temperature, from which we are able to constrain the dust geometry, origin, and heating mechanism. We place a 90% confidence lower limit on the dust masses for SNe 2004et and 2017eaw of >0.014 and >4e-4 M_sun, respectively. More dust may exist at even colder temperatures or may be obscured by high optical depths. We conclude dust formation in the ejecta to be the most plausible and consistent scenario. The observed dust is radiatively heated to ~100-150 K by ongoing shock interaction with the circumstellar medium. Regardless of the best fit or heating mechanism adopted, the inferred dust mass for SN 2004et is the second highest (next to SN 1987A) inferred dust mass in extragalactic SNe thus far, promoting the prospect of SNe as potential significant sources of dust in the Universe.

astro-ph.HE↗

arXiv2022

Prospects of Searching for Type Ia Supernovae with 2.5-m Wide Field Survey Telescope

Type Ia Supernovae (SNe Ia) are the thermonuclear explosion of a carbon-oxygen white dwarf (WD) and are well-known as a distance indicator. However, it is still unclear how WDs increase their mass near the Chandrasekhar limit and how the thermonuclear runaway happens. The observational clues associated with these open questions, such as the photometric data within hours to days since the explosion, are scarce. Thus, an essential way is to discover SNe Ia at specific epochs with optimal surveys. The 2.5-m Wide Field Survey Telescope (WFST) is an upcoming survey facility deployed in western China. In this paper, we assess the detecability of SNe Ia with mock observations of WFST. Followed by the volumetric rate, we generate a spectral series of SNe Ia based on a data-based model and introduce the line-of-sight extinction to calculate the brightness from the observer. By comparing with the detection limit of WFST, which is affected by the observing conditions, we can count the number of SNe Ia discovered by mock WFST observations. We expect that WFST can find more than $3.0\times10^{4}$ pre-maximum SNe Ia within one-year running. In particular, WFST could discover about 45 bright SNe Ia, 99 early-phase SNe Ia, or $1.1\times10^{4}$ well-observed SNe Ia with the hypothesized Wide, Deep, or Medium mode, respectively, suggesting WFST will be an influential facility in time-domain astronomy.

astro-ph.SR↗

arXiv2022

Pairwise Instance Relation Augmentation for Long-tailed Multi-label Text Classification

Multi-label text classification (MLTC) is one of the key tasks in natural language processing. It aims to assign multiple target labels to one document. Due to the uneven popularity of labels, the number of documents per label follows a long-tailed distribution in most cases. It is much more challenging to learn classifiers for data-scarce tail labels than for data-rich head labels. The main reason is that head labels usually have sufficient information, e.g., a large intra-class diversity, while tail labels do not. In response, we propose a Pairwise Instance Relation Augmentation Network (PIRAN) to augment tailed-label documents for balancing tail labels and head labels. PIRAN consists of a relation collector and an instance generator. The former aims to extract the document pairwise relations from head labels. Taking these relations as perturbations, the latter tries to generate new document instances in high-level feature space around the limited given tailed-label instances. Meanwhile, two regularizers (diversity and consistency) are designed to constrain the generation process. The consistency-regularizer encourages the variance of tail labels to be close to head labels and further balances the whole datasets. And diversity-regularizer makes sure the generated instances have diversity and avoids generating redundant instances. Extensive experimental results on three benchmark datasets demonstrate that PIRAN consistently outperforms the SOTA methods, and dramatically improves the performance of tail labels.

cs.CL↗

arXiv2022

Faster Last-iterate Convergence of Policy Optimization in Zero-Sum Markov Games

Multi-Agent Reinforcement Learning (MARL) -- where multiple agents learn to interact in a shared dynamic environment -- permeates across a wide range of critical applications. While there has been substantial progress on understanding the global convergence of policy optimization methods in single-agent RL, designing and analysis of efficient policy optimization algorithms in the MARL setting present significant challenges, which unfortunately, remain highly inadequately addressed by existing theory. In this paper, we focus on the most basic setting of competitive multi-agent RL, namely two-player zero-sum Markov games, and study equilibrium finding algorithms in both the infinite-horizon discounted setting and the finite-horizon episodic setting. We propose a single-loop policy optimization method with symmetric updates from both agents, where the policy is updated via the entropy-regularized optimistic multiplicative weights update (OMWU) method and the value is updated on a slower timescale. We show that, in the full-information tabular setting, the proposed method achieves a finite-time last-iterate linear convergence to the quantal response equilibrium of the regularized problem, which translates to a sublinear last-iterate convergence to the Nash equilibrium by controlling the amount of regularization. Our convergence results improve upon the best known iteration complexities, and lead to a better understanding of policy optimization in competitive Markov games.

cs.GT↗

arXiv2022

BiT: Robustly Binarized Multi-distilled Transformer

Modern pre-trained transformers have rapidly advanced the state-of-the-art in machine learning, but have also grown in parameters and computational complexity, making them increasingly difficult to deploy in resource-constrained environments. Binarization of the weights and activations of the network can significantly alleviate these issues, however, is technically challenging from an optimization perspective. In this work, we identify a series of improvements that enables binary transformers at a much higher accuracy than what was possible previously. These include a two-set binarization scheme, a novel elastic binary activation function with learned parameters, and a method to quantize a network to its limit by successively distilling higher precision models into lower precision students. These approaches allow for the first time, fully binarized transformer models that are at a practical level of accuracy, approaching a full-precision BERT baseline on the GLUE language understanding benchmark within as little as 5.9%. Code and models are available at: https://github.com/facebookresearch/bit.

cs.LG↗

arXiv2022

FedShuffle: Recipes for Better Use of Local Work in Federated Learning

The practice of applying several local updates before aggregation across clients has been empirically shown to be a successful approach to overcoming the communication bottleneck in Federated Learning (FL). Such methods are usually implemented by having clients perform one or more epochs of local training per round while randomly reshuffling their finite dataset in each epoch. Data imbalance, where clients have different numbers of local training samples, is ubiquitous in FL applications, resulting in different clients performing different numbers of local updates in each round. In this work, we propose a general recipe, FedShuffle, that better utilizes the local updates in FL, especially in this regime encompassing random reshuffling and heterogeneity. FedShuffle is the first local update method with theoretical convergence guarantees that incorporates random reshuffling, data imbalance, and client sampling - features that are essential in large-scale cross-device FL. We present a comprehensive theoretical analysis of FedShuffle and show, both theoretically and empirically, that it does not suffer from the objective function mismatch that is present in FL methods that assume homogeneous updates in heterogeneous FL setups, such as FedAvg (McMahan et al., 2017). In addition, by combining the ingredients above, FedShuffle improves upon FedNova (Wang et al., 2020), which was previously proposed to solve this mismatch. Similar to Mime (Karimireddy et al., 2020), we show that FedShuffle with momentum variance reduction (Cutkosky & Orabona, 2019) improves upon non-local methods under a Hessian similarity assumption.

cs.LG↗

arXiv2022

Federated Learning with Partial Model Personalization

We consider two federated learning algorithms for training partially personalized models, where the shared and personal parameters are updated either simultaneously or alternately on the devices. Both algorithms have been proposed in the literature, but their convergence properties are not fully understood, especially for the alternating variant. We provide convergence analyses of both algorithms in the general nonconvex setting with partial participation and delineate the regime where one dominates the other. Our experiments on real-world image, text, and speech datasets demonstrate that (a) partial personalization can obtain most of the benefits of full model personalization with a small fraction of personal parameters, and, (b) the alternating update algorithm often outperforms the simultaneous update algorithm by a small but consistent margin.

cs.LG↗

arXiv2022

Grad-GradaGrad? A Non-Monotone Adaptive Stochastic Gradient Method

The classical AdaGrad method adapts the learning rate by dividing by the square root of a sum of squared gradients. Because this sum on the denominator is increasing, the method can only decrease step sizes over time, and requires a learning rate scaling hyper-parameter to be carefully tuned. To overcome this restriction, we introduce GradaGrad, a method in the same family that naturally grows or shrinks the learning rate based on a different accumulation in the denominator, one that can both increase and decrease. We show that it obeys a similar convergence rate as AdaGrad and demonstrate its non-monotone adaptation capability with experiments.

cs.LG↗

arXiv2022

On Continual Model Refinement in Out-of-Distribution Data Streams

Real-world natural language processing (NLP) models need to be continually updated to fix the prediction errors in out-of-distribution (OOD) data streams while overcoming catastrophic forgetting. However, existing continual learning (CL) problem setups cannot cover such a realistic and complex scenario. In response to this, we propose a new CL problem formulation dubbed continual model refinement (CMR). Compared to prior CL settings, CMR is more practical and introduces unique challenges (boundary-agnostic and non-stationary distribution shift, diverse mixtures of multiple OOD data clusters, error-centric streams, etc.). We extend several existing CL approaches to the CMR setting and evaluate them extensively. For benchmarking and analysis, we propose a general sampling algorithm to obtain dynamic OOD data streams with controllable non-stationarity, as well as a suite of metrics measuring various aspects of online performance. Our experiments and detailed analysis reveal the promise and challenges of the CMR problem, supporting that studying CMR in dynamic OOD streams can benefit the longevity of deployed NLP models in production.

cs.CL↗

arXiv2022

A Mid-infrared Study of Superluminous Supernovae

We present the mid-infrared (MIR) light curves (LC) of 10 superluminous supernovae (SLSNe) at $z<0.12$ based on WISE data at 3.4 and 4.6 $μ$m. Three of them, including PS15br, SN 2017ens, and SN 2017err show rebrightening which started at 200--400 days and ended at 600--1000 days, indicating the presence of dust. In four of the left seven SLSNe, dust emission was detected with monochromatic luminosities of $10^7\sim10^8\ L_\odot$ at epochs of 100--500 days based on MIR colors $W1-W2\sim1$. Among the three SLSNe which show rebrightening, we further analysed PS15br and SN 2017ens. We modeled the SEDs at 500--700 days, which gives dust temperatures of 600--1100 K, dust masses of $\gtrsim 10^{-2}\ M_\odot$, and luminosities of $10^8\sim10^9$ $L_\odot$. Considering the time delay and the huge amount of energy released, the emitting dust can hardly be pre-existing dust heated whether collisionally by shocks or radiatively by peak SLSN luminosity or shock emission. Instead, it can be newly formed dust additionally heated by the interaction of circum-stellar medium, indicated by features in their spectra and slowly declining bolometric LCs. The dust masses appear to be ten times greater than those formed in normal core-collapse supernovae at similar epochs. Combining with the analysis of SN 2018bsz by Chen et al. (2022), we suggest that SLSNe have higher dust formation efficiency, although future observations are required to reach a final conclusion.

astro-ph.HE↗

arXiv2022

On the Convergence Rates of Policy Gradient Methods

We consider infinite-horizon discounted Markov decision problems with finite state and action spaces and study the convergence rates of the projected policy gradient method and a general class of policy mirror descent methods, all with direct parametrization in the policy space. First, we develop a theory of weak gradient-mapping dominance and use it to prove sharper sublinear convergence rate of the projected policy gradient method. Then we show that with geometrically increasing step sizes, a general class of policy mirror descent methods, including the natural policy gradient method and a projected Q-descent method, all enjoy a linear rate of convergence without relying on entropy or other strongly convex regularization. Finally, we also analyze the convergence rate of an inexact policy mirror descent method and estimate its sample complexity under a simple generative model.

math.OC↗

arXiv2022

Chemical variations across the TMC-1 boundary: molecular tracers from translucent phase to dense phase

We investigated the chemical evolutions of gas phase and grain surface species across the Taurus molecular cloud-1 (TMC-1) filament from translucent phase to dense phase. By comparing observations with modeling results from an up-to-date chemical network, we examined the conversion processes for the carbon-, oxygen-, nitrogen- and sulfur-bearing species, i.e.from their initial atomic form to their main molecular reservoir form both in the gas phase and on the grain surface. The conversion processes were found to depend on the species and A$_V$. The effect of initial carbon to oxygen elemental abundances ratio (C/O) by varying O on the chemistry was explored, and an initial carbon elemental abundance of 2.5 $\times$ 10$^{-4}$ and a C/O ratio of 0.5 could best reproduce the abundances of most observed molecules at TMC-1 CP, where more than 90 molecules have been identified. Based on the TMC-1 condition, we predicted a varied grain ice composition during the evolutions of molecular clouds, with H$_2$O ice as the dominant ice composition at A$_V$ $>$ 4 mag, CO$_2$ ice as the dominant ice composition at A$_V$ $<$ 4 mag, while CO ice severely decreased at A$_V$ around 4--5 mag.

astro-ph.GA↗

arXiv2021

Accelerated Bregman Proximal Gradient Methods for Relatively Smooth Convex Optimization

We consider the problem of minimizing the sum of two convex functions: one is differentiable and relatively smooth with respect to a reference convex function, and the other can be nondifferentiable but simple to optimize. We investigate a triangle scaling property of the Bregman distance generated by the reference convex function and present accelerated Bregman proximal gradient (ABPG) methods that attain an $O(k^{-γ})$ convergence rate, where $γ\in(0,2]$ is the triangle scaling exponent (TSE) of the Bregman distance. For the Euclidean distance, we have $γ=2$ and recover the convergence rate of Nesterov's accelerated gradient methods. For non-Euclidean Bregman distances, the TSE can be much smaller (say $γ\leq 1$), but we show that a relaxed definition of intrinsic TSE is always equal to 2. We exploit the intrinsic TSE to develop adaptive ABPG methods that converge much faster in practice. Although theoretical guarantees on a fast convergence rate seem to be out of reach in general, our methods obtain empirical $O(k^{-2})$ rates in numerical experiments on several applications and provide posterior numerical certificates for the fast rates.

math.OC↗

arXiv2021

Cross-Platform Simulation Architecture with application to truck platooning impact assessment

Simulation-based traffic impact assessment studies of advanced technologies such as truck platooning need to be carried out to ascertain their benefits for traffic efficiency, safety and environment. To reduce uncertainty in the results of such simulation-based studies, the same simulation studies can be performed in different simulation software. Many traffic simulation software packages (Aimsun, SymuVia, Vissim, SUMO) are currently available for traffic impact assessment of new technologies such as truckplatooning. However, to fully model and simulate the functionalities of such advanced technologies in different simulation environments, several extensions need to be made to the simulation platforms. In most cases, these extensions have to be programmed in different programming languages (C++, Python) and each simulator has its own simulator specific API. This makes it difficult to reuse software written for a specific functionality in one simulation platform in a different simulation platform. To overcome this issue, this paper presents a novel architecture for cross-platform simulation. The architecture is designed such that a specific functionality such as truck-platooning or any other functionality is made platform independent. We designed a cross-platform architecture for simulating a truck-platooning functionality using Vissim and SymuVia simulation software to determine the traffic flow effects of multi-brand truck platooning in the context of the EU project ENSEMBLE. In this draft paper, we present the structure of the framework as well as some preliminary results from a simple simulation performed with the cross-platform simulator.

eess.SY↗

arXiv2021

Stochastic Variance-Reduced Prox-Linear Algorithms for Nonconvex Composite Optimization

We consider minimization of composite functions of the form $f(g(x))+h(x)$, where $f$ and $h$ are convex functions (which can be nonsmooth) and $g$ is a smooth vector mapping. In addition, we assume that $g$ is the average of finite number of component mappings or the expectation over a family of random component mappings. We propose a class of stochastic variance-reduced prox-linear algorithms for solving such problems and bound their sample complexities for finding an $ε$-stationary point in terms of the total number of evaluations of the component mappings and their Jacobians. When $g$ is a finite average of $N$ components, we obtain sample complexity $\mathcal{O}(N+ N^{4/5}ε^{-1})$ for both mapping and Jacobian evaluations. When $g$ is a general expectation, we obtain sample complexities of $\mathcal{O}(ε^{-5/2})$ and $\mathcal{O}(ε^{-3/2})$ for component mappings and their Jacobians respectively. If in addition $f$ is smooth, then improved sample complexities of $\mathcal{O}(N+N^{1/2}ε^{-1})$ and $\mathcal{O}(ε^{-3/2})$ are derived for $g$ being a finite average and a general expectation respectively, for both component mapping and Jacobian evaluations.

math.OC↗

arXiv2021

Multi-Level Composite Stochastic Optimization via Nested Variance Reduction

We consider multi-level composite optimization problems where each mapping in the composition is the expectation over a family of random smooth mappings or the sum of some finite number of smooth mappings. We present a normalized proximal approximate gradient (NPAG) method where the approximate gradients are obtained via nested stochastic variance reduction. In order to find an approximate stationary point where the expected norm of its gradient mapping is less than $ε$, the total sample complexity of our method is $O(ε^{-3})$ in the expectation case, and $O(N+\sqrt{N}ε^{-2})$ in the finite-sum case where $N$ is the total number of functions across all composition levels. In addition, the dependence of our total sample complexity on the number of composition levels is polynomial, rather than exponential as in previous work.

math.OC↗

arXiv2021

Infrared Echoes of Optical Tidal Disruption Events: ~1% Dust Covering Factor or Less at sub-parsec Scale

The past decade has experienced an explosive increase of optically-discovered tidal disruption events (TDEs) with the advent of modern time-domain surveys. However, we still lack a comprehensive observational view of their infrared (IR) echoes in spite of individual detections. To this end, we have conducted a statistical study of IR variability of the 23 optical TDEs discovered between 2009 and 2018 utilizing the full public dataset of Wide-field Infrared Survey Explorer. The detection of variability is performed on the difference images, yielding out 11 objects with significant (>$3σ$) variability in at least one band while dust emission can be only fitted in 8 objects. Their peak dust luminosity is around $10^{41}$-$10^{42}$ erg/s, corresponding to a dust covering factor $f_c\sim0.01$ at scale of sub-parsec. The only exception is the disputed source ASASSN-15lh, which shows an ultra-high dust luminosity ($\sim10^{43.5}$ erg/s) and make its nature even elusive. Other non-detected objects show even lower $f_c$, which could be one more order of magnitude lower. The derived $f_c$ is generally much smaller than those of dusty tori in active galactic nuclei (AGNs), suggesting either a dearth of dust or a geometrically thin and flat disk in the vicinity of SMBHs. Our results also indicate that the optical TDE sample (post-starburst galaxies overrepresented) is seriously biased to events with little dust at sub-pc scale while TDEs in dusty star-forming systems could be more efficiently unveiled by IR echoes.

astro-ph.GA↗

arXiv2021

Does Head Label Help for Long-Tailed Multi-Label Text Classification

Multi-label text classification (MLTC) aims to annotate documents with the most relevant labels from a number of candidate labels. In real applications, the distribution of label frequency often exhibits a long tail, i.e., a few labels are associated with a large number of documents (a.k.a. head labels), while a large fraction of labels are associated with a small number of documents (a.k.a. tail labels). To address the challenge of insufficient training data on tail label classification, we propose a Head-to-Tail Network (HTTN) to transfer the meta-knowledge from the data-rich head labels to data-poor tail labels. The meta-knowledge is the mapping from few-shot network parameters to many-shot network parameters, which aims to promote the generalizability of tail classifiers. Extensive experimental results on three benchmark datasets demonstrate that HTTN consistently outperforms the state-of-the-art methods. The code and hyper-parameter settings are released for reproducibility

cs.CL↗