arXiv subjects

Zhiyuan Li

Publications and source records attributed to Zhiyuan Li.

At least 235 records · Page 13Linked to original sources

arXiv2024

Uncovering an Excess of X-ray Point Sources in the Halos of Virgo Late-type Galaxies

We present a systematic search for extraplanar X-ray point sources around 19 late-type, highly inclined disk galaxies residing in the Virgo cluster, based on archival Chandra observations reaching a source detection sensitivity of $L\rm(0.5- 8~keV)\sim10^{38}\rm~erg~s^{-1}$. Based on the cumulative source surface density distribution as a function of projected vertical distance from the disk mid-plane, we identify a statistically significant ($\sim3.3σ$) excess of $\sim20$ X-ray sources within a projected vertical off-disk distance of $0.92'-2.5'$ ($\sim4.4-12\ \mathrm{kpc}$), the presence of which cannot be explained by the bulk stellar content of the individual galaxies, nor by the cosmic X-ray background. On the other hand, there is no significant evidence for an excess of extraplanar X-ray sources in a comparison sample of field late-type edge-on galaxies, for which Chandra observations reaching a similar source detection sensitivity are available. We discuss possible origins for the observed excess, which include low-mass X-ray binaries (LMXBs) associated with globular clusters, supernova-kicked LMXBs, high-mass X-ray binaries born in recent star formation induced by ram pressure stripping of the disk gas, as well as a class of intra-cluster X-ray sources previously identified around early-type member galaxies of Virgo. We find that none of these X-ray populations can naturally dominate the observed extraplanar excess, although supernova-kicked LMXBs and the effect of ram pressure are most likely relevant

astro-ph.GA↗

arXiv2024

The Marginal Value of Momentum for Small Learning Rate SGD

Momentum is known to accelerate the convergence of gradient descent in strongly convex settings without stochastic gradient noise. In stochastic optimization, such as training neural networks, folklore suggests that momentum may help deep learning optimization by reducing the variance of the stochastic gradient update, but previous theoretical analyses do not find momentum to offer any provable acceleration. Theoretical results in this paper clarify the role of momentum in stochastic settings where the learning rate is small and gradient noise is the dominant source of instability, suggesting that SGD with and without momentum behave similarly in the short and long time horizons. Experiments show that momentum indeed has limited benefits for both optimization and generalization in practical training regimes where the optimal learning rate is not very large, including small- to medium-batch training from scratch on ImageNet and fine-tuning language models on downstream tasks.

cs.LG↗

arXiv2024

An X-ray Census of Active Galactic Nuclei in the Virgo and Fornax Clusters of Galaxies with SRG/eROSITA

We present a uniform and sensitive X-ray census of active galactic nuclei (AGNs) in the two nearest galaxy clusters, Virgo and Fornax, utilizing the newly released X-ray source catalogs from the first all-sky scan of SRG/eROSITA. A total of 50 and 10 X-ray sources are found positionally coincident with the nuclei of member galaxies in Virgo and Fornax, respectively, down to a 0.2-2.3 keV luminosity of $\sim10^{39}\rm~erg~s^{-1}$ and reaching out to a projected distance well beyond the virial radius of both clusters. The majority of the nuclear X-ray sources are newly identified. There is weak evidence that the nuclear X-ray sources are preferentially found in late-type hosts. Several hosts are dwarf galaxies with a stellar mass below $\sim10^{9}\rm~M_\odot$. We find that contamination by non-nuclear X-ray emission can be neglected in most cases, indicating the dominance of a genuine AGN. In the meantime, no nuclear X-ray source exhibits a luminosity higher than a few times $10^{41}\rm~erg~s^{-1}$. The X-ray AGN occupation rate is only $\sim$ 3% in both clusters, apparently much lower than that in field galaxies inferred from previous X-ray studies. Both aspects suggest that the cluster environment effectively suppresses AGN activity. The findings of this census have important implications on the interplay between galaxies and their central massive black holes in cluster environments.

astro-ph.GA↗

arXiv2024

Implicit Bias of AdamW: $\ell_\infty$ Norm Constrained Optimization

Adam with decoupled weight decay, also known as AdamW, is widely acclaimed for its superior performance in language modeling tasks, surpassing Adam with $\ell_2$ regularization in terms of generalization and optimization. However, this advantage is not theoretically well-understood. One challenge here is that though intuitively Adam with $\ell_2$ regularization optimizes the $\ell_2$ regularized loss, it is not clear if AdamW optimizes a specific objective. In this work, we make progress toward understanding the benefit of AdamW by showing that it implicitly performs constrained optimization. More concretely, we show in the full-batch setting, if AdamW converges with any non-increasing learning rate schedule whose partial sum diverges, it must converge to a KKT point of the original loss under the constraint that the $\ell_\infty$ norm of the parameter is bounded by the inverse of the weight decay factor. This result is built on the observation that Adam can be viewed as a smoothed version of SignGD, which is the normalized steepest descent with respect to $\ell_\infty$ norm, and a surprising connection between normalized steepest descent with weight decay and Frank-Wolfe.

cs.LG↗

arXiv2024

Dichotomy of Early and Late Phase Implicit Biases Can Provably Induce Grokking

Recent work by Power et al. (2022) highlighted a surprising "grokking" phenomenon in learning arithmetic tasks: a neural net first "memorizes" the training set, resulting in perfect training accuracy but near-random test accuracy, and after training for sufficiently longer, it suddenly transitions to perfect test accuracy. This paper studies the grokking phenomenon in theoretical setups and shows that it can be induced by a dichotomy of early and late phase implicit biases. Specifically, when training homogeneous neural nets with large initialization and small weight decay on both classification and regression tasks, we prove that the training process gets trapped at a solution corresponding to a kernel predictor for a long time, and then a very sharp transition to min-norm/max-margin predictors occurs, leading to a dramatic change in test accuracy.

cs.LG↗

arXiv2024

CHANG-ES. XXX. 10 kpc Radio Lobes in The Sombrero Galaxy

We report the discovery of the 10 kilo-parsec (kpc) scale radio lobes in the Sombrero galaxy (NGC 4594), using data from the Continuum Halos in Nearby Galaxies - an Expanded Very Large Array (VLA) Survey (CHANG-ES) project. We further examine the balance between the magnetic pressure inside the lobes and the thermal pressure of the ambient hot gas. At the radii $r$ of ~(1-10) kpc, the magnetic pressure inside the lobes and the thermal pressure of the ambient hot gas are generally in balance. This implies that the jets could expand into the surroundings at least to r ~ 10 kpc. The feedback from the active galactic nucleus (AGN) jet responsible for the large-scale lobes may help to explain the unusually high X-ray luminosity of this massive quiescent isolated disk galaxy, although more theoretical work is needed to further examine this possibility.

astro-ph.GA↗

arXiv2024

Sophia: A Scalable Stochastic Second-order Optimizer for Language Model Pre-training

Given the massive cost of language model pre-training, a non-trivial improvement of the optimization algorithm would lead to a material reduction on the time and cost of training. Adam and its variants have been state-of-the-art for years, and more sophisticated second-order (Hessian-based) optimizers often incur too much per-step overhead. In this paper, we propose Sophia, Second-order Clipped Stochastic Optimization, a simple scalable second-order optimizer that uses a light-weight estimate of the diagonal Hessian as the pre-conditioner. The update is the moving average of the gradients divided by the moving average of the estimated Hessian, followed by element-wise clipping. The clipping controls the worst-case update size and tames the negative impact of non-convexity and rapid change of Hessian along the trajectory. Sophia only estimates the diagonal Hessian every handful of iterations, which has negligible average per-step time and memory overhead. On language modeling with GPT models of sizes ranging from 125M to 1.5B, Sophia achieves a 2x speed-up compared to Adam in the number of steps, total compute, and wall-clock time, achieving the same perplexity with 50% fewer steps, less total compute, and reduced wall-clock time. Theoretically, we show that Sophia, in a much simplified setting, adapts to the heterogeneous curvatures in different parameter dimensions, and thus has a run-time bound that does not depend on the condition number of the loss.

cs.LG↗

arXiv2024

Observational Evidence for Hot Wind Impact on pc-scale in Low-luminosity Active Galactic Nucleus

Supermassive black holes in galaxies spend majority of their lifetime in the low-luminosity regime, powered by hot accretion flow. Strong winds launched from the hot accretion flow have the potential to play an important role in active galactic nuclei (AGN) feedback. Direct observational evidence for these hot winds with temperature around 10 keV, has been obtained through the detection of highly ionized iron emission lines with Doppler shifts in two prototypical low-luminosity AGNs, namely M81* and NGC 7213. In this work, we further identify blueshifted H-like O/Ne emission lines in the soft X-ray spectra of these two sources. These lines are interpreted to be associated with additional outflowing components possessing velocity around several $10^3$ km/s and lower temperature (~0.2-0.4 keV). Blue-shifted velocity and the X-ray intensity of these additional outflowing components are hard to be explained by previously detected hot wind freely propagating to larger radii. Through detailed numerical simulations, we find the newly detected blue-shifted emission lines would come from circumnuclear gas shock-heated by the hot wind instead. Hot wind can provide larger ram pressure force on the clumpy circumnuclear gas than the gravitational force from central black hole, effectively impeding the black hole accretion of gas. Our results provide strong evidences for the energy and momentum feedback by the hot AGN wind.

astro-ph.HE↗

arXiv2024

Traditional Transformation Theory Guided Model for Learned Image Compression

Recently, many deep image compression methods have been proposed and achieved remarkable performance. However, these methods are dedicated to optimizing the compression performance and speed at medium and high bitrates, while research on ultra low bitrates is limited. In this work, we propose a ultra low bitrates enhanced invertible encoding network guided by traditional transformation theory, experiments show that our codec outperforms existing methods in both compression and reconstruction performance. Specifically, we introduce the Block Discrete Cosine Transformation to model the sparsity of features and employ traditional Haar transformation to improve the reconstruction performance of the model without increasing the bitstream cost.

eess.IV↗

arXiv2024

A Fermi-LAT Study of Globular Cluster Dynamical Evolution in the Milky Way: Millisecond Pulsars as the Probe

Using archival {\it Fermi}-LAT data with a time span of $\sim12$ years, we study the population of Millisecond Pulsars (MSPs) in Globular Clusters (GlCs) and investigate their dependence on cluster dynamical evolution in the Milky Way Galaxy. We show that the $γ$-ray luminosity ($L_γ$) and emissivity ($ε_γ=L_γ/M$) are good indicators of the population and abundance of MSPs in GlCs, and they are highly dependent on the dynamical evolution history of the host clusters. Specifically speaking, the dynamically older GlCs with more compact structures are more likely to have larger $L_γ$ and $ε_γ$, and these trends can be summarized as strong correlations with cluster stellar encounter rate $Γ$ and the specific encounter rate ($Λ=Γ/M$), with $L_γ\propto Γ^{0.70\pm0.11}$ and $ε_γ\propto Λ^{0.73\pm0.13}$ for dynamically normal GlCs. However, as GlCs evolve into deep core collapse, these trends are found to be reversed, implying that strong encounters may have lead to the ejection of MSPs from core-collapsed Systems. Besides, the GlCs are found to exhibit larger $ε_γ$ with increasing stellar mass function slope, decreasing tidal radius and distances from the Galactic Center (GC). These correlations indicate that, as GlCs losing kinetic energy and spiral in towards GC, tidal stripping and mass segregation have a preference in leading to the loss of normal stars from GlCs, while MSPs are more likely to concentrate to cluster center and be deposited into the GC. Moreover, we gauge $ε_γ$ of GlCs is $\sim10-1000$ times larger than the Galactic bulge, the latter is thought to reside thousands of unresolved MSPs and may responsible for the GC $γ$-ray excess, which support that GlCs are generous contributors to the population of MSPs in the GC.

astro-ph.GA↗

arXiv2024

Backpropagation Through Agents

A fundamental challenge in multi-agent reinforcement learning (MARL) is to learn the joint policy in an extremely large search space, which grows exponentially with the number of agents. Moreover, fully decentralized policy factorization significantly restricts the search space, which may lead to sub-optimal policies. In contrast, the auto-regressive joint policy can represent a much richer class of joint policies by factorizing the joint policy into the product of a series of conditional individual policies. While such factorization introduces the action dependency among agents explicitly in sequential execution, it does not take full advantage of the dependency during learning. In particular, the subsequent agents do not give the preceding agents feedback about their decisions. In this paper, we propose a new framework Back-Propagation Through Agents (BPTA) that directly accounts for both agents' own policy updates and the learning of their dependent counterparts. This is achieved by propagating the feedback through action chains. With the proposed framework, our Bidirectional Proximal Policy Optimisation (BPPO) outperforms the state-of-the-art methods. Extensive experiments on matrix games, StarCraftII v2, Multi-agent MuJoCo, and Google Research Football demonstrate the effectiveness of the proposed method.

cs.MA↗

arXiv2023

Joint Self-Supervised and Supervised Contrastive Learning for Multimodal MRI Data: Towards Predicting Abnormal Neurodevelopment

The integration of different imaging modalities, such as structural, diffusion tensor, and functional magnetic resonance imaging, with deep learning models has yielded promising outcomes in discerning phenotypic characteristics and enhancing disease diagnosis. The development of such a technique hinges on the efficient fusion of heterogeneous multimodal features, which initially reside within distinct representation spaces. Naively fusing the multimodal features does not adequately capture the complementary information and could even produce redundancy. In this work, we present a novel joint self-supervised and supervised contrastive learning method to learn the robust latent feature representation from multimodal MRI data, allowing the projection of heterogeneous features into a shared common space, and thereby amalgamating both complementary and analogous information across various modalities and among similar subjects. We performed a comparative analysis between our proposed method and alternative deep multimodal learning approaches. Through extensive experiments on two independent datasets, the results demonstrated that our method is significantly superior to several other deep multimodal learning methods in predicting abnormal neurodevelopment. Our method has the capability to facilitate computer-aided diagnosis within clinical practice, harnessing the power of multimodal data.

eess.IV↗

arXiv2023

X-Ray Constraints on the Hot Gaseous Corona of Edge-on Late-type Galaxies in Virgo

We present a systematic study of the putative hot gas corona around late-type galaxies (LTGs) residing in the Virgo cluster, based on archival Chandra observations. Our sample consists of 21 nearly edge-on galaxies representing a star formation rate (SFR) range of ($0.2-3\rm~M_\odot~yr^{-1}$) a stellar mass ($M_*$) range of $(0.2-10) \times 10^{10}\rm~M_{\odot}$, the majority of which have not been explored with high-sensitivity X-ray observations so far. Significant extraplanar diffuse X-ray (0.5-2 keV) emission is detected in only three LTGs, which are also the three galaxies with the highest SFR. A stacking analysis is performed for the remaining galaxies without individual detection, dividing the whole sample into two subsets based on SFR, stellar mass, or specific SFR. Only the high-SFR bin yields a significant detection, which has a value of $L\rm_X \sim3\times10^{38}\rm~erg~s^{-1}$ per galaxy. The stacked extraplanar X-ray signals of the Virgo LTGs are consistent with the empirical $L\rm_X - SFR$ and $L\rm_X - M_*$ relations found among highly inclined disk galaxies in the field, but appear to be systematically lower than that of a comparison sample of simulated cluster star-formation galaxies identified from the Illustris-TNG100 simulation. The apparent paucity of hot gas coronae in the sampled Virgo LTGs might be understood as the net outcome of the long-lasting effect of ram pressure stripping exerted by the hot intra-cluster medium and in-disk star-forming activity acting on shorter timescales. A better understanding of the roles of environmental effects in regulating the hot gas content of cluster galaxies invites sensitive X-ray observations for a large galaxy sample.

astro-ph.GA↗

arXiv2023

A Statistical Study of Soft X-ray Flares on Solar-type Stars

The statistical characteristic of stellar flares at optical bands has received an extensive study, but it remains to be studied at soft X-ray bands, in particular for solar-type stars. Here, we present a statistical study of soft X-ray flares on solar-type stars, which can help understand multi-wavelength behaviors of stellar flares. We mainly use Chandra Source Catalog Release 2.0, which includes a number of flaring stars with denoted variability, and Gaia Data Release 3, which includes necessary information for classifying stars. We also develop a set of methods for identifying and classifying stellar soft X-ray flares and estimating their properties. A detailed statistical investigation for 129 flare samples on 103 nearby solar-type stars as selected yields the following main results. (1) The flare energy emitted at the soft X-ray band in our sample ranges from $\sim 10^{33}$ to $\sim 10^{37} \ \mathrm{erg}$, and the majority of them are superflares with the most energetic one having energy of $6.0_{-4.7}^{+3.2} \times 10^{37} \ \mathrm{erg}$. (2) The flare duration is related to its energy as formulated by $T_\mathrm{duration,SXR} \propto E_\mathrm{flare,SXR}^{\ 0.201 \pm 0.024}$, which is different from those derived at optical and NIR bands, indicating distinct radiation mechanisms at different bands. (3) The frequency distribution of stellar flares as a function of energy is formulated as $\mathrm{d} N_\mathrm{flare} / \mathrm{d} E_\mathrm{flare,SXR} \propto E_\mathrm{flare,SXR}^{\ -1.77}$, which is similar to the results found at other bands and on other types of stars, indicating that the energy emitted at the soft X-ray band could be a constant fraction of the full-band bolometric energy.

astro-ph.SR↗

arXiv2023

A Chandra Survey of Milky Way Globular Clusters -- IV. Periodic X-ray sources

We present a systematic search for periodic X-ray sources in 10 Galactic globular clusters (GCs) utilizing deep archival Chandra observations. By applying the Gregory-Loredo algorithm, we detect 28 periodic signals among 27 independent X-ray sources in 6 GCs, which include 21 newly discovered ones in the X-ray band. The remaining 4 GCs exhibit no periodic X-ray sources, mainly due to a relatively lower sensitivity of the data. Through analysis of their X-ray timing and spectral properties, complemented with available optical and ultraviolet information, we identify 21 of these periodic sources as cataclysmic variables (CVs). Combining with 11 periodic CVs in 47 Tuc similarly identified in the X-ray band, we compile the most comprehensive sample to date of GC CVs with a probable orbital period. The scarcity of old, short-period CVs in GCs compared to the Galactic inner bulge and solar neighborhood, can be attributed to both a selection effect favouring younger, dynamically-formed systems and the hindrance of CV formation through primordial binary evolution by stellar dynamical interactions common to the GC environment. Additionally, we identify a significant fraction of the GC CVs, most with an orbital period below or within the CV period gap, as probable magnetic CVs, but in the meantime there is a deficiency of luminous intermediate polars in the GC sample compared to the solar neighborhood.

astro-ph.HE↗

arXiv2023

Simultaneous uniqueness and numerical inversion for an inverse problem in the time-domain diffuse optical tomography with fluorescence

In this work, an inverse problem on the determination of multiple coefficients arising from the time-domain diffuse optical tomography with fluorescence (DOT-FDOT) is investigated. We simultaneously recover the distribution of background absorption coefficient, photon diffusion coefficient as well as the fluorescence absorption in biological tissue by the time-dependent boundary measurements. We build the uniqueness theorem of this multiple coefficients simultaneous inverse problem. After that, the numerical inversions are considered. We introduce an accelerated Landweber iterative algorithm and give several numerical examples illustrating the performance of the proposed inversion schemes.

math.NA↗

arXiv2023

Complex Organ Mask Guided Radiology Report Generation

The goal of automatic report generation is to generate a clinically accurate and coherent phrase from a single given X-ray image, which could alleviate the workload of traditional radiology reporting. However, in a real-world scenario, radiologists frequently face the challenge of producing extensive reports derived from numerous medical images, thereby medical report generation from multi-image perspective is needed. In this paper, we propose the Complex Organ Mask Guided (termed as COMG) report generation model, which incorporates masks from multiple organs (e.g., bones, lungs, heart, and mediastinum), to provide more detailed information and guide the model's attention to these crucial body regions. Specifically, we leverage prior knowledge of the disease corresponding to each organ in the fusion process to enhance the disease identification phase during the report generation process. Additionally, cosine similarity loss is introduced as target function to ensure the convergence of cross-modal consistency and facilitate model optimization.Experimental results on two public datasets show that COMG achieves a 11.4% and 9.7% improvement in terms of BLEU@4 scores over the SOTA model KiUT on IU-Xray and MIMIC, respectively. The code is publicly available at https://github.com/GaryGuTC/COMG_model.

cs.CV↗

arXiv2023

CAHA/PPAK Integral-field Spectroscopic Observations of M81. II. Testing Photoionization Models in A Spatially-resolved LINER

The origin of the low-ionization nuclear emission-line region (LINER) prevalent in local galaxies and its relationship with supermassive black holes are debated for decades. We preform a comprehensive evaluation of traditional photoionization models against the circumnuclear ionized gas in M81, for which recent CAHA/PPAK integral-field spectroscopic observations reveal a LINER characteristic out to a galactocentric radius of ~1 kpc. Constructed with the photoionization code cloudy, the models have the novel aspect of their primary parameters being well constrained by extensive observations of a prototypical low-luminosity active galactic nucleus (LLAGN) and an old stellar bulge in M81. Additionally, these models incorporate a reasonably broad range of uncertain nebular properties. It is found that the integrated photoionization by the LLAGN and hot, low-mass stars distributed in the bulge can roughly reproduce the observed radial intensity distributions of the Halpha, Hbeta and [N ii] lines, with the bulge stars dominating the ionizing flux at radii >~200 pc. However, the models generally fail to reproduce a similarly declining profile of the [Oiii] line or an accordingly flat profile of the [O iii]/Hbeta ratio. This clearly points to a deficiency of ionizing photons in the outer regions despite an extended photoionization source. The discrepancy might be alleviated if much of the observed [O iii] line arose from a bulge-filling, low-density gas surrounding a denser, Halpha-emitting disk, or by a higher AGN luminosity in the recent past. The case of M81 has important implications for the ionization mechanism of LINERs and low-ionization emission-line regions in general.

astro-ph.GA↗