arXiv subjects

Hong-Yu Chen

Publications and source records attributed to Hong-Yu Chen.

At least 19 recordsLinked to original sources

arXiv2026

Pre-localization of Massive Black Hole Binaries in the Millihertz Band

The space-borne gravitational-wave (GW) detectors will open a new mass and redshift regime, allowing us to observe massive black hole binaries (MBHBs) throughout the Universe. A subset of these systems is expected to produce electromagnetic (EM) counterparts, offering a unique opportunity to follow the continuous evolution of massive black holes through joint GW and EM observations. Realizing this potential, however, requires low-latency, high-throughput data-analysis pipelines that can extract reliable source parameters and sky localizations from space-borne data streams fast enough to trigger EM follow-up. In this work we develop a fast, normalising flow-based inference pipeline designed for early-warning analysis of MBHB signals in a TianQin-like configuration. Our method combines a learned embedding of the detector time series with a neural spline flow (NSF) to perform amortized Bayesian inference, producing posterior samples for the main source parameters in roughly one minute per event. For a representative MBHB whose merger occurs $\sim 15$ minutes after the end of the analyzed GW observation, the pipeline achieves pre-merger sky localizations of order $\sim 20~\mathrm{deg}^2$, recovers the same number of sky modes as a reference parallel-tempered Markov chain Monte Carlo (PTMCMC) analysis, and yields parameter uncertainties of comparable scale, while still operating within a practically useful pre-merger warning window. These results demonstrate that NSF-based inference can deliver accurate, near-real-time parameter estimation for space-borne MBHB GW signals, and that the resulting early-warning localizations are sufficiently precise to make rapid EM follow-up.

gr-qc↗

arXiv2025

StarEmbed: Benchmarking Time Series Foundation Models on Astronomical Observations of Variable Stars

Current time series foundation model (TSFM) training corpora largely omit data with certain complexities like irregular temporal sampling. Astronomical time series of stellar fluxes (light curves) are available in immense quantities and exhibit irregular sampling, multiple variates, and heteroskedasticity. We introduce StarEmbed, the first public benchmark for light curves comprised of real observations of ~40,000 stars expert-labeled across seven classes and evaluations in clustering, classification, and out-of-distribution (OOD) source detection. We benchmark TSFMs with differing architecture and training strategies as well as domain-specific transformers. Our results demonstrate that the Chronos family, despite being pre-trained on regularly sampled non-astronomical data, yields state-of-the-art (SOTA) performance in light curve clustering and OOD detection. While no TSFM strictly surpasses the classification performance of the long-established domain baseline, they do demonstrate excellent generalization abilities. StarEmbed marks a step toward universal light curve embeddings and improved TSFM performance on challenging data.

astro-ph.SR↗

arXiv2025

AdvEvo-MARL: Shaping Internalized Safety through Adversarial Co-Evolution in Multi-Agent Reinforcement Learning

LLM-based multi-agent systems excel at planning, tool use, and role coordination, but their openness and interaction complexity also expose them to jailbreak, prompt-injection, and adversarial collaboration. Existing defenses fall into two lines: (i) self-verification that asks each agent to pre-filter unsafe instructions before execution, and (ii) external guard modules that police behaviors. The former often underperforms because a standalone agent lacks sufficient capacity to detect cross-agent unsafe chains and delegation-induced risks; the latter increases system overhead and creates a single-point-of-failure-once compromised, system-wide safety collapses, and adding more guards worsens cost and complexity. To solve these challenges, we propose AdvEvo-MARL, a co-evolutionary multi-agent reinforcement learning framework that internalizes safety into task agents. Rather than relying on external guards, AdvEvo-MARL jointly optimizes attackers (which synthesize evolving jailbreak prompts) and defenders (task agents trained to both accomplish their duties and resist attacks) in adversarial learning environments. To stabilize learning and foster cooperation, we introduce a public baseline for advantage estimation: agents within the same functional group share a group-level mean-return baseline, enabling lower-variance updates and stronger intra-group coordination. Across representative attack scenarios, AdvEvo-MARL consistently keeps attack-success rate (ASR) below 20%, whereas baselines reach up to 38.33%, while preserving-and sometimes improving-task accuracy (up to +3.67% on reasoning tasks). These results show that safety and utility can be jointly improved without relying on extra guard agents or added system overhead.

cs.AI↗

arXiv2025

Evo-MARL: Co-Evolutionary Multi-Agent Reinforcement Learning for Internalized Safety

Multi-agent systems (MAS) built on multimodal large language models exhibit strong collaboration and performance. However, their growing openness and interaction complexity pose serious risks, notably jailbreak and adversarial attacks. Existing defenses typically rely on external guard modules, such as dedicated safety agents, to handle unsafe behaviors. Unfortunately, this paradigm faces two challenges: (1) standalone agents offer limited protection, and (2) their independence leads to single-point failure-if compromised, system-wide safety collapses. Naively increasing the number of guard agents further raises cost and complexity. To address these challenges, we propose Evo-MARL, a novel multi-agent reinforcement learning (MARL) framework that enables all task agents to jointly acquire defensive capabilities. Rather than relying on external safety modules, Evo-MARL trains each agent to simultaneously perform its primary function and resist adversarial threats, ensuring robustness without increasing system overhead or single-node failure. Furthermore, Evo-MARL integrates evolutionary search with parameter-sharing reinforcement learning to co-evolve attackers and defenders. This adversarial training paradigm internalizes safety mechanisms and continually enhances MAS performance under co-evolving threats. Experiments show that Evo-MARL reduces attack success rates by up to 22% while boosting accuracy by up to 5% on reasoning tasks-demonstrating that safety and utility can be jointly improved.

cs.AI↗

arXiv2025

Evaluating statistical significance for massive black hole binary mergers with space-based gravitational wave detectors

Important scientific discoveries should be backed by high statistical significance. In the 2030s, multiple space-based gravitational wave detectors are expected to operate. While many works aim to achieve quick and reliable detection and parameter estimation of millihertz gravitational wave sources, dedicated studies are lacking to assess the significance of space-based detectors. In this work, we propose a framework to assess the statistical significance of massive black hole binaries (MBHBs) detections with space-based gravitational wave detectors. We apply this algorithm to simulated data with Gaussian stationary noise and the complex LDC-2a dataset to measure the false alarm rate and significance of MBHB signals. We also analyze factors affecting the significance of MBHBs and design a method to mitigate multi-source confusion interference. In Gaussian noise conditions, MBHBs with a signal-to-noise ratio of about 7 can achieve $3 \sigma$ significance, and those with a signal-to-noise ratio of about 8 achieve $4 \sigma$. Our analysis demonstrates that all MBHB signals in the LDC-2a dataset have a significance exceeding $4.62 \sigma$.

gr-qc↗

arXiv2025

Universal Approximation with Softmax Attention

We prove that with linear transformations, both (i) two-layer self-attention and (ii) one-layer self-attention followed by a softmax function are universal approximators for continuous sequence-to-sequence functions on compact domains. Our main technique is a new interpolation-based method for analyzing attention's internal mechanism. This leads to our key insight: self-attention is able to approximate a generalized version of ReLU to arbitrary precision, and hence subsumes many known universal approximators. Building on these, we show that two-layer multi-head attention alone suffices as a sequence-to-sequence universal approximator. In contrast, prior works rely on feed-forward networks to establish universal approximation in Transformers. Furthermore, we extend our techniques to show that, (softmax-)attention-only layers are capable of approximating various statistical models in-context. We believe these techniques hold independent interest.

cs.LG↗

arXiv2025

Transformers versus the EM Algorithm in Multi-class Clustering

LLMs demonstrate significant inference capacities in complicated machine learning tasks, using the Transformer model as its backbone. Motivated by the limited understanding of such models on the unsupervised learning problems, we study the learning guarantees of Transformers in performing multi-class clustering of the Gaussian Mixture Models. We develop a theory drawing strong connections between the Softmax Attention layers and the workflow of the EM algorithm on clustering the mixture of Gaussians. Our theory provides approximation bounds for the Expectation and Maximization steps by proving the universal approximation abilities of multivariate mappings by Softmax functions. In addition to the approximation guarantees, we also show that with a sufficient number of pre-training samples and an initialization, Transformers can achieve the minimax optimal rate for the problem considered. Our extensive simulations empirically verified our theory by revealing the strong learning capacities of Transformers even beyond the assumptions in the theory, shedding light on the powerful inference capacities of LLMs.

stat.ML↗

arXiv2025

Transformers Simulate MLE for Sequence Generation in Bayesian Networks

Transformers have achieved significant success in various fields, notably excelling in tasks involving sequential data like natural language processing. Despite these achievements, the theoretical understanding of transformers' capabilities remains limited. In this paper, we investigate the theoretical capabilities of transformers to autoregressively generate sequences in Bayesian networks based on in-context maximum likelihood estimation (MLE). Specifically, we consider a setting where a context is formed by a set of independent sequences generated according to a Bayesian network. We demonstrate that there exists a simple transformer model that can (i) estimate the conditional probabilities of the Bayesian network according to the context, and (ii) autoregressively generate a new sample according to the Bayesian network with estimated conditional probabilities. We further demonstrate in extensive experiments that such a transformer does not only exist in theory, but can also be effectively obtained through training. Our analysis highlights the potential of transformers to learn complex probabilistic models and contributes to a better understanding of large language models as a powerful class of sequence generators.

stat.ML↗

arXiv2025

Learning Spectral Methods by Transformers

Transformers demonstrate significant advantages as the building block of modern LLMs. In this work, we study the capacities of Transformers in performing unsupervised learning. We show that multi-layered Transformers, given a sufficiently large set of pre-training instances, are able to learn the algorithms themselves and perform statistical estimation tasks given new instances. This learning paradigm is distinct from the in-context learning setup and is similar to the learning procedure of human brains where skills are learned through past experience. Theoretically, we prove that pre-trained Transformers can learn the spectral methods and use the classification of bi-class Gaussian mixture model as an example. Our proof is constructive using algorithmic design techniques. Our results are built upon the similarities of multi-layered Transformer architecture with the iterative recovery algorithms used in practice. Empirically, we verify the strong capacity of the multi-layered (pre-trained) Transformer on unsupervised learning through the lens of both the PCA and the Clustering tasks performed on the synthetic and real-world datasets.

stat.ML↗

arXiv2024

Signal-to-noise Ratio Analytic Formulae of the Inspiral Binary Black Holes in TianQin

Binary black holes are one of the important sources for the TianQin gravitational wave project. Our research has revealed that, for TianQin, the signal-to-noise ratio of inspiral binary black holes can be computed analytically. This finding is expected to greatly simplify the estimation of detection capabilities for binary black holes. In this paper, we demonstrated the signal-to-noise ratio relationships from stellar-mass black holes to massive black holes. With the all-sky average condition, the signal-to-noise ratio for most binary black hole signals can be determined with a relative error of $\lesssim10\%$, with notable deviations only for chirp masses near $1000~M_\odot$. In contrast, the signal-to-noise ratio without the average includes an additional term, which we refer to as the response factor. Although this term is not easily calculated analytically, we provide a straightforward estimation method with an error margin of $1\sigma$ within 2\%.

astro-ph.GA↗

arXiv2024

Gravitational Wave Astronomy With TianQin

The opening of the gravitational wave window has significantly enhanced our capacity to explore the universe's most extreme and dynamic sector. In the mHz frequency range, a diverse range of compact objects, from the most massive black holes at the farthest reaches of the Universe to the lightest white dwarfs in our cosmic backyard, generate a complex and dynamic symphony of gravitational wave signals. Once recorded by gravitational wave detectors, these unique fingerprints have the potential to decipher the birth and growth of cosmic structures over a wide range of scales, from stellar binaries and stellar clusters to galaxies and large-scale structures. The TianQin space-borne gravitational wave mission is scheduled for launch in the 2030s, with an operational lifespan of five years. It will facilitate pivotal insights into the history of our universe. This document presents a concise overview of the detectable sources of TianQin, outlining their characteristics, the challenges they present, and the expected impact of the TianQin observatory on our understanding of them.

astro-ph.GA↗

arXiv2024

GWnext 2024: Meeting Summary

GWnext 2024 was a meeting held in the Kavli Institute for Astronomy and Astrophysics at Peking University in March $4^\text{th} - 8^\text{th}$, 2024. In the meeting researchers at different career stages -- with a particular focus on early career scientists -- working on the different aspects of gravitational wave (GW) astronomy gathered to discuss the current status as well as prospects of the field. The meeting was divided into three core sessions: Astrophysics, GW Theory, and Detection. Each session consisted of introductory talks and extended discussion sessions. Moreover, there was a poster session where students could present their results. In this paper, we summarize the results presented during the meeting and present the most important outcomes.

astro-ph.HE↗

arXiv2024

Dealing with data gaps for TianQin with massive black hole binary signal

Space-borne gravitational wave detectors like TianQin might encounter data gaps due to factors like micrometeoroid collisions or hardware failures. Such events will cause discontinuity in the data, presenting challenges to the data analysis for TianQin, especially for massive black hole binary mergers. Since the signal-to-noise ratio (SNR) accumulates in a non-linear way, a gap near the merger could lead to a significant loss of SNR. It could introduce bias in the estimate of noise properties, and the results of the parameter estimation. In this work, using simulated TianQin data with injected a massive black hole binary merger, we study the window function method, and for the first time, the inpainting method to cope with the data gap, and an iterative estimate scheme is designed to properly estimate the noise spectrum. We find that both methods can properly estimate noise and signal parameters. The easy-to-implement window function method can already perform well, except that it will sacrifice some SNR due to the adoption of the window. The inpainting method is slower, but it can minimize the impact of the data gap.

gr-qc↗

arXiv2024

Outlier-Efficient Hopfield Layers for Large Transformer-Based Models

We introduce an Outlier-Efficient Modern Hopfield Model (termed $\mathrm{OutEffHop}$) and use it to address the outlier inefficiency problem of {training} gigantic transformer-based models. Our main contribution is a novel associative memory model facilitating \textit{outlier-efficient} associative memory retrievals. Interestingly, this memory model manifests a model-based interpretation of an outlier-efficient attention mechanism (${\rm Softmax}_1$): it is an approximation of the memory retrieval process of $\mathrm{OutEffHop}$. Methodologically, this allows us to introduce novel outlier-efficient Hopfield layers as powerful alternatives to traditional attention mechanisms, with superior post-quantization performance. Theoretically, the Outlier-Efficient Modern Hopfield Model retains and improves the desirable properties of standard modern Hopfield models, including fixed point convergence and exponential storage capacity. Empirically, we demonstrate the efficacy of the proposed model across large-scale transformer-based and Hopfield-based models (including BERT, OPT, ViT, and STanHop-Net), benchmarking against state-of-the-art methods like $\mathtt{Clipped\_Softmax}$ and $\mathtt{Gated\_Attention}$. Notably, $\mathrm{OutEffHop}$ achieves an average reduction of 22+\% in average kurtosis and 26+\% in the maximum infinity norm of model outputs across four models. Code is available at \href{https://github.com/MAGICS-LAB/OutEffHop}{GitHub}; models are on \href{https://huggingface.co/collections/magicslabnu/outeffhop-6610fcede8d2cda23009a98f}{Hugging Face Hub}; future updates are on \href{https://arxiv.org/abs/2404.03828}{arXiv}.

cs.LG↗

arXiv2023

GWSpace: a multi-mission science data simulator for space-based gravitational wave detection

Space-based gravitational wave detectors such as TianQin, LISA, and TaiJi have the potential to outperform themselves through joint observation. To achieve this, it is desirable to practice joint data analysis in advance on simulated data that encodes the intrinsic correlation among the signals found in different detectors that operate simultaneously. In this paper, we introduce GWSpace, a package that can simulate the joint detection data from TianQin, LISA, and TaiJi. The software is not a groundbreaking work that starts from scratch. Rather, we use as many open-source resources as possible, tailoring them to the needs of simulating the multi-mission science data and putting everything into a ready-to-go and easy-to-use package. We shall describe the main components, the construction, and a few examples of application of the package. A common coordinate system, namely the Solar System Barycenter (SSB) coordinate system, is utilized to calculate spacecraft orbits for all three missions. The paper also provides a brief derivation of the detection process and outlines the general waveform of sources detectable by these detectors.

gr-qc↗

arXiv2023

Near Real-Time Gravitational Wave Data Analysis of the Massive Black Hole Binary with TianQin

Space-borne gravitational wave (GW) detectors can detect the merger of massive black holes. The early warning and localization of GW events before merging can be used to inform electromagnetic telescopes and conduct multimessenger observations. However, this requires real-time data transmission and analysis capabilities. The geocentric orbit of the space-borne GW detector TianQin makes it possible to conduct real-time data transmission. In this study, we develop a search and localization pipeline for massive black hole binaries (MBHBs) with TianQin under both regular and real-time data transmission modes. We demonstrate that, with real-time data transmission, MBHBs can be accurately localized on the fly. With the approaching merger, each analysis can be finished in only 40 min. For an MBHB system at a distance of 1 Gpc, if we receive data every hour, then we can pinpoint its location to within less than 1 deg$^2$ on the final day before the merger.

gr-qc↗

arXiv2020

Absence of Superconductivity in Nd$_{0.8}$Sr$_{0.2}$NiO$_x$ Thin Films without Chemical Reduction

The recently reported superconductivity 9-15 K in Nd0.8Sr0.2NiO2/SrTiO3 heterostructures that were fabricated by a soft-chemical topotactic reduction approach based on precursor Nd0.8Sr0.2NiO3 thin films deposited on SrTiO3 substrates, has excited an immediate surge of research interest. To explore an alternative physical path instead of chemical reduction for realizing superconductivity in this compound, using pulsed laser deposition, we systematically fabricated 63 Nd0.8Sr0.2NiOx (NSNO) thin films at a wide range of oxygen partial pressures on various different oxide substrates. Transport measurements did not find any signature of superconductivity in all the 63 thin-film samples. With reducing the oxygen content in the NSNO films by lowering the deposition oxygen pressure, the NSNO films are getting more resistive and finally become insulating. Furthermore, we tried to cap a 20-nm-thick amorphous LaAlO3 layer on a Nd0.8Sr0.2NiO3 thin film deposited at a high oxygen pressure of 150 mTorr to create oxygen vacancies on its surface and did not succeed in higher conductivity either. Our experimental results together with the recent report on the absence of superconductivity in synthesized bulk Nd0.8Sr0.2NiO2 crystals suggest that the chemical reduction approach could be unique for yielding superconductivity in NSNO/SrTiO3 heterostructures. However, SrTiO3 substrates could be reduced to generate oxygen vacancies during the chemical reduction process as well, which may thus partially contribute to conductivity.

cond-mat.supr-con↗

arXiv2018

Two-photon exchange effects in $e^+e^- \rightarrow π^+π^-$ and time-like pion electromagnetic form factor

The two-photon-exchange (TPE) effects in the process $e^+e^- \rightarrow π^+π^-$ at large momentum transfer are discussed within the perturbative QCD (pQCD). The contributions from the twist-2 and twist-3 distribution amplitudes (DAs) of pion are considered in the estimation. Different with the results under the one-photon-exchange (OPE) approximation, the TPE effects result in an asymmetry of the differential cross section on the scattering angle. The precise measurement of this asymmetry by the further experiment is an precise test of pQCD at large momentum transfer. The time-like electromagnetic form factor of pion at the leading order of pQCD is re-discussed and the comparison of our results with those in the references are presented.

hep-ph↗