arXiv subjects

Zihao Li

Publications and source records attributed to Zihao Li.

At least 163 records · Page 9Linked to original sources

arXiv2024

PyG-SSL: A Graph Self-Supervised Learning Toolkit

Graph Self-Supervised Learning (SSL) has emerged as a pivotal area of research in recent years. By engaging in pretext tasks to learn the intricate topological structures and properties of graphs using unlabeled data, these graph SSL models achieve enhanced performance, improved generalization, and heightened robustness. Despite the remarkable achievements of these graph SSL methods, their current implementation poses significant challenges for beginners and practitioners due to the complex nature of graph structures, inconsistent evaluation metrics, and concerns regarding reproducibility hinder further progress in this field. Recognizing the growing interest within the research community, there is an urgent need for a comprehensive, beginner-friendly, and accessible toolkit consisting of the most representative graph SSL algorithms. To address these challenges, we present a Graph SSL toolkit named PyG-SSL, which is built upon PyTorch and is compatible with various deep learning and scientific computing backends. Within the toolkit, we offer a unified framework encompassing dataset loading, hyper-parameter configuration, model training, and comprehensive performance evaluation for diverse downstream tasks. Moreover, we provide beginner-friendly tutorials and the best hyper-parameters of each graph SSL algorithm on different graph datasets, facilitating the reproduction of results. The GitHub repository of the library is https://github.com/iDEA-iSAIL-Lab-UIUC/pyg-ssl.

cs.LG↗

arXiv2024

A unique window into the Epoch of Reionisation: A double-peaked Lyman-$α$ emitter in the proximity zone of a quasar at $z\sim 6.6$

We present a study of a double-peaked Ly$α$ emitter, named LAE-11, found in the proximity zone of QSO J0910-0414 at $z\sim6.6$. We use a combination of deep photometric data from Subaru Telescope, HST, and JWST with spectroscopic data from Keck/DEIMOS, NIRCam WFSS and NIRSpec MSA to characterise the ionising and general properties of the galaxy, as well as the quasar environment surrounding it. We detect Ly$α$, H$β$, [OIII] doublet, and H$α$ emission lines in the various spectral datasets. The presence of a double-peaked Ly$α$ line in the spectrum allows us to characterise the opening angle and lifetime of the QSO as $θ_Q>49.62$° and $t_Q > 3.8\times10^5$ years. LAE-11 is a fairly bright (M$_\rm{UV} = -19.84^{+0.14}_{-0.16}$), blue galaxy with a UV slope of $β= -2.61^{+0.06}_{-0.08}$. Since the galaxy is located in a quasar-ionised region, we have a unique opportunity to measure the escape fraction of Lyman Continuum photons using the un-attenuated double-peaked Ly$α$ emission profile and its equivalent width at such high redshift. We employ diagnostics which do not rely on the detection of Ly$α$ for comparison, and find that all tracers of ionising photon leakage agree within 1$σ$ uncertainty. We measure a moderate escape of Lyman Continuum photons from LAE-11 of $f_\rm{esc}^\rm{LyC} = (8 - 33)\%$. Detections of both H$α$ and H$β$ emission lines allow for separate measurements of the ionising photon production efficiency, resulting with $\log(ξ_\rm{ion}/\rm{Hz~erg^{-1}}) = 25.59\pm0.08$ and $25.65\pm0.09$, for H$α$ and H$β$, respectively. The total ionising output of LAE-11, $\log(f_\rm{esc}^\rm{LyC}ξ_\rm{ion, Hα}/\rm{Hz~erg^{-1}}) = 24.85^{+0.24}_{-0.34}$, is higher than the value of $24.3 - 24.8$ which is traditionally assumed as needed to drive Reionisation forward.

astro-ph.GA↗

arXiv2024

Language Ranker: A Metric for Quantifying LLM Performance Across High and Low-Resource Languages

The development of Large Language Models (LLMs) relies on extensive text corpora, which are often unevenly distributed across languages. This imbalance results in LLMs performing significantly better on high-resource languages like English, German, and French, while their capabilities in low-resource languages remain inadequate. Currently, there is a lack of quantitative methods to evaluate the performance of LLMs in these low-resource languages. To address this gap, we propose the Language Ranker, an intrinsic metric designed to benchmark and rank languages based on LLM performance using internal representations. By comparing the LLM's internal representation of various languages against a baseline derived from English, we can assess the model's multilingual capabilities in a robust and language-agnostic manner. Our analysis reveals that high-resource languages exhibit higher similarity scores with English, demonstrating superior performance, while low-resource languages show lower similarity scores, underscoring the effectiveness of our metric in assessing language-specific capabilities. Besides, the experiments show that there is a strong correlation between the LLM's performance in different languages and the proportion of those languages in its pre-training corpus. These insights underscore the efficacy of the Language Ranker as a tool for evaluating LLM performance across different languages, particularly those with limited resources.

cs.CL↗

arXiv2024

A SPectroscopic survey of biased halos In the Reionization Era (ASPIRE): Spectroscopically Complete Census of Obscured Cosmic Star Formation Rate Density at $z=4-6$

We present a stringent measurement of the dust-obscured star-formation rate density (SFRD) at $z=4-6$ from the ASPIRE JWST Cycle-1 medium and ALMA Cycle-9 large program. We obtained JWST/NIRCam grism spectroscopy and ALMA 1.2-mm continuum map along 25 independent quasar sightlines, covering a total survey area of $\sim$35 arcmin$^2$ where we search for dusty star-forming galaxies (DSFGs) at $z = 0 - 7$. We identify eight DSFGs in seven fields at $z=4-6$ through the detection of H$α$ or [O III] $λ$5008 lines, including fainter lines such as H$β$, [O III] $λ$4960, [N II] $λ$6585, [S II] $λλ$6718,6733 for six sources. With this spectroscopically complete DSFG sample at $z=4-6$ and negligible impact from cosmic variance (shot noise), we measure the infrared luminosity function (IRLF) down to $L_\mathrm{IR} \sim 2\times10^{11}$ $L_\odot$. We find flattening of IRLF at $z=4-6$ towards the faint end (power-law slope $α= 0.59_{-0.45}^{+0.39}$). We determine the dust-obscured cosmic SFRD at this epoch as $\log[ρ_\mathrm{SFR,IR} / (\mathrm{M}_\odot\,\mathrm{yr}^{-1}\,\mathrm{Mpc}^{-3})] = -1.52_{-0.13}^{+0.14}$. This is significantly higher than previous determination using ALMA data in the Hubble Ultra Deep Field, which is void of DSFGs at $z=4-6$ because of strong cosmic variance (shot noise). We conclude that the majority ($66\pm7$%) of cosmic star formation at $z \sim 5$ is still obscured by dust. We also discuss the uncertainty of SFRD propagated from far-IR spectral energy distribution and IRLF at the bright end, which will need to be resolved with future ALMA and JWST observations.

astro-ph.GA↗

arXiv2024

Why Anticipatory Sensing Matters in Commercial ACC Systems under Cut-In Scenarios: A Perspective from Stochastic Safety Analysis

This study presents an analytical solution for the vehicle state evolution of Adaptive Cruise Control (ACC) systems under cut-in scenarios, incorporating sensing delays and anticipation using the Lambert W function. The theoretical analysis demonstrates that the vehicle state evolution and the corresponding safety of ACC in cut-in situations are influenced by multiple factors, including the original leading vehicle's state, the initial conditions of the cut-in vehicle, subsequent cut-in maneuvers, sensing delays, and the ACC's anticipation capabilities. To quantitatively assess these influences, a series of numerical experiments were conducted to perform a stochastic safety analysis of ACC systems, accounting for embedded sensing delays and anticipation, using empirically calibrated control parameters from real-world data. The experiments revealed that the impact of sensing delays on ACC is multifaceted. Specifically, sensing delays negatively affect ACC stability, with the severity increasing as the delay lengthens. Furthermore, collision risk in cut-in scenarios becomes more significant with sensing delays, particularly when the cut-in vehicle is slower than the following vehicle and when cut-ins are aggressive. However, anticipation plays a crucial role in mitigating these risks. Even with a 0.6-second anticipation, collision risk can be reduced by 91% in highly adverse scenarios. Finally, both sensing delays and anticipation have effects that intensify with their duration. An anticipation period of 2 seconds effectively ensures safety in aggressive cut-in conditions, even in the presence of sensing delays.

eess.SY↗

arXiv2024

One-Layer Transformer Provably Learns One-Nearest Neighbor In Context

Transformers have achieved great success in recent years. Interestingly, transformers have shown particularly strong in-context learning capability -- even without fine-tuning, they are still able to solve unseen tasks well purely based on task-specific prompts. In this paper, we study the capability of one-layer transformers in learning one of the most classical nonparametric estimators, the one-nearest neighbor prediction rule. Under a theoretical framework where the prompt contains a sequence of labeled training data and unlabeled test data, we show that, although the loss function is nonconvex when trained with gradient descent, a single softmax attention layer can successfully learn to behave like a one-nearest neighbor classifier. Our result gives a concrete example of how transformers can be trained to implement nonparametric machine learning algorithms, and sheds light on the role of softmax attention in transformer models.

cs.LG↗

arXiv2024

Constraints on Cosmological Models with Gamma-Ray Bursts in Cosmology-Independent Way

In this paper, we present a cosmology-independent method to constrain cosmological models from the latest 221 gamma-ray bursts (GRBs) sample, including 49 GRBs from Fermi catalog with the Amati relation (the $E_{\rm p}$-${E}_{\rm iso}$ correlation), which are calibrated by using a Gaussian process from the Pantheon+ type Ia supernovae (SNe Ia) sample. With 182 GRBs at $0.8\le z\le8.2$ in the Hubble diagram and the latest observational Hubble data (OHD) by the Markov Chain Monte Carlo (MCMC) method, we obtained $Ω_{\rm m}$ = $0.348^{+0.048}_{-0.066}$ and $h$ = $0.680^{+0.029}_{-0.029}$ for the flat $Λ$CDM model, and $Ω_{\rm m}$ = $0.318^{+0.067}_{-0.059}$, $h$ = $0.704^{+0.055}_{-0.068}$, $w$ = $-1.21^{+0.32}_{-0.67}$ for the flat $w$CDM model. These results are consistent with those in which the coefficients of the Amati relation and the cosmological parameters fitted simultaneously.

astro-ph.CO↗

arXiv2024

PageRank Bandits for Link Prediction

Link prediction is a critical problem in graph learning with broad applications such as recommender systems and knowledge graph completion. Numerous research efforts have been directed at solving this problem, including approaches based on similarity metrics and Graph Neural Networks (GNN). However, most existing solutions are still rooted in conventional supervised learning, which makes it challenging to adapt over time to changing customer interests and to address the inherent dilemma of exploitation versus exploration in link prediction. To tackle these challenges, this paper reformulates link prediction as a sequential decision-making process, where each link prediction interaction occurs sequentially. We propose a novel fusion algorithm, PRB (PageRank Bandits), which is the first to combine contextual bandits with PageRank for collaborative exploitation and exploration. We also introduce a new reward formulation and provide a theoretical performance guarantee for PRB. Finally, we extensively evaluate PRB in both online and offline settings, comparing it with bandit-based and graph-based methods. The empirical success of PRB demonstrates the value of the proposed fusion approach. Our code is released at https://github.com/jiaruzouu/PRB.

cs.LG↗

arXiv2024

Global Convergence in Training Large-Scale Transformers

Despite the widespread success of Transformers across various domains, their optimization guarantees in large-scale model settings are not well-understood. This paper rigorously analyzes the convergence properties of gradient flow in training Transformers with weight decay regularization. First, we construct the mean-field limit of large-scale Transformers, showing that as the model width and depth go to infinity, gradient flow converges to the Wasserstein gradient flow, which is represented by a partial differential equation. Then, we demonstrate that the gradient flow reaches a global minimum consistent with the PDE solution when the weight decay regularization parameter is sufficiently small. Our analysis is based on a series of novel mean-field techniques that adapt to Transformers. Compared with existing tools for deep networks (Lu et al., 2020) that demand homogeneity and global Lipschitz smoothness, we utilize a refined analysis assuming only $\textit{partial homogeneity}$ and $\textit{local Lipschitz smoothness}$. These new techniques may be of independent interest.

stat.ML↗

arXiv2024

Best-of-Both-Worlds Fair Allocation of Indivisible and Mixed Goods

We study the problem of fairly allocating either a set of indivisible goods or a set of mixed divisible and indivisible goods (i.e., mixed goods) to agents with additive utilities, taking the best-of-both-worlds perspective of guaranteeing fairness properties both ex ante and ex post. The ex-post fairness notions considered in this paper are relaxations of envy-freeness, specifically, EFX for indivisible-goods allocation, and EFM for mixed-goods allocation. For two agents, we show that there is a polynomial-time randomized algorithm that achieves ex-ante envy-freeness and ex-post EFX / EFM simultaneously. For $n$ agents with bi-valued utilities, we show there exist randomized allocations that are (i) ex-ante proportional and ex-post EFM, and (ii) ex-ante envy-free, ex-post EFX, and ex-post fractionally Pareto optimal.

cs.GT↗

arXiv2024

Real-time Vehicle-to-Vehicle Communication Based Network Cooperative Control System through Distributed Database and Multimodal Perception: Demonstrated in Crossroads

The autonomous driving industry is rapidly advancing, with Vehicle-to-Vehicle (V2V) communication systems highlighting as a key component of enhanced road safety and traffic efficiency. This paper introduces a novel Real-time Vehicle-to-Vehicle Communication Based Network Cooperative Control System (VVCCS), designed to revolutionize macro-scope traffic planning and collision avoidance in autonomous driving. Implemented on Quanser Car (Qcar) hardware platform, our system integrates the distributed databases into individual autonomous vehicles and an optional central server. We also developed a comprehensive multi-modal perception system with multi-objective tracking and radar sensing. Through a demonstration within a physical crossroad environment, our system showcases its potential to be applied in congested and complex urban environments.

cs.RO↗

arXiv2024

fAmulet: Finding Finalization Failure Bugs in Polygon zkRollup

Zero-knowledge layer 2 protocols emerge as a compelling approach to overcoming blockchain scalability issues by processing transactions through the transaction finalization process. During this process, transactions are efficiently processed off the main chain. Besides, both the transaction data and the zero-knowledge proofs of transaction executions are reserved on the main chain, ensuring the availability of transaction data as well as the correctness and verifiability of transaction executions. Hence, any bugs that cause the transaction finalization failure are crucial, as they impair the usability of these protocols and the scalability of blockchains. In this work, we conduct the first systematic study on finalization failure bugs in zero-knowledge layer 2 protocols, and define two kinds of such bugs. Besides, we design fAmulet, the first tool to detect finalization failure bugs in Polygon zkRollup, a prominent zero-knowledge layer 2 protocol, by leveraging fuzzing testing. To trigger finalization failure bugs effectively, we introduce a finalization behavior model to guide our transaction fuzzer to generate and mutate transactions for inducing diverse behaviors across each component (e.g., Sequencer) in the finalization process. Moreover, we define bug oracles according to the distinct bug definitions to accurately detect bugs. Through our evaluation, fAmulet can uncover twelve zero-day finalization failure bugs in Polygon zkRollup, and cover at least 20.8% more branches than baselines. Furthermore, through our preliminary study, fAmulet uncovers a zero-day finalization failure bug in Scroll zkRollup, highlighting the generality of fAmulet to be applied to other zero-knowledge layer 2 protocols. At the time of writing, all our uncovered bugs have been confirmed and fixed by Polygon zkRollup and Scroll zkRollup teams.

cs.CR↗

arXiv2024

A Comparison of Language Modeling and Translation as Multilingual Pretraining Objectives

Pretrained language models (PLMs) display impressive performances and have captured the attention of the NLP community. Establishing best practices in pretraining has, therefore, become a major focus of NLP research, especially since insights gained from monolingual English models may not necessarily apply to more complex multilingual models. One significant caveat of the current state of the art is that different works are rarely comparable: they often discuss different parameter counts, training data, and evaluation methodology. This paper proposes a comparison of multilingual pretraining objectives in a controlled methodological environment. We ensure that training data and model architectures are comparable, and discuss the downstream performances across 6 languages that we observe in probing and fine-tuning scenarios. We make two key observations: (1) the architecture dictates which pretraining objective is optimal; (2) multilingual translation is a very effective pretraining objective under the right conditions. We make our code, data, and model weights available at \texttt{\url{https://github.com/Helsinki-NLP/lm-vs-mt}}.

cs.CL↗

arXiv2024

A SPectroscopic survey of biased halos In the Reionization Era (ASPIRE): JWST Supports Earlier Reionization around [OIII] Emitters

Understanding when and how reionization happened is crucial for studying the early structure formation and the properties of first galaxies in the Universe. At $z>5.5$, the observed IGM optical depth shows a significant scatter, indicating an inhomogeneous reionization process. However, the nature of the inhomogeneous reionization remains debated. ASPIRE is a JWST Cycle 1 program that has spectroscopically identified $>400$ [OIII] emitters in 25 quasar fields at $z>6.5$. Combined with deep ground-based optical spectroscopy of ASPIRE quasars, ASPIRE program provides the current largest sample for IGM-galaxy connection studies during cosmic reionization. We present the first results of IGM effective optical depth measurements around [OIII] emitters using 14 ASPIRE quasar fields. We find the IGM transmission is tightly related with reionization-era galaxies to the extent that significant excess of Ly$α$ transmission exists around [OIII] emitters. We measure the stacked IGM effective optical depth of IGM patches associated with [OIII] emitters and find they reach the same IGM effective optical depth at least dz~0.1 ahead of those IGM patches where no [OIII] emitters are detected, supporting earlier reionization around [OIII] emitters. Our results indicate an enhancement in IGM Ly$α$ transmission around [OIII] emitters at scales beyond 25 $h^{-1}$ cMpc, consistent with the predicted topology of reionization from fluctuating UV background (UVB) models.

astro-ph.CO↗

arXiv2024

Large Language Models for Biomedical Text Simplification: Promising But Not There Yet

In this system report, we describe the models and methods we used for our participation in the PLABA2023 task on biomedical abstract simplification, part of the TAC 2023 tracks. The system outputs we submitted come from the following three categories: 1) domain fine-tuned T5-like models including Biomedical-T5 and Lay-SciFive; 2) fine-tuned BARTLarge model with controllable attributes (via tokens) BART-w-CTs; 3) ChatGPTprompting. We also present the work we carried out for this task on BioGPT finetuning. In the official automatic evaluation using SARI scores, BeeManc ranks 2nd among all teams and our model LaySciFive ranks 3rd among all 13 evaluated systems. In the official human evaluation, our model BART-w-CTs ranks 2nd on Sentence-Simplicity (score 92.84), 3rd on Term-Simplicity (score 82.33) among all 7 evaluated systems; It also produced a high score 91.57 on Fluency in comparison to the highest score 93.53. In the second round of submissions, our team using ChatGPT-prompting ranks the 2nd in several categories including simplified term accuracy score 92.26 and completeness score 96.58, and a very similar score on faithfulness score 95.3 to re-evaluated PLABA-base-1 (95.73) via human evaluations. Our codes, fine-tuned models, prompts, and data splits from the system development stage will be available at https://github.com/ HECTA-UoM/PLABA-MU

cs.CL↗

arXiv2024

A SPectroscopic survey of biased halos In the Reionization Era (ASPIRE): Broad-line AGN at $z=4-5$ revealed by JWST/NIRCam WFSS

Low-luminosity AGNs with low-mass black holes (BHs) in the early universe are fundamental to understanding the BH growth and their co-evolution with the host galaxies. Utilizing JWST NIRCam Wide Field Slitless Spectroscopy (WFSS), we perform a systematic search for broad-line ${\rm Hα}$ emitters (BHAEs) at $z\approx 4-5$ in 25 fields of the ASPIRE (A SPectroscopic survey of biased halos In the Reionization Era) project, covering a total area of 275 arcmin$^2$. We identify 16 BHAEs with FWHM of the broad components spanning from $\sim$ 1000 km s$^{-1}$ to 3000 km s$^{-1}$. Assuming the broad linewidths arise due to Doppler broadening around BHs, the implied BH masses range from $10^7$ to $10^{8}~M_\odot$, with broad ${\rm Hα}$-converted bolometric luminosity of $10^{44.5}-10^{45.5}$ erg s$^{-1}$ and Eddington ratios of $0.07-0.47$. The spatially extended structure of the F200W stacked image may trace the stellar light from the host galaxies. The ${\rm Hα}$ luminosity function indicates an increasing AGN fraction towards the higher ${\rm Hα}$ luminosities. We find possible evidence for clustering of BHAEs: two sources are at the same redshift with a projected separation of 519 kpc; one BHAE appears as a composite system residing in an overdense region with three close companion ${\rm Hα}$ emitters. Three BHAEs exhibit blueshifted absorption troughs indicative of the presence of high-column-density gas. We find the broad-line and photometrically selected BHAE samples exhibit different distributions in the optical continuum slopes, which can be attributed to their different selection methods. The ASPIRE broad-line ${\rm Hα}$ sample provides a good database for future studies of faint AGN populations at high redshift.

astro-ph.GA↗

arXiv2024

Unlocking the Potential: Benchmarking Large Language Models in Water Engineering and Research

Recent advancements in Large Language Models (LLMs) have sparked interest in their potential applications across various fields. This paper embarked on a pivotal inquiry: Can existing LLMs effectively serve as "water expert models" for water engineering and research tasks? This study was the first to evaluate LLMs' contributions across various water engineering and research tasks by establishing a domain-specific benchmark suite, namely, WaterER. Herein, we prepared 983 tasks related to water engineering and research, categorized into "wastewater treatment", "environmental restoration", "drinking water treatment and distribution", "sanitation", "anaerobic digestion" and "contaminants assessment". We evaluated the performance of seven LLMs (i.e., GPT-4, GPT-3.5, Gemini, GLM-4, ERNIE, QWEN and Llama3) on these tasks. We highlighted the strengths of GPT-4 in handling diverse and complex tasks of water engineering and water research, the specialized capabilities of Gemini in academic contexts, Llama3's strongest capacity to answer Chinese water engineering questions and the competitive performance of Chinese-oriented models like GLM-4, ERNIE and QWEN in some water engineering tasks. More specifically, current LLMs excelled particularly in generating precise research gaps for papers on "contaminants and related water quality monitoring and assessment". Additionally, they were more adept at creating appropriate titles for research papers on "treatment processes for wastewaters", "environmental restoration", and "drinking water treatment". Overall, this study pioneered evaluating LLMs in water engineering and research by introducing the WaterER benchmark to assess the trustworthiness of their predictions. This standardized evaluation framework would also drive future advancements in LLM technology by using targeting datasets, propelling these models towards becoming true "water expert".

cs.CL↗

arXiv2024

Mapping the Empirical Evidence of the GDPR (In-)Effectiveness: A Systematic Review

In the realm of data protection, a striking disconnect prevails between traditional domains of doctrinal, legal, theoretical, and policy-based inquiries and a burgeoning body of empirical evidence. Much of the scholarly and regulatory discourse remains entrenched in abstract legal principles or normative frameworks, leaving the empirical landscape uncharted or minimally engaged. Since the birth of EU data protection law, a modest body of empirical evidence has been generated but remains widely scattered and unexamined. Such evidence offers vital insights into the perception, impact, clarity, and effects of data protection measures but languishes on the periphery, inadequately integrated into the broader conversation. To make a meaningful connection, we conduct a comprehensive review and synthesis of empirical research spanning nearly three decades (1995- March 2022), advocating for a more robust integration of empirical evidence into the evaluation and review of the GDPR, while laying a methodological foundation for future empirical research.

cs.CY↗