Searcharxiv⌕ Search

arXiv subjects

Tian Li

Publications and source records attributed to Tian Li.

At least 55 records · Page 3Linked to original sources

Quantifying the Privacy Implications of High-Fidelity Synthetic Network Traffic

To address the scarcity and privacy concerns of network traffic data, various generative models have been developed to produce synthetic traffic. However, synthetic traffic is not inherently privacy-preserving, and the extent to which it leaks sensitive information, and how to measure such leakage, remain largely unexplored. This challenge is further compounded by the diversity of model architectures, which shape how traffic is represented and synthesized. We introduce a comprehensive set of privacy metrics for synthetic network traffic, combining standard approaches like membership inference attacks (MIA) and data extraction attacks with network-specific identifiers and attributes. Using these metrics, we systematically evaluate the vulnerability of different representative generative models and examine the factors that influence attack success. Our results reveal substantial variability in privacy risks across models and datasets. MIA success ranges from 0% to 88%, and up to 100% of network identifiers can be recovered from generated traffic, highlighting serious privacy vulnerabilities. We further identify key factors that significantly affect attack outcomes, including training data diversity and how well the generative model fits the training data. These findings provide actionable guidance for designing and deploying generative models that minimize privacy leakage, establishing a foundation for safer synthetic network traffic generation.

cs.AI↗

CSST Strong Lensing Preparation: Cosmological Constraints Forecast from CSST Galaxy-Scale Strong Lensing

Strong gravitational lensing by galaxies is a powerful tool for studying cosmology and galaxy structure. The China Space Station Telescope (CSST) will revolutionize this field by discovering up to $\sim$100,000 galaxy-scale strong lenses, a huge increase over current samples. To harness the statistical power of this vast dataset, we forecast its cosmological constraining power using the gravitational-dynamical mass combination method. We create a realistic simulated lens sample and test how uncertainties in redshift and velocity dispersion measurements affect results under ideal, optimistic, and pessimistic scenarios. We find that increasing the sample size from 100 to 10,000 systems dramatically improves precision: in the $Λ$CDM model, the uncertainty on the matter density parameter, $Ω_m$, drops from 0.2 to 0.01; in the $w$CDM model, the uncertainty on the dark energy equation of state, $w$, decreases from 0.3 to 0.04. With 10,000 lenses, our constraints on dark energy are twice as tight as those from the latest DESI BAO measurements. We also compare two parameter estimation techniques -- MultiNest sampling and Bayesian Hierarchical Modeling (BHM). While both achieve similar precision, BHM provides more robust estimates of intrinsic lens parameters, whereas MultiNest is about twice as fast. This work establishes an efficient and scalable framework for cosmological analysis with next-generation strong lensing surveys.

astro-ph.CO↗

Large-scale automatic carbon ion treatment planning for head and neck cancers via parallel multi-agent reinforcement learning

Head-and-neck cancer (HNC) planning is difficult because multiple critical organs-at-risk (OARs) are close to complex targets. Intensity-modulated carbon-ion therapy (IMCT) offers superior dose conformity and OAR sparing but remains slow due to relative biological effectiveness (RBE) modeling, leading to laborious, experience-based, and often suboptimal tuning of many treatment-planning parameters (TPPs). Recent deep learning (DL) methods are limited by data bias and plan feasibility, while reinforcement learning (RL) struggles to efficiently explore the exponentially large TPP search space. We propose a scalable multi-agent RL (MARL) framework for parallel tuning of 45 TPPs in IMCT. It uses a centralized-training decentralized-execution (CTDE) QMIX backbone with Double DQN, Dueling DQN, and recurrent encoding (DRQN) for stable learning in a high-dimensional, non-stationary environment. To enhance efficiency, we (1) use compact historical DVH vectors as state inputs, (2) apply a linear action-to-value transform mapping small discrete actions to uniform parameter adjustments, and (3) design an absolute, clinically informed piecewise reward aligned with plan scores. A synchronous multi-process worker system interfaces with the PHOENIX TPS for parallel optimization and accelerated data collection. On a head-and-neck dataset (10 training, 10 testing), the method tuned 45 parameters simultaneously and produced plans comparable to or better than expert manual ones (relative plan score: RL $85.93\pm7.85%$ vs Manual $85.02\pm6.92%$), with significant (p-value $<$ 0.05) improvements for five OARs. The framework efficiently explores high-dimensional TPP spaces and generates clinically competitive IMCT plans through direct TPS interaction, notably improving OAR sparing.

cs.LG↗

Efficient Adaptive Federated Optimization

Adaptive optimization is critical in federated learning, where enabling adaptivity on both the server and client sides has proven essential for achieving optimal performance. However, the scalability of such jointly adaptive systems is often hindered by resource limitations in communication and memory. In this paper, we introduce a class of efficient adaptive algorithms, named $FedAda^2$ and its enhanced version $FedAda^2$++, designed specifically for large-scale, cross-device federated environments. $FedAda^2$ optimizes communication efficiency by avoiding the transfer of preconditioners between the server and clients. Additionally, $FedAda^2$++ extends this approach by incorporating memory-efficient adaptive optimizers on the client side, further reducing on-device memory usage. Theoretically, we demonstrate that $FedAda^2$ and $FedAda^2$++ achieve the same convergence rates for general, non-convex objectives as its more resource-intensive counterparts that directly integrate joint adaptivity. Extensive empirical evaluations on image and text datasets demonstrate both the advantages of joint adaptivity and the effectiveness of $FedAda^2$/$FedAda^2$++.

cs.LG↗

Zeroth-Order Sharpness-Aware Learning with Exponential Tilting

Classic zeroth-order optimization approaches typically optimize for a smoothed version of the original function, i.e., the expected objective under randomly perturbed model parameters. This can be interpreted as encouraging the loss values in the perturbation set to be small on average. Popular sharpness-aware minimization (SAM) objectives, however, typically focus on the largest loss within the neighborhood to arrive at flat minima more effectively. In this work, we connect zeroth-order optimization (and its corresponding objectives) with SAM approaches explicitly, through an exponential tilting objective that provides a smooth transition between the average- and the max-loss formulations. We explore new zeroth-order algorithms to solve a soft SAM objective parameterized by a tilting parameter $t$. We provide precise characterizations of the sharpness notions of the tilted SAM framework. Practically, our approach can be used as a gradient-free and memory-efficient alternative to SAM variants, and it achieves better generalization compared to vanilla zeroth-order baselines on a wide range of downstream tasks, including classification, multiple choice QA, and language generation.

cs.LG↗

Line-of-sight effects on double source plane lenses

Weak gravitational lensing perturbations have a non-negligible impact on strong lensing observables, and several degeneracies exist between the properties of the main lens, line of sight, and cosmology. In this work, we consider the impact of the line of sight on double-source-plane lenses (DSPLs), a rare class of lens systems in which two sources at different redshifts are lensed by the same foreground galaxy, and which enable competitive constraints on the dark energy equation of state. Generating and sampling statistically representative lines of sight from N-body simulations, we show that line-of-sight perturbations add a $\sim1\%$ uncertainty to measurements of the cosmological scaling factor $η$ (a ratio of angular diameter distance ratios), which is subdominant but non-negligible compared to the measurement error. We also show that the line-of-sight shear experienced by images of the two sources can differ significantly in both magnitude and direction. Including a line-of-sight error budget, we measure $w=-1.17^{+0.19}_{-0.21}$ from the Jackpot DSPL in combination with Planck. We show that the line of sight is expected to introduce an additional scatter in the constraints possible with a larger sample of DSPLs from Euclid, but that this scatter is subdominant compared to other sources of error.

astro-ph.CO↗

The revolution in strong lensing discoveries from Euclid

Strong gravitational lensing offers a powerful and direct probe of dark matter, galaxy evolution and cosmology, yet strong lenses are rare: only 1 in roughly 10,000 massive galaxies can lens a background source into multiple images. The European Space Agency's Euclid telescope, with its unique combination of high-resolution imaging and wide-area sky coverage, is set to transform this field. In its first quick data release, covering just 0.45% of the full survey area, around 500 high-quality strong lens candidates have been identified using a synergy of machine learning, citizen science and expert visual inspection. This dataset includes exotic systems such as compound lenses and edge-on disk lenses, demonstrating Euclid's capacity to probe the lens parameter space. The machine learning models developed to discover strong lenses in Euclid data are able to find lenses with high purity rates, confirming that the mission's forecast of discovering over 100,000 strong lenses is achievable during its 6-year mission. This will increase the number of known strong lenses by two orders of magnitude, transforming the science that can be done with strong lensing.

astro-ph.GA↗

Efficient Distributed Optimization under Heavy-Tailed Noise

Distributed optimization has become the default training paradigm in modern machine learning due to the growing scale of models and datasets. To mitigate communication overhead, local updates are often applied before global aggregation, resulting in a nested optimization approach with inner and outer steps. However, heavy-tailed stochastic gradient noise remains a significant challenge, particularly in attention-based models, hindering effective training. In this work, we propose TailOPT, an efficient framework designed to address heavy-tailed noise by leveraging adaptive optimization or clipping techniques. We establish convergence guarantees for the TailOPT framework under heavy-tailed noise with potentially unbounded gradient variance and local updates. Among its variants, we highlight a memory and communication efficient instantiation which we call $Bi^2Clip$, which performs coordinate-wise clipping at both the inner and outer optimizers, achieving adaptive-like performance (e.g., Adam) without the cost of maintaining or transmitting additional gradient statistics. Empirically, TailOPT, including $Bi^2Clip$, demonstrates superior performance on several language tasks and models, outperforming state-of-the-art methods.

cs.LG↗

Differentially Private Federated Clustering with Random Rebalancing

Federated clustering aims to group similar clients into clusters and produce one model for each cluster. Such a personalization approach typically improves model performance compared with training a single model to serve all clients, but can be more vulnerable to privacy leakage. Directly applying client-level differentially private (DP) mechanisms to federated clustering could degrade the utilities significantly. We identify that such deficiencies are mainly due to the difficulties of averaging privacy noise within each cluster (following standard privacy mechanisms), as the number of clients assigned to the same clusters is uncontrolled. To this end, we propose a simple and effective technique, named RR-Cluster, that can be viewed as a light-weight add-on to many federated clustering algorithms. RR-Cluster achieves reduced privacy noise via randomly rebalancing cluster assignments, guaranteeing a minimum number of clients assigned to each cluster. We analyze the tradeoffs between decreased privacy noise variance and potentially increased bias from incorrect assignments and provide convergence bounds for RR-Clsuter. Empirically, we demonstrate the RR-Cluster plugged into strong federated clustering algorithms results in significantly improved privacy/utility tradeoffs across both synthetic and real-world datasets.

cs.LG↗

CSST Strong Lensing Preparation: Fast Modeling of Galaxy-Galaxy Strong Lenses in the Big Data Era

Galaxy-galaxy strong lensing provides a powerful probe of galaxy formation, evolution, and the properties of dark matter and dark energy. However, conventional lens-modeling approaches are computationally expensive and require fine-tuning to avoid local optima, rendering them impractical for the hundreds of thousands of lenses expected from surveys such as Euclid, CSST, and Roman Space Telescopes. To overcome these challenges, we introduce TinyLensGPU, a GPU-accelerated lens-modeling tool that employs XLA-based acceleration with JAX and a neural-network-enhanced nested sampling algorithm, nautilus-sampler. Tests on 1,000 simulated galaxy-galaxy lenses demonstrate that on an RTX 4060 Ti GPU, TinyLensGPU achieves likelihood evaluations approximately 2,000 times faster than traditional methods. Moreover, the nautilus-sampler reduces the number of likelihood evaluations by a factor of 3, decreasing the overall modeling time per lens from several days to roughly 3 minutes. Application to 63 SLACS lenses observed by the Hubble Space Telescope recovers Einstein radii consistent with the literature values (within $\lesssim 5\%$ deviation), which is within known systematic uncertainties. Catastrophic failures, where the sampler becomes trapped in local optima, occur in approximately 5\% of the simulated cases and 10\% of the SLACS sample. We argue that such issues are inherent to automated lens modeling but can be mitigated by incorporating prior knowledge from machine learning techniques. This work thus marks a promising step toward the efficient analysis of strong lenses in the era of big data. The code and data are available online: https://github.com/caoxiaoyue/TinyLensGpu.

astro-ph.IM↗

iShumei-Chinchunmei at SemEval-2025 Task 4: A balanced forgetting and retention multi-task framework using effective unlearning loss

As the Large Language Model (LLM) gains widespread adoption, increasing attention has been given to the challenge of making LLM forget non-compliant data memorized during its pre-training. Machine Unlearning focuses on efficiently erasing sensitive information from LLM under limited computational resources. To advance research in this area, SemEval 2025 Task 4: "Unlearning Sensitive Content from Large Language Models" introduces three unlearning datasets and establishes a benchmark by evaluating both forgetting effectiveness and the preservation of standard capabilities. In this work, we propose a more controllable forgetting loss, Effective Unlearning Loss, and explore its integration with various techniques to achieve more efficient and controlled unlearning. Our system ultimately ranked 5th on the competition leaderboard.

cs.CL↗

Chinchunmei at SemEval-2025 Task 11: Boosting the Large Language Model's Capability of Emotion Perception using Contrastive Learning

The SemEval-2025 Task 11, Bridging the Gap in Text-Based Emotion Detection, introduces an emotion recognition challenge spanning over 28 languages. This competition encourages researchers to explore more advanced approaches to address the challenges posed by the diversity of emotional expressions and background variations. It features two tracks: multi-label classification (Track A) and emotion intensity prediction (Track B), covering six emotion categories: anger, fear, joy, sadness, surprise, and disgust. In our work, we systematically explore the benefits of two contrastive learning approaches: sample-based (Contrastive Reasoning Calibration) and generation-based (DPO, SimPO) contrastive learning. The sample-based contrastive approach trains the model by comparing two samples to generate more reliable predictions. The generation-based contrastive approach trains the model to differentiate between correct and incorrect generations, refining its prediction. All models are fine-tuned from LLaMa3-Instruct-8B. Our system achieves 9th place in Track A and 6th place in Track B for English, while ranking among the top-tier performing systems for other languages.

cs.CL↗

Unveiling a 36 Billion Solar Mass Black Hole at the Centre of the Cosmic Horseshoe Gravitational Lens

Supermassive black holes (SMBHs) are found at the centre of every massive galaxy, with their masses tightly connected to their host galaxies through a co-evolution over cosmic time. For massive ellipticals, the SMBH mass ($M_\text{BH}$) strongly correlates with the central stellar velocity dispersion ($σ_e$), via the $M_\text{BH}-σ_e$ relation. However, SMBH mass measurements have traditionally relied on central stellar dynamics in nearby galaxies ($z < 0.1$), limiting our ability to explore the SMBHs across cosmic time. In this work, we present a self-consistent analysis combining 2D stellar dynamics and lens modelling of the Cosmic Horseshoe gravitational lens system ($z = 0.44$), one of the most massive galaxies ever observed. Using integral-field spectroscopic data from MUSE and high-resolution imaging from HST, we model the radial arc and stellar kinematics, constraining the galaxy's central mass distribution and SMBH mass. Bayesian model comparison yields a $5σ$ detection of an ultramassive black hole (UMBH) with $\log_{10}(M_\text{BH}/M_{\odot}) = 10.56^{+0.07}_{-0.08} \pm (0.12)^\text{sys}$, consistent across various systematic tests. Our findings place the Cosmic Horseshoe $\sim$$1.5σ$ above the $M_\text{BH}-σ_e$ relation, supporting an emerging trend observed in BGCs and other massive galaxies. This suggests a steeper $M_\text{BH}-σ_e$ relationship at the highest masses, potentially driven by a different co-evolution of SMBHs and their host galaxies. Future surveys will uncover more radial arcs, enabling the detection of SMBHs over a broader redshift and mass range. These discoveries will further refine our understanding of the $M_\text{BH}-σ_e$ relation and its evolution across cosmic time.

astro-ph.GA↗

Stochastic Alignments: Matching an Observed Trace to Stochastic Process Models

Process mining leverages event data extracted from IT systems to generate insights into the business processes of organizations. Such insights benefit from explicitly considering the frequency of behavior in business processes, which is captured by stochastic process models. Given an observed trace and a stochastic process model, conventional alignment-based conformance checking techniques face a fundamental limitation: They prioritize matching the trace to a model path with minimal deviations, which may, however, lead to selecting an unlikely path. In this paper, we study the problem of matching an observed trace to a stochastic process model by identifying a likely model path with a low edit distance to the trace. We phrase this as an optimization problem and develop a heuristic-guided path-finding algorithm to solve it. Our open-source implementation demonstrates the feasibility of the approach and shows that it can provide new, useful diagnostic insights for analysts.

cs.FL↗

MTLM: Incorporating Bidirectional Text Information to Enhance Language Model Training in Speech Recognition Systems

Automatic speech recognition (ASR) systems normally consist of an acoustic model (AM) and a language model (LM). The acoustic model estimates the probability distribution of text given the input speech, while the language model calibrates this distribution toward a specific knowledge domain to produce the final transcription. Traditional ASR-specific LMs are typically trained in a unidirectional (left-to-right) manner to align with autoregressive decoding. However, this restricts the model from leveraging the right-side context during training, limiting its representational capacity. In this work, we propose MTLM, a novel training paradigm that unifies unidirectional and bidirectional manners through 3 training objectives: ULM, BMLM, and UMLM. This approach enhances the LM's ability to capture richer linguistic patterns from both left and right contexts while preserving compatibility with standard ASR autoregressive decoding methods. As a result, the MTLM model not only enhances the ASR system's performance but also support multiple decoding strategies, including shallow fusion, unidirectional/bidirectional n-best rescoring. Experiments on the LibriSpeech dataset show that MTLM consistently outperforms unidirectional training across multiple decoding strategies, highlighting its effectiveness and flexibility in ASR applications.

cs.CL↗

Tilted Sharpness-Aware Minimization

Sharpness-Aware Minimization (SAM) has been demonstrated to improve the generalization performance of overparameterized models by seeking flat minima on the loss landscape through optimizing model parameters that incur the largest loss within a neighborhood. Nevertheless, such min-max formulations are computationally challenging especially when the problem is highly non-convex. Additionally, focusing only on the worst-case local solution while ignoring potentially many other local solutions may be suboptimal when searching for flat minima. In this work, we propose Tilted SAM (TSAM), a smoothed generalization of SAM inspired by exponential tilting that effectively assigns higher priority to local solutions that incur larger losses. TSAM is parameterized by a tilt hyperparameter $t$ and reduces to SAM as $t$ approaches infinity. We show that TSAM is smoother than SAM and thus easier to optimize, and it explicitly favors flatter minima. We develop algorithms motivated by the discretization of Hamiltonian dynamics to solve TSAM. Empirically, TSAM arrives at flatter local minima and results in superior test performance than the baselines of SAM and ERM across a range of image and text tasks.

cs.LG↗

Generalization and Robustness of the Tilted Empirical Risk

The generalization error (risk) of a supervised statistical learning algorithm quantifies its prediction ability on previously unseen data. Inspired by exponential tilting, \citet{li2020tilted} proposed the {\it tilted empirical risk} (TER) as a non-linear risk metric for machine learning applications such as classification and regression problems. In this work, we examine the generalization error of the tilted empirical risk in the robustness regime under \textit{negative tilt}. Our first contribution is to provide uniform and information-theoretic bounds on the {\it tilted generalization error}, defined as the difference between the population risk and the tilted empirical risk, under negative tilt for unbounded loss function under bounded $(1+ε)$-th moment of loss function for some $ε\in(0,1]$ with a convergence rate of $O(n^{-ε/(1+ε)})$ where $n$ is the number of training samples, revealing a novel application for TER under no distribution shift. Secondly, we study the robustness of the tilted empirical risk with respect to noisy outliers at training time and provide theoretical guarantees under distribution shift for the tilted empirical risk. We empirically corroborate our findings in simple experimental setups where we evaluate our bounds to select the value of tilt in a data-driven manner.

stat.ML↗

Topology-Aware Knowledge Propagation in Decentralized Learning

Decentralized learning enables collaborative training of models across naturally distributed data without centralized coordination or maintenance of a global model. Instead, devices are organized in arbitrary communication topologies, in which they can only communicate with neighboring devices. Each device maintains its own local model by training on its local data and integrating new knowledge via model aggregation with neighbors. Therefore, knowledge is propagated across the topology via successive aggregation rounds. We study, in particular, the propagation of out-of-distribution (OOD) knowledge. We find that popular decentralized learning algorithms struggle to propagate OOD knowledge effectively to all devices. Further, we find that both the location of OOD data within a topology, and the topology itself, significantly impact OOD knowledge propagation. We then propose topology-aware aggregation strategies to accelerate (OOD) knowledge propagation across devices. These strategies improve OOD data accuracy, compared to topology-unaware baselines, by 123% on average across models in a topology.

cs.LG↗