arXiv subjects

Li Chen

Publications and source records attributed to Li Chen.

At least 235 records · Page 13Linked to original sources

arXiv2025

Incentive-Compatible Federated Learning with Stackelberg Game Modeling

Federated Learning (FL) has gained prominence as a decentralized machine learning paradigm, allowing clients to collaboratively train a global model while preserving data privacy. Despite its potential, FL faces significant challenges in heterogeneous environments, where varying client resources and capabilities can undermine overall system performance. Existing approaches primarily focus on maximizing global model accuracy, often at the expense of unfairness among clients and suboptimal system efficiency, particularly in non-IID (non-Independent and Identically Distributed) settings. In this paper, we introduce FLamma, a novel Federated Learning framework based on adaptive gamma-based Stackelberg game, designed to address the aforementioned limitations and promote fairness. Our approach allows the server to act as the leader, dynamically adjusting a decay factor while clients, acting as followers, optimally select their number of local epochs to maximize their utility. Over time, the server incrementally balances client influence, initially rewarding higher-contributing clients and gradually leveling their impact, driving the system toward a Stackelberg Equilibrium. Extensive simulations on both IID and non-IID datasets show that our method significantly improves fairness in accuracy distribution without compromising overall model performance or convergence speed, outperforming traditional FL baselines.

cs.LG↗

arXiv2024

"I Recall the Past": Exploring How People Collaborate with Generative AI to Create Cultural Heritage Narratives

Visitors to cultural heritage sites often encounter official information, while local people's unofficial stories remain invisible. To explore expression of local narratives, we conducted a workshop with 20 participants utilizing Generative AI (GenAI) to support visual narratives, asking them to use Stable Diffusion to create images of familiar cultural heritage sites, as well as images of unfamiliar ones for comparison. The results revealed three narrative strategies and highlighted GenAI's strengths in illuminating, amplifying, and reinterpreting personal narratives. However, GenAI showed limitations in meeting detailed requirements, portraying cultural features, and avoiding bias, which were particularly pronounced with unfamiliar sites due to participants' lack of local knowledge. To address these challenges, we recommend providing detailed explanations, prompt engineering, and fine-tuning AI models to reduce uncertainties, using objective references to mitigate inaccuracies from participants' inability to recognize errors or misconceptions, and curating datasets to train AI models capable of accurately portraying cultural features.

cs.HC↗

arXiv2024

From Pairwise to Ranking: Climbing the Ladder to Ideal Collaborative Filtering with Pseudo-Ranking

Intuitively, an ideal collaborative filtering (CF) model should learn from users' full rankings over all items to make optimal top-K recommendations. Due to the absence of such full rankings in practice, most CF models rely on pairwise loss functions to approximate full rankings, resulting in an immense performance gap. In this paper, we provide a novel analysis using the multiple ordinal classification concept to reveal the inevitable gap between a pairwise approximation and the ideal case. However, bridging the gap in practice encounters two formidable challenges: (1) none of the real-world datasets contains full ranking information; (2) there does not exist a loss function that is capable of consuming ranking information. To overcome these challenges, we propose a pseudo-ranking paradigm (PRP) that addresses the lack of ranking information by introducing pseudo-rankings supervised by an original noise injection mechanism. Additionally, we put forward a new ranking loss function designed to handle ranking information effectively. To ensure our method's robustness against potential inaccuracies in pseudo-rankings, we equip the ranking loss function with a gradient-based confidence mechanism to detect and mitigate abnormal gradients. Extensive experiments on four real-world datasets demonstrate that PRP significantly outperforms state-of-the-art methods.

cs.IR↗

arXiv2024

Unlocking the Hidden Treasures: Enhancing Recommendations with Unlabeled Data

Collaborative filtering (CF) stands as a cornerstone in recommender systems, yet effectively leveraging the massive unlabeled data presents a significant challenge. Current research focuses on addressing the challenge of unlabeled data by extracting a subset that closely approximates negative samples. Regrettably, the remaining data are overlooked, failing to fully integrate this valuable information into the construction of user preferences. To address this gap, we introduce a novel positive-neutral-negative (PNN) learning paradigm. PNN introduces a neutral class, encompassing intricate items that are challenging to categorize directly as positive or negative samples. By training a model based on this triple-wise partial ranking, PNN offers a promising solution to learning complex user preferences. Through theoretical analysis, we connect PNN to one-way partial AUC (OPAUC) to validate its efficacy. Implementing the PNN paradigm is, however, technically challenging because: (1) it is difficult to classify unlabeled data into neutral or negative in the absence of supervised signals; (2) there does not exist any loss function that can handle set-level triple-wise ranking relationships. To address these challenges, we propose a semi-supervised learning method coupled with a user-aware attention model for knowledge acquisition and classification refinement. Additionally, a novel loss function with a two-step centroid ranking approach enables handling set-level rankings. Extensive experiments on four real-world datasets demonstrate that, when combined with PNN, a wide range of representative CF models can consistently and significantly boost their performance. Even with a simple matrix factorization, PNN can achieve comparable performance to sophisticated graph neutral networks.

cs.IR↗

arXiv2024

Decoding fairness: a reinforcement learning perspective

Behavioral experiments on the ultimatum game (UG) reveal that we humans prefer fair acts, which contradicts the prediction made in orthodox Economics. Existing explanations, however, are mostly attributed to exogenous factors within the imitation learning framework. Here, we adopt the reinforcement learning paradigm, where individuals make their moves aiming to maximize their accumulated rewards. Specifically, we apply Q-learning to UG, where each player is assigned two Q-tables to guide decisions for the roles of proposer and responder. In a two-player scenario, fairness emerges prominently when both experiences and future rewards are appreciated. In particular, the probability of successful deals increases with higher offers, which aligns with observations in behavioral experiments. Our mechanism analysis reveals that the system undergoes two phases, eventually stabilizing into fair or rational strategies. These results are robust when the rotating role assignment is replaced by a random or fixed manner, or the scenario is extended to a latticed population. Our findings thus conclude that the endogenous factor is sufficient to explain the emergence of fairness, exogenous factors are not needed.

cs.LG↗

arXiv2024

Class-RAG: Real-Time Content Moderation with Retrieval Augmented Generation

Robust content moderation classifiers are essential for the safety of Generative AI systems. In this task, differences between safe and unsafe inputs are often extremely subtle, making it difficult for classifiers (and indeed, even humans) to properly distinguish violating vs. benign samples without context or explanation. Scaling risk discovery and mitigation through continuous model fine-tuning is also slow, challenging and costly, preventing developers from being able to respond quickly and effectively to emergent harms. We propose a Classification approach employing Retrieval-Augmented Generation (Class-RAG). Class-RAG extends the capability of its base LLM through access to a retrieval library which can be dynamically updated to enable semantic hotfixing for immediate, flexible risk mitigation. Compared to model fine-tuning, Class-RAG demonstrates flexibility and transparency in decision-making, outperforms on classification and is more robust against adversarial attack, as evidenced by empirical studies. Our findings also suggest that Class-RAG performance scales with retrieval library size, indicating that increasing the library size is a viable and low-cost approach to improve content moderation.

cs.AI↗

arXiv2024

Model-GLUE: Democratized LLM Scaling for A Large Model Zoo in the Wild

As Large Language Models (LLMs) excel across tasks and specialized domains, scaling LLMs based on existing models has garnered significant attention, which faces the challenge of decreasing performance when combining disparate models. Various techniques have been proposed for the aggregation of pre-trained LLMs, including model merging, Mixture-of-Experts, and stacking. Despite their merits, a comprehensive comparison and synergistic application of them to a diverse model zoo is yet to be adequately addressed. In light of this research gap, this paper introduces Model-GLUE, a holistic LLM scaling guideline. First, our work starts with a benchmarking of existing LLM scaling techniques, especially selective merging, and variants of mixture. Utilizing the insights from the benchmark results, we formulate an optimal strategy for the selection and aggregation of a heterogeneous model zoo characterizing different architectures and initialization.Our methodology involves the clustering of mergeable models and optimal merging strategy selection, and the integration of clusters through a model mixture. Finally, evidenced by our experiments on a diverse Llama-2-based model zoo, Model-GLUE shows an average performance enhancement of 5.61%, achieved without additional training. Codes are available at: https://github.com/Model-GLUE/Model-GLUE.

cs.LG↗

arXiv2024

ALMA Observations of Massive Clouds in the Central Molecular Zone: External-Pressure-Confined Dense Cores and Salpeter-like Core Mass Functions

We present Atacama Large Millimeter/submillimeter Array (ALMA) Band 6 (1.3 mm) observations of dense cores in three massive molecular clouds within the Central Molecular Zone (CMZ) of the Milky Way, including the Dust Ridge cloud e, Sgr C, and the 20 km s-1 cloud, at a spatial resolution of 2000 au. Among the 834 cores identified from the 1.3 mm continuum, we constrain temperatures and linewidths of 253 cores using local thermodynamic equilibrium (LTE) methods to fit the H2CO and/or CH3CN spectra. We determine their masses using the 1.3 mm dust continuum and derived temperatures, and then evaluate their virial parameters using the H2CO and/or CH3CN linewidths and construct the core mass functions (CMFs). We find that the contribution of external pressure is crucial for the virial equilibrium of the dense cores in the three clouds, which contrasts with the environment in the Galactic disk where dense cores are already bound even without the contribution of external pressure. We also find that the CMFs show a Salpeter-like slope in the high-mass (>~3-6 Msun) end, a change from previous works with our new temperature estimates. Combined with the possible top-heavy initial mass functions (IMFs) in the CMZ, our result suggests that gas accretion and further fragmentation may play important roles in transforming the CMF to the IMF.

astro-ph.GA↗

arXiv2024

Integrating Secondary Structures Information into Triangular Spatial Relationships (TSR) for Advanced Protein Classification

Protein structures represent the key to deciphering biological functions. The more detailed form of similarity among these proteins is sometimes overlooked by the conventional structural comparison methods. In contrast, further advanced methods, such as Triangular Spatial Relationship (TSR), have been demonstrated to make finer differentiations. Still, the classical implementation of TSR does not provide for the integration of secondary structure information, which is important for a more detailed understanding of the folding pattern of a protein. To overcome these limitations, we developed the SSE-TSR approach. The proposed method integrates secondary structure elements (SSEs) into TSR-based protein representations. This allows an enriched representation of protein structures by considering 18 different combinations of helix, strand, and coil arrangements. Our results show that using SSEs improves the accuracy and reliability of protein classification to varying degrees. We worked with two large protein datasets of 9.2K and 7.8K samples, respectively. We applied the SSE-TSR approach and used a neural network model for classification. Interestingly, introducing SSEs improved performance statistics for Dataset 1, with accuracy moving from 96.0% to 98.3%. For Dataset 2, where the performance statistics were already good, further small improvements were found with the introduction of SSE, giving an accuracy of 99.5% compared to 99.4%. These results show that SSE integration can dramatically improve TSR key discrimination, with significant benefits in datasets with low initial accuracies and only incremental gains in those with high baseline performance. Thus, SSE-TSR is a powerful bioinformatics tool that improves protein classification and understanding of protein function and interaction.

cs.LG↗

arXiv2024

Finite-Precision Arithmetic Transceiver for Massive MIMO Systems

Efficient implementation of massive multiple-input-multiple-output (MIMO) transceivers is essential for the next-generation wireless networks. To reduce the high computational complexity of the massive MIMO transceiver, in this paper, we propose a new massive MIMO architecture using finite-precision arithmetic. First, we conduct the rounding error analysis and derive the lower bound of the achievable rate for single-input-multiple-output (SIMO) using maximal ratio combining (MRC) and multiple-input-single-output (MISO) systems using maximal ratio transmission (MRT) with finite-precision arithmetic. Then, considering the multi-user scenario, the rounding error analysis of zero-forcing (ZF) detection and precoding is derived by using the normal equations (NE) method. The corresponding lower bounds of the achievable sum rate are also derived and asymptotic analyses are presented. Built upon insights from these analyses and lower bounds, we propose a mixed-precision architecture for massive MIMO systems to offset performance gaps due to finite-precision arithmetic. The corresponding analysis of rounding errors and computational costs is obtained. Simulation results validate the derived bounds and underscore the superiority of the proposed mixed-precision architecture to the conventional structure.

cs.IT↗

arXiv2024

Strategic Responses to Personalized Pricing and Demand for Privacy: An Experiment

We consider situations where consumers are aware that a statistical model determines the price of a product based on their observed behavior. Using a novel experiment varying the context similarity between participant data and a product, we find that participants manipulate their responses to a survey about personal characteristics, and manipulation is more successful when the contexts are similar. Moreover, participants demand less privacy, and make less optimal privacy choices when the contexts are less similar. Our findings highlight the importance of data privacy policies in the age of big data, where behavior in seemingly unrelated contexts might affect prices.

econ.GN↗

arXiv2024

Unveiling the Binary Nature of NGC 2323

As a well-known open cluster, NGC 2323 (also called M50) has been widely investigated for over a hundred years and has always been considered a classical single cluster. In this work, with the help of Gaia DR3, we study the binary structure nature of this cluster. Although indistinguishable in the spatial space, the small but undeniable difference in the proper motion indicates that they may be two individual clusters. After investigating the properties of the two clusters, it is found that they have very close positions (three-dimensional $Δ$pos = 12.3 pc, $σ_{Δ\mathrm{pos}} = 3.4$ pc) and similar tangential velocities (two-dimensional $Δ$V = 2.2 km s$^{-1}$, $σ_{Δ\mathrm{V}} = 0.02$ km s$^{-1}$), indicating the existence of their physical association. Moreover, the best isochrone fitting ages of the two clusters are the same (158 Myr), further proving their possibly common origin. To comprehensively understand the formation and evolution of this binary cluster, we employ the PETAR $N$-body code to trace back their birthplace and deduce their dynamical evolutionary fate. With observational mean cluster properties, the simulations suggest that they may form together, and then orbit each other as a binary cluster for over 200 Myr. After that, because of their gradual mass loss, the two clusters will eventually separate and evolve into two independent clusters. Meanwhile, the numerical $N$-body simulation suggests that the less massive cluster is unlikely to be the cluster tidal tails created by the differential rotation of the Milky Way.

astro-ph.GA↗

arXiv2024

Pistis-RAG: Enhancing Retrieval-Augmented Generation with Human Feedback

RAG systems face limitations when semantic relevance alone does not guarantee improved generation quality. This issue becomes particularly evident due to the sensitivity of large language models (LLMs) to the ordering of few-shot prompts, which can affect model performance. To address this challenge, aligning LLM outputs with human preferences using structured feedback, such as options to copy, regenerate, or dislike, offers a promising method for improvement. This feedback is applied to the entire list of inputs rather than giving specific ratings for individual documents, making it a Listwide Labels Learning-to-Rank task. To address this task, we propose Pistis-RAG, a new RAG framework designed with a content-centric approach to better align LLMs with human preferences. Pistis-RAG effectively utilizes human feedback, enhancing content ranking and generation quality. To validate our framework, we use public datasets to simulate human feedback, allowing us to evaluate and refine our method effectively. Experimental results indicate that Pistis-RAG improves alignment with human preferences relative to the baseline RAG system, showing a 6.06% increase in MMLU (English) and a 7.08% increase in C-EVAL (Chinese) accuracy metrics. These results highlight Pistis-RAG's effectiveness in overcoming the limitations associated with traditional RAG approaches.

cs.IR↗

arXiv2024

Vista: A Generalizable Driving World Model with High Fidelity and Versatile Controllability

World models can foresee the outcomes of different actions, which is of paramount importance for autonomous driving. Nevertheless, existing driving world models still have limitations in generalization to unseen environments, prediction fidelity of critical details, and action controllability for flexible application. In this paper, we present Vista, a generalizable driving world model with high fidelity and versatile controllability. Based on a systematic diagnosis of existing methods, we introduce several key ingredients to address these limitations. To accurately predict real-world dynamics at high resolution, we propose two novel losses to promote the learning of moving instances and structural information. We also devise an effective latent replacement approach to inject historical frames as priors for coherent long-horizon rollouts. For action controllability, we incorporate a versatile set of controls from high-level intentions (command, goal point) to low-level maneuvers (trajectory, angle, and speed) through an efficient learning strategy. After large-scale training, the capabilities of Vista can seamlessly generalize to different scenarios. Extensive experiments on multiple datasets show that Vista outperforms the most advanced general-purpose video generator in over 70% of comparisons and surpasses the best-performing driving world model by 55% in FID and 27% in FVD. Moreover, for the first time, we utilize the capacity of Vista itself to establish a generalizable reward for real-world action evaluation without accessing the ground truth actions.

cs.CV↗

arXiv2024

Imaginary-Temperature Zeros for Quantum Phase Transitions

While the zeros of complex partition functions, such as Lee-Yang zeros and Fisher zeros, have been pivotal in characterizing temperature-driven phase transitions, extending this concept to zero temperature remains an open question. In this work, we propose a solution to this issue by calculating the imaginary-temperature zeros (ITZs), which are defined as the roots of the imaginary-temperature partition function. We illustrate the analytical properties of ITZs in the transverse-field Ising chain, showing that the ITZs' distribution can distinguish between various phases and signify the critical exponents. Universal singular behaviors manifest in such quantities as the edge density of ITZs and the magnetization, with the scaling exponents remarkably differing from those in Lee-Yang theory. We further illuminate the consistency between ITZs and the zeros of the spectral form factor, which offers a practical path for the experimental detection of ITZs.

quant-ph↗

arXiv2024

Mean-Field Control for Diffusion Aggregation system with Coulomb Interaction

The mean-field control problem for a multi-dimensional diffusion-aggregation system with Coulomb interaction (the so called parabolic elliptic Keller-Segel system) is considered. The existence of optimal control is proved through the $Γ$-convergence of the corresponding control problem of the interacting particle system. There are three building blocks in the whole argument. Firstly, for the optimal control problem on the particle level, instead of using classical method for stochastic system, we study directly the control problem of high-dimensional parabolic equation, i.e. the Liouville equation of it. Secondly, we obtain a strong propagation of chaos result for the interacting particle system by combining the convergence in probability and relative entropy method. Due to this strong mean field limit result, we avoid giving compact support requirement for control functions, which has been often used in the literature. Thirdly, because of strong aggregation effect, additional difficulties arise from control function in obtaining the well-posedness theory of the diffusion-aggregation equation, so that the known method cannot be directly applied. Instead, we use a combination of local existence result and bootstrap argument to obtain the global solution in the sub-critical regime.

math.OC↗

arXiv2024

Closed-Loop Visuomotor Control with Generative Expectation for Robotic Manipulation

Despite significant progress in robotics and embodied AI in recent years, deploying robots for long-horizon tasks remains a great challenge. Majority of prior arts adhere to an open-loop philosophy and lack real-time feedback, leading to error accumulation and undesirable robustness. A handful of approaches have endeavored to establish feedback mechanisms leveraging pixel-level differences or pre-trained visual representations, yet their efficacy and adaptability have been found to be constrained. Inspired by classic closed-loop control systems, we propose CLOVER, a closed-loop visuomotor control framework that incorporates feedback mechanisms to improve adaptive robotic control. CLOVER consists of a text-conditioned video diffusion model for generating visual plans as reference inputs, a measurable embedding space for accurate error quantification, and a feedback-driven controller that refines actions from feedback and initiates replans as needed. Our framework exhibits notable advancement in real-world robotic tasks and achieves state-of-the-art on CALVIN benchmark, improving by 8% over previous open-loop counterparts. Code and checkpoints are maintained at https://github.com/OpenDriveLab/CLOVER.

cs.RO↗

arXiv2024

Deep Transfer Learning-based Detection for Flash Memory Channels

The NAND flash memory channel is corrupted by different types of noises, such as the data retention noise and the wear-out noise, which lead to unknown channel offset and make the flash memory channel non-stationary. In the literature, machine learning-based methods have been proposed for data detection for flash memory channels. However, these methods require a large number of training samples and labels to achieve a satisfactory performance, which is costly. Furthermore, with a large unknown channel offset, it may be impossible to obtain enough correct labels. In this paper, we reformulate the data detection for the flash memory channel as a transfer learning (TL) problem. We then propose a model-based deep TL (DTL) algorithm for flash memory channel detection. It can effectively reduce the training data size from $10^6$ samples to less than 104 samples. Moreover, we propose an unsupervised domain adaptation (UDA)-based DTL algorithm using moment alignment, which can detect data without any labels. Hence, it is suitable for scenarios where the decoding of error-correcting code fails and no labels can be obtained. Finally, a UDA-based threshold detector is proposed to eliminate the need for a neural network. Both the channel raw error rate analysis and simulation results demonstrate that the proposed DTL-based detection schemes can achieve near-optimal bit error rate (BER) performance with much less training data and/or without using any labels.

cs.IT↗