arXiv subjects

Ke Yang

Publications and source records attributed to Ke Yang.

At least 73 records · Page 4Linked to original sources

arXiv2025

ADEPT: A DEbiasing PrompT Framework

Several works have proven that finetuning is an applicable approach for debiasing contextualized word embeddings. Similarly, discrete prompts with semantic meanings have shown to be effective in debiasing tasks. With unfixed mathematical representation at the token level, continuous prompts usually surpass discrete ones at providing a pre-trained language model (PLM) with additional task-specific information. Despite this, relatively few efforts have been made to debias PLMs by prompt tuning with continuous prompts compared to its discrete counterpart. Furthermore, for most debiasing methods that alter a PLM's original parameters, a major problem is the need to not only decrease the bias in the PLM but also to ensure that the PLM does not lose its representation ability. Finetuning methods typically have a hard time maintaining this balance, as they tend to violently remove meanings of attribute words. In this paper, we propose ADEPT, a method to debias PLMs using prompt tuning while maintaining the delicate balance between removing biases and ensuring representation ability. To achieve this, we propose a new training criterion inspired by manifold learning and equip it with an explicit debiasing term to optimize prompt tuning. In addition, we conduct several experiments with regard to the reliability, quality, and quantity of a previously proposed attribute training corpus in order to obtain a clearer prototype of a certain attribute, which indicates the attribute's position and relative distances to other words on the manifold. We evaluate ADEPT on several widely acknowledged debiasing benchmarks and downstream tasks, and find that it achieves competitive results while maintaining (and in some cases even improving) the PLM's representation ability. We further visualize words' correlation before and after debiasing a PLM, and give some possible explanations for the visible effects.

cs.CL↗

arXiv2025

AgentOccam: A Simple Yet Strong Baseline for LLM-Based Web Agents

Autonomy via agents using large language models (LLMs) for personalized, standardized tasks boosts human efficiency. Automating web tasks (like booking hotels within a budget) is increasingly sought after. Fulfilling practical needs, the web agent also serves as an important proof-of-concept example for various agent grounding scenarios, with its success promising advancements in many future applications. Prior research often handcrafts web agent strategies (e.g., prompting templates, multi-agent systems, search methods, etc.) and the corresponding in-context examples, which may not generalize well across all real-world scenarios. On the other hand, there has been limited study on the misalignment between a web agent's observation/action representation and the pre-training data of the LLM it's based on. This discrepancy is especially notable when LLMs are primarily trained for language completion rather than tasks involving embodied navigation actions and symbolic web elements. Our study enhances an LLM-based web agent by simply refining its observation and action space to better align with the LLM's capabilities. This approach enables our base agent to significantly outperform previous methods on a wide variety of web tasks. Specifically, on WebArena, a benchmark featuring general-purpose web interaction tasks, our agent AgentOccam surpasses the previous state-of-the-art and concurrent work by 9.8 (+29.4%) and 5.9 (+15.8%) absolute points respectively, and boosts the success rate by 26.6 points (+161%) over similar plain web agents with its observation and action space alignment. We achieve this without using in-context examples, new agent roles, online feedback or search strategies. AgentOccam's simple design highlights LLMs' impressive zero-shot performance on web tasks, and underlines the critical role of carefully tuning observation and action spaces for LLM-based agents.

cs.AI↗

arXiv2025

An In-Depth Investigation of Data Collection in LLM App Ecosystems

LLM app (tool) ecosystems are rapidly evolving to support sophisticated use cases that often require extensive user data collection. Given that LLM apps are developed by third parties and anecdotal evidence indicating inconsistent enforcement of policies by LLM platforms, sharing user data with these apps presents significant privacy risks. In this paper, we aim to bring transparency in data practices of LLM app ecosystems. We examine OpenAI's GPT app ecosystem as a case study. We propose an LLM-based framework to analyze the natural language specifications of GPT Actions (custom tools) and assess their data collection practices. Our analysis reveals that Actions collect excessive data across 24 categories and 145 data types, with third-party Actions collecting 6.03% more data on average. We find that several Actions violate OpenAI's policies by collecting sensitive information, such as passwords, which is explicitly prohibited by OpenAI. Lastly, we develop an LLM-based privacy policy analysis framework to automatically check the consistency of data collection by Actions with disclosures in their privacy policies. Our measurements indicate that the disclosures for most of the collected data types are omitted, with only 5.8% of Actions clearly disclosing their data collection practices.

cs.CR↗

arXiv2025

JIR-Arena: The First Benchmark Dataset for Just-in-time Information Recommendation

Just-in-time Information Recommendation (JIR) is a service designed to deliver the most relevant information precisely when users need it, , addressing their knowledge gaps with minimal effort and boosting decision-making and efficiency in daily life. Advances in device-efficient deployment of foundation models and the growing use of intelligent wearable devices have made always-on JIR assistants feasible. However, there has been no systematic effort to formally define JIR tasks or establish evaluation frameworks. To bridge this gap, we present the first mathematical definition of JIR tasks and associated evaluation metrics. Additionally, we introduce JIR-Arena, a multimodal benchmark dataset featuring diverse, information-request-intensive scenarios to evaluate JIR systems across critical dimensions: i) accurately inferring user information needs, ii) delivering timely and relevant recommendations, and iii) avoiding irrelevant content that may distract users. Developing a JIR benchmark dataset poses challenges due to subjectivity in estimating user information needs and uncontrollable system variables affecting reproducibility. To address these, JIR-Arena: i) combines input from multiple humans and large AI models to approximate information need distributions; ii) assesses JIR quality through information retrieval outcomes using static knowledge base snapshots; and iii) employs a multi-turn, multi-entity validation framework to improve objectivity and generality. Furthermore, we implement a baseline JIR system capable of processing real-time information streams aligned with user inputs. Our evaluation of this baseline system on JIR-Arena indicates that while foundation model-based JIR systems simulate user needs with reasonable precision, they face challenges in recall and effective content retrieval. To support future research in this new area, we fully release our code and data.

cs.IR↗

arXiv2025

DexCtrl: Towards Sim-to-Real Dexterity with Adaptive Controller Learning

Dexterous manipulation has seen remarkable progress in recent years, with policies capable of executing many complex and contact-rich tasks in simulation. However, transferring these policies from simulation to real world remains a significant challenge. One important issue is the mismatch in low-level controller dynamics, where identical trajectories can lead to vastly different contact forces and behaviors when control parameters vary. Existing approaches often rely on manual tuning or controller randomization, which can be labor-intensive, task-specific, and introduce significant training difficulty. In this work, we propose a framework that jointly learns actions and controller parameters based on the historical information of both trajectory and controller. This adaptive controller adjustment mechanism allows the policy to automatically tune control parameters during execution, thereby mitigating the sim-to-real gap without extensive manual tuning or excessive randomization. Moreover, by explicitly providing controller parameters as part of the observation, our approach facilitates better reasoning over force interactions and improves robustness in real-world scenarios. Experimental results demonstrate that our method achieves improved transfer performance across a variety of dexterous tasks involving variable force conditions.

cs.RO↗

arXiv2025

Contrasting magnetism in VPS3 and CrI3 monolayers with the common honeycomb S = 3/2 spin lattice

Two-dimensional (2D) magnetic materials are promising candidates for spintronics and quantum technologies. One extensively studied example is the ferromagnetic (FM) CrI$_3$ monolayer with the honeycomb Cr$^{3+}$ ($t_{2g}^3$, $S$ = 3/2) spin lattice, while VPS$_3$ has a same honeycomb $S$ = 3/2 spin lattice (V$^{2+}$, $t_{2g}^3$) but displays N$\acute{e}$el antiferromagnetism (AFM). In this work, we study the electronic structure and particularly the contrasting magnetism of VPS$_3$ and CrI$_3$ monolayers. We find that VPS$_3$ is a Mott-Hubbard insulator but CrI$_3$ is a charge-transfer insulator, and therefore their magnetic exchange mechanisms are essentially different. The first nearest-neighbor (1NN) direct $d$-$d$ exchange dominates in VPS$_3$, thus leading to a strong antiferromagnetic (AF) coupling. However, the formation of vanadium vacancies, associated with instability of the low-valence V$^{2+}$ ions, suppresses the AF coupling and thus strongly reduces the N$\acute{e}$el temperature ($T_{\text{N}}$) in line with the experimental observation. In contrast, our results reveal that the major 1NN $d$-$p$-$d$ superexchanges in CrI$_3$ via different channels give rise to competing FM and AF couplings, ultimately resulting in a weak FM coupling as observed experimentally. After revisiting several important superexchange channels reported in the literature, based on our MLWFs and tight-binding analyses, we note that some antiphase contributions must be subtly and simultaneously considered, and thus we provide a deeper insight into the FM coupling of CrI$_3$. Moreover, we identify and compare the major contributions to the magnetic anisotropy, i.e., a weak shape anisotropy in VPS$_3$ and a relatively strong exchange anisotropy in CrI$_3$.

cond-mat.mtrl-sci↗

arXiv2025

First Extraction of Transverse Momentum Dependent Helicity Distributions

We report on the first global analysis of transverse momentum dependent helicity distributions of the proton. The analysis is performed at next-to-leading order with the evolution factor at next-to-next-to-leading-logarithmic accuracy. Nonzero signals are determined for up and down quarks and their $k_T$-integrated polarization are consistent with analyses in collinear factorization, while the distributions of other flavors are loosely constrained by existing data. With increasing transverse momentum, quarks at large $x$ become less polarized while those at small $x$ become more polarized.

hep-ph↗

arXiv2025

ResiComp: Loss-Resilient Image Compression via Dual-Functional Masked Visual Token Modeling

Recent advancements in neural image codecs (NICs) are of significant compression performance, but limited attention has been paid to their error resilience. These resulting NICs tend to be sensitive to packet losses, which are prevalent in real-time communications. In this paper, we investigate how to elevate the resilience ability of NICs to combat packet losses. We propose ResiComp, a pioneering neural image compression framework with feature-domain packet loss concealment (PLC). Motivated by the inherent consistency between generation and compression, we advocate merging the tasks of entropy modeling and PLC into a unified framework focused on latent space context modeling. To this end, we take inspiration from the impressive generative capabilities of large language models (LLMs), particularly the recent advances of masked visual token modeling (MVTM). During training, we integrate MVTM to mirror the effects of packet loss, enabling a dual-functional Transformer to restore the masked latents by predicting their missing values and conditional probability mass functions. Our ResiComp jointly optimizes compression efficiency and loss resilience. Moreover, ResiComp provides flexible coding modes, allowing for explicitly adjusting the efficiency-resilience trade-off in response to varying Internet or wireless network conditions. Extensive experiments demonstrate that ResiComp can significantly enhance the NIC's resilience against packet losses, while exhibits a worthy trade-off between compression efficiency and packet loss resilience.

eess.IV↗

arXiv2025

Multiferroic Metallic Monolayer Cu(CrSe2)2

The two-dimensional (2D) Cu(CrSe$_2$)$_2$ monolayer stands out for its combined ferromagnetic (FM), ferroelectric (FE), and metallic properties, marking itself as a prominent 2D multiferroic metal. This work studies those properties and the relevant physics, using density functional calculations, Monte Carlo simulations, and $ab$ $initio$ molecular dynamics. Our results show that Cu(CrSe$_2$)$_2$ monolayer is in the Cr$^{3+}$ $t_{2g}^3$ state with $S$ = 3/2 and Cu$^{1+}$ $3d^{10}$ with $S$ = 0. The observed in-plane magnetic anisotropy primarily arises from exchange anisotropy, which is associated with the Cr-Se-Cr itinerant ferromagnetism. In contrast, both single-ion anisotropy and shape magnetic anisotropy contribute negligibly. The Dzyaloshinskii-Moriya interaction is also quite weak, only about 3\% of the intralayer exchange parameters. Our Monte Carlo simulations show a FM Curie temperature ($T_{\rm C}$) of 190 K. Moreover, the monolayer exhibits a vertical FE polarization of 1.79 pC/m and a FE polarization switching barrier of 182 meV/f.u., and the FE state remains stable above room temperature as shown by $ab$ $initio$ molecular dynamics simulations. Furthermore, a magnetoelectric coupling is partially manifested by a magnetization rotation from in-plane to out-of-plane associated with a FE-to-paraelectric transition. The magnetization rotation can also be induced by either hole or electron doping, and the hole doping increases the $T_{\rm C}$ up to 238 K. In addition, tensile strain reduces the FE polarization but enhances $T_{\rm C}$ to 290 K, while a compressive strain gives an opposite effect. Therefore, the multiferroic metallic Cu(CrSe$_2$)$_2$ monolayer may be explored for advanced multifunctional electronic devices.

cond-mat.mtrl-sci↗

arXiv2025

Persona-DB: Efficient Large Language Model Personalization for Response Prediction with Collaborative Data Refinement

The increasing demand for personalized interactions with large language models (LLMs) calls for methodologies capable of accurately and efficiently identifying user opinions and preferences. Retrieval augmentation emerges as an effective strategy, as it can accommodate a vast number of users without the costs from fine-tuning. Existing research, however, has largely focused on enhancing the retrieval stage and devoted limited exploration toward optimizing the representation of the database, a crucial aspect for tasks such as personalization. In this work, we examine the problem from a novel angle, focusing on how data can be better represented for more data-efficient retrieval in the context of LLM customization. To tackle this challenge, we introduce Persona-DB, a simple yet effective framework consisting of a hierarchical construction process to improve generalization across task contexts and collaborative refinement to effectively bridge knowledge gaps among users. In the evaluation of response prediction, Persona-DB demonstrates superior context efficiency in maintaining accuracy with a significantly reduced retrieval size, a critical advantage in scenarios with extensive histories or limited context windows. Our experiments also indicate a marked improvement of over 10% under cold-start scenarios, when users have extremely sparse data. Furthermore, our analysis reveals the increasing importance of collaborative knowledge as the retrieval capacity expands.

cs.CL↗

arXiv2024

TinyHelen's First Curriculum: Training and Evaluating Tiny Language Models in a Simpler Language Environment

Training language models (LMs) and their application agents is increasingly costly due to large datasets and models, making test failures difficult to bear. Simplified language environments serve as primordial training and testing grounds, retaining essential commonsense and communication skills but in a more digestible form, potentially enhancing the learning efficiency of LMs, and thus reducing the required model size and data volume for effective training and evaluation. In these simplified language environments, workable strategies for small models, datasets, and agents may be adaptable to larger models, datasets, and agents in complex language environments. To create such environments, we focus on two aspects: i) minimizing language dataset noise and complexity, and ii) preserving the essential text distribution characteristics. Unlike previous methods, we propose a pipeline to refine text data by eliminating noise, minimizing vocabulary, and maintaining genre-specific patterns (e.g., for books, conversation, code, etc.). Implementing this pipeline with large LMs, we have created a leaner suite of LM training and evaluation datasets: 71M Leaner-Pretrain, 7M Leaner-Instruct, Leaner-Glue for assessing linguistic proficiency, and Leaner-Eval for testing instruction-following ability. Our experiments show that leaner pre-training boosts LM learning efficiency. Tiny LMs trained on these datasets outperform those trained on original datasets in instruction-following across different language granularity levels. Moreover, the Leaner-Pretrain dataset's alignment with conventional large LM training sets enables resource-optimized analysis of how learning objectives, model architectures, and training techniques impact performance on language modeling and downstream tasks. Our code and datasets are available at https://github.com/EmpathYang/TinyHelen.git.

cs.CL↗

arXiv2024

ORBIT: Cost-Effective Dataset Curation for Large Language Model Domain Adaptation with an Astronomy Case Study

Recent advances in language modeling demonstrate the need for high-quality domain-specific training data, especially for tasks that require specialized knowledge. General-purpose models, while versatile, often lack the depth needed for expert-level tasks because of limited domain-specific information. Domain adaptation training can enhance these models, but it demands substantial, high-quality data. To address this, we propose ORBIT, a cost-efficient methodology for curating massive, high-quality domain-specific datasets from noisy web sources, tailored for training specialist large language models. Using astronomy as a primary case study, we refined the 1.3T-token FineWeb-Edu dataset into a high-quality, 10B-token subset focused on astronomy. Fine-tuning \textsc{LLaMA-3-8B} on a 1B-token astronomy subset improved performance on the MMLU astronomy benchmark from 69\% to 76\% and achieved top results on AstroBench, an astronomy-specific benchmark. Moreover, our model (Orbit-LLaMA) outperformed \textsc{LLaMA-3-8B-base}, with GPT-4o evaluations preferring it in 73\% of cases across 1000 astronomy-specific questions. Additionally, we validated ORBIT's generalizability by applying it to law and medicine, achieving a significant improvement of data quality compared to an unfiltered baseline. We open-source the ORBIT methodology, including the curated datasets, the codebase, and the resulting model at \href{https://github.com/ModeEric/ORBIT-Llama}{https://github.com/ModeEric/ORBIT-Llama}.

cs.CL↗

arXiv2024

Linear Perturbations and Stability Analysis in $f(T)$ Braneworld Scenario

We conduct a detailed analysis of the full linear perturbations in the braneworld scenario within $f(T)$ gravity. By decomposing the perturbations of the theory into the scalar, transverse vector, antisymmetric pseudotensor, and symmetric transverse-traceless tensor modes, we derive the quadratic action for each mode. The results indicate that there is a total of one scalar, one massless vector, and one tensor propagating degrees of freedom. Consequently, in comparison to general relativity, no additional degrees of freedom appear under the flat braneworld background in the linearized theory. For a thick brane model with $f(T)=T+ αT^2$, we find that it exhibits stability against linear perturbations.

hep-th↗

arXiv2024

Inflation with shallow dip and primordial black holes

Primordial black holes may arise through ultra slow-roll inflation. In this work we study a toy model of ultra slow-roll inflation with a shallow dip. The ultra slow-roll stage enhances the curvature perturbations and thus the primordial scalar power spectrum. We analyze the features of the power spectrum numerically and analytically, and then give a rough estimate of the lower and upper bound of the enhancement. These large perturbations also produce second order gravitational waves, which are in the scope of future observations.

astro-ph.CO↗

arXiv2024

Direct Fractional Auction

This paper designs a market algorithm for fractional ownership of an indivisible asset. It provides an efficient market mechanism, named Direct Fractional Auction (DFA) that offers valuable assets to both small and large investors who can become partial owners of such assets. Additionally, it introduces procedures and algorithms with DFA to determine the optimal winning combinations of unaffiliated bidders. DFA, on the one hand, transfers the partial ownership to the winners and, on the other, redirects the proceeds from the auctions to the sellers. We show that the DFA algorithm works more efficiently than the Greedy algorithm in maximizing the seller's value. We also demonstrate the possibility of reducing the complexity of the problem using the "pruning" method for data pre-processing.

econ.GN↗

arXiv2024

Bright dipolar excitons in twisted black phosphorus homostructures

Bright dipolar excitons, which contain electrical dipoles and have high oscillator strength, are an ideal platform for studying correlated quantum phenomena. They usually rely on carrier tunneling between two quantum wells or two layers to hybridize with nondipolar excitons to gain oscillator strength. In this work, we uncovered a new type of bright infrared dipolar exciton by stacking 90°-twisted black phosphorus (BP) structures. These excitons, inherent to the reconstructed band structure, exhibit high oscillator strength. Most importantly, they inherit the linear polarization from BP, which allows light polarization to be used to select the dipole direction. Moreover, the dipole moment and resonance energy can be widely tuned by the thickness of the BP. Our results demonstrate a useful platform for exploring tunable correlated dipolar excitons.

cond-mat.mes-hall↗

arXiv2024

Electrically charged black holes in gravity with a background Kalb-Ramond field

We derive the exact solutions for electrically charged black holes both in the absence and presence of a cosmological constant in the gravitational theory with Lorentz violation induced by a background Kalb-Ramond (KR) field. The corresponding thermodynamic properties are investigated. It is found that the standard first law of thermodynamics and the Smarr formula remain valid for the charged KR black holes. Nevertheless, the Lorentz-violating effect influences their ranges of local thermodynamic stability and the first- and second-order phase transition points. Furthermore, to examine the impact of Lorentz violation on the motion of test particles in the spacetime, we analyze the shadow and the innermost stable circular orbit (ISCO) of these black holes. Our results reveal that both the shadow and ISCO radii exhibit a high sensitivity to the Lorentz-violating parameter $\ell$, with a decrease observed as $\ell$ increases.

gr-qc↗

arXiv2024

Mean Field Study of Superconductivity in the Square Lattice $t$-$J$ Model with Three-Site Hopping

It remains an open question whether the two-dimensional single-band pure Hubbard model and its related pure $t$-$J$ model truly capture the superconducting order in cuprates. Recent numerical studies on this issue have raised a notable disparity in superconducting order between the pure Hubbard model and the pure $t$-$J$ model. Inspired by these, we investigate the role of the three-site hopping term in $d$-wave superconductivity, such a term is usually neglected in the effective Hamiltonian of the Hubbard model, though its amplitude is of the same order as the superexchange coupling $J$ in the $t$-$J$ model. Our slave-boson mean-field solution demonstrates the suppression of $d$-wave superconducting order by incorporating the three-site hopping term, consistent with numerical observations by the density matrix renormalization group. This suppression could be understood as a result of competition between superexchange interaction and three-site hopping, the former favors $d$-wave pairing while the latter favors $s$-wave pairing. We also discussed its role in quasiparticle dispersion and boson-condensation temperature. Our findings may offer an alternative understanding of the recent numerical contrasting findings in the strong coupling regime: the absent or weak superconductivity in the pure Hubbard model, while the robust superconductivity in the $t$-$J$ model without including the three-site hopping term.

cond-mat.str-el↗