Searcharxiv⌕ Search

arXiv subjects

Yifei Zhao

Publications and source records attributed to Yifei Zhao.

At least 19 recordsLinked to original sources

RPMem: Learning Long-Term Recurrent Parametric Memory Across Sessions for LLM Agents

Long-running LLM agents require memory that persists and evolves across sessions. Text-based memory retrieves and reconstructs past interactions at every query, making long-horizon performance increasingly dependent on retrieval quality and contextual reasoning as histories grow. Parametric memory encodes experience directly into model computation, but existing approaches provide limited support for cross-session memory evolution. Their coupling to a specific backbone further restricts memory reuse after model replacement. We introduce RPMem, a two-stage architecture that compiles each session into a model-independent latent memory through forward computation and selectively integrates it with retained memory via a task-trained recurrent gate. The consolidated memory is then mapped to backbone-specific low-rank adaptation (LoRA) parameters, allowing the encoding capability to transfer when the backbone is replaced. Evaluation across three long-term memory benchmarks and five diverse backbones demonstrates broad generalization with near-constant update cost and memory footprint. With Qwen3-8B on PERMA, RPMem reaches 85.52%, outperforming the strongest parametric and text-based baselines by 5.32 and 12.98 percentage points, respectively. Ablations validate the complementary roles of session compilation and cross-session consolidation, while dynamics analyses reveal that the gate acquires task-specific memory integration strategies. These results establish RPMem as a lifecycle-independent parametric memory framework that maintains evolving cross-session memory that remains reusable across backbone replacements. Our code is available at https://github.com/Quark-Medical/rpmem/tree/main.

cs.CL↗

Depth-0 supercuspidal L-packets of tame covers

We generalize DeBacker and Reeder's construction (or rather, its refinement by Kaletha) of depth-0 supercuspidal L-packets to tame covers of unramified reductive groups defined by étale levels. These include, in particular, all tame covers associated to Brylinski-Deligne data. We prove that our L-packets satisfy invariance of formal degrees and an appropriate notion of stability.

math.RT↗

OpenMarkX: An Extensible Platform for Watermarking Open-Source Model and Agent Outputs

We introduce OpenMarkX, an extensible platform for watermarking open-source model and agent outputs under standard inference. It takes models, compatible methods, private keys, and evaluation settings, and produces watermarked weights, detection results, and quality and robustness reports. Its 16 text-watermark configurations are complemented by agent, vision-language, image, and physics-field workflows. Our proposed method, OpenMark, addresses learnability and robustness through joint model--policy learning and perturbation-aware training. On several open-source LLMs, OpenMark achieves 0.92 TPR at 1% FPR while reducing perplexity by 20.5% on average relative to the strongest prior method, and remains more detectable after such modifications, e.g., retaining 0.57 TPR at 5% FPR after 0.5-ratio model merging, compared with at most 0.34 for prior methods.

cs.CR↗

RedPhys: An Extensible Red-Teaming Platform for Physical-World Models and Agents

Physical-world models (PhyModels) perform grounded perception, reasoning, or decision tasks; physical-world agents (PhyAgents) combine these models with planning or tool use. We introduce RedPhys, an extensible red-teaming platform, instantiated here with vision-language models and agents. Shared interfaces make tasks, test methods, defenses, and victim systems configurable and extensible, producing robustness reports and datasets for future alignment. The platform emphasizes adversarial testing that seeks physically grounded functional failures, while also supporting non-adversarial input perturbations without an attack objective. The benchmark integrates 12 red-teaming methods, 3 model-agnostic defenses, and 13 PhyModels across seven physical domains under a practical black-box threat model. An agentic target-generation pipeline derives a scene-specific, physically grounded target, so targeted methods pursue the same intended error. Our evaluation shows that when attackers control the text prompt or add auxiliary visual instructions, injection exposes substantial instruction-following vulnerabilities; among bounded visual-transfer attacks, multimodal co-optimization transfers best; single-pass generative attacks approach iterative methods without per-image optimization; and neither model scale nor physical-reasoning post-training confers robustness. Code is available at https://github.com/loopleap-lab/RedPhys.

cs.CV↗

Game-Creation-Agents: Autonomous Game Production

Game-Creation-Agents (GCA) turns a natural-language request into a runnable Phaser 4 browser game. The pipeline first builds a multimodal game design document that couples mechanics and scene contracts to grounded visual references. It then projects that document into a runtime scaffold, machine-readable contracts, and deterministic tests. Eight specialized agents next carry out production: scenes are implemented serially, assets are produced in parallel, and one reduced manifest joins the two branches into a single build. Build-linked real-UI checks and a code-blind critic then verify that build and decide whether the game may ship. Each blocker returns to the agent that owns the affected artifact for a bounded repair, and the game is released once all the gates pass. We evaluate finished games separately, through their rendered interfaces alone, over prompt-matched tasks run under a common protocol. GCA achieves stronger requirement satisfaction, core mechanics, content depth, and functional visuals than a comparable open pipeline. In a side-by-side human study, raters also score GCA's games higher on every dimension and prefer them overall.

cs.AI↗

Do 3D Large Language Models Really Use 3D Evidence?

Recent 3D Large Language Models (3D-LLMs) have shown strong performance on scene question answering with extensive post-training. Yet, existing benchmarks rarely evaluate how 3D-LLMs respond to referential ambiguity, insufficient scene evidence, or false presuppositions, obscuring whether their responses are grounded in 3D evidence. In this work, we introduce UNICORNS, a benchmark for evaluating the reliability of 3D-LLM responses across four evidence conditions. UNICORNS contains 11,387 questions across 557 ScanNet scans, covering eight challenge categories alongside answerable controls. A response passes only if it satisfies the required response behavior and contains no scene claim unsupported by the available evidence. Evaluation of six open-sourced 3D-LLMs reveals a pronounced performance gap: even the strongest zero-shot model achieves 61.22% accuracy on answerable controls, but only 40.94% across the eight challenge categories. To address this gap, we further propose HORN, an obligation-aware post-training method combining group-relative task rewards, leaf-conditioned obligation rewards, and supervised control replay. Extensive experiments demonstrate that HORN reaches $75.1 \pm 1.0\%$ across the challenge categories and $77.2 \pm 0.8\%$ overall, outperforming all baselines without degrading answerable-control accuracy. Our findings enable a systematic evaluation and improvement of evidence-grounded 3D reasoning for 3D-LLMs (code: https://anonymous.4open.science/r/unicorns-6B38/).

cs.CV↗

Uncertainty Gates the Gradient: Adaptive Updates for Generalized Few-Shot 3D Segmentation

In generalized few-shot 3D point cloud segmentation, support annotations are often compromised by sparsity and occlusion. Blindly adapting to such low quality samples risks learning spurious geometric patterns rather than robust novel concepts. Although 3D Vision-Language Models have been introduced to sup plement supervision, the potential of uncertainty to actively govern adaptation dynamics remains underexplored. Existing signal-level methods like loss reweight ing fail to prevent unreliable episodes from steering the optimization trajectory. To address this, we propose Uncertainty-Gated Adaptive (UGA), to the best of our knowledge the first framework that elevates uncertainty to update-level gating for generalized few-shot 3D point cloud segmentation. The core mod ule, Uncertainty-Gated Update (UG-Update), estimates episodic reliability via intra-class dispersion over merged support and pseudo-labeled novel supervision, and selectively gates residual gradient updates. This encourages the model to learn novel semantics from reliable episodes while attenuating updates during unreliable ones. Complementing this, we introduce an Uncertainty-Based Pseudo Label Filter (UB-Filter) to enforce learning from high-confidence dense signals. On ScanNet200 and ScanNet++ (1-/5-shot), UGA sets new state of the art with an average +3.08 HM IoU over the strongest prior baseline.

cs.CV↗

Phase Kernel Lifts Capacity of Dense Associative Memory

Dense associative memory underlies modern attention mechanisms; however, Modern Hopfield Networks (MHNs) with linear kernels yield overlapping attractor basins and spurious states under high memory load, limiting their theoretically exponential storage capacity in practice. We construct a class of nonlinear phase kernels that modify the energy landscape and retrieval dynamics, and prove that these kernels preserve exponential storage capacity and local fixed-point convergence under given conditions on kernel smoothness, memory separation, and memory distributions. The framework is instantiated by the Sigmoidal Phase Hopfield Network (S-Hop), which guarantees monotone energy descent and mitigates practical capacity loss. S-Hop has $8.3\times$ and $4\times$ gains in one-step noisy retrieval load over Uniform Hopfield Networks (U-Hop) on CIFAR-10 and TinyImageNet respectively, and reaches a $50\times$ gain in first-failure load for ImageNet. The results demonstrate that phase kernels provide a principled route to robust dense associative memory.

cs.LG↗

MemCalib: Benchmarking and Optimizing Memory Use in LLM Agents

The effectiveness of agent memory ultimately depends on whether the underlying LLM gives each memory in context an appropriate degree of influence over its response. Yet this capability has remained largely overlooked. To assess this capability, we introduce MemCalib, a benchmark grounded in realistic memory-system scenarios for evaluating memory use and advancing optimization algorithms. Results on the MemCalib test set reveal that frontier open- and closed-source models struggle to use memory appropriately. They frequently over-use or under-use memory rather than matching each proposition's actual use to its target level, leading to biased, low-quality responses. Experiments with common post-training algorithms, including group relative policy optimization and on-policy self-distillation, further reveal a clear directional skew: trained models improve in one direction while deteriorating in the other. We therefore propose MemCalib-RL, an ordered bidirectional counterfactual credit-assignment algorithm that separates over- and under-use signals and localizes their credit to response tokens through exact atom ablation. Results across model families and scales (Qwen3-8B, Ministral-3-8B-Instruct, and Qwen3.5-35B-A3B) show that MemCalib-RL achieves the best overall performance while better balancing over-use and under-use, with gains generalizing beyond MemCalib in external benchmark evaluation. Further experiments support its design choices and robustness and provide insight into its training dynamics.

cs.LG↗

Projection of purification performance for the RELICS experiment

The RELICS (REactor neutrino LIquid xenon Coherent elastic Scattering) experiment employs a dual-phase liquid xenon time projection chamber to search for Coherent Elastic Neutrino-Nucleus Scattering (CE$ν$NS) induced by reactor neutrinos. To detect these sub-keV nuclear recoils and minimize signal attenuation, it is critical to maintain a sufficiently low impurity concentration in the detector. This work presents a comprehensive purity evolution model developed to describe impurity migration inside the detector. Utilizing measured material outgassing rates as input parameters, the model incorporates non-uniform transport mechanisms of the impurities, including circulation, vaporization, and condensation. The model is validated using data from a dedicated prototype detector. Based on this validated model, projections for the purification performance of the upcoming RELICS-10 and RELICS-50 detectors are provided.

physics.ins-det↗

AdS - de Sitter transition in interacting dark energy models

Recent cosmological results, including the persistent $H_0$ tension and indications of dynamical dark energy, have raised the possibility that the standard $Λ$-Cold Dark Matter ($Λ$CDM) model may be incomplete. At the same time, theoretical developments based on swampland conjectures indicate that the realization of a stable de Sitter vacuum within string theory is challenging. Motivated by these observational and theoretical considerations, we investigate an interacting $Λ$ scenario characterized by a general linear coupling in the dark sector. At the background level, this model naturally allows the dark energy density to evolve from negative to positive values, opening the possibility of a transition from an anti-de Sitter-like phase to the de Sitter-like phase inferred by current observations. We derive the analytic background solutions and perturbation equations, discuss the physically acceptable region of parameter space, and constrain the model with DES-Dovekie type Ia supernovae, DESI BAO and CMB data. Our analysis suggests a parameter space allowing the dark energy density to change sign while maintaining positive dark matter density, motivating further investigation of an interaction-driven anti-de Sitter to de Sitter transition.

astro-ph.CO↗

Energy-guided Recursive Model

Recursive models show promise on reasoning and language tasks, yet their test-time scaling lacks a principled criterion for selecting trajectories or determining recurrent depth. We introduce \textbf{Energy-guided Recursive Model (ERM)}, which uses Hopfield-type memories of valid local and global structures to assign intrinsic energies to candidate trajectories. These energies guide candidate selection and suggest an effective range of recurrent depths, implying that deeper recurrence does not necessarily improve reasoning accuracy. They also enable sampling methods such as parallel tempering to improve exploration. For reasoning tasks, ERM achieves optimal solutions on Sudoku ($98.97\%$), Pencil Puzzle Bench (PPBench, $88.04\%$) and Maze ($99.30\%$), reaching the best accuracy in recursive modeling. On language modeling, ERM reduces RedPajama-V2 perplexity by $1.74\%$ with marginal inference overhead. The results support energy guidance as a practical framework for improving test-time scaling in recursive models.

cs.LG↗

SceneActBench: Can Agents Act on the 3D Scenes They See?

Vision-language model (VLM) agents increasingly use tools to act on 3D scenes rather than only describe them. Existing 3D benchmarks score textual responses or single-object operations, leaving agent action on complete multi-object 3D scenes under evaluated. We present SceneActBench, a benchmark for visually conditioned action across five 3D tasks under a unified agent-environment loop. Given PNG images or sampled video frames and, where applicable, supplied 3D assets, an agent acts on a 3D environment. We evaluate each final output against hidden ground truth with task-specific geometric metrics. SceneActBench comprises five tasks built from 210 source instances, yielding 520 task cases including paired input conditions. Every task runs through one fixed agent loop to keep the comparison fair. Across eleven proprietary VLM configurations, Overall scores span 38.6-50.2, and none performs consistently well across tasks. We further analyse where and how failures manifest.

cs.AI↗

Distribution Preference Optimization: A Fine-grained Perspective for LLM Unlearning

As Large Language Models (LLMs) demonstrate remarkable capabilities learned from vast corpora, concerns regarding data privacy and safety are receiving increasing attention. LLM unlearning, which aims to remove the influence of specific data while preserving overall model utility, is becoming an important research area. One of the mainstream unlearning classes is optimization-based methods, which achieve forgetting directly through fine-tuning, exemplified by Negative Preference Optimization (NPO). However, NPO's effectiveness is limited by its inherent lack of explicit positive preference signals. Attempts to introduce such signals by constructing preferred responses often necessitate domain-specific knowledge or well-designed prompts, fundamentally restricting their generalizability. In this paper, we shift the focus to the distribution-level, directly targeting the next-token probability distribution instead of entire responses, and derive a novel unlearning algorithm termed \textbf{Di}stribution \textbf{P}reference \textbf{O}ptimization (DiPO). We show that the requisite preference distribution pairs for DiPO, which are distributions over the model's output tokens, can be constructed by selectively amplifying or suppressing the model's high-confidence output logits, thereby effectively overcoming NPO's limitations. We theoretically prove the consistency of DiPO's loss function with the desired unlearning direction. Extensive experiments demonstrate that DiPO achieves a strong trade-off between model utility and forget quality. Notably, DiPO attains the highest forget quality on the TOFU benchmark, and maintains leading scalability and sustainability in utility preservation on the MUSE benchmark.

cs.LG↗

SIF: Semantically In-Distribution Fingerprints for Large Vision-Language Models

The public accessibility of large vision-language models (LVLMs) raises serious concerns about unauthorized model reuse and intellectual property infringement. Existing ownership verification methods often rely on semantically abnormal queries or out-of-distribution responses as fingerprints, which can be easily detected and removed by adversaries. We expose this vulnerability through a Semantic Divergence Attack (SDA), which identifies and filters fingerprint queries by measuring semantic divergence between a suspect model and a reference model, showing that existing fingerprints are not semantic-preserving and are therefore easy to detect and bypass. To address these limitations, we propose SIF (Semantically In-Distribution Fingerprints), a non-intrusive ownership verification framework that requires no parameter modification. SIF introduces Semantic-Aligned Fingerprint Distillation (SAFD), which transfers text watermarking signals into the visual modality to produce semantically coherent yet fingerprinted responses. In addition, Robust-Fingerprint Optimization (RFO) enhances robustness by simulating worst-case representation perturbations, making the fingerprints resilient to model modifications such as fine-tuning and quantization. Extensive experiments on LLaVA-1.5 and Qwen2.5-VL demonstrate that SIF achieves strong stealthiness and robustness, providing a practical solution for LVLM copyright protection. Code is available at https://github.com/UCF-ML-Research/SIF-VLM-Fingerprint

cs.CV↗

Uncertainty-aware Prototype Learning with Variational Inference for Few-shot Point Cloud Segmentation

Few-shot 3D semantic segmentation aims to generate accurate semantic masks for query point clouds with only a few annotated support examples. Existing prototype-based methods typically construct compact and deterministic prototypes from the support set to guide query segmentation. However, such rigid representations are unable to capture the intrinsic uncertainty introduced by scarce supervision, which often results in degraded robustness and limited generalization. In this work, we propose UPL (Uncertainty-aware Prototype Learning), a probabilistic approach designed to incorporate uncertainty modeling into prototype learning for few-shot 3D segmentation. Our framework introduces two key components. First, UPL introduces a dual-stream prototype refinement module that enriches prototype representations by jointly leveraging limited information from both support and query samples. Second, we formulate prototype learning as a variational inference problem, regarding class prototypes as latent variables. This probabilistic formulation enables explicit uncertainty modeling, providing robust and interpretable mask predictions. Extensive experiments on the widely used ScanNet and S3DIS benchmarks show that our UPL achieves consistent state-of-the-art performance under different settings while providing reliable uncertainty estimation. The code is available at https://fdueblab-upl.github.io/.

cs.CV↗

Learning Hierarchical Orthogonal Prototypes for Generalized Few-Shot 3D Point Cloud Segmentation

Generalized few-shot 3D point cloud segmentation aims to adapt to novel classes from only a few annotations while maintaining strong performance on base classes, but this remains challenging due to the inherent stability-plasticity trade-off: adapting to novel classes can interfere with shared representations and cause base-class forgetting. We present HOP3D, a unified framework that learns hierarchical orthogonal prototypes with an entropy-based few-shot regularizer to enable robust novel-class adaptation without degrading base-class performance. HOP3D introduces hierarchical orthogonalization that decouples base and novel learning at both the gradient and representation levels, effectively mitigating base-novel interference. To further enhance adaptation under sparse supervision, we incorporate an entropy-based regularizer that leverages predictive uncertainty to refine prototype learning and promote balanced predictions. Extensive experiments on ScanNet200 and ScanNet++ demonstrate that HOP3D consistently outperforms state-of-the-art baselines under both 1-shot and 5-shot settings. The code is available at https://fdueblab-hop3d.github.io/.

cs.CV↗