Searcharxiv⌕ Search

arXiv subjects

Jiaying Lei

Publications and source records attributed to Jiaying Lei.

7 recordsLinked to original sources

Drift-Aware Multimodal User Representation Learning via Multi-Scale Temporal Modeling and Sparse Mixture-of-Experts

Understanding user preferences from noisy and temporally evolving social media behaviors is fundamentally challenging due to interest drift, where user preferences shift across time and exhibit both multi-scale temporal patterns and diverse co-existing interests. To address this, we propose DUMoE, a unified framework for drift-aware multimodal user representation learning. Our model consists of (i) a temporal dynamics-aware backbone that captures and integrates static profiles, short-term behavioral signals, and long-term dependencies into a coherent representation, and (ii) a sparse mixture-of-experts (MoE) interest adapter that disentangles multiple latent interests via expert specialization and adaptive routing. Each expert models a distinct interest subspace, while a gating network dynamically selects and aggregates a sparse subset of relevant experts for each user. To enable stable and effective optimization, we further introduce a three-stage training strategy that decouples backbone learning, expert specialization, and gating optimization. Extensive experiments on real-world social media datasets show that DUMoE consistently outperforms state-of-the-art methods on both user interest prediction and interaction prediction tasks.Code is available at https://github.com/idvxlab/DUMoE.

cs.LG↗

From Seeing to Remembering: Brain-Informed Agentic Text-to-Video Generation for Enhancing Memorability

Generating videos that faithfully convey intended semantics while remaining memorable to viewers is valuable for applications such as advertising, education, and storytelling. However, memorability-oriented text-to-video generation remains underexplored, as human memory involves complex neural processes that are difficult to model and translate into generation guidance. In this paper, we propose MemCrafter, brain-informed approach that enhances video memorability through iteratively agentic prompt refinement. The agentic framework contains two key components. First, we construct a cortical region-based neuro-memorability signal that predicts video memorability from estimated neural responses, providing both an optimization objective and interpretable region-level information. Second, we develop a sensitivity-aware feedback module, motivated by a conceptual sensitivity analysis of the optimization objective, to derive actionable guidance for prompt refinement while preserving semantic fidelity. Extensive experiments demonstrate that MemCrafter consistently improves predicted video memorability while maintaining semantic alignment and perceptual quality.

cs.CV↗

From Watching to Steering: Enabling Human-Coding Agent Collaboration through Execution Traces

Recent advances in large language models (LLMs) have enabled coding agents to autonomously perform complex software development tasks, yet their long-running execution remains difficult for users to inspect, understand, and steer. We present Vibetrace, an interactive trace visualization and intervention system for agentic programming built upon OpenCode. Vibetrace introduces a human-agent collaboration paradigm of Observe, Understand, Intervene, and Evolve. It (1) visualizes agent workflows as interactive execution traces for inspecting actions, intermediate states, and failures; (2) enables users to fork execution from prior steps and provide targeted corrections without restarting validated progress; and (3) distills traces and human feedback into reusable skills for continual agent evolution. On SWE-bench Verified, distilled skills reduce input tokens by 23.0% and cost by 15.5%, while improving pass rate from 65.0% to 68.0%. User studies further demonstrate improved accuracy and efficiency over conventional log-based interaction.

cs.HC↗

HypoForge: A Self-Improving Multi-Agent Framework for Automated Hypothesis Generation and Testing via Scientific Skill Learning

Large language models (LLMs) have enabled AI scientist systems to automate scientific discovery, yet existing approaches most rely on static prompting or fixed workflows and fail to accumulate experience for continual improvement. We propose HypoForge, an experience-guided multi-agent framework that learns reusable scientific skills for automated hypothesis generation and hypothesis testing. HypoForge is built on the observation that these two stages involve different supervision signals. For hypothesis generation, where explicit feedback is unavailable, HypoForge adopts an adversarial generator--discriminator mechanism to improve reasoning through comparative critique. For hypothesis testing, where empirical feedback is available, HypoForge learns testing skills from execution outcomes and ground-truth results. By matching skill learning strategies with stage-specific supervision, HypoForge enables continual improvement without fine-tuning foundation models. Experiments on hypothesis generation and testing benchmarks show that HypoForge consistently outperforms existing AI scientist frameworks and skill-level variants. Further analysis demonstrates the effectiveness of the proposed stage-specific skill learning paradigms.

cs.MA↗

From Latent Influence to Language: Diffusion-Oriented Content Generation via Audience-Susceptible Features

The rapid growth of multimodal user-generated content on social media has made information diffusion a critical factor for advertisers and brand marketers. However, manually tailoring content to resonate with specific audiences is labor-intensive and heuristic-driven. While recent generative models offer promising capabilities for automatic content generation, existing approaches for diffusion-oriented content generation still struggle to effectively translate numeric diffusion influence signals into actionable guidance that captures latent audience susceptibility and accounts for heterogeneous audience interests. To address these challenges, we propose DOCG-AS, a three-stage framework for diffusion-oriented content generation. It first performs implicit feature optimization on the realistic content manifold to discover an optimal propagation feature vector. Then, it explicitly decodes this vector using a learnable decoder into interpretable audience-susceptible features described in natural language, providing guidance for content generation. Finally, it leverages multiple sets of audience-susceptible features obtained from different optimization initializations to rewrite the user's input into the final multimodal content. Experiments demonstrate that DOCG-AS consistently outperforms state-of-the-art baselines in terms of predicted diffusion influence.

cs.SI↗

CogBlender: Towards Continuous Cognitive Intervention in Text-to-Image Generation

Beyond conveying semantic information, images also possess cognitive properties that elicit specific psychological responses from viewers, such as memory encoding or emotional reactions. Although modern text-to-image (T2I) models generate semantically coherent content effectively, they struggle to control cognitive properties (e.g., valence, memorability) and often fail to align with the user's psychological intent. To bridge the gap, we introduce CogBlender, an algorithm that enables continuous and multi-dimensional intervention on cognitive properties through a novel two-stage approach. First, we construct discrete cognition-aware rewritten prompts-variants of the input prompt that represent distinct extreme cognitive states. Second, we translate these discrete prompts into continuous control signals by interpolating within the velocity-field domain of flow-matching models. By dynamically blending the velocity fields predicted from these prompts according to the target cognitive scores, CogBlender smoothly steers the generative trajectory to realize the desired cognitive properties in the final image. Extensive experiments across four cognitive properties (i.e., valence, arousal, dominance, and memorability) demonstrate that CogBlender achieves effective cognitive intervention.

cs.CV↗

Designed to Spread: A Generative Approach to Enhance Information Diffusion

Social media has fundamentally transformed how people access information and form social connections, with content expression playing a critical role in driving information diffusion. While prior research has focused largely on network structures and tipping point identification, it provides limited tools for automatically generating content tailored for virality within a specific audience. To fill this gap, we propose the novel task of DOCG and introduce an information enhancement algorithm for generating content optimized for diffusion. Our method includes an influence indicator that enables content-level diffusion assessment without requiring access to network topology, and an information editor that employs reinforcement learning to explore interpretable editing strategies. The editor leverages generative models to produce semantically faithful, audience-aware textual or visual content. Experiments on real-world social media datasets and user study demonstrate that our approach significantly improves diffusion effectiveness while preserving the core semantics of the original content.

cs.SI↗