arXiv subjects

Siqi Cao

Publications and source records attributed to Siqi Cao.

4 recordsLinked to original sources

arXiv2026

From Models to Systems: A Comprehensive Survey of Efficient Multimodal Learning

The rapid expansion of multimodal models has surfaced formidable bottlenecks in computation, memory, and deployment, catalyzing the rise of Efficient Multimodal Learning (EML) as a pivotal research frontier. Despite intensive progress, a cohesive understanding of what, how, and where efficiency is manifested across the learning stack remains fragmented. This survey systematizes the EML landscape by introducing the first structured, model-to-system taxonomy. We distill insights from over 300 seminal works into three hierarchical levels--model, algorithm, and system--addressing architectural parsimony, execution refinement, and hardware-aware orchestration, respectively. Moving beyond a purely categorical review, we offer a methodological synthesis of the vertical synergies between these layers, elucidating how cross-layer co-design contributes to the fundamental "Efficiency-Utility-Privacy" trade-off. Through an integrative case study of Multimodal Large Language Models (MLLMs), we trace the field's evolutionary trajectory from initial structural adjustments to modern full-stack resource orchestration. Furthermore, we provide a holistic discussion and application-specific optimization blueprints for diverse domains and posit a paradigm shift toward self-regulating intelligence, where efficiency is an intrinsic, emergent property of the model's fundamental design rather than a post-hoc constraint. Finally, we present open challenges and future directions that will define the trajectory of EML research. This survey establishes a structured framework for multimodal systems that are not only high-performing and generalizable but natively efficient and ready for ubiquitous deployment. A continuously updated version is available at https://github.com/pwang322/Efficient-Multimodal-Learning-Survey.

cs.MM↗

arXiv2026

Knowledge-guided Disentanglement with Atomic Actions for Action Recognition

Action recognition in complex scenes often involves multiple concurrent fine-grained actions, making it challenging to model internal action structures. Most existing methods rely on holistic representations, which are insufficient for capturing subtle interactions and fine-grained semantics. While recent prompt-based approaches introduce disentanglement, they lack explicit semantic guidance, and methods based solely on visual or structured cues remain coarse-grained. In this paper, we propose Knowledge-guided Disentanglement with Atomic Actions (KDA), which leverages fine-grained semantic knowledge to enhance action representations and enable more precise disentanglement. Specifically, we use Large Language Models (LLMs) to decompose action labels into atomic actions, providing explicit spatial-temporal semantics. A Knowledge Injection Module (KIM) first integrates atomic action knowledge into video features. Based on this enhanced representation, a Knowledge Disentanglement Module (KDM) further disentangles atomic action knowledge to produce more precise semantic guidance for action disentanglement. A Knowledge Disentanglement Loss (KD Loss) is introduced to encourage clearer disentanglement of knowledge components within KDM. Extensive experiments demonstrate that KDA improves feature discriminability and achieves state-of-the-art performance on multi-label action recognition benchmarks. Moreover, KIM and KDM can be readily integrated into other methods, demonstrating strong generality.

cs.CV↗

arXiv2025

Heterogeneity-Aware Federated Causal Inference Leveraging Effect-Measure Transportability

Federated learning of causal estimands offers a powerful strategy to improve estimation efficiency by leveraging data from multiple study sites while preserving privacy. Existing literature has primarily focused on the average treatment effect using single data source, whereas our work addresses a broader class of causal measures across multiple sources. We derive and compare semiparametrically efficient estimators under two transportability assumptions, which impose different restrictions on the data likelihood and illustrate the efficiency-robustness tradeoff. This estimator also permits the incorporation of flexible machine learning algorithms for nuisance functions while maintaining parametric convergence rates and nominal coverage. To further handle scenarios where some source sites violate transportability, we propose a Post-Federated Weighting Selection (PFWS) framework, which is a two-step procedure that adaptively identifies compatible sites and achieves the semiparametric efficiency bound asymptotically. This framework mitigates the efficiency loss of weighting methods and the instability and computational burden of direct site selection in finite samples. Through extensive simulations and real-data analysis, we demonstrate that our PFWS framework achieves superior variance efficiency compared with the target-only analyses across diverse transportability scenarios.

stat.ME↗

arXiv2022

Can AI detect pain and express pain empathy? A review from emotion recognition and a human-centered AI perspective

Sensory and emotional experiences such as pain and empathy are essential for mental and physical health. Cognitive neuroscience has been working on revealing mechanisms underlying pain and empathy. Furthermore, as trending research areas, computational pain recognition and empathic artificial intelligence (AI) show progress and promise for healthcare or human-computer interaction. Although AI research has recently made it increasingly possible to create artificial systems with affective processing, most cognitive neuroscience and AI research do not jointly address the issues of empathy in AI and cognitive neuroscience. The main aim of this paper is to introduce key advances, cognitive challenges and technical barriers in computational pain recognition and the implementation of artificial empathy. Our discussion covers the following topics: How can AI recognize pain from unimodal and multimodal information? Is it crucial for AI to be empathic? What are the benefits and challenges of empathic AI? Despite some consensus on the importance of AI, including empathic recognition and responses, we also highlight future challenges for artificial empathy and possible paths from interdisciplinary perspectives. Furthermore, we discuss challenges for responsible evaluation of cognitive methods and computational techniques and show approaches to future work to contribute to affective assistants capable of empathy.

cs.AI↗