arXiv subjects

Kai Shi

Publications and source records attributed to Kai Shi.

15 recordsLinked to original sources

arXiv2026

ProMSA:Progressive Multimodal Search Agents for Knowledge-Based Visual Question Answering

Knowledge-based Visual Question Answering (KB-VQA) requires models to combine image understanding with external knowledge. Most prior methods use a fixed retrieve-then-generate pipeline with a pre-selected retriever and a static top-k setting, which is not adaptive during reasoning. We propose ProMSA, a progressive multimodal search agent for KB-VQA. Given an image-question pair, the agent iteratively chooses image search, text search, or stop, under explicit tool-call budgets and with deduplication to avoid redundant retrieval. For training, we first use rejection-sampling SFT to learn valid tool-use formats, then optimize the agent with TN-GSPO, a sequence-level RL objective that normalizes updates by both generation length and tool-interaction depth. Experiments on E-VQA and InfoSeek show consistent gains over strong RAG and agent baselines, and improved retrieval and end-to-end accuracy. The code is available at https://github.com/DingWu1021/Promsa.

cs.CV↗

arXiv2026

MemExplorer: Navigating the Heterogeneous Memory Design Space for Agentic Inference NPUs

Emerging agentic LLM workloads are driving rapidly growing demand on both memory capacity and bandwidth, with different phases of inference (e.g., prefill and decode) imposing distinct requirements. Industry is responding by composing heterogeneous accelerators into single interconnected systems, as exemplified by NVIDIA's Vera Rubin platform, where each device brings its own memory architecture. This heterogeneity is further compounded by a widening landscape of available memory technologies: high-density on-chip SRAM, HBM, LPDDR, GDDR, and emerging options such as high-bandwidth flash (HBF), each offering different capacity, bandwidth, and power trade-offs. Identifying the right memory architecture for next-generation inference accelerators requires navigating a vast and rapidly evolving design space, in which the interplay between workload characteristics, NPU design dimensions, and memory system design remains largely underexplored. To address this challenge, we present MemExplorer, a new memory system synthesizer for heterogeneous NPU systems. MemExplorer provides a unified abstraction for modeling diverse memory technologies across different hierarchy levels (e.g., on-chip and off-chip) and automatically determines an efficient heterogeneous memory system together with NPU design choices (e.g., matrix engine size) to balance throughput and power between prefilling and decoding devices in a multi-device NPU system. Experimental results show that, under the same power budget for agentic workloads, MemExplorer achieves up to 2.3x higher energy efficiency than the baseline NPU and 3.23x higher than H100 in the prefill-only setting. Under equivalent performance targets in the decode setting, it further delivers up to 1.93x and 2.72x higher power efficiency over the baseline NPU and H100, respectively.

cs.AR↗

arXiv2026

When Models Judge Themselves: Unsupervised Self-Evolution for Multimodal Reasoning

Recent progress in multimodal large language models has led to strong performance on reasoning tasks, but these improvements largely rely on high-quality annotated data or teacher-model distillation, both of which are costly and difficult to scale. To address this, we propose an unsupervised self-evolution training framework for multimodal reasoning that achieves stable performance improvements without using human-annotated answers or external reward models. For each input, we sample multiple reasoning trajectories and jointly model their within group structure. We use the Actor's self-consistency signal as a training prior, and introduce a bounded Judge based modulation to continuously reweight trajectories of different quality. We further model the modulated scores as a group level distribution and convert absolute scores into relative advantages within each group, enabling more robust policy updates. Trained with Group Relative Policy Optimization (GRPO) on unlabeled data, our method consistently improves reasoning performance and generalization on five mathematical reasoning benchmarks, offering a scalable path toward self-evolving multimodal models. The code are available at https://github.com/OPPO-Mente-Lab/LLM-Self-Judge.

cs.CV↗

arXiv2025

DaMo: Data Mixing Optimizer in Fine-tuning Multimodal LLMs for Mobile Phone Agents

Mobile Phone Agents (MPAs) have emerged as a promising research direction due to their broad applicability across diverse scenarios. While Multimodal Large Language Models (MLLMs) serve as the foundation for MPAs, their effectiveness in handling multiple mobile phone tasks simultaneously remains limited. Although multitask supervised fine-tuning (SFT) is widely adopted for multitask learning, existing approaches struggle to determine optimal training data compositions for peak performance. To address this challenge, we propose DaMo (Data Mixture Optimizer) - a novel solution employing a trainable network that predicts optimal data mixtures by forecasting downstream task performance for any given dataset ratio. To support comprehensive evaluation, we introduce PhoneAgentBench, the first specialized benchmark to evaluate MLLMs on multimodal mobile phone tasks, comprising 1235 QA pairs spanning diverse real-world industrial mobile application scenarios. Demonstrating strong predictive capability (R^2=0.81) in small-scale pilot experiments, DaMo efficiently extrapolates optimal data mixing configurations. Our results show DaMo achieves a 3.38% performance improvement on PhoneAgentBench compared to alternative methods. Furthermore, extensive experiments across established benchmarks including BFCL-v3, MME-Reasoning, MME-Perception, and OCRBench reveal DaMo's superior generalization, outperforming other approaches by 2.57% in terms of average score. When used solely for MLLM optimization on the BFCL-v3 task, DaMo improves the metrics by 12.47% than other methods. Notably, DaMo maintains robust scalability, preserving its effectiveness when applied to other model architectures. The code and dataset are available at https://github.com/OPPO-Mente-Lab/DaMo.git

cs.CV↗

arXiv2025

UAQFact: Evaluating Factual Knowledge Utilization of LLMs on Unanswerable Questions

Handling unanswerable questions (UAQ) is crucial for LLMs, as it helps prevent misleading responses in complex situations. While previous studies have built several datasets to assess LLMs' performance on UAQ, these datasets lack factual knowledge support, which limits the evaluation of LLMs' ability to utilize their factual knowledge when handling UAQ. To address the limitation, we introduce a new unanswerable question dataset UAQFact, a bilingual dataset with auxiliary factual knowledge created from a Knowledge Graph. Based on UAQFact, we further define two new tasks to measure LLMs' ability to utilize internal and external factual knowledge, respectively. Our experimental results across multiple LLM series show that UAQFact presents significant challenges, as LLMs do not consistently perform well even when they have factual knowledge stored. Additionally, we find that incorporating external knowledge may enhance performance, but LLMs still cannot make full use of the knowledge which may result in incorrect responses.

cs.CL↗

arXiv2025

ReviewInstruct: A Review-Driven Multi-Turn Conversations Generation Method for Large Language Models

The effectiveness of large language models (LLMs) in conversational AI is hindered by their reliance on single-turn supervised fine-tuning (SFT) data, which limits contextual coherence in multi-turn dialogues. Existing methods for generating multi-turn dialogue data struggle to ensure both diversity and quality in instructions. To address this, we propose Review-Instruct, a novel framework that synthesizes multi-turn conversations through an iterative "Ask-Respond-Review" process involving three agent roles: a Candidate, multiple Reviewers, and a Chairman. The framework iteratively refines instructions by incorporating Reviewer feedback, enhancing dialogue diversity and difficulty. We construct a multi-turn dataset using the Alpaca dataset and fine-tune the LLaMA2-13B model. Evaluations on MT-Bench, MMLU-Pro, and Auto-Arena demonstrate significant improvements, achieving absolute gains of 2.9\% on MMLU-Pro and 2\% on MT-Bench compared to prior state-of-the-art models based on LLaMA2-13B. Ablation studies confirm the critical role of the Review stage and the use of multiple Reviewers in boosting instruction diversity and difficulty. Our work highlights the potential of review-driven, multi-agent frameworks for generating high-quality conversational data at scale.

cs.CL↗

arXiv2023

$sl(2,\mathds{C})\times D$ symmetry and conformal primary basis for massless fields

Alternative to the embedding formalism, we provide a group theoretic approach to the conformal primary basis for the massless field with arbitrary helicity. To this end, we first point out that $sl(2,\mathds{C})$ isometry gets enhanced to $sl(2,\mathds{C})\times D$ symmetry for the solution space of the massless field with arbitrary helicity. Then associated with $sl(2,\mathds{C})\times D$ symmetry, we introduce the novel quadratic Casimirs and relevant tensor/spinor fields to derive 2 explicit constraints on the bulk dilatation and $sl(2,\mathds{C})$ Casimirs. With this, we further argue that the candidate conformal primary basis can be constructed out of the infinite tower of the descendants of the left and right highest (lowest) conformal primary wavefunction of $sl(2,\mathds{C})$ Lie algebra, and the corresponding celestial conformal weights are determined by the bulk scaling dimension through solving out the exact on-shell conformal primary wavefunctions, where on top of the two kinds of familiar-looking on-shell conformal primary wavefunctions, we also obtain another set of independent on-shell conformal primary wavefunctions for the massless field with helicity $|s|\ge 1$. In passing, we also develop the relationship between the 4D Lorentz Lie algebra and 2D conformal Lie algebra from scratch, and present an explicit derivation for the two important properties associated with the conformal primary wavefunctions.

hep-th↗

arXiv2023

Monadic Deep Learning

The Java and Scala community has built a very successful big data ecosystem. However, most of neural networks running on it are modeled in dynamically typed programming languages. These dynamically typed deep learning frameworks treat neural networks as differentiable expressions that contain many trainable variable, and perform automatic differentiation on those expressions when training them. Until 2019, none of the learning frameworks in statically typed languages provided the expressive power of traditional frameworks. Their users are not able to use custom algorithms unless creating plenty of boilerplate code for hard-coded back-propagation. We solved this problem in DeepLearning.scala 2. Our contributions are: 1. We discovered a novel approach to perform automatic differentiation in reverse mode for statically typed functions that contain multiple trainable variable, and can interoperate freely with the metalanguage. 2. We designed a set of monads and monad transformers, which allow users to create monadic expressions that represent dynamic neural networks. 3. Along with these monads, we provide some applicative functors, to perform multiple calculations in parallel. With these features, users of DeepLearning.scala were able to create complex neural networks in an intuitive and concise way, and still maintain type safety.

cs.PL↗

arXiv2023

Dynamic and Thermodynamic Stability of Charged Perfect Fluid Stars

We perform a thorough analysis of the dynamic and thermodynamic stability for the charged perfect fluid star by applying the Wald formalism to the Lagrangian formulation of Einstein-Maxwell-charged fluid system. As a result, we find that neither the presence of the additional electromagnetic field nor the Lorentz force experienced by the charged fluid makes any obstruction to the key steps towards the previous results obtained for the neutral perfect fluid star. Therefore, the criterion for the dynamic stability of our charged star in dynamic equilibrium within the symplectic complement of the trivial perturbaions with the ADM $3$-momentum unchanged is given by the non-negativity of the canonical energy associated with the timelike Killing field, where it is further shown for both non-axisymmetric and axisymmetric perturbations that the dynamic stability against these restricted perturbations also implies the dynamic stability against more generic perturbations. On the other hand, the necessary condition for the thermodynamic stability of our charged star in thermodynamic equilibrium is given by the positivity of the canonical energy of all the linear on-shell perturbations with the ADM angular momentum unchanged in the comoving frame, which is equivalent to the positivity of the canonical energy associated with the timelike Killing field when restricted onto the axisymmetric perturbations. As a by-product, we further establish the equivalence of the dynamic and thermodynamic stability with respect to the spherically symmetric perturbations of the static, spherically symmetric isentropic charged star.

gr-qc↗

arXiv2023

$SL(2,R)\times U(1)$ symmetry and quasinormal modes in the self-dual warped AdS black hole

The algebraic approach to the spectrum of quasinormal modes has been made as simple as possible for the BTZ black hole by the strategy developed in \cite{Zhang}. By working with the self-dual warped AdS black hole, we demonstrate in an explicit way that such a strategy can be well adapted to those warped AdS balck holes with the $SL(2,R)\times U(1)$ isometry. To this end, we first introduce two associated tensor fields with the quadratic Casimir of $SL(2,R)\times U(1)$ Lie algebra in the self-dual warped AdS black hole and show that they correspond essentially to the metric and volume element up to a constant prefactor, respectively. Then without appealing to any concrete coordinate system, we can further show that the solutions to the equations of motion for the scalar, vector, spinor fields all fall into the representations of the $SL(2,R)\times U(1)$ Lie algebra by a purely abstract tensor and spinor analysis. Accordingly, the corresponding spectrum of quasinormal modes for each fixed azimuthal quantum number can be derived algebraically as the infinite tower of descendants of the highest weight mode of the $SL(2,R)$ Lie subalgebra.

hep-th↗

arXiv2022

Thermodynamic equilibrium condition and the first law of thermodynamics for charged perfect fluids in electromagnetic and gravitational fields

We provide a proof of the necessary and sufficient condition on the profile of the temperature, chemical potential, and angular velocity for a charged perfect fluid in dynamic equilibrium to be in thermodynamic equilibrium not only in fixed but also in dynamical electromagnetic and gravitational fields. In passing, we also present the corresponding expression for the first law of thermodynamics for such a charged star.

gr-qc↗

arXiv2021

Covariant phase space with null boundaries

By imposing the boundary condition associated with the boundary structure of the null boundaries rather than the usual one, we find that the key requirement in Harlow-Wu's algorithm fails to be met in the whole covariant phase space. Instead, it can be satisfied in its submanifold with the null boundaries given by the expansion free and shear free hypersurfaces in Einstein's gravity, which can be regarded as the origin of the non-triviality of null boundaries in terms of Wald-Zoupas's prescription. But nevertheless, by sticking to the variational principle as our guiding principle and adapting Harlow-Wu's algorithm to the aforementioned submanifold, we successfully reproduce the Hamiltonians obtained previously by Wald-Zoupas' prescription, where not only are we endowed with the expansion free and shear free null boundary as the natural stand point for the definition of the Hamiltonian in the whole covariant phase space, but also led naturally to the correct boundary term for such a definition.

hep-th↗

arXiv2020

Ultrafast optical circuit switching for data centers using integrated soliton microcombs

Networks inside current data centers comprise a hierarchy of power-hungry electronic packet switches interconnected via optical fibers and transceivers. As the scaling of such electrically-switched networks approaches a plateau, a power-efficient solution is to implement a flat network with optical circuit switching (OCS), without electronic switches and a reduced number of transceivers due to direct links among servers. One of the promising ways of implementing OCS is by using tunable lasers and arrayed waveguide grating routers. Such an OCS-network can offer high bandwidth and low network latency, and the possibility of photonic integration results in an energy-efficient, compact, and scalable photonic data center network. To support dynamic data center workloads efficiently, it is critical to switch between wavelengths in sub nanoseconds (ns). Here we demonstrate ultrafast photonic circuit switching based on a microcomb. Using a photonic integrated Si3N4 microcomb in conjunction with semiconductor optical amplifiers (SOAs), sub ns (< 500 ps) switching of more than 20 carriers is achieved. Moreover, the 25-Gbps non-return to zero (NRZ) and 50-Gbps four-level pulse amplitude modulation (PAM-4) burst mode transmission systems are shown. Further, on-chip Indium phosphide (InP) based SOAs and arrayed waveguide grating (AWG) are used to show sub-ns switching along with 25-Gbps NRZ burst mode transmission providing a path toward a more scalable and energy-efficient wavelength-switched network for future data centers.

physics.app-ph↗

arXiv2018

Comparison of Low Complexity Coherent Receivers for UDWDM-PONs ($λ$-to-the-user)

It is predicted that demand in optical access networks will reach multi-Gb/s per user. However, the limited performance of the direct detection receiver technology currently used in the optical network units at the customers' premises restricts data rates/user. Therefore, the concept of coherent-enabled access networks has attracted attention in recent years, as this technology offers high receiver sensitivity, inherent frequency selectivity, and linear field detection enabling the full compensation of linear channel impairments. However, the complexity of conventional (dual-polarisation digital) coherent receivers has so far prevented their introduction into access networks. Thus, to exploit the benefits of coherent technology in the ONUs, low complexity coherent receivers, suitable for implementation in ONUs, are needed. In this paper, the recently proposed low complexity coherent (i.e., polarisation-independent Alamouti-coding heterodyne) receiver is, for the first time, compared in terms of its minimum receiver sensitivity with five previously reported receiver designs, including a detailed discussion on their advantages and limitations. It is shown that the Alamouti-coding based receiver approach allows the lowest number of photons per bit (PPB) transmitted (with a lower bound of 15.5 PPB in an ideal system simulations) whilst requiring the lowest optical receiver hardware complexity. It also exhibits comparable complexity to the currently deployed direct-detection receivers, which typically require >1000 PPB. Finally, a comparison of experimentally achieved receiver sensitivities and transmission distances using these receivers is presented. The highest spectral efficiency and longest transmission distance at the highest bit rate reported using the Alamouti-coding receiver, which is also the only one, to date, to have been demonstrated in a full system bidirectional transmission.

eess.SP↗

arXiv2013

A Low-Fluorine Solution with the F/Ba Mole Ratio of 2 for the Fabrication of YBCO Films

In the reported low-fluorine MOD-YBCO studies, the lowest F/Ba mole ratio of the precursor solution was 4.5. However, further lowering the F/Ba ratio is important according to the researches of YBCO thick film. On the other hand, the F/Ba ratio is necessary to be at least 2 for the full conversion of the Ba precursor to BaF_2 to avoid the formation of BaCO_3, which is detrimental to the superconducting performance. In this study, a novel solution with the F/Ba mole ratio of 2 was developed, in which the fluorine content was only about 10.3% of that used in the conventional TFA-MOD method. Attenuated total reflectance-Fourier transformed-infrared spectra(ATR-FT-IR) revealed that BaCO_3 was remarkably suppressed in the as-pyrolyzed film and eliminated at 700 Celsius degree. Thus YBCO films with a critical current density (J_c) over 5 MA cm^{-2} (77 K, 0 T, 200 nm thickness) could be obtained on LAO single crystal substrates. In-situ FT-IR spectra showed that no obvious fluorinated gaseous by-products were detected in the pyrolysis step, which indicated that all of the F atoms might remain in the film as fluorides. X-ray diffraction (XRD) θ/2θ-scan showed that BaF_2, but neither YF_3 nor CuF_2, was detected in the films quenched at 400 - 800 Celsius degree. The formation priority of BaF_2 over YF_3 and CuF_2 was interpreted by the chemical equilibrium of the potential reactions. Our study could enlarge the synthesis window of the precursor solution for MOD-YBCO fabrication and open a gate to study the fluorine content in the precursor solution continuously and systematically.

cond-mat.mtrl-sci↗