arXiv subjects

Jianhua Zhu

Publications and source records attributed to Jianhua Zhu.

15 recordsLinked to original sources

arXiv2026

EgoSIS: From Factorized Visual Ego-Transitions to Motion-Canonical Spatial Evidence for UAV Reasoning

UAV video question answering requires separating camera motion from changes in the scene, but RGB-only multimodal models receive no explicit, stable reference for that separation. We present EgoSIS, a pose-free adapter that converts RGB-derived bidirectional flow into motion-canonical visual evidence in three stages. Factorized Visual Ego-Transitions (FVET) fits a robust image-plane transition and exposes motion, residual-support, and reliability factors. Reliability-Gated Ego-Transition Memory (ReTEM) uses reliability-weighted updates for a bounded history and re-anchors it at cuts or sustained uncertainty. Ego-Aligned Spatial Evidence (EASE) warps supported visual features into each segment's local anchor and injects four spatial evidence tokens per visual slice through zero-initialized residuals, without changing Qwen's visual-token count. On SIS-Bench, EgoSIS-8B obtains 89.9\% perception, 82.5\% perception-plus-memory, and 76.2\% overall accuracy, with the largest gains concentrated in self-awareness perception and memory. The adapter thus provides an interpretable interface between optical flow and spatial reasoning.

cs.CV↗

arXiv2026

GraphPL: Leveraging GNN for Efficient and Robust Modalities Imputation in Patchwork Learning

Current research on distributed multi-modal learning typically assumes that clients can access complete information across all modalities, which may not hold in practice. In this paper, we explore patchwork learning, in which the modalities available to different clients vary, and the objective is to impute the missing modalities for each client in an unsupervised manner. Existing methods are shown not to fully utilize the modality information as they tend to rely on only a subset of the observed modalities. To address this issue, we propose GraphPL, which combines graph neural networks with patchwork learning to flexibly integrate all observed modalities and remains robust with noisy inputs. Experimental results show that GraphPL achieves SOTA performance on benchmark datasets. Our results on real-world distributed electronic health record dataset show GraphPL learns strong downstream features and enables tasks like disease prediction via superior modality imputation.

cs.LG↗

arXiv2026

Clair Obscur: an Illumination-Aware Method for Real-World Image Vectorization

Image vectorization aims to convert raster images into editable, scalable vector representations while preserving visual fidelity. Existing vectorization methods struggle to represent complex real-world images, often producing fragmented shapes at the cost of semantic conciseness. In this paper, we propose COVec, an illumination-aware vectorization framework inspired by the Clair-Obscur principle of light-shade contrast. COVec is the first to introduce intrinsic image decomposition in the vector domain, separating an image into albedo, shade, and light layers in a unified vector representation. A semantic-guided initialization and two-stage optimization refine these layers with differentiable rendering. Experiments on various datasets demonstrate that COVec achieves higher visual fidelity and significantly improved editability compared to existing methods. The code will be released at https://github.com/decade-de/COVec.

cs.CV↗

arXiv2025

Uni-MuMER: Unified Multi-Task Fine-Tuning of Vision-Language Model for Handwritten Mathematical Expression Recognition

Handwritten Mathematical Expression Recognition (HMER) remains a persistent challenge in Optical Character Recognition (OCR) due to the inherent freedom of symbol layouts and variability in handwriting styles. Prior methods have faced performance bottlenecks by proposing isolated architectural modifications, making them difficult to integrate coherently into a unified framework. Meanwhile, recent advances in pretrained vision-language models (VLMs) have demonstrated strong cross-task generalization, offering a promising foundation for developing unified solutions. In this paper, we introduce Uni-MuMER, which fully fine-tunes a VLM for the HMER task without modifying its architecture, effectively injecting domain-specific knowledge into a generalist framework. Our method integrates three data-driven tasks: Tree-Aware Chain-of-Thought (Tree-CoT) for structured spatial reasoning, Error-Driven Learning (EDL) for reducing confusion among visually similar characters, and Symbol Counting (SC) for improving recognition consistency in long expressions. Experiments on the CROHME and HME100K datasets show that Uni-MuMER achieves super state-of-the-art performance, outperforming the best lightweight specialized model SSAN by 16.31\% and the top-performing VLM Gemini2.5-flash by 24.42\% under zero-shot setting. Our datasets, models, and code are open-sourced at: {https://github.com/BFlameSwift/Uni-MuMER

cs.CV↗

arXiv2025

Do Large Language Models Excel in Complex Logical Reasoning with Formal Language?

Large Language Models (LLMs) have been shown to achieve breakthrough performance on complex logical reasoning tasks. Nevertheless, most existing research focuses on employing formal language to guide LLMs to derive reliable reasoning paths, while systematic evaluations of these capabilities are still limited. In this paper, we aim to conduct a comprehensive evaluation of LLMs across various logical reasoning problems utilizing formal languages. From the perspective of three dimensions, i.e., spectrum of LLMs, taxonomy of tasks, and format of trajectories, our key findings are: 1) Thinking models significantly outperform Instruct models, especially when formal language is employed; 2) All LLMs exhibit limitations in inductive reasoning capability, irrespective of whether they use a formal language; 3) Data with PoT format achieves the best generalization performance across other languages. Additionally, we also curate the formal-relative training data to further enhance the small language models, and the experimental results indicate that a simple rejected fine-tuning method can better enable LLMs to generalize across formal languages and achieve the best overall performance. Our codes and reports are available at https://github.com/jiangjin1999/FormalEval.

cs.CL↗

arXiv2025

Seed1.5-VL Technical Report

We present Seed1.5-VL, a vision-language foundation model designed to advance general-purpose multimodal understanding and reasoning. Seed1.5-VL is composed with a 532M-parameter vision encoder and a Mixture-of-Experts (MoE) LLM of 20B active parameters. Despite its relatively compact architecture, it delivers strong performance across a wide spectrum of public VLM benchmarks and internal evaluation suites, achieving the state-of-the-art performance on 38 out of 60 public benchmarks. Moreover, in agent-centric tasks such as GUI control and gameplay, Seed1.5-VL outperforms leading multimodal systems, including OpenAI CUA and Claude 3.7. Beyond visual and video understanding, it also demonstrates strong reasoning abilities, making it particularly effective for multimodal reasoning challenges such as visual puzzles. We believe these capabilities will empower broader applications across diverse tasks. In this report, we mainly provide a comprehensive review of our experiences in building Seed1.5-VL across model design, data construction, and training at various stages, hoping that this report can inspire further research. Seed1.5-VL is now accessible at https://www.volcengine.com/ (Volcano Engine Model ID: doubao-1-5-thinking-vision-pro-250428)

cs.CV↗

arXiv2025

Generative Evaluation of Complex Reasoning in Large Language Models

With powerful large language models (LLMs) demonstrating superhuman reasoning capabilities, a critical question arises: Do LLMs genuinely reason, or do they merely recall answers from their extensive, web-scraped training datasets? Publicly released benchmarks inevitably become contaminated once incorporated into subsequent LLM training sets, undermining their reliability as faithful assessments. To address this, we introduce KUMO, a generative evaluation framework designed specifically for assessing reasoning in LLMs. KUMO synergistically combines LLMs with symbolic engines to dynamically produce diverse, multi-turn reasoning tasks that are partially observable and adjustable in difficulty. Through an automated pipeline, KUMO continuously generates novel tasks across open-ended domains, compelling models to demonstrate genuine generalization rather than memorization. We evaluated 23 state-of-the-art LLMs on 5,000 tasks across 100 domains created by KUMO, benchmarking their reasoning abilities against university students. Our findings reveal that many LLMs have outperformed university-level performance on easy reasoning tasks, and reasoning-scaled LLMs reach university-level performance on complex reasoning challenges. Moreover, LLM performance on KUMO tasks correlates strongly with results on newly released real-world reasoning benchmarks, underscoring KUMO's value as a robust, enduring assessment tool for genuine LLM reasoning capabilities.

cs.CL↗

arXiv2025

SketchRef: a Multi-Task Evaluation Benchmark for Sketch Synthesis

Sketching is a powerful artistic technique for capturing essential visual information about real-world objects and has increasingly attracted attention in image synthesis research. However, the field lacks a unified benchmark to evaluate the performance of various synthesis methods. To address this, we propose SketchRef, the first comprehensive multi-task evaluation benchmark for sketch synthesis. SketchRef fully leverages the shared characteristics between sketches and reference photos. It introduces two primary tasks: category prediction and structural consistency estimation, the latter being largely overlooked in previous studies. These tasks are further divided into five sub-tasks across four domains: animals, common things, human body, and faces. Recognizing the inherent trade-off between recognizability and simplicity in sketches, we are the first to quantify this balance by introducing a recognizability calculation method constrained by simplicity, mRS, ensuring fair and meaningful evaluations. To validate our approach, we collected 7,920 responses from art enthusiasts, confirming the effectiveness of our proposed evaluation metrics. Additionally, we evaluate the performance of existing sketch synthesis methods on our benchmark, highlighting their strengths and weaknesses. We hope this study establishes a standardized benchmark and offers valuable insights for advancing sketch synthesis algorithms.

cs.CV↗

arXiv2024

TAMER: Tree-Aware Transformer for Handwritten Mathematical Expression Recognition

Handwritten Mathematical Expression Recognition (HMER) has extensive applications in automated grading and office automation. However, existing sequence-based decoding methods, which directly predict $\LaTeX$ sequences, struggle to understand and model the inherent tree structure of $\LaTeX$ and often fail to ensure syntactic correctness in the decoded results. To address these challenges, we propose a novel model named TAMER (Tree-Aware Transformer) for handwritten mathematical expression recognition. TAMER introduces an innovative Tree-aware Module while maintaining the flexibility and efficient training of Transformer. TAMER combines the advantages of both sequence decoding and tree decoding models by jointly optimizing sequence prediction and tree structure prediction tasks, which enhances the model's understanding and generalization of complex mathematical expression structures. During inference, TAMER employs a Tree Structure Prediction Scoring Mechanism to improve the structural validity of the generated $\LaTeX$ sequences. Experimental results on CROHME datasets demonstrate that TAMER outperforms traditional sequence decoding and tree decoding models, especially in handling complex mathematical structures, achieving state-of-the-art (SOTA) performance.

cs.CV↗

arXiv2024

ICAL: Implicit Character-Aided Learning for Enhanced Handwritten Mathematical Expression Recognition

Significant progress has been made in the field of handwritten mathematical expression recognition, while existing encoder-decoder methods are usually difficult to model global information in $LaTeX$. Therefore, this paper introduces a novel approach, Implicit Character-Aided Learning (ICAL), to mine the global expression information and enhance handwritten mathematical expression recognition. Specifically, we propose the Implicit Character Construction Module (ICCM) to predict implicit character sequences and use a Fusion Module to merge the outputs of the ICCM and the decoder, thereby producing corrected predictions. By modeling and utilizing implicit character information, ICAL achieves a more accurate and context-aware interpretation of handwritten mathematical expressions. Experimental results demonstrate that ICAL notably surpasses the state-of-the-art(SOTA) models, improving the expression recognition rate (ExpRate) by 2.25\%/1.81\%/1.39\% on the CROHME 2014/2016/2019 datasets respectively, and achieves a remarkable 69.06\% on the challenging HME100k test set. We make our code available on the GitHub: https://github.com/qingzhenduyu/ICAL

cs.CV↗

arXiv2024

CLoG: Benchmarking Continual Learning of Image Generation Models

Continual Learning (CL) poses a significant challenge in Artificial Intelligence, aiming to mirror the human ability to incrementally acquire knowledge and skills. While extensive research has focused on CL within the context of classification tasks, the advent of increasingly powerful generative models necessitates the exploration of Continual Learning of Generative models (CLoG). This paper advocates for shifting the research focus from classification-based CL to CLoG. We systematically identify the unique challenges presented by CLoG compared to traditional classification-based CL. We adapt three types of existing CL methodologies, replay-based, regularization-based, and parameter-isolation-based methods to generative tasks and introduce comprehensive benchmarks for CLoG that feature great diversity and broad task coverage. Our benchmarks and results yield intriguing insights that can be valuable for developing future CLoG methods. Additionally, we will release a codebase designed to facilitate easy benchmarking and experimentation in CLoG publicly at https://github.com/linhaowei1/CLoG. We believe that shifting the research focus to CLoG will benefit the continual learning community and illuminate the path for next-generation AI-generated content (AIGC) in a lifelong learning paradigm.

cs.LG↗

arXiv2024

One-dimensional Dexter-type excitonic topological phase transition

Recently topogical excitons have attracted much attention. However, studies on the topological properties of excitons in one dimension are still rare. Here we have computed the Zak phase for a generic one-dimensional dimerised excitonic model. Tuning relevant hopping parameters gives rise to a rich spectrum of physics, including non-trivial topological phase in uniform chain unlike the conventional Su-Shcrieffer-Heeger model, topologically nontrivial flat bands, and exotic fractional phase. a new concept of ``composite chiral site" was developed to interpret the Zak phase of $π$ in our calculations. Our finite-chain calculations substantiate topological edge states, providing more information about their characteristics. Most importantly, in the first time, a topological phase transition assisted by the Dexter electron exchange process has been found.

cond-mat.mes-hall↗

arXiv2023

Co-existence of charge density wave and anti-ferromagnetic coupling in the spin-chain compound Ba$_6$Cr$_2$S$_{10}$

Here we have performed detailed first principles calculations for the electronic structure and magnetic properties of Ba$_6$Cr$_2$S$_{10}$ to study the origin of the anti-ferromagnetic exchange interaction between spins on Cr ions for the spin-chain compound Ba$_6$Cr$_2$S$_{10}$ synthesised recently. Most importantly, we have found the co-existence of a charge density wave phase along one line and an anti-ferromagnetic spin chain along another. The dimerization of sulfur atoms loosely bonded with Ba atoms drives the system into an insulating state owing to the formation of charge density wave. Meanwhile, the small size of the effective Hubbard $U$ parameter ($\sim 0.5$ eV) due to electrostatic screening mainly accounts for the anti-ferromagnetic ground state. This co-existence equips us with a platform to tune the charge and spin degrees of freedom independently. Moreover, there exists a next-nearest-neighbouring anti-ferromagnetic interaction along the chain, which could bring forward spin frustration and hence quantum spin liquid.

cond-mat.str-el↗

arXiv2021

Multi-hole models for deterministically placed acceptor arrays in silicon

We compute the electronic structure of acceptor clusters in silicon by using three methods to include electron correlations: the full configuration interaction, the Heitler-London approximation, and the unrestricted Hartree-Fock method. We show both the HL approach and the UHF method are good approximations to the ground state of the full CI calculation for a pair of acceptors and for finite linear chains. The total energies for finite linear chains show the formation of a 4-fold degenerate ground state when there is a weak bond at the end of the chain, which is shown to be a manifold of topological edge states. We identify a change in the angular momentum composition of the ground state at a critical pattern of bond lengths and show it is related to a crossing in the Fock matrix eigenvalues. We also test the symmetry of the UHF solution and compare it to the full CI; the symmetry is broken under almost all the arrangements by the formation of a magnetic state in UHF, and we find further broken symmetries for some particular arrangements related to crossings or potential crossings between the Fock-matrix eigenvalues. We also compute the charge distributions across the acceptors obtained from the eigenvectors of the Fock matrix: with weak bonds at the chain ends, two holes are localized at either end of the chain while the others have a nearly uniform distribution over the middle; this implies the existence of the non-trivial edge states. We apply the UHF method to treat an infinite linear chain with periodic boundary conditions. We find the band structures in the UHF approximation and compute the Zak phases for the occupied Fock-matrix eigenvalues; we find they do not correctly predict the topological edge states in this interacting system. We find direct study of the quantum numbers characterising the edge states provides a better insight into their topological nature.

cond-mat.mes-hall↗

arXiv2020

A linear combination of atomic orbitals (LCAO) model for deterministically placed acceptor arrays in silicon

We develop a tight-binding model based on linear combination of atomic orbitals (LCAO) methods to describe the electronic structure of arrays of acceptors, where the underlying basis states are derived from an effective-mass-theory solution for a single acceptor in either the spherical approximation or the cubic model. Our model allows for arbitrarily strong spin-orbit coupling in the valence band of the semiconductor. We have studied pairs and dimerised linear chains of acceptors in silicon in the `independent-hole' approximation, and investigated the conditions for the existence of topological edge states in the chains. For the finite chain we find a complex interplay between electrostatic effects and the dimerisation, with the long-range Coulomb attraction of the hole to the acceptors splitting off states localised at the end acceptors from the rest of the chain. A further pair of states then splits off from each band, to form a pair localised on the next-to-end acceptors, for one sense of the bond alternation and merges into the bulk bands for the other sense of the alternation. We confirm the topologically non-trivial nature of these next-to-end localised states by calculating the Zak phase. We argue that for the more physically accessible case of one hole per acceptor these long-range electrostatic effects will be screened out; we show this by treating a simple phenomenologically screened model in which electrostatic contributions from beyond the nearest neighbours of acceptor each pair are removed. Topological states are now found on the end acceptors of the chains. In some cases the termination of the chain required to produce topological states is not the one expected on the basis of simple geometry (short versus long bonds); we argue this is because of a non-monotonic relationship between the bond length and the effective Hamiltonian matrix elements between the acceptors.

cond-mat.mes-hall↗