arXiv subjects

Biao Wu

Publications and source records attributed to Biao Wu.

At least 55 records · Page 3Linked to original sources

arXiv2025

PresentAgent: Multimodal Agent for Presentation Video Generation

We present PresentAgent, a multimodal agent that transforms long-form documents into narrated presentation videos. While existing approaches are limited to generating static slides or text summaries, our method advances beyond these limitations by producing fully synchronized visual and spoken content that closely mimics human-style presentations. To achieve this integration, PresentAgent employs a modular pipeline that systematically segments the input document, plans and renders slide-style visual frames, generates contextual spoken narration with large language models and Text-to-Speech models, and seamlessly composes the final video with precise audio-visual alignment. Given the complexity of evaluating such multimodal outputs, we introduce PresentEval, a unified assessment framework powered by Vision-Language Models that comprehensively scores videos across three critical dimensions: content fidelity, visual clarity, and audience comprehension through prompt-based evaluation. Our experimental validation on a curated dataset of 30 document-presentation pairs demonstrates that PresentAgent approaches human-level quality across all evaluation metrics. These results highlight the significant potential of controllable multimodal agents in transforming static textual materials into dynamic, effective, and accessible presentation formats. Code will be available at https://github.com/AIGeeksGroup/PresentAgent.

cs.CV↗

arXiv2025

Curriculum Learning with Quality-Driven Data Selection

The impressive multimodal capabilities demonstrated by OpenAI's GPT-4 have generated significant interest in the development of Multimodal Large Language Models (MLLMs). Visual instruction tuning of MLLMs with machine-generated instruction-following data has shown to enhance zero-shot capabilities across various tasks. However, there has been limited exploration into controlling the quality of the instruction data.Current methodologies for data selection in MLLMs often rely on single, unreliable scores or use downstream tasks for selection, which is time-consuming and can lead to potential overfitting on the chosen evaluation datasets. To mitigate these limitations, we propose a novel data selection methodology that utilizes image-text correlation and model perplexity to evaluate and select data of varying quality. This approach leverages the distinct distribution of these two attributes, mapping data quality into a two-dimensional space that allows for the selection of data based on their location within this distribution. By utilizing this space, we can analyze the impact of task type settings, used as prompts, on data quality. Additionally, this space can be used to construct multi-stage subsets of varying quality to facilitate curriculum learning. Our research includes comprehensive experiments conducted on various datasets. The results emphasize substantial enhancements in five commonly assessed capabilities compared to using the complete dataset. Our codes, data, and models are publicly available at: https://anonymous.4open.science/r/EHIT-31B4

cs.LG↗

arXiv2025

RepNet-VSR: Reparameterizable Architecture for High-Fidelity Video Super-Resolution

As a fundamental challenge in visual computing, video super-resolution (VSR) focuses on reconstructing highdefinition video sequences from their degraded lowresolution counterparts. While deep convolutional neural networks have demonstrated state-of-the-art performance in spatial-temporal super-resolution tasks, their computationally intensive nature poses significant deployment challenges for resource-constrained edge devices, particularly in real-time mobile video processing scenarios where power efficiency and latency constraints coexist. In this work, we propose a Reparameterizable Architecture for High Fidelity Video Super Resolution method, named RepNet-VSR, for real-time 4x video super-resolution. On the REDS validation set, the proposed model achieves 27.79 dB PSNR when processing 180p to 720p frames in 103 ms per 10 frames on a MediaTek Dimensity NPU. The competition results demonstrate an excellent balance between restoration quality and deployment efficiency. The proposed method scores higher than the previous champion algorithm of MAI video super-resolution challenge.

eess.IV↗

arXiv2025

MMCLIP: Cross-modal Attention Masked Modelling for Medical Language-Image Pre-Training

Vision-and-language pretraining (VLP) in the medical field utilizes contrastive learning on image-text pairs to achieve effective transfer across tasks. Yet, current VLP approaches with the masked modeling strategy face two challenges when applied to the medical domain. First, current models struggle to accurately reconstruct key pathological features due to the scarcity of medical data. Second, most methods only adopt either paired image-text or image-only data, failing to exploit the combination of both paired and unpaired data. To this end, this paper proposes the MMCLIP (Masked Medical Contrastive Language-Image Pre-Training) framework to enhance pathological learning and feature learning via unpaired data. First, we introduce the attention-masked image modeling (AttMIM) and entity-driven masked language modeling module (EntMLM), which learns to reconstruct pathological visual and textual tokens via multi-modal feature interaction, thus improving medical-enhanced features. The AttMIM module masks a portion of the image features that are highly responsive to textual features. This allows MMCLIP to improve the reconstruction of highly similar image data in medicine efficiency. Second, our MMCLIP capitalizes unpaired data to enhance multimodal learning by introducing disease-kind prompts. The experimental results show that MMCLIP achieves SOTA for zero-shot and fine-tuning classification performance on five datasets. Our code will be available at https://github.com/AIGeeksGroup/MMCLIP.

cs.CV↗

arXiv2025

On cross-2-intersecting families

Two families $\mathcal A\subseteq\binom{[n]}{k}$ and $\mathcal B\subseteq\binom{[n]}{\ell}$ are called cross-$t$-intersecting if $|A\cap B|\geq t$ for all $A\in\mathcal A$, $B\in\mathcal B$. Let $n$, $k$ and $\ell$ be positive integers such that $n\geq 3.38\ell$ and $\ell\geq k\geq 2$. In this paper, we will determine the upper bound of $|\mathcal A||\mathcal B|$ for cross-$2$-intersecting families $\mathcal A\subseteq\binom{[n]}{k}$ and $\mathcal B\subseteq\binom{[n]}{\ell}$. The structures of the extremal families attaining the upper bound are also characterized. The similar result obtained by Tokushige can be considered as a special case of ours when $k=\ell$, but under a more strong condition $n>3.42k$. Moreover, combined with the results obtained in this paper, the complicated extremal structures attaining the upper bound for nontrivial cases can be relatively easy to reach with similar techniques.

math.CO↗

arXiv2025

Motion Anything: Any to Motion Generation

Conditional motion generation has been extensively studied in computer vision, yet two critical challenges remain. First, while masked autoregressive methods have recently outperformed diffusion-based approaches, existing masking models lack a mechanism to prioritize dynamic frames and body parts based on given conditions. Second, existing methods for different conditioning modalities often fail to integrate multiple modalities effectively, limiting control and coherence in generated motion. To address these challenges, we propose Motion Anything, a multimodal motion generation framework that introduces an Attention-based Mask Modeling approach, enabling fine-grained spatial and temporal control over key frames and actions. Our model adaptively encodes multimodal conditions, including text and music, improving controllability. Additionally, we introduce Text-Music-Dance (TMD), a new motion dataset consisting of 2,153 pairs of text, music, and dance, making it twice the size of AIST++, thereby filling a critical gap in the community. Extensive experiments demonstrate that Motion Anything surpasses state-of-the-art methods across multiple benchmarks, achieving a 15% improvement in FID on HumanML3D and showing consistent performance gains on AIST++ and TMD. See our project website https://steve-zeyu-zhang.github.io/MotionAnything

cs.CV↗

arXiv2025

Thermal Entanglement in Disordered Spin Chains: Localization, Thresholds, and the Quantum-to-Classical Crossover

We investigate the mixed-state entanglement between two spins embedded in the XXZ Heisenberg chain under thermal equilibrium. By deriving an analytical expression for the entanglement of two-spin thermal states and extending this analysis to larger spin chains, we demonstrate that mixed-state entanglement is profoundly shaped by both disorder and temperature. Our results reveal a sharp distinction between many-body localized (MBL) and ergodic phases, with entanglement vanishing above different finite temperature thresholds. Furthermore, by analyzing non-adjacent spins, we uncover an approximate exponential decay of entanglement with separation. This work advances the understanding of the quantum-to-classical transition by linking the entanglement properties of small subsystems to the broader thermal environment, offering a explanation for the absence of entanglement in macroscopic systems. These findings provide critical insights into quantum many-body physics, bridging concepts from thermalization, localization, and quantum information theory.

quant-ph↗

arXiv2025

Variational method for $\mathbb{Z}_K$ wavefunctions in spin-$J$ PXP model

We investigate the approach of time-dependent variational principle (TDVP) for the one-dimensional spin-$J$ PXP model with detuning, which is relevant for programmable Rydberg atom arrays. The variational manifold is chosen as the minimally entangled $\mathbb{Z}_K$ matrix-product-states (MPS). We demonstrate that variational dynamics and variational error can be expressed as rapidly convergent series in the thermodynamic limit. In particular, for $J=1/2$ and the limiting case $J\rightarrow \infty$, the TDVP results become exact and significantly simplified.

quant-ph↗

arXiv2024

Exact extremal non-trivial cross-intersecting families

Two families $\mathcal{A}$ and $\mathcal{B}$ of sets are called cross-intersecting if each pair of sets $A\in \mathcal{A}$ and $B\in \mathcal{B}$ has nonempty intersection. Let $\cal{A}$ and ${\cal B}$ be two cross-intersecting families of $k$-subsets and $\ell$-subsets of $[n]$. Matsumoto and Tokushige [J. Combin. Theory Ser. A 52 (1989) 90--97] studied the extremal problem of the size $|\cal{A}||\cal{B}|$ and obtained the uniqueness of extremal families whenever $n\ge 2 \ell\ge 2k$, building on the work of Pyber. This paper will explore the second extremal size of $|\cal{A}||\cal{B}|$ and obtain that if $\mathcal{A}$ and $\mathcal{B}$ are not the subfamilies of Matsumoto--Tokushige's extremal families, then, for $n\ge 2\ell >2k$ or $n> 2\ell=2k$, \begin{itemize} \item[1)]either $|\cal{A}||\cal{B}|\le \left({\binom{n-1}{k-1}}+{\binom{n-2 }{k-1}}\right){\binom{n-2}{\ell-2}}$ with the unique extremal families (up to isomorphism) \[\mbox{$\mathcal{A}=\{A\in {\binom{[n]}{k}}: 1\in A \: \rm{ or} \: 2\in A\}$ \quad and \quad $\mathcal{B}=\{B\in {\binom{[n]}{\ell}}: [2] \subseteq B\}$};\] \item[2)] or $|\cal{A}||\cal{B}|\le \left({\binom{n-1}{k-1}}+1\right)\left({\binom{n-1}{\ell-1}}-{\binom{n-k-1}{\ell-1}}\right)$ with the unique extremal families (up to isomorphism) \[\mbox{$\mathcal{A}=\{A\in {\binom{[n]}{k}}: 1\in A\}\cup \{[2,k+1] \}$\quad and \quad $\mathcal{B}=\{B\in {\binom{[n]}{\ell}}: 1\in B, B\cap [2,k+1]\neq \emptyset \}$.}\] \end{itemize} The bound ``$n\ge 2\ell >2k$ or $n> 2\ell=2k$" is sharp for $n$. To achieve the above results, we establish some size-sensitive inequalities for cross-intersecting families. As by-products, we will recover the main results of Frankl and Kupavskii [European J. Combin. 62 (2017) 263--271].

math.CO↗

arXiv2024

On a conjecture of Tokushige for cross-$t$-intersecting families

Two families of sets $\mathcal{A}$ and $\mathcal{B}$ are called cross-$t$-intersecting if $|A\cap B|\ge t$ for all $A\in \mathcal{A}$, $B\in \mathcal{B}$. An active problem in extremal set theory is to determine the maximum product of sizes of cross-$t$-intersecting families. This incorporates the classical Erdős--Ko--Rado (EKR) problem. In the present paper, we prove that if $\mathcal{A}$ and $\mathcal{B}$ are cross-$t$-intersecting families of $\binom {[n]}k$ with $k\ge t\ge 3$ and $n\ge (t+1)(k-t+1)$, then $|\mathcal{A}||\mathcal{B}|\le {\binom{n-t}{k-t}}^2$; moreover, if $n>(t+1)(k-t+1)$, then equality holds if and only if $\mathcal{A}=\mathcal{B}$ is a maximum $t$-intersecting subfamily of $\binom{[n]}{k}$. This confirms a conjecture of Tokushige for $t\ge 3$.

math.CO↗

arXiv2024

Act Like a Radiologist: Radiology Report Generation across Anatomical Regions

Automating radiology report generation can ease the reporting workload for radiologists. However, existing works focus mainly on the chest area due to the limited availability of public datasets for other regions. Besides, they often rely on naive data-driven approaches, e.g., a basic encoder-decoder framework with captioning loss, which limits their ability to recognise complex patterns across diverse anatomical regions. To address these issues, we propose X-RGen, a radiologist-minded report generation framework across six anatomical regions. In X-RGen, we seek to mimic the behaviour of human radiologists, breaking them down into four principal phases: 1) initial observation, 2) cross-region analysis, 3) medical interpretation, and 4) report formation. Firstly, we adopt an image encoder for feature extraction, akin to a radiologist's preliminary review. Secondly, we enhance the recognition capacity of the image encoder by analysing images and reports across various regions, mimicking how radiologists gain their experience and improve their professional ability from past cases. Thirdly, just as radiologists apply their expertise to interpret radiology images, we introduce radiological knowledge of multiple anatomical regions to further analyse the features from a clinical perspective. Lastly, we generate reports based on the medical-aware features using a typical auto-regressive text decoder. Both natural language generation (NLG) and clinical efficacy metrics show the effectiveness of X-RGen on six X-ray datasets. Our code and checkpoints are available at: https://github.com/YtongXie/X-RGen.

cs.CV↗

arXiv2024

Stabilities for non-uniform $t$-intersecting families

The study of intersection problems on families of sets is one of the most important topics in extremal combinatorics. As we all know, the extremal problems involving certain intersection constraints are equivalent to that with the union properties by taking complement of sets. A family of sets is called $s$-union if the union of any two sets in this family has size at most $s$. Katona [Acta Math. Hungar. 15 (1964)] provided the maximum size of an $s$-union family of sets of $[n]$, and he also determined the extremal families up to isomorphism. Recently, Frankl [J. Combin. Theory Ser. B 122 (2017) 869--876] sharpened this result by establishing the maximum size of an $s$-union family that is not a subfamily of the so-called Katona family. In this paper, we determine the maximum size of an $s$-union family that is neither contained in the Katona family nor in the Frankl family. Moreover, we characterize all extremal families achieving the upper bounds.

math.CO↗

arXiv2024

Quantum Hamiltonian Algorithms for Maximum Independent Sets

With qubits encoded into atomic ground and Rydberg states and situated on the vertexes of a graph, the conditional quantum dynamics of Rydberg blockade, which inhibits simultaneous excitation of nearby atoms, has been employed recently to find maximum independent sets following an adiabatic evolution algorithm hereafter denoted by HV [Science 376, 1209 (2022)]. An alternative algorithm, short named the PK algorithm, reveals that the independent sets diffuse over a media graph governed by a non-abelian gauge matrix of an emergent PXP model. This work shows the above two algorithms are mathematically equivalent, despite of their seemingly different physical implementations. More importantly, we demonstrated that although the two are mathematically equivalent, the PK algorithm exhibits more efficient and resource-saving performance. Within the same range of experimental parameters, our numerical studies suggest that the PK algorithm performs at least 25% better on average and saves at least $6\times10^6$ measurements ($\sim 900$ hours of continuous operation) for each graph when compared to the HV algorithm. We further consider the measurement error and point out that this may cause the oscillations in the performance of the HV's optimization process.

quant-ph↗

arXiv2024

The Power of Lorentz Quantum Computer

We demonstrate the superior capabilities of the recently proposed Lorentz quantum computer (LQC) compared to conventional quantum computers. We introduce an associated computational complexity class termed bounded-error Lorentz quantum polynomial-time (BLQP), demonstrating its equivalence to the complexity class ${\text P}^{\sharp \text{P}}$. We present LQC algorithms that efficiently solve the problem of maximum independent set, PP (probabilistic polynomial-time), and consequently ${\text P}^{\sharp \text{P}}$, all within polynomial time. Additionally, we show that the quantum computing with postselection proposed by Aaronson can be efficiently simulated by LQC, but not vice versa.

quant-ph↗

arXiv2024

Motion Avatar: Generate Human and Animal Avatars with Arbitrary Motion

In recent years, there has been significant interest in creating 3D avatars and motions, driven by their diverse applications in areas like film-making, video games, AR/VR, and human-robot interaction. However, current efforts primarily concentrate on either generating the 3D avatar mesh alone or producing motion sequences, with integrating these two aspects proving to be a persistent challenge. Additionally, while avatar and motion generation predominantly target humans, extending these techniques to animals remains a significant challenge due to inadequate training data and methods. To bridge these gaps, our paper presents three key contributions. Firstly, we proposed a novel agent-based approach named Motion Avatar, which allows for the automatic generation of high-quality customizable human and animal avatars with motions through text queries. The method significantly advanced the progress in dynamic 3D character generation. Secondly, we introduced a LLM planner that coordinates both motion and avatar generation, which transforms a discriminative planning into a customizable Q&A fashion. Lastly, we presented an animal motion dataset named Zoo-300K, comprising approximately 300,000 text-motion pairs across 65 animal categories and its building pipeline ZooGen, which serves as a valuable resource for the community. See project website https://steve-zeyu-zhang.github.io/MotionAvatar/

cs.CV↗

arXiv2024

Quantum Entanglement and Chocolates

Two statistical ensembles of chocolates are constructed to mimic two quantum entangled states, the spin singlet state and the GHZ state. Despite great efforts to achieve the closest possible resemblance in terms of probability distribution and correlation, subtle differences remain. The differences reveal and illustrate a fundamental characteristic that distinguishes quantum entanglement from classical correlation.

quant-ph↗

arXiv2024

Semi-supervised Video Semantic Segmentation Using Unreliable Pseudo Labels for PVUW2024

Pixel-level Scene Understanding is one of the fundamental problems in computer vision, which aims at recognizing object classes, masks and semantics of each pixel in the given image. Compared with image scene parsing, video scene parsing introduces temporal information, which can effectively improve the consistency and accuracy of prediction,because the real-world is actually video-based rather than a static state. In this paper, we adopt semi-supervised video semantic segmentation method based on unreliable pseudo labels. Then, We ensemble the teacher network model with the student network model to generate pseudo labels and retrain the student network. Our method achieves the mIoU scores of 63.71% and 67.83% on development test and final test respectively. Finally, we obtain the 1st place in the Video Scene Parsing in the Wild Challenge at CVPR 2024.

cs.CV↗

arXiv2024

2nd Place Solution for PVUW Challenge 2024: Video Panoptic Segmentation

Video Panoptic Segmentation (VPS) is a challenging task that is extends from image panoptic segmentation.VPS aims to simultaneously classify, track, segment all objects in a video, including both things and stuff. Due to its wide application in many downstream tasks such as video understanding, video editing, and autonomous driving. In order to deal with the task of video panoptic segmentation in the wild, we propose a robust integrated video panoptic segmentation solution. We use DVIS++ framework as our baseline to generate the initial masks. Then,we add an additional image semantic segmentation model to further improve the performance of semantic classes.Finally, our method achieves state-of-the-art performance with a VPQ score of 56.36 and 57.12 in the development and test phases, respectively, and ultimately ranked 2nd in the VPS track of the PVUW Challenge at CVPR2024.

cs.CV↗