Searcharxiv⌕ Search

arXiv subjects

Bo He

Publications and source records attributed to Bo He.

At least 19 recordsLinked to original sources
arXiv2026

Metalens stereoscopic optical palpation for imaging cancer mechanics

Accurate intraoperative margin assessment is critical for complete resection of solid tumours. However, surgeons routinely rely on manual palpation, which is highly subjective and fundamentally limited by poor spatial resolution, and pre-operative imaging lacks real-time feedback. While emerging optical elastography techniques offer quantitative mechanical contrast, their reliance on bulky optical components necessitates cumbersome alignment and hinders integration into miniaturised surgical instruments. Here, we report metalens stereoscopic optical palpation (MSOP), a nanophotonic imaging platform leveraging a binocular metalens integrated with a single CMOS sensor to achieve high-fidelity, self-aligned elastography. By replacing complex bulk optics with a compact stereoscopic architecture, MSOP enables robust, high-contrast mapping of tissue surface stress. Following validation in heterogeneous silicone phantoms, we demonstrate MSOP's translational potential by characterising the mechanical signatures of malignancies in fresh mouse pancreatic cancer models, as well as excised human breast and liver specimens. By providing reliable, label-free mechanical contrast across diverse oncological landscapes, MSOP offers a compact tool for precise intraoperative margin delineation to reduce re-excision rates and improve surgical outcomes.

physics.optics↗
arXiv2026

Internal Data Repetition Destroys Language Models

Language models are running out of high-quality training data, and even aggressively deduplicated corpora retain some amount of repetition. Earlier controlled studies predated Chinchilla-style scaling laws and could only measure the cost of repetition indirectly. We revisit repetition in the Chinchilla era, using a fitted no-repetition scaling law to report Compute-Equivalent Gain and Compute-Equivalent Loss. We show that under this modernized paradigm, repetition damage is systematic in three ways. First, holding compute allocated to repeated data constant, eval loss peaks at an intermediate repeat count $\Rep$; repeating a moderately sized subset a moderate number of times damages performance more than repeating a large subset a few times or a small subset many times. Second, the location of this peak is well-fit by a power law in model size; this scaling law reveals that the most damaging number of repeated data grows more quickly than compute. Finally, when repeated documents consume 10\% of the FLOPs budget in a controlled exact-document repetition setting, the compute-equivalent loss can be large: on FineWeb-Edu-Dedup, the most damaging repeat count for a Qwen3-style 344M-parameter model at $\OT=1$ matches the loss of a no-repetition run using 67% of the FLOPs. We demonstrate that these phenomena are not language-model-specific, and can be analytically understood in a simple statistical model: a misspecified linear regression with verbatim duplicates reproduces the same qualitative loss peak, quantifying how such peaks can arise from a statistical tradeoff between memorization and generalization. Our findings add precision to the study of duplication in language models, allowing practitioners to quantify the wasted compute incurred by the presence and repeat structure of duplicates in pretraining corpora.

cs.LG↗
arXiv2026

IDEAL: In-DEpth ALignment Makes A Discrete Representation AutoEncoder

Built on pretrained vision foundation models (VFMs), representation autoencoders (RAEs) have recently emerged as a promising approach for constructing semantically rich latent spaces for image generation. However, their reconstruction quality often remains suboptimal, largely because deep VFM representations do not preserve sufficient fine-grained visual detail. This limitation becomes even more severe after discretization, where missing low-level information is difficult to recover. In fact, we observe that shallow VFM features retain considerably richer local appearance and structural detail, which complements the high-level semantics carried by deep features used in existing RAEs. Motivated by this complementary property, we propose Ideal, an In-depth Alignment framework for discrete representation autoencoding. By jointly aligning quantized tokens with both shallow and deep VFM features, Ideal enables the resulting discrete visual tokens to preserve both visual fidelity and rich semantics. Extensive experiments demonstrate that Ideal yields superior reconstruction performance, achieving 0.61 rFID on ImageNet and outperforming the previous best method by 0.28. When used for autoregressive image generation, Ideal further produces a gFID of 1.89, establishing a new state of the art for autoregressive image generation.

cs.CV↗
arXiv2026

FluxMem: Adaptive Hierarchical Memory for Streaming Video Understanding

This paper presents FluxMem, a training-free framework for efficient streaming video understanding. FluxMem adaptively compresses redundant visual memory through a hierarchical, two-stage design: (1) a Temporal Adjacency Selection (TAS) module removes redundant visual tokens across adjacent frames, and (2) a Spatial Domain Consolidation (SDC) module further merges spatially repetitive regions within each frame into compact representations. To adapt effectively to dynamic scenes, we introduce a self-adaptive token compression mechanism in both TAS and SDC, which automatically determines the compression rate based on intrinsic scene statistics rather than manual tuning. Extensive experiments demonstrate that FluxMem achieves new state-of-the-art results on existing online video benchmarks, reaching 76.4 on StreamingBench and 67.2 on OVO-Bench under real-time settings, while reducing latency by 69.9% and peak GPU memory by 34.5% on OVO-Bench. Furthermore, it maintains strong offline performance, achieving 73.1 on MLVU while using 65% fewer visual tokens.

cs.CV↗
arXiv2026

Scale Dependent Data Duplication

Data duplication during pretraining can degrade generalization and lead to memorization, motivating aggressive deduplication pipelines. However, at web scale, it is unclear what constitutes a ``duplicate'': beyond surface-form matches, semantically equivalent documents (e.g. translations) may induce redundant training signals once models become sufficiently capable. Practically, this means that semantic duplicates operate increasingly like exact duplicates during training. We present evidence that duplication is scale-dependent in two ways. First, as model capability increases, cross-entropy loss gradients for semantically equivalent documents become more aligned. Smaller models, by contrast, produce gradients that reflect surface similarity (e.g., shared tokens) rather than semantic similarity. Second, we embedded all 192 million FineWeb-Edu-Dedup documents using EmbeddingGemma-300m. For moderate corpus sizes, the cosine similarity between nearest-neighbors follows an isotropic power law baseline. However, as corpus size grows to hundreds of billions of tokens, the nearest-neighbor similarities deviate sharply, indicating accelerated semantic collisions. Finally, controlled pretraining on data sampled with replacement from pools of finite unique documents shows that limited uniqueness yields mild degradation for small models, but rapidly increasing loss penalties for larger models, breaking naive scaling extrapolation. We derive explicit scaling laws that allow practitioners to estimate deviation from expected scaling due to limited semantic uniqueness of the pretraining corpus. Our results identify and resolve an unstudied source of scale-dependence, allowing for more accurate prediction at scale.

cs.LG↗
arXiv2026

VideoLoom: A Video Large Language Model for Joint Spatial-Temporal Understanding

This paper presents VideoLoom, a unified Video Large Language Model (Video LLM) for joint spatial-temporal understanding. To facilitate the development of fine-grained spatial and temporal localization capabilities, we curate LoomData-8.7k, a human-centric video dataset with temporally grounded and spatially localized captions. With this, VideoLoom achieves state-of-the-art or highly competitive performance across a variety of spatial and temporal benchmarks (e.g., 63.1 J&F on ReVOS for referring video object segmentation, and 48.3 R1@0.7 on Charades-STA for temporal grounding). In addition, we introduce LoomBench, a novel benchmark consisting of temporal, spatial, and compositional video-question pairs, enabling a comprehensive evaluation of Video LLMs from diverse aspects. Collectively, these contributions offer a universal and effective suite for joint spatial-temporal video understanding, setting a new standard in multimodal intelligence.

cs.CV↗
arXiv2025

The diophantine equation $(2^{k}-1)(b^{k}-1)=y^{q}$

In this paper, we consider the exponential Diophantine equation \( (2^k-1)(b^k-1)=y^q \) with $k\ge 2$, odd integer $b$ and an odd prime exponent $q$ and obtain effective upper bounds for $q$ in terms of $b$. In particular, we show that $q\le \log_2(b+1)$ holds apart from a finite, explicitly determined set of exceptional pairs $(b,q)$ when $3\le b<10^6$. As an application, we prove that the related equation \( (2^k-1)(b^k-1)=x^n, \) has no positive integer solution $(k,x,n)$ for several specific odd values of $b$, including $b\in\{5,7,11,13,21,23,27,29\}$.

math.NT↗
arXiv2025

Absolute summability from diagonal operators to Carleson embeddings and Hankel operators

The present paper consists of three parts, each building on the preceding one. In Part I, we characterize the scalar sequences $\mathbf b=\{b_k\}$ for which the diagonal operator $\mathscr{M}_{\mathbf b}\bigl(\{a_k\}\bigr):=\{b_ka_k\}$ is $r$-summing from $\ell^p$ to $\ell^q$ for \(1\le p,q\le \infty\) and \(1\le r<\infty\). This resolves a problem left open in Garling's 1974 classification of \(r\)-summing diagonal operators. In particular, in the previously unresolved exceptional range $\{(p,q,r):1 \max\{p',q\}\},$ we identify a new intermediate exponent \(\kappa\in(\max\{p',q\},r)\) and prove that \(\mathscr M_{\mathbf b}\) is \(r\)-summing if and only if \(\mathbf b\in\ell^\kappa\). In Part II, by developing two general transference principles for $r$-summability and building on the characterization from Part I, we characterize the positive Borel measures $\mu$ for which the Carleson embedding operator $J_\mu$ is $r$-summing from the weighted Bergman space $A_\alpha^p(\mathbb B_n)$ to $L^q(\mathbb B_n,d\mu)$ for $ 1\leq p,q<\infty$ , $1\leq r<\infty$, and $\alpha>-1$, thereby extending the existing theory to the full off-diagonal range. In Part III, building on the characterization from Part II, we characterize the symbols \(f\) and \(g\) for which the big and little Hankel operators \(H_f^\beta\) and \(h_{\bar g}^\beta\), respectively, are \(r\)-summing from \(A_\alpha^p(\mathbb B_n)\) to \(L^q(\mathbb B_n,dv_\beta)\), where $1\leq p<\infty$, $1 -1$, and \(dv_\beta(z):=c_\beta(1-|z|^2)^\beta\,dv(z)\). These characterizations appear to be new even in the diagonal case $p=q$ and $\alpha=\beta$.

math.FA↗
arXiv2025

NeRV-Diffusion: Diffuse Implicit Neural Representations for Video Synthesis

We present NeRV-Diffusion, an implicit latent video diffusion model that synthesizes videos via generating neural network weights. The generated weights can be rearranged as the parameters of a convolutional neural network, which forms an implicit neural representation (INR), and decodes into videos with frame indices as the input. Our framework consists of two stages: 1) A hypernetworkbased tokenizer that encodes raw videos from pixel space to neural parameter space, where the bottleneck latent serves as INR weights to decode. 2) An implicit diffusion transformer that denoises on the latent INR weights. In contrast to traditional video tokenizers that encode videos into frame-wise feature maps, NeRV-Diffusion compresses and generates a video holistically as a unified neural network. This enables efficient and high-quality video synthesis via obviating temporal cross-frame attentions in the denoiser and decoding video latent with dedicated decoders. To achieve Gaussian-distributed INR weights with high expressiveness, we reuse the bottleneck latent across all NeRV layers, as well as reform its weight assignment, upsampling connection and input coordinates. We also introduce SNR-adaptive loss weighting and scheduled sampling for effective training of the implicit diffusion model. NeRV-Diffusion reaches superior video generation quality over previous INR-based models and comparable performance to most recent state-of-the-art non-implicit models on real-world video benchmarks including UCF-101 and Kinetics-600. It also brings a smooth INR weight space that facilitates seamless interpolations between frames or videos.

cs.CV↗
arXiv2025

Absolutely summing Carleson embeddings on weighted Fock spaces with $A_{\infty}$-type weights

In this paper, we investigate the $r$-summing Carleson embeddings on weighted Fock spaces $F^p_{\alpha,w}$. By using duality arguments, translating techniques and block diagonal operator skills, we completely characterize the $r$-summability of the natural embeddings $I_d:F^p_{\alpha,w}\to L^p_{\alpha}(\mu)$ for any $r\geq1$ and $p>1$, where $w$ is a weight on the complex plane $\mathbb{C}$ that satisfies an $A_p$-type condition. As applications, we establish some results on the $r$-summability of differentiation and integration operators, Volterra-type operators and composition operators. Especially, we completely characterize the boundedness of Volterra-type operators and composition operators on vector-valued Fock spaces for all $1<p<\infty$, which were left open before for the case $1<p<2$.

math.FA↗
arXiv2025

OIPR: Evaluation for Time-series Anomaly Detection Inspired by Operator Interest

With the growing adoption of time-series anomaly detection (TAD) technology, numerous studies have employed deep learning-based detectors to analyze time-series data in the fields of Internet services, industrial systems, and sensors. The selection and optimization of anomaly detectors strongly rely on the availability of an effective evaluation for TAD performance. Since anomalies in time-series data often manifest as a sequence of points, conventional metrics that solely consider the detection of individual points are inadequate. Existing TAD evaluators typically employ point-based or event-based metrics to capture the temporal context. However, point-based evaluators tend to overestimate detectors that excel only in detecting long anomalies, while event-based evaluators are susceptible to being misled by fragmented detection results. To address these limitations, we propose OIPR (Operator Interest-based Precision and Recall metrics), a novel TAD evaluator with area-based metrics. It models the process of operators receiving detector alarms and handling anomalies, utilizing area under the operator interest curve to evaluate TAD performance. Furthermore, we build a special scenario dataset to compare the characteristics of different evaluators. Through experiments conducted on the special scenario dataset and five real-world datasets, we demonstrate the remarkable performance of OIPR in extreme and complex scenarios. It achieves a balance between point and event perspectives, overcoming their primary limitations and offering applicability to broader situations.

cs.LG↗
arXiv2025

Pix2Cap-COCO: Advancing Visual Comprehension via Pixel-Level Captioning

We present Pix2Cap-COCO, the first panoptic pixel-level caption dataset designed to advance fine-grained visual understanding. To achieve this, we carefully design an automated annotation pipeline that prompts GPT-4V to generate pixel-aligned, instance-specific captions for individual objects within images, enabling models to learn more granular relationships between objects and their contexts. This approach results in 167,254 detailed captions, with an average of 22.94 words per caption. Building on Pix2Cap-COCO, we introduce a novel task, panoptic segmentation-captioning, which challenges models to recognize instances in an image and provide detailed descriptions for each simultaneously. To benchmark this task, we design a robust baseline based on X-Decoder. The experimental results demonstrate that Pix2Cap-COCO is a particularly challenging dataset, as it requires models to excel in both fine-grained visual understanding and detailed language generation. Furthermore, we leverage Pix2Cap-COCO for Supervised Fine-Tuning (SFT) on large multimodal models (LMMs) to enhance their performance. For example, training with Pix2Cap-COCO significantly improves the performance of GPT4RoI, yielding gains in CIDEr +1.4%, ROUGE +0.4%, and SPICE +0.5% on Visual Genome dataset, and strengthens its region understanding ability on the ViP-BENCH, with an overall improvement of +5.1%, including notable increases in recognition accuracy +11.2% and language generation quality +22.2%.

cs.CV↗
arXiv2024

Deadline and Priority Constrained Immersive Video Streaming Transmission Scheduling

Deadline-aware transmission scheduling in immersive video streaming is crucial. The objective is to guarantee that at least a certain block in multi-links is fully delivered within their deadlines, which is referred to as delivery ratio. Compared with existing models that focus on maximizing throughput and ultra-low latency, which makes bandwidth resource allocation and user satisfaction locally optimized, immersive video streaming needs to guarantee more high-priority block delivery within personalized deadlines. In this paper, we propose a deadline and priority-constrained immersive video streaming transmission scheduling scheme. It builds an accurate bandwidth prediction model that can sensitively assist scheduling decisions. It divides video streaming into various media elements and performs scheduling based on the user's personalized latency sensitivity thresholds and the media element's priority. We evaluate our scheme via trace-driven simulations. Compared with existing models, the results further demonstrate the superiority of our scheme with 12{\%}-31{\%} gains in quality of experience (QoE).

cs.NI↗
arXiv2024

UMono: Physical Model Informed Hybrid CNN-Transformer Framework for Underwater Monocular Depth Estimation

Underwater monocular depth estimation serves as the foundation for tasks such as 3D reconstruction of underwater scenes. However, due to the influence of light and medium, the underwater environment undergoes a distinctive imaging process, which presents challenges in accurately estimating depth from a single image. The existing methods fail to consider the unique characteristics of underwater environments, leading to inadequate estimation results and limited generalization performance. Furthermore, underwater depth estimation requires extracting and fusing both local and global features, which is not fully explored in existing methods. In this paper, an end-to-end learning framework for underwater monocular depth estimation called UMono is presented, which incorporates underwater image formation model characteristics into network architecture, and effectively utilize both local and global features of underwater image. Experimental results demonstrate that the proposed method is effective for underwater monocular depth estimation and outperforms the existing methods in both quantitative and qualitative analyses.

cs.CV↗
arXiv2024

MA-LMM: Memory-Augmented Large Multimodal Model for Long-Term Video Understanding

With the success of large language models (LLMs), integrating the vision model into LLMs to build vision-language foundation models has gained much more interest recently. However, existing LLM-based large multimodal models (e.g., Video-LLaMA, VideoChat) can only take in a limited number of frames for short video understanding. In this study, we mainly focus on designing an efficient and effective model for long-term video understanding. Instead of trying to process more frames simultaneously like most existing work, we propose to process videos in an online manner and store past video information in a memory bank. This allows our model to reference historical video content for long-term analysis without exceeding LLMs' context length constraints or GPU memory limits. Our memory bank can be seamlessly integrated into current multimodal LLMs in an off-the-shelf manner. We conduct extensive experiments on various video understanding tasks, such as long-video understanding, video question answering, and video captioning, and our model can achieve state-of-the-art performances across multiple datasets. Code available at https://boheumd.github.io/MA-LMM/.

cs.CV↗
arXiv2024

OmniVid: A Generative Framework for Universal Video Understanding

The core of video understanding tasks, such as recognition, captioning, and tracking, is to automatically detect objects or actions in a video and analyze their temporal evolution. Despite sharing a common goal, different tasks often rely on distinct model architectures and annotation formats. In contrast, natural language processing benefits from a unified output space, i.e., text sequences, which simplifies the training of powerful foundational language models, such as GPT-3, with extensive training corpora. Inspired by this, we seek to unify the output space of video understanding tasks by using languages as labels and additionally introducing time and box tokens. In this way, a variety of video tasks could be formulated as video-grounded token generation. This enables us to address various types of video tasks, including classification (such as action recognition), captioning (covering clip captioning, video question answering, and dense video captioning), and localization tasks (such as visual object tracking) within a fully shared encoder-decoder architecture, following a generative framework. Through comprehensive experiments, we demonstrate such a simple and straightforward idea is quite effective and can achieve state-of-the-art or competitive results on seven video benchmarks, providing a novel perspective for more universal video understanding. Code is available at https://github.com/wangjk666/OmniVid.

cs.CV↗