arXiv subjects

Peng Hou

Publications and source records attributed to Peng Hou.

13 recordsLinked to original sources

arXiv2026

Human and AI collaboration for pulmonary nodule segmentation

Medical expert annotators are scarce, and blind reliance on artificial intelligence (AI) can be misleading, motivating approaches in which humans, particularly junior medical trainees or even non-medical personnel, collaborate with AI to achieve robust medical segmentation. Although the Segment Anything Model (SAM) shows promise for general-purpose image segmentation, its performance in human-AI collaboration for specialized medical tasks has not been thoroughly evaluated. Here we present Hi-Seg, a human-in-the-loop segmentation framework for pulmonary nodules built on SAM. Humans iteratively refine prompts through trial-and-error learning and semantic reasoning, progressively guiding SAM toward higher-quality masks. Using chest CT scans from 1,179 patients across 12 centers, we conducted the first large-scale external validation of collaborative human-SAM segmentation. Across all annotator groups, Hi-Seg achieved a mean Dice score of almost 85%, outperforming five state-of-the-art deep learning models by 10-22% and 13 SAM variants by 1-29%. Hi-Seg improved segmentation accuracy while reducing annotation time for medical annotators, and briefly trained non-medical annotators achieved performance comparable to that of the junior medical student. These findings suggest that human-in-the-loop segmentation can reduce clinician workload, enable scalable crowdsourced annotation, and transform clinical workflows by facilitating the safe and efficient integration of foundation models into routine clinical practice.

cs.CV↗

arXiv2026

CharTide: Data-Centric Chart-to-Code Generation via Tri-Perspective Tuning and Inquiry-Driven Evolution

Chart-to-code generation demands strict visual precision and syntactic correctness from Vision-Language Models (VLMs). However, existing approaches are fundamentally constrained by data-centric limitations: despite the availability of growing chart-to-code datasets, simply scaling homogeneous chart-code pairs conflates visual perception with program logic, preventing models from fully leveraging the richness of multimodal supervision. We present CharTide, a novel data-centric framework that systematically redesigns both training and alignment data for chart-to-code generation. First, we construct a 2M-sample dataset via a Tri-Perspective Tuning strategy, explicitly decoupling training into visual perception, pure-text code logic, and modality fusion streams, enabling a 7B model to surpass specialized baselines using only supervised data. Second, we reformulate alignment as a data verification problem rather than a heuristic scoring task. To this end, we introduce an Inquiry-Driven RL framework grounded in the principle of information invariance: a downstream model should yield consistent answers to identical visual queries across both original and generated charts. Moving beyond rigid rule matching or VLM scoring, we employ a frozen Inspector to objectively verify generated charts through atomic QA tasks, providing verifiable reward signals based on answer accuracy. Experiments on ChartMimic, Plot2Code, and ChartX show that CharTide-7B/8B significantly outperforms open-source baselines, surpasses GPT-4o, and is competitive with GPT-5.

cs.CV↗

arXiv2026

SEA-Vision: A Multilingual Benchmark for Comprehensive Document and Scene Text Understanding in Southeast Asia

Multilingual document and scene text understanding plays an important role in applications such as search, finance, and public services. However, most existing benchmarks focus on high-resource languages and fail to evaluate models in realistic multilingual environments. In Southeast Asia, the diversity of languages, complex writing systems, and highly varied document types make this challenge even greater. We introduce SEA-Vision, a benchmark that jointly evaluates Document Parsing and Text-Centric Visual Question Answering (TEC-VQA) across 11 Southeast Asian languages. SEA-Vision contains 15,234 document parsing pages from nine representative document types, annotated with hierarchical page-, block-, and line-level labels. It also provides 7,496 TEC-VQA question-answer pairs that probe text recognition, numerical calculation, comparative analysis, logical reasoning, and spatial understanding. To make such multilingual, multi-task annotation feasible, we design a hybrid pipeline for Document Parsing and TEC-VQA. It combines automated filtering and scoring with MLLM-assisted labeling and lightweight native-speaker verification, greatly reducing manual labeling while maintaining high quality. We evaluate several leading multimodal models and observe pronounced performance degradation on low-resource Southeast Asian languages, highlighting substantial remaining gaps in multilingual document and scene text understanding. We believe SEA-Vision will help drive global progress in document and scene text understanding.

cs.CL↗

arXiv2026

CCCaption: Dual-Reward Reinforcement Learning for Complete and Correct Image Captioning

Image captioning remains a fundamental task for vision language understanding, yet ground-truth supervision still relies predominantly on human-annotated references. Because human annotations reflect subjective preferences and expertise, ground-truth captions are often incomplete or even incorrect, which in turn limits caption models. We argue that caption quality should be assessed by two objective aspects: completeness (does the caption cover all salient visual facts?) and correctness (are the descriptions true with respect to the image?). To this end, we introduce CCCaption: a dual-reward reinforcement learning framework with a dedicated fine-tuning corpus that explicitly optimizes these properties to generate \textbf{C}omplete and \textbf{C}orrect \textbf{Captions}. For completeness, we use diverse LVLMs to disentangle the image into a set of visual queries, and reward captions that answer more of these queries, with a dynamic query sampling strategy to improve training efficiency. For correctness, we penalize captions that contain hallucinations by validating the authenticity of sub-caption queries, which are derived from the caption decomposition. Our symmetric dual-reward optimization jointly maximizes completeness and correctness, guiding models toward captions that better satisfy these objective criteria. Extensive experiments across standard captioning benchmarks show consistent improvements, offering a principled path to training caption models beyond human-annotation imitation.

cs.CV↗

arXiv2026

Towards On-Policy SFT: Distribution Discriminant Theory and its Applications in LLM Training

Supervised fine-tuning (SFT) is computationally efficient but often yields inferior generalization compared to reinforcement learning (RL). This gap is primarily driven by RL's use of on-policy data. We propose a framework to bridge this chasm by enabling On-Policy SFT. We first present \textbf{\textit{Distribution Discriminant Theory (DDT)}}, which explains and quantifies the alignment between data and the model-induced distribution. Leveraging DDT, we introduce two complementary techniques: (i) \textbf{\textit{In-Distribution Finetuning (IDFT)}}, a loss-level method to enhance generalization ability of SFT, and (ii) \textbf{\textit{Hinted Decoding}}, a data-level technique that can re-align the training corpus to the model's distribution. Extensive experiments demonstrate that our framework achieves generalization performance surpassing prominent offline RL algorithms, including DPO and SimPO, while maintaining the efficiency of an SFT pipeline. The proposed framework thus offers a practical alternative in domains where RL is infeasible. We open-source the code here: https://github.com/zhangmiaosen2000/Towards-On-Policy-SFT

cs.LG↗

arXiv2025

MUG-V 10B: High-efficiency Training Pipeline for Large Video Generation Models

In recent years, large-scale generative models for visual content (\textit{e.g.,} images, videos, and 3D objects/scenes) have made remarkable progress. However, training large-scale video generation models remains particularly challenging and resource-intensive due to cross-modal text-video alignment, the long sequences involved, and the complex spatiotemporal dependencies. To address these challenges, we present a training framework that optimizes four pillars: (i) data processing, (ii) model architecture, (iii) training strategy, and (iv) infrastructure for large-scale video generation models. These optimizations delivered significant efficiency gains and performance improvements across all stages of data preprocessing, video compression, parameter scaling, curriculum-based pretraining, and alignment-focused post-training. Our resulting model, MUG-V 10B, matches recent state-of-the-art video generators overall and, on e-commerce-oriented video generation tasks, surpasses leading open-source baselines in human evaluations. More importantly, we open-source the complete stack, including model weights, Megatron-Core-based large-scale training code, and inference pipelines for video generation and enhancement. To our knowledge, this is the first public release of large-scale video generation training code that exploits Megatron-Core to achieve high training efficiency and near-linear multi-node scaling, details are available in https://github.com/Shopee-MUG/MUG-V.

cs.CV↗

arXiv2025

Data-driven modeling of wind farm wake flow based on multi-scale feature recognition

Accurate, efficient prediction of wind flow with wake effects is crucial for wind-farm layout and power forecasting. Existing approaches-physical measurements, numerical simulations, physics-based models, and data-driven models-face trade-offs: the first two are time- and resource-intensive; physics-based models can lack accuracy due to limited physics; data-driven methods leverage abundant, high-quality data and are increasingly popular. We propose a rapid, data-driven wake-flow model inspired by video-frame interpolation and the principle of similarity. Field data are transformed into images; multi-scale feature recognition then identifies, matches, and interpolates wake structures using Scale-Invariant Feature Transform (SIFT) and Dynamic Time Warping (DTW) to generate intermediate flow fields. Six representative mini wind-farm cases validate the approach, spanning variations in turbine spacing, turbine size, combined spacing-size variations, different turbine counts, and wind-direction misalignment. Across cases, the method achieves a mean absolute percentage error (MAPE) of 0.68-2.28%. Because it flexibly computes both 2D and 3D wake fields, the method offers substantial computational-efficiency gains over large-eddy simulation (LES) and Meteodyn WT when 2D accuracy suffices for industrial needs. Accordingly, it provides a practical alternative to measurements, high-fidelity simulations, and simplified physics-based models, enabling efficient expansion of wake-flow databases for wind-farm design and power prediction while balancing speed and accuracy.

physics.flu-dyn↗

arXiv2024

Annotation-Free Curb Detection Leveraging Altitude Difference Image

Road curbs are considered as one of the crucial and ubiquitous traffic features, which are essential for ensuring the safety of autonomous vehicles. Current methods for detecting curbs primarily rely on camera imagery or LiDAR point clouds. Image-based methods are vulnerable to fluctuations in lighting conditions and exhibit poor robustness, while methods based on point clouds circumvent the issues associated with lighting variations. However, it is the typical case that significant processing delays are encountered due to the voluminous amount of 3D points contained in each frame of the point cloud data. Furthermore, the inherently unstructured characteristics of point clouds poses challenges for integrating the latest deep learning advancements into point cloud data applications. To address these issues, this work proposes an annotation-free curb detection method leveraging Altitude Difference Image (ADI), which effectively mitigates the aforementioned challenges. Given that methods based on deep learning generally demand extensive, manually annotated datasets, which are both expensive and labor-intensive to create, we present an Automatic Curb Annotator (ACA) module. This module utilizes a deterministic curb detection algorithm to automatically generate a vast quantity of training data. Consequently, it facilitates the training of the curb detection model without necessitating any manual annotation of data. Finally, by incorporating a post-processing module, we manage to achieve state-of-the-art results on the KITTI 3D curb dataset with considerably reduced processing delays compared to existing methods, which underscores the effectiveness of our approach in curb detection tasks.

cs.CV↗

arXiv2023

Transformer-empowered Multi-modal Item Embedding for Enhanced Image Search in E-Commerce

Over the past decade, significant advances have been made in the field of image search for e-commerce applications. Traditional image-to-image retrieval models, which focus solely on image details such as texture, tend to overlook useful semantic information contained within the images. As a result, the retrieved products might possess similar image details, but fail to fulfil the user's search goals. Moreover, the use of image-to-image retrieval models for products containing multiple images results in significant online product feature storage overhead and complex mapping implementations. In this paper, we report the design and deployment of the proposed Multi-modal Item Embedding Model (MIEM) to address these limitations. It is capable of utilizing both textual information and multiple images about a product to construct meaningful product features. By leveraging semantic information from images, MIEM effectively supplements the image search process, improving the overall accuracy of retrieval results. MIEM has become an integral part of the Shopee image search platform. Since its deployment in March 2023, it has achieved a remarkable 9.90% increase in terms of clicks per user and a 4.23% boost in terms of orders per user for the image search feature on the Shopee e-commerce platform.

cs.CV↗

arXiv2021

Distributionally Robust Chance-Constrained Flexibility Planning for Integrated Energy System

Inflexible combined heat and power (CHP) plants and uncertain wind power production result in excess power in distribution networks, which leads to inverse power flow challenging grid operations. Power-to-X facilities such as electrolysers and electric boilers can offer extra flexibility to the integrated energy system. In this regard, we aim to jointly determine the optimal Power-to-X facility sizing and integrated energy system operations in this study. To account for wind power uncertainties, a distributionally robust chance-constrained model is developed to characterize wind power uncertainties using ambiguity sets. Linear decision rules are applied to analytically express real-time recourse actions when uncertainties are exposed, which allows the propagation of wind power uncertainties to gas and heat systems. Accordingly, the developed three-stage distributionally robust chance-constrained model is converted into a computationally tractable single-stage mixed-integer conic model. A case study validates the effectiveness of introducing the electrolyser and electric boiler into the integrated energy system, with respect to the decreased system cost, expanded CHP plant flexibility and reduced inverse power flow. The developed distributionally robust optimization model exhibits better effectiveness and robustness compared to a chance-constrained optimization model assuming wind forecast errors follow Gaussian distribution. Detailed profit analysis reveals that although the overall system cost is minimized, the profit is distributed unevenly across various stakeholders in the system.

eess.SY↗

arXiv2019

A network-constrained rolling transactive energy model for EV aggregators participating in balancing marke

The increasing adoption of renewable energy sources increases the need for balancing power. The security concerns of the distribution system operators are increasing due to fast adoption of distributed energy resources. So far, various operational models are proposed to manage the distributed energy resources for economic benefit. However, there is lack of an efficient operational model that balances the security, market, and uncertainty management. With the focus on the aggregators operation, this paper developed an operational model for continuous operation of aggregators interactively with the grid operators. A rolling optimization model is developed to manage the uncertainties from prediction errors. The network security is ensured continuously through an iterative price negotiation process between the distribution network operator and the aggregators. The optimality of the solution is guaranteed by convexification of the mixed integer formulation.

eess.SY↗

arXiv2019

An Interactive Transactive Energy Mechanism Integrating Grid Operators, Aggregators and Prosumers

With the decreasing cost of solar photovoltaics (PV) and battery storage systems, more and more prosumers appear in the distribution systems. Accompanying with it is the trend of using home energy management systems (HEMS). HEMS technologies can help the households to schedule their energy prosumption with aims such as reduced electricity bills or increased self-sufficiency. However, their economic-driven operation can affect the grid security. Therefore, it is paramount to design a framework that can accommodate the interests of the key stakeholders in distribution systems, namely the grid operators, aggregators, and prosumers. In this paper, a novel transactive energy based operational framework is proposed. On the upper level, aggregators will interact with distribution grid operators through transactive approach to ensure the grid interests are satisfied. If there are grid issues, the aggregator will interact with the prosumers through a designed price adder. The simulation results indicate that the proposed framework can effectively accommodate the prosumers operation in distribution systems while keeping the key stakeholders interests.

eess.SY↗

arXiv2019

Hearing Lips: Improving Lip Reading by Distilling Speech Recognizers

Lip reading has witnessed unparalleled development in recent years thanks to deep learning and the availability of large-scale datasets. Despite the encouraging results achieved, the performance of lip reading, unfortunately, remains inferior to the one of its counterpart speech recognition, due to the ambiguous nature of its actuations that makes it challenging to extract discriminant features from the lip movement videos. In this paper, we propose a new method, termed as Lip by Speech (LIBS), of which the goal is to strengthen lip reading by learning from speech recognizers. The rationale behind our approach is that the features extracted from speech recognizers may provide complementary and discriminant clues, which are formidable to be obtained from the subtle movements of the lips, and consequently facilitate the training of lip readers. This is achieved, specifically, by distilling multi-granularity knowledge from speech recognizers to lip readers. To conduct this cross-modal knowledge distillation, we utilize an efficacious alignment scheme to handle the inconsistent lengths of the audios and videos, as well as an innovative filtering strategy to refine the speech recognizer's prediction. The proposed method achieves the new state-of-the-art performance on the CMLR and LRS2 datasets, outperforming the baseline by a margin of 7.66% and 2.75% in character error rate, respectively.

cs.CV↗