Searcharxiv⌕ Search

arXiv subjects

Wei Yao

Publications and source records attributed to Wei Yao.

At least 19 recordsLinked to original sources

FORGE: Force-Aware Multi-Granularity Control for Efficient Humanoid Loco-Manipulation

Humanoid loco-manipulation requires coordinated locomotion, balance regulation, and manipulation under frequent contacts, making teleoperated demonstration collection difficult: a practical controller must simultaneously provide precision, stability, and contact awareness. We propose FORGE, a force-aware multi-granularity control system with a coarse-to-fine command space spanning velocity navigation, planar foot placement, wrist-pose tracking, and optional force-conditioning inputs. Policy training requires no full-body human motion trajectories or joint-angle references; deployment consumes only sparse bilateral wrist poses relative to the pelvis, coarse motion commands, and optional desired wrist-load inputs. For sustained contact, FORGE combines a momentum observer with a learned correction trained using simulator contact-force supervision. At deployment, it estimates low-frequency, wrist-equivalent interaction forces from proprioception and reconstructed low-level arm PD torques, without wrist force/torque sensors. A quasi-static virtual-target loop uses this estimate and an optional desired applied-force input to adapt the wrist target. A frozen locomotion prior and asymmetric residual actions stabilize manipulation-induced whole-body corrections. Experiments evaluate unified-control stability, task-space tracking, quasi-static wrist-equivalent force estimation, squeezing forces during sustained contact with a rigid box, and downstream diffusion-policy execution of a squat-and-lift task across payload conditions. Website: https://bb0928.github.io/forge/.

cs.RO↗

Resolving Primitive-Sharing Ambiguity in Long-Tailed TLS-Based Industrial MEP Point Cloud Segmentation via Spatial Context Constraints

In terrestrial laser scanning (TLS)-based mechanical, electrical, and plumbing (MEP) point cloud segmentation, safety-critical components such as reducers and valves are persistently misclassified, blocking reliable engineering knowledge extraction. This stems from a dual crisis--extreme class imbalance (215:1) compounded by geometric ambiguity, since most tail classes share cylindrical primitives with dominant head classes--that existing frequency-based re-weighting methods cannot resolve. We propose spatial context constraints that exploit neighborhood prediction consistency to disambiguate locally similar structures. Our approach extends Class-Balanced (CB) Loss with two architecture-agnostic mechanisms: Boundary-CB, an entropy-based constraint that emphasizes ambiguous boundaries and encodes an MEP assembly-topology prior, and Density-CB, a density-based constraint that compensates for scan-dependent variations and encodes TLS sensor-physics knowledge. Both operate at the loss level and require no backbone modifications; we evaluate them with ResPointNet++. On the Industrial3D dataset (612.7M labelled points from water treatment facilities), our method achieves 55.74% mIoU on the evaluation split, which also served as the validation split, exceeding the strongest of three backbone baselines (39.83-52.48% mIoU), with a 21.7% relative improvement on tail-class performance (29.59% vs. 24.32%) while preserving head-class accuracy (88.14%). Reducer improves from 0% to 21.12% IoU and valve by 24.3% relative. Evaluated per area, the gain comes from the area rich in these primitive-sharing components (tail mIoU 21.4% to 26.5%), while the other area's mIoU is 0.6 points lower. Spatial context constraints thus reduce primitive-sharing errors in the target MEP setting, supporting more reliable identification of safety-critical components for Digital Twin and Scan-to-BIM applications.

cs.CV↗

Newton-Based Methods for Bilevel Optimization Beyond Lower-Level Strong Convexity

Bilevel optimization provides a framework for machine learning tasks such as hyperparameter optimization, data reweighting, and meta-learning. Existing Newton-based bilevel methods typically rely on the classical hypergradient formula, which is generally unavailable when lower-level solutions are nonunique and the lower-level Hessian is singular. We develop two Newton-based methods beyond lower-level strong convexity by exploiting curvature shared by the lower-level and multiplier updates. Newton-based GALET handles potentially singular lower-level curvature under a positive spectral-gap condition, exploiting its range structure to avoid the squared-Hessian multiplier system used in GALET. Newton-based BAMM uses a strongly convex aggregation model when the lower-level objective is convex and the upper-level objective is strongly convex in the lower-level variable. We establish convergence guarantees and improved complexity bounds relative to GALET and sl-BAMM under the corresponding assumptions. We further introduce GNBO, a generalized Newton-based framework that captures the shared-curvature structure of the proposed methods and several existing Newton and quasi-Newton bilevel methods. Experiments on synthetic problems, data hyper-cleaning, and GPT-2 data reweighting evaluate the proposed methods across different bilevel learning tasks.

math.OC↗

Regime-Aware Router Learning for Multi-Domain Medical Image Generation with Diffusion Experts

Heterogeneous medical image data motivate domain-specialized diffusion experts, but expert specialization alone does not determine how their predictions should be combined during sampling. Learning a global router is particularly challenging when the server has limited, potentially unrepresentative images. We study two router-learning strategies for a fixed expert pool. Server-Only learns input- and timestep-dependent expert assignments from server images with component labels. Anchor-Delta (AD) uses the same images and frozen experts to obtain additional evidence from denoising losses. It distills targets that combine this evidence with the current router's assignments, while a proxy gate controls a subsequent refinement step. Component labels describe data origin; expert losses measure empirical prediction performance. We evaluate these distinct signals through routing analyses and paired medical image generation. An AD-trained router improves generation metrics over Server-Only in a heterogeneous four-domain setting, and a separate server-data study shows gains in held-out routing over a corrected-routing comparator. Additional paired evaluations show that the effect of inference-time evidence mixing varies across benchmarks. These findings establish server-data availability and expert performance as distinct considerations in learning routers for multi-domain diffusion models.

cs.CV↗

XAI-Guided Fourier Domain Generalisation for Pure-Simulation Sim-to-Real Deep Reinforcement Learning

Deep reinforcement learning has shown strong potential for vision-based autonomous driving, but policies trained in simulation often suffer severe performance degradation when deployed in the real world. Existing sim-to-real methods commonly address this gap through image-transfer-based domain adaptation or policy-agnostic domain randomisation. However, the former requires laborious acquisition of target-domain data and is often limited to the deployment domain, whereas the latter introduces unguided perturbations that may not reflect the underlying sim-to-real discrepancy, leading to suboptimal transfer performance. To overcome these limitations, we propose XFD, an explainable artificial intelligence (XAI)-guided Fourier domain generalisation framework for pure-simulation sim-to-real reinforcement learning. The proposed framework extracts explainability attribution maps from simulated observations, and converts these attributions into policy-conditioned Fourier-domain interventions. By perturbing appearance-sensitive spectral components while preserving task-defining spatial structures, XFD constructs robustness-enhancing visual domains entirely within simulation. Subsequently, the proposed robust policy optimisation guides the policy towards attending to task-critical features while suppressing sensitivity to irrelevant visual information. Extensive experiments in simulation and zero-shot real-world deployment demonstrate that XFD consistently improves traveled distance and reward over standard policy and randomly domain-randomised baseline, especially on unseen maps with cluttered backgrounds. Attribution-based analysis further proves that the proposed policy preserves more consistent task-relevant visual evidence across simulation and real-world observations.

cs.AI↗

A Composite Representation Theorem for Solution Mappings in Regularized Composite Optimization with Applications to Hausdorff Lipschitz Stability

We study the geometry and stability of solution mappings in regularized composite optimization, allowing multiple minimizers. We establish a composite representation that separates optimal linear images from their corresponding primal solutions through an image mapping and a generalized basis pursuit mapping. Under a fiber-attainment condition, the image mapping admits a reduced optimization formulation; neither representation requires convexity. For convex regularizers, suitable qualification conditions yield a Fenchel-conjugate representation of the reduced value function and a geometric characterization of the generalized basis pursuit mapping through affine fibers and a fixed lifting set. Using these representations, we establish sufficient conditions for local Hausdorff Lipschitz continuity of possibly set-valued solution mappings. We further show that the primal-space sufficient conditions imply the corresponding image-space conditions, with an additional injectivity requirement arising in the former. The results cover $\ell_1$-type and structured piecewise linear-quadratic regularizers under suitable assumptions. For classical $\ell_1$ regularization, our approach recovers an existing stability result without a finite sign-pattern decomposition. Numerical examples illustrate the recovery of complete solution sets, including a case with a nonpolyhedral graph of the generalized basis pursuit mapping.

math.OC↗

Bayesian Fusion of Active Contour Models and ConvNet Priors for Standing Dead Tree Segmentation

Instance segmentation is a core computer vision task with great practical significance. Recent advances, driven by large-scale benchmark datasets, have yielded good general-purpose Convolutional Neural Network (CNN)-based methods. Natural Resource Monitoring (NRM) utilizes remote sensing imagery with generally known scale and containing multiple overlapping instances of the same class, wherein the object contours are jagged and highly irregular. This is in stark contrast with the regular man-made objects found in classic benchmark datasets. We address this problem and propose a novel instance segmentation method geared towards NRM imagery. We formulate the problem as Bayesian maximum a posteriori inference which, in learning the individual object contours, incorporates shape, location, and position priors from state-of-the-art CNN architectures, driving a simultaneous level-set evolution of multiple object contours. We employ loose coupling between the CNNs that supply the priors and the active contour process, allowing a drop-in replacement of new network architectures. Moreover, we introduce a novel prior for contour shape, namely, a class of Deep Shape Models based on architectures from Generative Adversarial Networks (GANs). These Deep Shape Models are in essence a non-linear generalization of the classic Eigenshape formulation. In experiments, we tackle the challenging, real-world problem of segmenting individual dead tree crowns and delineating precise contours. We compare our method to two leading general-purpose instance segmentation methods - Mask R-CNN and K-net - on color infrared aerial imagery. Results show our approach to significantly outperform both methods in terms of reconstruction quality of tree crown contours. Furthermore, use of the GAN-based deep shape model prior yields significant improvement of all results over the vanilla Eigenshape prior.

cs.CV↗

Spectral Consistency-Guided Multiview Point Cloud Registration for Low-Overlap Scenes

Multiview point cloud registration is particularly challenging in low-overlap scenes, where reliable correspondences are limited and incorrect pairwise transformations can affect global pose estimation. In addition, registering all scan pairs is computationally expensive because many pairs provide weak geometric information. To address these problems, we propose GMPCR, a non-learning-based spectral consistency-guided framework for efficient and robust multiview point cloud registration. GMPCR builds a refined second-order compatibility structure from initial correspondences and uses its dominant spectral response to evaluate both correspondence reliability and scan-pair confidence. This allows unreliable correspondences to be filtered and informative scan pairs to be selected before relative transformation estimation, leading to a sparse pose graph and reduced pairwise registration cost. For each retained scan pair, maximal-clique-based hypothesis generation is used to estimate reliable relative transformations. The resulting pose graph is further refined by an adaptive history-aware synchronization scheme, in which the effect of residual history is adjusted according to changes in the global rotation residual. A recovery mechanism also allows down-weighted edges to regain confidence when their global consistency improves. Experiments on 3DMatch, 3DLoMatch, ScanNet, and ETH demonstrate the effectiveness of GMPCR. It achieves registration recalls of 97.2% and 89.6% on 3DMatch and 3DLoMatch, respectively, while maintaining competitive performance on ScanNet and ETH. The results show that GMPCR provides a favorable balance among registration accuracy, robustness to low overlap, and computational efficiency. The code is publicly available at https://github.com/swccj/gmpcr.

cs.CV↗

Semi-Supervised Hyperspectral Image Classification with Edge-Aware Superpixel Label Propagation and Adaptive Pseudo-Labeling

Significant progress has been made in semi-supervised hyperspectral image (HSI) classification regarding feature extraction and classification performance. However, due to high annotation costs and limited sample availability, semi-supervised learning still faces challenges such as boundary label diffusion and pseudo-label instability. To address these issues, this paper proposes a novel semi-supervised hyperspectral classification framework integrating spatial prior information with a dynamic learning mechanism. First, we design an Edge-Aware Superpixel Label Propagation (EASLP) module. By integrating edge intensity penalty with neighborhood correction strategy, it mitigates label diffusion from superpixel segmentation while enhancing classification robustness in boundary regions. Second, we introduce a Dynamic History-Fused Prediction (DHP) method. By maintaining historical predictions and dynamically weighting them with current results, DHP smoothens pseudo-label fluctuations and improves temporal consistency and noise resistance. Concurrently, incorporating condifence and consistency measures, the Adaptive Tripartite Sample Categorization (ATSC) strategy implements hierarchical utilization of easy, ambiguous, and hard samples, leading to enhanced pseudo-label quality and learning efficiency. The Dynamic Reliability-Enhanced Pseudo-Label Framework (DREPL), composed of DHP and ATSC, strengthens pseudo-label stability across temporal and sample domains. Through synergizes operation with EASLP, it achieves spatio-temporal consistency optimization. Evaluations on four benchmark datasets demonstrate its capability to maintain superior classification performance.

cs.CV↗

Preference-Oriented Aggregation of Heterogeneous Distributed Energy Resources for Reserve Dispatch

Aggregating distributed energy resources (DERs) aims to encode their collective flexibility into a single set for efficient grid dispatch. However, existing aggregation methods are overly conservative for heterogeneous DERs due to two main challenges: 1) dimensional heterogeneity, which complicates the combination of flexibilities across different time dimensions, and 2) type heterogeneity, where diverse and irregular DER profiles hinder accurate approximations, resulting in significant flexibility loss. To resolve these challenges, this paper propose a novel preference-oriented aggregation method for reserve dispatch. For dimensional heterogeneity, we extend existing techniques by reformulating the Minkowski sum as a polytope projection problem using a matrix transformation technique. By unifying DERs in a higher-dimensional space and projecting them back into the aggregate feasible region, the proposed technique effectively aggregates dimensionally heterogeneous DERs. For type heterogeneity, we further develop a distributed aggregation-dispatch coordination framework that incorporates reserve dispatch preferences into aggregation. This framework effectively captures the critical, active aggregate flexibility prioritized in optimal reserve dispatch, thereby significantly reducing the flexibility loss when aggregating type-heterogeneous DERs. Numerical tests validate the effectiveness of our method in addressing both heterogeneities and highlight its promising potential for power systems with high reserve requirements.

eess.SY↗

Continuous-Time Aggregation of Massive Flexible HVAC Loads Considering Uncertainty for Reserve Provision in Power System Dispatch

Heating, ventilation, and air conditioning (HVAC) loads, with their rapid response capabilities, can provide considerable intra-hour flexibility on the demand side for reserve provision in order to follow the fast variations of renewables. However, scheduling massive HVACs is challenging due to computation complexity and the uncertainty of outdoor temperature. In this paper, we first introduce a novel continuous-time (CT) aggregation model to reveal the potential intra-hour flexibility of HVACs. For accurate aggregation, a new affine transformation is designed to handle the heterogeneity in high-dimensional feasible region. Further, for reliable aggregation in practical environment, the outdoor temperature uncertainty is constructed by distributionally robust chance constrains and integrated into the aggregation model. Secondly, for the tractable calculation of the proposed CT aggregation model, a cascade of tailored reformulation techniques is proposed, including the Bernstein polynomial spline, polytope projection, and linearization transformation. Thirdly, a customized hierarchical dispatch framework is proposed via incorporating the proposed CT aggregation model into reserve provision in power system dispatch, so as to efficiently schedule massive HVACs to cope with the renewable uncertainty. Case studies verify the effectiveness and scalability of the proposed CT aggregation model in aggregation accuracy, intra-hour flexibility utilization, and uncertainty handling.

eess.SY↗

Weak-to-Strong Generalization via Bregman Bias-Variance Decomposition

Weak-to-strong generalization (W2SG) is the phenomenon in which a powerful student model, trained on labels produced by a weaker teacher, ultimately outperforms the teacher on the target task. In this work, we theoretically investigate how W2SG can arise via a generalized bias-variance decomposition under Bregman divergence. We show that the expected population risk gap between the student and the teacher is characterized by the expected misfit between the two models. Unlike earlier misfit-based analyses, our theory removes several restrictive assumptions, e.g., it does not require the student hypothesis class to be convex. Our results indicate that W2SG is more likely when the student effectively approximates the teacher's posterior mean. Specializing to squared loss, we provide a sufficient condition (illustrated through a concrete example) under which the student converges to its posterior mean teacher; in particular, increasing the student model size can ensure this convergence. For cross-entropy loss, our analysis further suggests that lowering the entropy of the student's predictive distribution can promote W2SG. We also find that the reverse cross-entropy, unlike the standard forward cross-entropy, is less sensitive to the teacher's predictive uncertainty. Finally, we verify these theoretical insights empirically and demonstrate that incorporating reverse cross-entropy consistently improves student performance.

cs.LG↗

COSTA: A Cluster-Centric Paradigm for Annotation-Free Open-Set Semantic Segmentation of Aerial Point Clouds with Domain Shifts

Semantic segmentation of aerial point cloud is trapped in a generalization crisis under distinct domain shifts. While test-time adaptation offers a privacy-preserving and computationally efficient way to adapt pre-trained models to unlabeled target-domain data during inference, existing methods, bound to closed-set label assumptions and non-scalable point-wise segmentation pipelines, still struggle with semantic shifts. We ask: can we adapt any given pre-trained aerial point cloud segmentation model to a shifted target domain at the inference phase alone, without additional training, while segmenting target-specific categories beyond the source label space on demand? This paper introduces COSTA, which breaks this limitation by shifting from closed-set point-wise adaptation to cluster-centric open-set semantic propagation. Our core discovery is that, once effectively adapted at test time, the rich feature distribution of aerial point clouds can be distilled into a compact set of well-separated semantic centroids that are transferable across label spaces. COSTA leverages this to reformulate open-set semantic segmentation as a cluster-level propagating process: it first bridges the domain gap through proven test-time adaptation, then groups each batch of target-domain points into a small set of semantic clusters based on the similarity distribution in the adapted feature space, and finally propagates high-confidence pseudo labels obtained from an open-vocabulary vision-language model to all points through cluster-level voting. This cluster-centric paradigm enables test-time adaptation of aerial point clouds under significant domain gaps with mixed semantic shifts. With DALES as the source domain, COSTA enables on-demand segmentation across three aerial point cloud benchmarks with distinct domains and heterogeneous category spaces, achieving up to 70.09% mIoU under this new setting.

cs.CV↗

SynAgent: Generalizable Cooperative Humanoid Manipulation via Solo-to-Cooperative Agent Synergy

Controllable cooperative humanoid manipulation is a fundamental yet challenging problem for embodied intelligence, due to severe data scarcity, complexities in multi-agent coordination, and limited generalization across objects. In this paper, we present SynAgent, a unified framework that enables scalable and physically plausible cooperative manipulation by leveraging Solo-to-Cooperative Agent Synergy to transfer skills from single-agent human-object interaction to multi-agent human-object-human scenarios. To maintain semantic integrity during motion transfer, we introduce an interaction-preserving retargeting method based on an Interact Mesh constructed via Delaunay tetrahedralization, which faithfully maintains spatial relationships among humans and objects. Building upon this refined data, we propose a single-agent pretraining and adaptation paradigm that distills synergistic collaborative behaviors from abundant single-human data through decentralized training and multi-agent PPO. Finally, we develop a trajectory-conditioned generative policy using a conditional VAE, trained via multi-teacher distillation from motion imitation priors to achieve stable and controllable object-level trajectory execution. Extensive experiments demonstrate that SynAgent significantly outperforms existing baselines in both cooperative imitation and trajectory-conditioned control, while generalizing across diverse object geometries. Codes and data will be available after publication. Project Page: https://yw0208.github.io/synagent/

cs.CV↗

A Provably Convergent Plug-and-Play Framework for Stochastic Bilevel Optimization

Bilevel optimization has recently attracted significant attention in machine learning due to its wide range of applications and advanced hierarchical optimization capabilities. In this paper, we propose a plug-and-play framework, named PnPBO, for developing and analyzing stochastic bilevel optimization methods. This framework integrates both modern unbiased and biased stochastic estimators into the single-loop bilevel optimization framework introduced in [9], with several improvements. In the implementation of PnPBO, all stochastic estimators for different variables can be independently incorporated, and an additional moving average technique is applied when using an unbiased estimator for the upper-level variable. In the theoretical analysis, we provide a unified convergence and complexity analysis for PnPBO, demonstrating that the adaptation of various stochastic estimators (including PAGE, ZeroSARAH, and mixed strategies) within the PnPBO framework achieves optimal sample complexity, comparable to that of single-level optimization. This resolves the open question of whether the optimal complexity bounds for solving bilevel optimization are identical to those for single-level optimization. Finally, we empirically validate our framework, demonstrating its effectiveness on several benchmark problems and confirming our theoretical findings.

math.OC↗

Industrial3D: A Water-Treatment TLS Point Cloud Dataset and Cross-Paradigm Benchmark for MEP Scene Understanding

Automated semantic understanding of dense terrestrial laser scanning (TLS) point clouds is a prerequisite for Scan-to-BIM, digital twin maintenance, and as-built verifcation. Yet for operational industrial mechanical, electrical, and plumbing (MEP) facilities, this challenge remains largely unsolved: water-treatment TLS scans exhibit extreme geometric ambiguity, severe occlusion, and extreme class imbalance that architectural benchmarks such as S3DIS and ScanNet cannot adequately represent. We present Industrial3D, a terrestrial LiDAR dataset with 612.7 million expert-labeled points at 6 mm resolution from 20 room scenes, 13 dataset areas, and 7 operational water treatment facilities. At 6.6x the scale of the closest comparable MEP dataset, Industrial3D provides the largest industrial MEP testbed for within-domain scene understanding. We further establish a cross-paradigm benchmark of nine methods across fully supervised, weakly supervised, unsupervised, and foundation-model settings. The best supervised method reaches 55.74% mIoU, whereas zero-shot Point-SAM reaches 15.79%, a 39.95 percentage-point gap that quantifes unresolved domain transfer for industrial TLS data. Analysis attributes this gap to a dual crisis: 215:1 statistical rarity and cylindrical geometric ambiguity between tail classes and head-class pipes. The dataset, benchmark code, and pre-trained models will be publicly released at https://github.com/pointcloudyc/Industrial3D.

cs.CV↗

Directional first order approach for a class of bilevel programs

In this paper, we study a class of bilevel optimization program, where the feasible set of the lower level program is independent of the upper level variable. For bilevel programs it is known that the first order reformulation of a bilevel program requires the convexity of the lower level program while reformulations involving the value function result in difficult optimization problems. In this paper we propose a directional first order approach which does not require convexity of the lower level program. First, we propose some conditions under which the lower level program can be equivalently characterized by its first order condition over a directional neighborhood around the local optimal condition. Next we give some conditions under which the classical first order optimality condition in the form of M-stationary condition still holds as a necessary optimality condition for the first order reformulation of the bilevel program even when the lower level program is nonconvex.

math.OC↗

On the Blessing of Pre-training in Weak-to-Strong Generalization

The paradigm of Weak-to-Strong Generalization (W2SG) suggests that a pre-trained strong model can surpass its weak supervisor, yet the decisive role of pre-training remains theoretically and empirically under-explored. In this work, we identify pre-training as the essential prerequisite for the emergence of W2SG. Theoretically, we formalize the W2SG problem within a high-dimensional single-index model framework using spiked Gaussian data, modeling pre-training as a spectral initialization step. Building upon prior impossibility results regarding the failure of learning under random initialization, we prove that W2SG is achievable when pre-training provides a geometric warm start that places the model within an "effective region" characterized by a perturbed strong-convexity geometry. Within this region, we derive a rigorous generalization bound that naturally captures the optimization dynamics: an initial performance improvement followed by a saturation bottleneck dictated by the weak supervisor's bias. Empirically, we first validate all our assumptions and theoretical insights through controlled synthetic simulations. Finally, through a massive-scale evaluation of hundreds of intermediate pre-training checkpoints from large language models, we demonstrate that W2SG is not an innate capability but emerges via a phase transition tightly coupled with the progression of pre-training.

cs.LG↗