arXiv subjects

Haibo Yang

Publications and source records attributed to Haibo Yang.

At least 19 recordsLinked to original sources

arXiv2026

Learning Dynamics of Zeroth-Order Optimization: A Kernel Perspective

Classical optimization theory establishes that zeroth-order (ZO) algorithms suffer from a dimension-dependent slowdown, with convergence rates typically scaling with the model dimension compared to first-order methods. However, in contrast to these theoretical expectations, a growing body of recent work demonstrates the successful application of ZO methods to fine-tuning Large Language Models (LLMs) with billions of parameters. To explain this paradox, we derive the one-step learning dynamics of ZO SGD, where the empirical Neural Tangent Kernel (eNTK) naturally emerges as the key term governing the learning behavior. Inspection of the eNTK produced by ZO SGD reveals that each element corresponds to the inner product of neural tangent vectors projected onto a random low-dimensional subspace. Thus, by invoking the Johnson-Lindenstrauss Lemma, our analysis shows that the fidelity of the ZO eNTK is governed primarily by the number of perturbations. Crucially, the approximation error depends on the model output size rather than the massive parameter dimension. This dimension-free property provides a theoretical justification for the scalability of ZO methods to LLMs finetuning tasks. We believe that this kernel-based framework offers a novel perspective for understanding ZO methods within the context of learning dynamics.

cs.LG↗

arXiv2026

Flow Matching for Offline Reinforcement Learning with Discrete Actions

Generative policies based on diffusion models and flow matching have shown strong promise for offline reinforcement learning (RL), but their applicability remains largely confined to continuous action spaces. To address a broader range of offline RL settings, we extend flow matching to a general framework that supports discrete action spaces with multiple objectives. Specifically, we replace continuous flows with continuous-time Markov chains, trained using a Q-weighted flow matching objective. We then extend our design to multi-agent settings, mitigating the exponential growth of joint action spaces via a factorized conditional path. We theoretically show that, under idealized conditions, optimizing this objective recovers the optimal policy. Extensive experiments further demonstrate that our method performs robustly across diverse settings and benchmarks, including high-dimensional control, multi-agent games, and dynamically changing preferences over multiple objectives, while outperforming traditional offline RL methods in practical multi-modal decision-making scenarios. Our discrete framework can also be applied to continuous-control problems through action quantization, providing a flexible trade-off between representational complexity and performance.

cs.LG↗

arXiv2026

HOSL: Hybrid-Order Split Learning for Memory-Constrained Edge Training

Split learning (SL) enables collaborative training of large language models (LLMs) between resource-constrained edge devices and compute-rich servers by partitioning model computation across the network boundary. However, existing SL systems predominantly rely on first-order (FO) optimization, which requires clients to store intermediate quantities such as activations for backpropagation. This results in substantial memory overhead, largely negating benefits of model partitioning. In contrast, zeroth-order (ZO) optimization eliminates backpropagation and significantly reduces memory usage, but often suffers from slow convergence and degraded performance. In this work, we propose HOSL, a novel Hybrid-Order Split Learning framework that addresses this fundamental trade-off between memory efficiency and optimization effectiveness by strategically integrating ZO optimization on the client side with FO optimization on the server side. By employing memory-efficient ZO gradient estimation at the client, HOSL eliminates backpropagation and activation storage, reducing client memory consumption. Meanwhile, server-side FO optimization ensures fast convergence and competitive performance. Theoretically, we show that HOSL achieves an $\mathcal{O}(\sqrt{d_c/TQ})$ rate, which depends on client-side model dimension $d_c$ rather than the full model dimension $d$, demonstrating that convergence improves as more computation is offloaded to the server. Extensive experiments on OPT models (125M and 1.3B parameters) across 6 tasks demonstrate that HOSL reduces client GPU memory by up to 3.7$\times$ compared to the FO method while achieving accuracy within 0.20%-4.23% of this baseline. Furthermore, HOSL outperforms the ZO baseline by up to 15.55%, validating the effectiveness of our hybrid strategy for memory-efficient training on edge devices.

cs.LG↗

arXiv2025

Dual-Model Framework for CHIKV Transmission Modeling: ODE and Petri Net Analysis of the 2025 Foshan Outbreak

This study constructs a dual-model framework integrating Ordinary Differential Equations (ODE) and Petri Nets (PN) to analyze the 2025 Chikungunya outbreak in Foshan City, China. We employ SEICR compartmental modeling to compare two distinct approaches under identical epidemiological scenarios and evaluate intervention effectiveness through three-phase fitting protocols. Both models demonstrate excellent accuracy with MAE of 18.77-18.91 cases and RMSE of 36.52-36.54 cases. Models predicted epidemic peaks at day 32 (406 cases), 3 days earlier than observed (day 35, 432 cases), with 6.0% peak value error. Reproduction number analysis revealed initial R0 of 14.67 (ODE)/13.90 (PN), with effective reproduction numbers decreasing through intervention phases: 7.85/7.86 after Phase 1, 7.59/7.56 after Phase 2, and 0.059 in Phase 3, achieving transmission blockade. Sensitivity analysis showed recovery rate) as the most sensitive parameter (Sobol index 0.9672), explaining 96.72% of R0 variation. This study presents the first systematic ODE-Petri Net comparison, providing a novel dual-model framework for vector-borne disease modeling with significant theoretical and practical value for epidemic control strategy formulation.

q-bio.PE↗

arXiv2025

Huizhou Hadron Spectrometer -- a Proposed High-rate Experimental Setup at the High Intensity Heavy-ion Accelerator Facility

The High-Intensity Heavy-Ion Accelerator Facility (HIAF), currently under construction in Huizhou, Guangdong Province, China, is projected to be completed by 2025. This facility will be capable of producing proton and heavy-ion beams with energies reaching several GeV, thereby offering a versatile platform for advanced fundamental physics research. Key scientific objectives include exploring physics beyond the Standard Model through the search for novel particles and interactions, testing fundamental symmetries, investigating exotic hadronic states such as di-baryons, pentaquark states and multi-strange hypernuclei, conducting precise measurements of hadron and hypernucleus properties, and probing the phase boundary and critical point of nuclear matter. To facilitate these investigations, we propose the development of a dedicated experimental apparatus at HIAF - the Huizhou Hadron Spectrometer (HHaS). This paper presents the conceptual design of HHaS, comprising a solenoid magnet, a five-dimensional silicon pixel tracker, a Low-Gain Avalanche Detector (LGAD) for time-of-flight measurements, and a Cherenkov-scintillation dual-readout electromagnetic calorimeter. The design anticipates an unprecedented event rate of 1-100 MHz, extensive particle acceptance, a track momentum resolution at 1% level, an electromagnetic energy resolution of ~3% @ 1 GeV and multi-particle identification capabilities. Such capabilities position HHaS as a powerful instrument for advancing experimental studies in particle and nuclear physics. The successful realization of HHaS is expected to significantly bolster the development of medium- and high-energy physics research within China.

hep-ex↗

arXiv2025

Towards Straggler-Resilient Split Federated Learning: An Unbalanced Update Approach

Split Federated Learning (SFL) enables scalable training on edge devices by combining the parallelism of Federated Learning (FL) with the computational offloading of Split Learning (SL). Despite its great success, SFL suffers significantly from the well-known straggler issue in distributed learning systems. This problem is exacerbated by the dependency between Split Server and clients: the Split Server side model update relies on receiving activations from clients. Such synchronization requirement introduces significant time latency, making straggler a critical bottleneck to the scalability and efficiency of the system. To mitigate this problem, we propose MU-SplitFed, a straggler-resilient SFL algorithm in zeroth-order optimization that decouples training progress from straggler delays via a simple yet effective unbalanced update mechanism. By enabling the server to perform $\tau$ local updates per client round, MU-SplitFed achieves a convergence rate of $O(\sqrt{d/(\tau T)})$ for non-convex objectives, demonstrating a linear speedup of $\tau$ in communication rounds. Experiments demonstrate that MU-SplitFed consistently outperforms baseline methods with the presence of stragglers and effectively mitigates their impact through adaptive tuning of $\tau$. The code for this project is available at https://github.com/Johnny-Zip/MU-SplitFed.

cs.DC↗

arXiv2025

Simulation of radiation environment for the beam monitor of CEE experiment

The cooling storage ring external-target experiment is a large-scale nuclear physics experiment, which aims to study the physics of heavy-ion collisions at low temperatures and high baryon densities. A beam monitor (BM) is placed in the beam line to monitor the beam status and to improve the reconstruction resolution of the primary vertices. The radiation dose and particle fluence stemming from the beam interactions with gases and detector materials affect the performance of the sensors and electronics of BM. This paper uses FLUKA Monte Carlo code to simulate the radiation environment of BM detector. Radiation quantities including the total ionizing dose, 1 MeV neutron equivalent fluence, high-energy hadron flux, thermal neutron flux, and nuclear fragment flux are presented. Results of alternative simulation setups, including adding shielding layers inside the BM, are also investigated.

physics.ins-det↗

arXiv2025

Design and performance of the prototype gaseous beam monitor with GEM and pixel sensors for the CSR external-target experiment

A gaseous beam monitor utilizing gas electron multiplier (GEM) and pixel sensors is being developed for the Cooling Storage Ring (CSR) External-target Experiment (CEE) at Heavy Ion Research Facility in Lanzhou (HIRFL). The beam monitor is mainly used to track each beam particle, providing an accurate reconstruction of the primary vertex of the collision. Two generations of the pixel sensors (named Topmetal-CEE) were produced, with the second generation's performance improving over the first one. The design and performance of the prototype are described in the paper. Characterization of the prototype with heavy-ion beams and laser beams are presented, showing a spatial resolution better than 50 $\mum$ and a time resolution better than 15 ns.

physics.ins-det↗

arXiv2025

Enabling Pareto-Stationarity Exploration in Multi-Objective Reinforcement Learning: A Multi-Objective Weighted-Chebyshev Actor-Critic Approach

In many multi-objective reinforcement learning (MORL) applications, being able to systematically explore the Pareto-stationary solutions under multiple non-convex reward objectives with theoretical finite-time sample complexity guarantee is an important and yet under-explored problem. This motivates us to take the first step and fill the important gap in MORL. Specifically, in this paper, we propose a \uline{M}ulti-\uline{O}bjective weighted-\uline{CH}ebyshev \uline{A}ctor-critic (MOCHA) algorithm for MORL, which judiciously integrates the weighted-Chebychev (WC) and actor-critic framework to enable Pareto-stationarity exploration systematically with finite-time sample complexity guarantee. Sample complexity result of MOCHA algorithm reveals an interesting dependency on $p_{\min}$ in finding an $\epsilon$-Pareto-stationary solution, where $p_{\min}$ denotes the minimum entry of a given weight vector $\mathbf{p}$ in WC-scarlarization. By carefully choosing learning rates, the sample complexity for each exploration can be $\tilde{\mathcal{O}}(\epsilon^{-2})$. Furthermore, simulation studies on a large KuaiRand offline dataset, show that the performance of MOCHA algorithm significantly outperforms other baseline MORL approaches.

cs.LG↗

arXiv2025

STIMULUS: Achieving Fast Convergence and Low Sample Complexity in Stochastic Multi-Objective Learning

Recently, multi-objective optimization (MOO) has gained attention for its broad applications in ML, operations research, and engineering. However, MOO algorithm design remains in its infancy and many existing MOO methods suffer from unsatisfactory convergence rate and sample complexity performance. To address this challenge, in this paper, we propose an algorithm called STIMULUS( stochastic path-integrated multi-gradient recursive e\ulstimator), a new and robust approach for solving MOO problems. Different from the traditional methods, STIMULUS introduces a simple yet powerful recursive framework for updating stochastic gradient estimates to improve convergence performance with low sample complexity. In addition, we introduce an enhanced version of STIMULUS, termed STIMULUS-M, which incorporates a momentum term to further expedite convergence. We establish $O(1/T)$ convergence rates of the proposed methods for non-convex settings and $O (\exp{-\mu T})$ for strongly convex settings, where $T$ is the total number of iteration rounds. Additionally, we achieve the state-of-the-art $O \left(n+\sqrt{n}\epsilon^{-1}\right)$ sample complexities for non-convex settings and $O\left(n+ \sqrt{n} \ln ({\mu/\epsilon})\right)$ for strongly convex settings, where $\epsilon>0$ is a desired stationarity error. Moreover, to alleviate the periodic full gradient evaluation requirement in STIMULUS and STIMULUS-M, we further propose enhanced versions with adaptive batching called STIMULUS+/ STIMULUS-M+ and provide their theoretical analysis.

cs.LG↗

arXiv2025

Converge Faster, Talk Less: Hessian-Informed Federated Zeroth-Order Optimization

Zeroth-order (ZO) optimization enables dimension-free communication in federated learning (FL), making it attractive for fine-tuning of large language models (LLMs) due to significant communication savings. However, existing ZO-FL methods largely overlook curvature information, despite its well-established benefits for convergence acceleration. To address this, we propose HiSo, a Hessian-informed ZO federated optimization method that accelerates convergence by leveraging global diagonal Hessian approximations, while strictly preserving scalar-only communication without transmitting any second-order information. Theoretically, for non-convex functions, we show that HiSo can achieve an accelerated convergence rate that is independent of the Lipschitz constant $L$ and model dimension $d$ under some Hessian approximation assumptions, offering a plausible explanation for the observed phenomenon of ZO convergence being much faster than its worst-case $\mathscr{O}(d)$-bound. Empirically, across diverse LLM fine-tuning benchmarks, HiSo delivers a 1$\sim$5$\times$ speedup in communication rounds over existing state-of-the-art ZO-FL baselines. This superior convergence not only cuts communication costs but also provides strong empirical evidence that Hessian information acts as an effective accelerator in federated ZO optimization settings. Our source code is provided at https://github.com/ZidongLiu/DeComFL.

cs.LG↗

arXiv2025

Exploring Pose-Guided Imitation Learning for Robotic Precise Insertion

Imitation learning is promising for robotic manipulation, but \emph{precise insertion} in the real world remains difficult due to contact-rich dynamics, tight clearances, and limited demonstrations. Many existing visuomotor policies depend on high-dimensional RGB/point-cloud observations, which can be data-inefficient and generalize poorly under pose variations. In this paper, we study pose-guided imitation learning by using object poses in $\mathrm{SE}(3)$ as compact, object-centric observations for precise insertion tasks. First, we propose a diffusion policy for precise insertion that observes the \emph{relative} $\mathrm{SE}(3)$ pose of the source object with respect to the target object and predicts a future relative pose trajectory as its action. Second, to improve robustness to pose estimation noise, we augment the pose-guided policy with RGBD cues. Specifically, we introduce a goal-conditioned RGBD encoder to capture the discrepancy between current and goal observations. We further propose a pose-guided residual gated fusion module, where pose features provide the primary control signal and RGBD features adaptively compensate when pose estimates are unreliable. We evaluate our methods on six real-robot precise insertion tasks and achieve high performance with only $7$--$10$ demonstrations per task. In our setup, the proposed policies succeed on tasks with clearances down to $0.01$~mm and demonstrate improved data efficiency and generalization over existing baselines. Code will be available at https://github.com/sunhan1997/PoseInsert.

cs.RO↗

arXiv2025

Exact and Linear Convergence for Federated Learning under Arbitrary Client Participation is Attainable

This work tackles the fundamental challenges in Federated Learning (FL) posed by arbitrary client participation and data heterogeneity, prevalent characteristics in practical FL settings. It is well-established that popular FedAvg-style algorithms struggle with exact convergence and can suffer from slow convergence rates since a decaying learning rate is required to mitigate these scenarios. To address these issues, we introduce the concept of stochastic matrix and the corresponding time-varying graphs as a novel modeling tool to accurately capture the dynamics of arbitrary client participation and the local update procedure. Leveraging this approach, we offer a fresh decentralized perspective on designing FL algorithms and present FOCUS, Federated Optimization with Exact Convergence via Push-pull Strategy, a provably convergent algorithm designed to effectively overcome the previously mentioned two challenges. More specifically, we provide a rigorous proof demonstrating that FOCUS achieves exact convergence with a linear rate regardless of the arbitrary client participation, establishing it as the first work to demonstrate this significant result.

cs.LG↗

arXiv2025

Do We Really Need to Design New Byzantine-robust Aggregation Rules?

Federated learning (FL) allows multiple clients to collaboratively train a global machine learning model through a server, without exchanging their private training data. However, the decentralized aspect of FL makes it susceptible to poisoning attacks, where malicious clients can manipulate the global model by sending altered local model updates. To counter these attacks, a variety of aggregation rules designed to be resilient to Byzantine failures have been introduced. Nonetheless, these methods can still be vulnerable to sophisticated attacks or depend on unrealistic assumptions about the server. In this paper, we demonstrate that there is no need to design new Byzantine-robust aggregation rules; instead, FL can be secured by enhancing the robustness of well-established aggregation rules. To this end, we present FoundationFL, a novel defense mechanism against poisoning attacks. FoundationFL involves the server generating synthetic updates after receiving local model updates from clients. It then applies existing Byzantine-robust foundational aggregation rules, such as Trimmed-mean or Median, to combine clients' model updates with the synthetic ones. We theoretically establish the convergence performance of FoundationFL under Byzantine settings. Comprehensive experiments across several real-world datasets validate the efficiency of our FoundationFL method.

cs.CR↗

arXiv2024

PSMGD: Periodic Stochastic Multi-Gradient Descent for Fast Multi-Objective Optimization

Multi-objective optimization (MOO) lies at the core of many machine learning (ML) applications that involve multiple, potentially conflicting objectives (e.g., multi-task learning, multi-objective reinforcement learning, among many others). Despite the long history of MOO, recent years have witnessed a surge in interest within the ML community in the development of gradient manipulation algorithms for MOO, thanks to the availability of gradient information in many ML problems. However, existing gradient manipulation methods for MOO often suffer from long training times, primarily due to the need for computing dynamic weights by solving an additional optimization problem to determine a common descent direction that can decrease all objectives simultaneously. To address this challenge, we propose a new and efficient algorithm called Periodic Stochastic Multi-Gradient Descent (PSMGD) to accelerate MOO. PSMGD is motivated by the key observation that dynamic weights across objectives exhibit small changes under minor updates over short intervals during the optimization process. Consequently, our PSMGD algorithm is designed to periodically compute these dynamic weights and utilizes them repeatedly, thereby effectively reducing the computational overload. Theoretically, we prove that PSMGD can achieve state-of-the-art convergence rates for strongly-convex, general convex, and non-convex functions. Additionally, we introduce a new computational complexity measure, termed backpropagation complexity, and demonstrate that PSMGD could achieve an objective-independent backpropagation complexity. Through extensive experiments, we verify that PSMGD can provide comparable or superior performance to state-of-the-art MOO algorithms while significantly reducing training time.

cs.LG↗

arXiv2024

Interpolation spaces of Besov hierarchical spaces and non-linearities defined by vertex K functional and grid topology

In 1967, Peetre proposed to give a precise description of the real interpolation space for Besov hierarchical spaces $l^{s,q}(A)$. In 1974, Cwikel proved that the Lions-Peetre formula for $(l^{q_0}(A_0), l^{q_1}(A_1))_{\theta,r}$ have no reasonable generalization for any $r\neq q$. In this paper, we apply wavelets to transform the study of real interpolation space into {\bf the study of nonlinear functional structure and nonlinear topological structure}. We solve completely Peetre's longstanding open problem.

math.FA↗

arXiv2024

Peetre conjecture on real interpolation spaces of Besov spaces and Grid K functional

In this paper, Peetre's conjecture about the real interpolation space of Besov space {\bf is solved completely } by using the classification of vertices of cuboids defined by {\bf wavelet coefficients and wavelet's grid structure}. Littlewood-Paley analysis provides only a decomposition of the function on the ring. We extend Lorentz's rearrangement function and Hunt's Marcinkiewicz interpolation theorem to more general cases. We use the method of calculating the topological quantity of the grid to replace the traditional methods of data classification such as gradient descent method and distributed algorithm. We developed a series of new techniques to solve this longstanding open problem. These skills make up for the deficiency of Lions-Peetre iterative theorem in dealing with strong nonlinearity. Using the properties of wavelet basis, a series of {\bf functional nonlinearities} are studied. Using the lattice property of wavelet, we study the lattice topology. By three kinds of {\bf topology nonlinearities}, we give the specific wavelet expression of K functional.

math.FA↗

arXiv2024

Hi3D: Pursuing High-Resolution Image-to-3D Generation with Video Diffusion Models

Despite having tremendous progress in image-to-3D generation, existing methods still struggle to produce multi-view consistent images with high-resolution textures in detail, especially in the paradigm of 2D diffusion that lacks 3D awareness. In this work, we present High-resolution Image-to-3D model (Hi3D), a new video diffusion based paradigm that redefines a single image to multi-view images as 3D-aware sequential image generation (i.e., orbital video generation). This methodology delves into the underlying temporal consistency knowledge in video diffusion model that generalizes well to geometry consistency across multiple views in 3D generation. Technically, Hi3D first empowers the pre-trained video diffusion model with 3D-aware prior (camera pose condition), yielding multi-view images with low-resolution texture details. A 3D-aware video-to-video refiner is learnt to further scale up the multi-view images with high-resolution texture details. Such high-resolution multi-view images are further augmented with novel views through 3D Gaussian Splatting, which are finally leveraged to obtain high-fidelity meshes via 3D reconstruction. Extensive experiments on both novel view synthesis and single view reconstruction demonstrate that our Hi3D manages to produce superior multi-view consistency images with highly-detailed textures. Source code and data are available at \url{https://github.com/yanghb22-fdu/Hi3D-Official}.

cs.CV↗