arXiv subjects

Jinghui Yuan

Publications and source records attributed to Jinghui Yuan.

18 recordsLinked to original sources

arXiv2026

HyperTransfer: Understanding the Equivalence between Base Optimizer and Hyperball

Hyperball optimizers constrain parameter norms and update only their directions, establishing a distinct paradigm for neural network optimization. Although this geometry appears fundamentally different from that of conventional Base Optimizers, which update both parameter norms and directions, we show that the two paradigms are dynamically equivalent for scale-invariant networks. Building on this equivalence, we propose HyperTransfer, which constructs a Hyperball optimizer that reproduces the dynamics of a target Base Optimizer using only its initialization and learning-rate schedule, without running the target optimizer itself. We further derive the inverse mapping and extend the framework to non-scale-invariant networks. Experiments show that both HyperTransfer and the inverse mapping produce loss trajectories nearly identical to those of their targets, suggesting that Hyperball dynamics are governed primarily by the induced effective learning-rate schedule and optimizer state.

cs.LG↗

arXiv2026

Doubly Stochastic Adaptive Neighbors Clustering via the Marcus Mapping

Clustering is a fundamental task in machine learning and data science, and similarity graph-based clustering is an important approach within this domain. Doubly stochastic symmetric similarity graphs provide numerous benefits for clustering problems and downstream tasks, yet learning such graphs remains a significant challenge. Marcus theorem states that a strictly positive symmetric matrix can be transformed into a doubly stochastic symmetric matrix by diagonal matrices. However, in clustering, learning sparse matrices is crucial for computational efficiency. We extend Marcus theorem by proposing the Marcus mapping, which indicates that certain sparse matrices can also be transformed into doubly stochastic symmetric matrices via diagonal matrices. Additionally, we introduce rank constraints into the clustering problem and propose the Doubly Stochastic Adaptive Neighbors Clustering algorithm based on the Marcus Mapping (ANCMM). This ensures that the learned graph naturally divides into the desired number of clusters. We validate the effectiveness of our algorithm through extensive comparisons with state-of-the-art algorithms. Finally, we explore the relationship between the Marcus mapping and optimal transport. We prove that the Marcus mapping solves a specific type of optimal transport problem.

cs.LG↗

arXiv2026

Achieving More with Less: A Tensor-Optimization-Powered Ensemble Method

Ensemble learning is a method that leverages weak learners to produce a strong learner. However, obtaining a large number of base learners requires substantial time and computational resources. Therefore, it is meaningful to study how to achieve the performance typically obtained with many base learners using only a few. We argue that to achieve this, it is essential to enhance both classification performance and generalization ability during the ensemble process. To increase model accuracy, each weak base learner needs to be more efficiently integrated. It is observed that different base learners exhibit varying levels of accuracy in predicting different classes. To capitalize on this, we introduce confidence tensors $\tilde{\mathbfΘ}$, where $\tilde{\mathbfΘ}_{rst}$ signifies the degree of confidence that the $t$-th base classifier assigns the sample to class $r$ while it actually belongs to class $s$. To the best of our knowledge, this is the first time an evaluation of the performance of base classifiers across different classes has been proposed. The proposed confidence tensor compensates for the strengths and weaknesses of each base classifier in different classes, enabling the method to achieve superior results with a smaller number of base learners. To enhance generalization performance, we design a smooth and partially convex objective function that leverages the concept of margin, making the strong learner more discriminative. Furthermore, it is proved that in the gradient matrix of the loss function, the sum of each column's elements is zero, allowing us to solve a constrained optimization problem using gradient-based methods.

cs.LG↗

arXiv2026

Double-Bounded Nonlinear Optimal Transport for Size Constrained Min Cut Clusterin

Min cut is an important graph partitioning method. However, current solutions to the min cut problem suffer from slow speeds, difficulty in solving, and often converge to simple solutions. To address these issues, we relax the min cut problem into a double-bounded constraint and, for the first time, treat the min cut problem as a double-bounded nonlinear optimal transport problem. Additionally, we develop a method for solving double bounded nonlinear optimal transport based on the Frank-Wolfe method (abbreviated as DNF). We prove that for convex problems satisfying Lipschitz smoothness, the DNF method can achieve a convergence rate of $\mathcal{O}(\frac{1}{t})$. We apply DNF to size-constrained min-cut clustering and evaluate it on eight benchmark datasets. DNF achieves competitive clustering performance and matches or outperforms the compared baselines on several datasets and metrics.

cs.LG↗

arXiv2026

Cooperation of Experts: Fusing Heterogeneous Information with Large Margin

Fusing heterogeneous information remains a persistent challenge in modern data analysis. While significant progress has been made, existing approaches often fail to account for the inherent heterogeneity of object patterns across different semantic spaces. To address this limitation, we propose the Cooperation of Experts (CoE) framework, which encodes multi-typed information into unified heterogeneous multiplex networks. By overcoming modality and connection differences, CoE provides a powerful and flexible model for capturing the intricate structures of real-world complex data. In our framework, dedicated encoders act as domain-specific experts, each specializing in learning distinct relational patterns in specific semantic spaces. To enhance robustness and extract complementary knowledge, these experts collaborate through a novel large margin mechanism supported by a tailored optimization strategy. Rigorous theoretical analyses guarantee the framework's feasibility and stability, while extensive experiments across diverse benchmarks demonstrate its superior performance and broad applicability. Our code is available at https://github.com/strangeAlan/CoE.

cs.LG↗

arXiv2026

Nora: Normalized Orthogonal Row Alignment for Scalable Matrix Optimizer

Matrix-based optimizers have demonstrated immense potential in training Large Language Models (LLMs), however, designing an ideal optimizer remains a formidable challenge. A superior optimizer must satisfy three core desiderata: efficiency, achieving Muon-like preconditioning to accelerate optimization; stability, strictly adhering to the scale-invariance inherent in neural networks; and speed, minimizing computational overhead. While existing methods address these aspects to varying degrees, they often fail to unify them, either incurring prohibitive computational costs like Muon, or allowing radial jitters that compromise stability like RMNP. To bridge this gap, we propose Nora, an optimizer that rigorously satisfies all three requirements. Nora achieves training stability by explicitly stabilizing weight norms and angular velocities through row-wise momentum projection onto the orthogonal complement of the weights. Simultaneously, by leveraging the block-diagonal dominance of the Transformer Hessian, Nora effectively approximates structured preconditioning while maintaining an optimal computational complexity of $\mathcal{O}(mn)$. Furthermore, we prove that Nora is a scalable optimizer and establish its corresponding scaling theorems. With a streamlined implementation requiring only two lines of code, our preliminary experiments validate Nora as an efficient and highly promising optimizer for large-scale training.

cs.LG↗

arXiv2026

Memory-Guided Trust-Region Bayesian Optimization (MG-TuRBO) for High Dimensions

Traffic simulation and digital-twin calibration is a challenging optimization problem with a limited simulation budget. Each trial requires an expensive simulation run, and the relationship between calibration inputs and model error is often nonconvex, and noisy. The problem becomes more difficult as the number of calibration parameters increases. We compare a commonly used automatic calibration method, a genetic algorithm (GA), with Bayesian optimization methods (BOMs): classical Bayesian optimization (BO), Trust-Region BO (TuRBO), Multi-TuRBO, and a proposed Memory-Guided TuRBO (MG-TuRBO) method. We compare performance on 2 real-world traffic simulation calibration problems with 14 and 84 decision variables, representing lower- and higher-dimensional (14D and 84D) settings. For BOMs, we study two acquisition strategies, Thompson sampling and a novel adaptive strategy. We evaluate performance using final calibration quality, convergence behavior, and consistency across runs. The results show that BOMs reach good calibration targets much faster than GA in the lower-D problem. MG-TuRBO performs comparably in our 14D setting, it demonstrates noticeable advantages in the 84D problem, particularly when paired with our adaptive strategy. Our results suggest that MG-TuRBO is especially useful for high-D traffic simulation calibration and potentially for high-D problems in general.

cs.LG↗

arXiv2026

Decoupled Orthogonal Dynamics: Regularization for Deep Network Optimizers

Is the standard weight decay in AdamW truly optimal? Although AdamW decouples weight decay from adaptive gradient scaling, a fundamental conflict remains: the Radial Tug-of-War. In deep learning, gradients tend to increase parameter norms to expand effective capacity while steering directions to learn features, whereas weight decay indiscriminately suppresses norm growth. This push--pull interaction induces radial oscillations, injecting noise into Adam's second-moment estimates and potentially degrading delicate tangential feature learning. We argue that magnitude and direction play distinct roles and should be decoupled in optimizer dynamics. We propose Orthogonal Dynamics Decoupling and instantiate it as AdamO: an SGD-style update handles the one-dimensional norm control, while Adam's adaptive preconditioning is confined to the tangential subspace. AdamO further incorporates curvature-adaptive radial step sizing and architecture-aware rules and projections for scale-invariant layers and low-dimensional parameters. Experiments on vision and language tasks show that AdamO improves generalization and stability over AdamW without introducing additional complex constraints.

cs.LG↗

arXiv2025

Riemannian Optimization on Relaxed Indicator Matrix Manifold

The indicator matrix plays an important role in machine learning, but optimizing it is an NP-hard problem. We propose a new relaxation of the indicator matrix and prove that this relaxation forms a manifold, which we call the Relaxed Indicator Matrix Manifold (RIM manifold). Based on Riemannian geometry, we develop a Riemannian toolbox for optimization on the RIM manifold. Specifically, we provide several methods of Retraction, including a fast Retraction method to obtain geodesics. We point out that the RIM manifold is a generalization of the double stochastic manifold, and it is much faster than existing methods on the double stochastic manifold, which has a complexity of $ \mathcal{O}(n^3) $, while RIM manifold optimization is $ \mathcal{O}(n) $ and often yields better results. We conducted extensive experiments, including image denoising, with millions of variables to support our conclusion, and applied the RIM manifold to Ratio Cut, we provide a rigorous convergence proof and achieve clustering results that outperform the state-of-the-art methods. Our Code in \href{https://github.com/Yuan-Jinghui/Riemannian-Optimization-on-Relaxed-Indicator-Matrix-Manifold}{here}.

cs.LG↗

arXiv2024

A Margin-Maximizing Fine-Grained Ensemble Method

Ensemble learning has achieved remarkable success in machine learning, but its reliance on numerous base learners limits its application in resource-constrained environments. This paper introduces an innovative "Margin-Maximizing Fine-Grained Ensemble Method" that achieves performance surpassing large-scale ensembles by meticulously optimizing a small number of learners and enhancing generalization capability. We propose a novel learnable confidence matrix, quantifying each classifier's confidence for each category, precisely capturing category-specific advantages of individual learners. Furthermore, we design a margin-based loss function, constructing a smooth and partially convex objective using the logsumexp technique. This approach improves optimization, eases convergence, and enables adaptive confidence allocation. Finally, we prove that the loss function is Lipschitz continuous, based on which we develop an efficient gradient optimization algorithm that simultaneously maximizes margins and dynamically adjusts learner weights. Extensive experiments demonstrate that our method outperforms traditional random forests using only one-tenth of the base learners and other state-of-the-art ensemble methods.

cs.LG↗

arXiv2024

GenAI-powered Multi-Agent Paradigm for Smart Urban Mobility: Opportunities and Challenges for Integrating Large Language Models (LLMs) and Retrieval-Augmented Generation (RAG) with Intelligent Transportation Systems

Leveraging recent advances in generative AI, multi-agent systems are increasingly being developed to enhance the functionality and efficiency of smart city applications. This paper explores the transformative potential of large language models (LLMs) and emerging Retrieval-Augmented Generation (RAG) technologies in Intelligent Transportation Systems (ITS), paving the way for innovative solutions to address critical challenges in urban mobility. We begin by providing a comprehensive overview of the current state-of-the-art in mobility data, ITS, and Connected Vehicles (CV) applications. Building on this review, we discuss the rationale behind RAG and examine the opportunities for integrating these Generative AI (GenAI) technologies into the smart mobility sector. We propose a conceptual framework aimed at developing multi-agent systems capable of intelligently and conversationally delivering smart mobility services to urban commuters, transportation operators, and decision-makers. Our approach seeks to foster an autonomous and intelligent approach that (a) promotes science-based advisory to reduce traffic congestion, accidents, and carbon emissions at multiple scales, (b) facilitates public education and engagement in participatory mobility management, and (c) automates specialized transportation management tasks and the development of critical ITS platforms, such as data analytics and interpretation, knowledge representation, and traffic simulations. By integrating LLM and RAG, our approach seeks to overcome the limitations of traditional rule-based multi-agent systems, which rely on fixed knowledge bases and limited reasoning capabilities. This integration paves the way for a more scalable, intuitive, and automated multi-agent paradigm, driving advancements in ITS and urban mobility.

cs.AI↗

arXiv2024

Multi-Task Curriculum Graph Contrastive Learning with Clustering Entropy Guidance

Recent advances in unsupervised deep graph clustering have been significantly promoted by contrastive learning. Despite the strides, most graph contrastive learning models face challenges: 1) graph augmentation is used to improve learning diversity, but commonly used random augmentation methods may destroy inherent semantics and cause noise; 2) the fixed positive and negative sample selection strategy is limited to deal with complex real data, thereby impeding the model's capability to capture fine-grained patterns and relationships. To reduce these problems, we propose the Clustering-guided Curriculum Graph contrastive Learning (CCGL) framework. CCGL uses clustering entropy as the guidance of the following graph augmentation and contrastive learning. Specifically, according to the clustering entropy, the intra-class edges and important features are emphasized in augmentation. Then, a multi-task curriculum learning scheme is proposed, which employs the clustering guidance to shift the focus from the discrimination task to the clustering task. In this way, the sample selection strategy of contrastive learning can be adjusted adaptively from early to late stage, which enhances the model's flexibility for complex data structure. Experimental results demonstrate that CCGL has achieved excellent performance compared to state-of-the-art competitors.

cs.LG↗

arXiv2018

Approach-Level Real-Time Crash Risk Analysis for Signalized Intersections

This study attempts to investigate the relationship between crash occurrence at signalized intersections and real-time traffic, signal timing, and weather characteristics based on 23 signalized intersections in Central Florida. The intersection and intersection-related crashes were collected and then divided into two types, i.e., within intersection crashes and intersection entrance crashes. Bayesian conditional logistic models were developed for these two kinds of crashes, respectively. For the within intersection models, the model results showed that the through volume from "A" approach (the traveling approach of at-fault vehicle), the left turn volume from "B" approach (near-side crossing approach), and the overall average flow ratio (OAFR) from "D" approach (far-side crossing approach), were found to have significant positive effects on the odds of crash occurrence. Moreover, the increased adaptability for the left turn signal timing of "B" approach and more priority for "A" approach could significantly decrease the odds of crash occurrence. For the intersection entrance models, average speed was found to have significant negative effect on the odds of crash occurrence. The longer average green time and longer average waiting time for the left turn phase, higher green ratio for the through phase, and higher adaptability for the through phase can significantly improve the safety performance of intersection entrance area. In addition, the average queue length on the through lanes was found to have positive effect on the odds of crash occurrence. These results are important in real-time safety applications at signalized intersections in the context of proactive traffic management.

stat.AP↗

arXiv2018

Investigating Safety Impacts of Roadway Network Features of Suburban Arterials in Shanghai, China

With the rapid changes in land use development along suburban arterials in Shanghai, there is also a corresponding increase in traffic demand along these arterials. With a preference toward increased accessibility and efficiency, these arterials have been installed with an increased number of signalized intersections and accesses to serve local traffic needs. The absence of a defined functional hierarchy along the road network, together with the non-uniform installation of signals and accesses tends to deteriorate arterial safety. Previous studies on arterial safety have generally been based on a single type of road entity (either intersection or roadway segment). These studies only analyzed partial safety impacts of signal spacing and access density, as these factors would significantly influence the safety performance of both intersections and roadway segments. Macro level safety modeling was usually applied to investigate the relationship between the zonal crash frequencies and demographics, road network features and traffic characteristics. In this study, a new modeling strategy was proposed to analyze the safety impacts of roadway network features (i.e., road network patterns, signal spacing and access density) of arterials by applying a macro level safety modeling technique. Bayesian Conditional Autoregressive models were developed for arterials covering 173 Traffic Analysis Zones in the suburban area in Shanghai. The results identified that the road network pattern with collector roads parallel to the arterials was shown to be associated with fewer crashes than those without parallel collectors. Higher signal density and access density also tended to increase crash frequencies on arterials.

stat.AP↗

arXiv2018

Modeling the Safety Effect of Access and Signal Density on Suburban Arterials: Using Macro Level Analysis Method

With rapidly increasing of the land development density along suburban arterials, much more irregular signal spacing appeared on suburban arterials, and high access density is commonly observed on suburban arterials. These issues tend to increase the risk of crash occurrence of arterials. By developing safety performance functions on road segments and intersections separately, the previous research analyzed the partial safety effects of the influence factors. In this study, Bayesian Conditional Autoregressive (CAR) models were developed at traffic analysis zone (TAZ) level for suburban arterials laid in suburban area in Shanghai. The model result showed that higher access and signal density tend to increase crash frequencies occurred on arterials. At this point, designing frontage roads paralleled to arterials to collect the access traffic instead of those intensive access could reduce crashes occurred on arterials.

stat.AP↗

arXiv2018

Real-Time Crash Risk Analysis of Urban Arterials Incorporating Bluetooth, Weather, and Adaptive Signal Control Data

Real-time safety analysis has become a hot research topic as it can reveal the relationship between real-time traffic characteristics and crash occurrence more accurately, and these results could be applied to improve active traffic management systems and enhance safety performance. Most of the previous studies have been applied to freeways and seldom to arterials. Therefore, this study attempts to examine the relationship between crash occurrence and real-time traffic and weather characteristics based on four urban arterials in Central Florida. Considering the substantial difference between the interrupted traffic flow on urban arterials and the free flow on freeways, the adaptive signal phasing was also introduced in this study. Bayesian conditional logistic models were developed by incorporating the Bluetooth, adaptive signal control, and weather data, which were extracted for a period of 20 minutes (four 5-minute interval) before the time of crash occurrence. Model comparison results indicate that the model based on 5-10 minute interval dataset is the most appropriate model. It reveals that the average speed, upstream volume, and rainy weather indicator were found to have significant effects on crash occurrence. Furthermore, both Bayesian logistic and Bayesian random effects logistic models were developed to compare with the Bayesian conditional logistic model, and the Bayesian conditional logistic model was found to be much better than the other two models. These results are important in real-time safety applications in the context of Integrated Active Traffic Management.

stat.AP↗

arXiv2018

Utilizing Bluetooth and Adaptive Signal Control Data for Urban Arterials Safety Analysis

Real-time safety analysis has become a hot research topic as it can more accurately reveal the relationships between real-time traffic characteristics and crash occurrence, and these results could be applied to improve active traffic management systems and enhance safety performance. Most of the previous studies have been applied to freeways and seldom to arterials. This study attempts to examine the relationship between crash occurrence and real-time traffic and weather characteristics based on four urban arterials in Central Florida. Considering the substantial difference between the interrupted urban arterials and the access controlled freeways, the adaptive signal phasing data was introduced in addition to the traditional traffic data. Bayesian conditional logistic models were developed by incorporating the Bluetooth, adaptive signal control, and weather data, which were extracted for a period of 20 minutes (four 5-minute intervals) before the time of crash occurrence. Model comparison results indicated that the model based on 5-10 minute interval dataset performs the best. It revealed that the average speed, upstream left-turn volume, downstream green ratio, and rainy indicator were found to have significant effects on crash occurrence. Furthermore, both Bayesian random parameters logistic and Bayesian random parameters conditional logistic models were developed to compare with the Bayesian conditional logistic model, and the Bayesian random parameters conditional logistic model was found to have the best model performance in terms of the AUC and DIC values. These results are important in real-time safety applications in the context of Integrated Active Traffic Management.

stat.AP↗

arXiv2018

Modelling of crash types at signalized intersections based on random effect model

Approach-level models were developed to accommodate the diversity of approaches within the same intersection. A random effect term, which indicates the intersection-specific effect, was incorporated into each crash type model to deal with the spatial correlation between different approaches within the same intersection. The model parameters were estimated under the Bayesian framework. Results show that different crash types are correlated with different groups of factors, and each factor shows diverse effects on different crash types, which indicates the importance of crash type models. Besides, the significance of random effect term confirms the existence of spatial correlations among different approaches within the same intersection.

stat.AP↗