arXiv subjects

Yanwen Zhu

Publications and source records attributed to Yanwen Zhu.

3 recordsLinked to original sources

arXiv2026

Ornstein-Uhlenbeck information particle: A new candidate of active agent

An information particle can acquire active-like motion through transforming the information entropy into effective self-propulsion velocity/force using the attached information engine. We consider an underdamped Brownian particle additionally driven by either a constant self-propulsion force or an information engine using Ornstein-Uhlenbeck (OU) bath feedback control, such particles are called self-propelled particle (SPP) or OU information particle (OUIP). Compared to the widely-investigated SPP, the OUIP shows a significant different dynamical pattern, including two types of moving mode: a slow-speed diffusion mode and a high-speed traveling mode. The specific evolution of OUIP can be adjusted flexibly between such two modes through the inertial effect, thus acquiring a rich and non-trivial motion behavior. By tuning the strength of fluctuation of the OU bath, a wide range of net velocity can be achieved for OUIP. We highlight that OUIP could be an exceptional candidate for active agent.

cond-mat.stat-mech↗

arXiv2021

MLR-SNet: Transferable LR Schedules for Heterogeneous Tasks

The learning rate (LR) is one of the most important hyper-parameters in stochastic gradient descent (SGD) algorithm for training deep neural networks (DNN). However, current hand-designed LR schedules need to manually pre-specify a fixed form, which limits their ability to adapt practical non-convex optimization problems due to the significant diversification of training dynamics. Meanwhile, it always needs to search proper LR schedules from scratch for new tasks, which, however, are often largely different with task variations, like data modalities, network architectures, or training data capacities. To address this learning-rate-schedule setting issues, we propose to parameterize LR schedules with an explicit mapping formulation, called \textit{MLR-SNet}. The learnable parameterized structure brings more flexibility for MLR-SNet to learn a proper LR schedule to comply with the training dynamics of DNN. Image and text classification benchmark experiments substantiate the capability of our method for achieving proper LR schedules. Moreover, the explicit parameterized structure makes the meta-learned LR schedules capable of being transferable and plug-and-play, which can be easily generalized to new heterogeneous tasks. We transfer our meta-learned MLR-SNet to query tasks like different training epochs, network architectures, data modalities, dataset sizes from the training ones, and achieve comparable or even better performance compared with hand-designed LR schedules specifically designed for the query tasks. The robustness of MLR-SNet is also substantiated when the training data are biased with corrupted noise. We further prove the convergence of the SGD algorithm equipped with LR schedule produced by our MLR-Net, with the convergence rate comparable to the best-known ones of the algorithm for solving the problem.

cs.LG↗

arXiv2020

Meta Feature Modulator for Long-tailed Recognition

Deep neural networks often degrade significantly when training data suffer from class imbalance problems. Existing approaches, e.g., re-sampling and re-weighting, commonly address this issue by rearranging the label distribution of training data to train the networks fitting well to the implicit balanced label distribution. However, most of them hinder the representative ability of learned features due to insufficient use of intra/inter-sample information of training data. To address this issue, we propose meta feature modulator (MFM), a meta-learning framework to model the difference between the long-tailed training data and the balanced meta data from the perspective of representation learning. Concretely, we employ learnable hyper-parameters (dubbed modulation parameters) to adaptively scale and shift the intermediate features of classification networks, and the modulation parameters are optimized together with the classification network parameters guided by a small amount of balanced meta data. We further design a modulator network to guide the generation of the modulation parameters, and such a meta-learner can be readily adapted to train the classification network on other long-tailed datasets. Extensive experiments on benchmark vision datasets substantiate the superiority of our approach on long-tailed recognition tasks beyond other state-of-the-art methods.

cs.CV↗