arXiv subjects

Yuhao Ding

Publications and source records attributed to Yuhao Ding.

22 records · Page 2Linked to original sources

arXiv2021

Escaping spurious local minimum trajectories in online time-varying nonconvex optimization

A major limitation of online algorithms that track the optimizers of time-varying nonconvex optimization problems is that they focus on a specific local minimum trajectory, which may lead to poor spurious local solutions. In this paper, we show that the natural temporal variation may help simple online tracking methods find and track time-varying global minima. To this end, we investigate the properties of a time-varying projected gradient flow system with inertia, which can be regarded as the continuous-time limit of (1) the optimality conditions for a discretized sequential optimization problem with a proximal regularization and (2) the online tracking scheme. We introduce the notion of the dominant trajectory and show that the inherent temporal variation could re-shape the landscape of the Lagrange functional and help a proximal algorithm escape the spurious local minimum trajectories if the global minimum trajectory is dominant. For a problem with twice continuously differentiable objective function and constraints, sufficient conditions are derived to guarantee that no matter how a local search method is initialized, it will track a time-varying global solution after some time. The results are illustrated on a benchmark example with many local minima.

math.OC↗

arXiv2020

CSB-RNN: A Faster-than-Realtime RNN Acceleration Framework with Compressed Structured Blocks

Recurrent neural networks (RNNs) have been widely adopted in temporal sequence analysis, where realtime performance is often in demand. However, RNNs suffer from heavy computational workload as the model often comes with large weight matrices. Pruning schemes have been proposed for RNNs to eliminate the redundant (close-to-zero) weight values. On one hand, the non-structured pruning methods achieve a high pruning rate but introducing computation irregularity (random sparsity), which is unfriendly to parallel hardware. On the other hand, hardware-oriented structured pruning suffers from low pruning rate due to restricted constraints on allowable pruning structure. This paper presents CSB-RNN, an optimized full-stack RNN framework with a novel compressed structured block (CSB) pruning technique. The CSB pruned RNN model comes with both fine pruning granularity that facilitates a high pruning rate and regular structure that benefits the hardware parallelism. To address the challenges in parallelizing the CSB pruned model inference with fine-grained structural sparsity, we propose a novel hardware architecture with a dedicated compiler. Gaining from the architecture-compilation co-design, the hardware not only supports various RNN cell types, but is also able to address the challenging workload imbalance issue and therefore significantly improves the hardware efficiency.

cs.DC↗

arXiv2019

Aggressive Local Search for Constrained Optimal Control Problems with Many Local Minima

This paper is concerned with numerically finding a global solution of constrained optimal control problems with many local minima. The focus is on the optimal decentralized control (ODC) problem, whose feasible set is recently shown to have an exponential number of connected components and consequently an exponential number of local minima. The rich literature of numerical algorithms for nonlinear optimization suggests that if a local search algorithm is initialized in an arbitrary connected component of the feasible set, it would search only within that component and find a stationary point there. This is based on the fact that numerical algorithms are designed to generate a sequence of points (via searching for descent directions and adjusting the step size), whose corresponding continuous path is trapped in a single connected component. In contrast with this perception rooted in convex optimization, we numerically illustrate that local search methods for non-convex constrained optimization can obliviously jump between different connected components to converge to a global minimum, via an aggressive step size adjustment using backtracking and the Armijio rule. To support the observations, we prove that from almost every arbitrary point in any connected component of the feasible set, it is possible to generate a sequence of points using local search to jump to different components and converge to a global solution. However, due to the NP-hardness of the problem, such fine-tuning of the parameters of a local search algorithm may need prior knowledge or be time consuming. This paper offers the first result on escaping non-global local solutions of constrained optimal control problems with complicated feasible sets.

math.OC↗

arXiv2019

Optimal Input Design for Affine Model Discrimination with Applications in Intention-Aware Vehicles

This paper considers the optimal design of input signals for the purpose of discriminating among a finite number of affine models with uncontrolled inputs and noise. Each affine model represents a different system operating mode, corresponding to unobserved intents of other drivers or robots, or to fault types or attack strategies, etc. The input design problem aims to find optimal separating/discriminating (controlled) inputs such that the output trajectories of all the affine models are guaranteed to be distinguishable from each other, despite uncertainty in the initial condition and uncontrolled inputs as well as the presence of process and measurement noise. We propose a novel formulation to solve this problem, with an emphasis on guarantees for model discrimination and optimality, in contrast to a previously proposed conservative formulation using robust optimization. This new formulation can be recast as a bilevel optimization problem and further reformulated as a mixed-integer linear program (MILP). Moreover, our fairly general problem setting allows the incorporation of objectives and/or responsibilities among rational agents. For instance, each driver has to obey traffic rules, while simultaneously optimizing for safety, comfort and energy efficiency. Finally, we demonstrate the effectiveness of our approach for identifying the intention of other vehicles in several driving scenarios.

math.OC↗