arXiv subjects

Andrew Holliday

Publications and source records attributed to Andrew Holliday.

6 recordsLinked to original sources

arXiv2025

Applications of deep reinforcement learning to urban transit network design

This thesis concerns the use of reinforcement learning to train neural networks to aid in the design of public transit networks. The Transit Network Design Problem (TNDP) is an optimization problem of considerable practical importance. Given a city with an existing road network and travel demands, the goal is to find a set of transit routes - each of which is a path through the graph - that collectively satisfy all demands, while minimizing a cost function that may depend both on passenger satisfaction and operating costs. The existing literature on this problem mainly considers metaheuristic optimization algorithms, such as genetic algorithms and ant-colony optimization. By contrast, we begin by taking a reinforcement learning approach, formulating the construction of a set of transit routes as a Markov Decision Process (MDP) and training a neural net policy to act as the agent in this MDP. We then show that, beyond using this policy to plan a transit network directly, it can be combined with existing metaheuristic algorithms, both to initialize the solution and to suggest promising moves at each step of a search through solution space. We find that such hybrid algorithms, which use a neural policy trained via reinforcement learning as a core component within a classical metaheuristic framework, can plan transit networks that are superior to those planned by either the neural policy or the metaheuristic algorithm. We demonstrate the utility of our approach by using it to redesign the transit network for the city of Laval, Quebec, and show that in simulation, the resulting transit network provides better service at lower cost than the existing transit network.

cs.LG↗

arXiv2024

Learning Heuristics for Transit Network Design and Improvement with Deep Reinforcement Learning

Planning a network of public transit routes is a challenging optimization problem. Metaheuristic algorithms search through the space of possible transit networks by applying heuristics that randomly alter routes in a network. Existing algorithms almost exclusively use heuristics that modify the network in purely random ways. In this work, we explore whether we can obtain better transit networks using more intelligent heuristics, that modify networks according to a learned preference function instead of at random. We use reinforcement learning to train graph neural nets to act as heuristics. These neural heuristics yield improved results on benchmark synthetic cities with 70 nodes or more, and achieve new state-of-the-art results on the challenging Mumford benchmark. They also improve upon a simulation of the real transit network in the city of Laval, Canada, achieving cost savings of up to 19% over the city's existing transit network.

cs.LG↗

arXiv2024

A Neural-Evolutionary Algorithm for Autonomous Transit Network Design

Planning a public transit network is a challenging optimization problem, but essential in order to realize the benefits of autonomous buses. We propose a novel algorithm for planning networks of routes for autonomous buses. We first train a graph neural net model as a policy for constructing route networks, and then use the policy as one of several mutation operators in a evolutionary algorithm. We evaluate this algorithm on a standard set of benchmarks for transit network design, and find that it outperforms the learned policy alone by up to 20% and a plain evolutionary algorithm approach by up to 53% on realistic benchmark instances.

cs.NE↗

arXiv2023

Uncertainty-aware hybrid paradigm of nonlinear MPC and model-based RL for offroad navigation: Exploration of transformers in the predictive model

In this paper, we investigate a hybrid scheme that combines nonlinear model predictive control (MPC) and model-based reinforcement learning (RL) for navigation planning of an autonomous model car across offroad, unstructured terrains without relying on predefined maps. Our innovative approach takes inspiration from BADGR, an LSTM-based network that primarily concentrates on environment modeling, but distinguishes itself by substituting LSTM modules with transformers to greatly elevate the performance our model. Addressing uncertainty within the system, we train an ensemble of predictive models and estimate the mutual information between model weights and outputs, facilitating dynamic horizon planning through the introduction of variable speeds. Further enhancing our methodology, we incorporate a nonlinear MPC controller that accounts for the intricacies of the vehicle's model and states. The model-based RL facet produces steering angles and quantifies inherent uncertainty. At the same time, the nonlinear MPC suggests optimal throttle settings, striking a balance between goal attainment speed and managing model uncertainty influenced by velocity. In the conducted studies, our approach excels over the existing baseline by consistently achieving higher metric values in predicting future events and seamlessly integrating the vehicle's kinematic model for enhanced decision-making. The code and the evaluation data are available at https://github.com/FARAZLOTFI/offroad_autonomous_navigation/).

cs.RO↗

arXiv2023

Neural Bee Colony Optimization: A Case Study in Public Transit Network Design

In this work we explore the combination of metaheuristics and learned neural network solvers for combinatorial optimization. We do this in the context of the transit network design problem, a uniquely challenging combinatorial optimization problem with real-world importance. We train a neural network policy to perform single-shot planning of individual transit routes, and then incorporate it as one of several sub-heuristics in a modified Bee Colony Optimization (BCO) metaheuristic algorithm. Our experimental results demonstrate that this hybrid algorithm outperforms the learned policy alone by up to 20% and the original BCO algorithm by up to 53% on realistic problem instances. We perform a set of ablations to study the impact of each component of the modified algorithm.

cs.NE↗

arXiv2018

Scale-Robust Localization Using General Object Landmarks

Visual localization under large changes in scale is an important capability in many robotic mapping applications, such as localizing at low altitudes in maps built at high altitudes, or performing loop closure over long distances. Existing approaches, however, are robust only up to about a 3x difference in scale between map and query images. We propose a novel combination of deep-learning-based object features and state-of-the-art SIFT point-features that yields improved robustness to scale change. This technique is training-free and class-agnostic, and in principle can be deployed in any environment out-of-the-box. We evaluate the proposed technique on the KITTI Odometry benchmark and on a novel dataset of outdoor images exhibiting changes in visual scale of $7\times$ and greater, which we have released to the public. Our technique consistently outperforms localization using either SIFT features or the proposed object features alone, achieving both greater accuracy and much lower failure rates under large changes in scale.

cs.RO↗