arXiv subjects

Ostap Okhrin

Publications and source records attributed to Ostap Okhrin.

At least 19 recordsLinked to original sources

arXiv2026

Second-Order Continuum Model for Disordered Traffic

In this study, a two-dimensional second-order macroscopic traffic flow model is proposed to capture the complex dynamics of disordered traffic by explicitly incorporating coupled longitudinal and lateral interactions. The framework extends the classical continuity equation to two spatial dimensions and introduces acceleration equations consisting of self-driven, interaction-induced, and road-boundary-related components. The macroscopic longitudinal acceleration is formulated based on the Full Velocity Difference Model (FVDM), while lateral dynamics are governed by interaction principles consistent with the Optimal Velocity Model (OVM). The explicit inclusion of road-boundary effects enables the representation of vehicle confinement and space-sharing behavior that are essential in lane-free and weakly lane-disciplined traffic systems. The model is examined through a series of numerical experiments in which longitudinal and lateral dynamics are analysed individually as well as simultaneously within a coupled two-dimensional setting under a range of initial conditions, including transitions between free-flow, medium congestion, and heavy congestion, and different lateral density configurations. The simulations demonstrate the model's ability to reproduce key traffic features such as shockwave propagation, lateral dispersion, vehicle rearrangement, and stable density evolution across the road width. The numerical scheme, based on upwind and Lax-Friedrichs discretisations, ensures stable solutions of the coupled partial differential equations. Overall, the proposed framework provides a robust macroscopic description of disordered traffic and offers a consistent basis for analysing two-dimensional vehicular flow dynamics.

physics.soc-ph↗

arXiv2026

Beyond Lanes: Traffic Flow Dynamics in Disordered Conditions Based on High-Resolution Trajectory Data

Disordered traffic flow is characterized by weak or non-existent lane discipline in the presence of strong vehicle heterogeneity and continuous lateral interactions, challenging traditional lane-based modeling assumptions. This study presents an empirical study of macroscopic and microscopic aspects of disordered traffic using high-resolution UAV trajectory data collected on an urban arterial. A two-dimensional extension of Edie's framework is applied to quantify aggregate traffic variables and produce a two-dimensional fundamental diagram, revealing that traffic states cannot be adequately represented using one-dimensional formulations and highlighting the persistent role of lateral redistribution. The propagation of congestion is estimated directly from the spatiotemporal speed fields, demonstrating the emergence of coherent stop-and-go waves and showing a similar dynamics as conventional lane-based flow, in spite of the heterogeneous vehicle interactions. At the microscopic level, steady-state follower-leader identification is used to examine desired time gaps and minimum lateral spacing, vehicle dimension distributions, and kinematic characteristics, revealing pronounced inter-class heterogeneity that explains disordered traffic behavior. The study provides an empirical framework linking vehicle-level interactions and aggregate traffic dynamics and establishes a data-driven basis for the calibration and validation of traffic models for disordered mixed traffic systems.

physics.soc-ph↗

arXiv2026

MTA-RL: Robust Urban Driving via Multi-modal Transformer-based 3D Affordances and Reinforcement Learning

Robust urban autonomous driving requires reliable 3D scene understanding and stable decision-making under dense interactions. However, existing end-to-end models lack interpretability, while modular pipelines suffer from error propagation across brittle interfaces. This paper proposes MTA-RL, the first framework that bridges perception and control through Multi-modal Transformer-based 3D Affordances and Reinforcement Learning (RL). Unlike previous fusion models that directly regress actions, RGB images and LiDAR point clouds are fused using a transformer architecture to predict explicit, geometry-aware affordance representations. These structured representations serve as a compact observation space, enabling the RL policy to operate purely on predicted driving semantics, which significantly improves sample efficiency and stability. Extensive evaluations in CARLA Town01-03 across varying densities (20-60 background vehicles) show that MTA-RL consistently outperforms state-of-the-art baselines. Trained solely on Town03, our method demonstrates superior zero-shot generalization in unseen towns, achieving up to a 9.0% increase in Route Completion, an 11.0% increase in Total Distance, and an 83.7% improvement in Distance Per Violation. Furthermore, ablation studies confirm that our multi-modal fusion and reward shaping are critical, significantly outperforming image-only and unshaped variants, demonstrating the effectiveness of MTA-RL for robust urban autonomous driving.

cs.CV↗

arXiv2025

Ethics-Aware Safe Reinforcement Learning for Rare-Event Risk Control in Interactive Urban Driving

Autonomous vehicles hold great promise for reducing traffic fatalities and improving transportation efficiency, yet their widespread adoption hinges on embedding credible and transparent ethical reasoning into routine and emergency maneuvers, particularly to protect vulnerable road users (VRUs) such as pedestrians and cyclists. Here, we present a hierarchical Safe Reinforcement Learning (Safe RL) framework that augments standard driving objectives with ethics-aware cost signals. At the decision level, a Safe RL agent is trained using a composite ethical risk cost, combining collision probability and harm severity, to generate high-level motion targets. A dynamic, risk-sensitive Prioritized Experience Replay mechanism amplifies learning from rare but critical, high-risk events. At the execution level, polynomial path planning coupled with Proportional-Integral-Derivative (PID) and Stanley controllers translates these targets into smooth, feasible trajectories, ensuring both accuracy and comfort. We train and validate our approach on closed-loop simulation environments derived from large-scale, real-world traffic datasets encompassing diverse vehicles, cyclists, and pedestrians, and demonstrate that it outperforms baseline methods in reducing risk to others while maintaining ego performance and comfort. This work provides a reproducible benchmark for Safe RL with explicitly ethics-aware objectives in human-mixed traffic scenarios. Our results highlight the potential of combining formal control theory and data-driven learning to advance ethically accountable autonomy that explicitly protects those most at risk in urban traffic environments. Across two interactive benchmarks and five random seeds, our policy decreases conflict frequency by 25-45% compared to matched task successes while maintaining comfort metrics within 5%.

cs.LG↗

arXiv2024

Autonomous Driving Small-Scale Cars: A Survey of Recent Development

While engaging with the unfolding revolution in autonomous driving, a challenge presents itself, how can we effectively raise awareness within society about this transformative trend? While full-scale autonomous driving vehicles often come with a hefty price tag, the emergence of small-scale car platforms offers a compelling alternative. These platforms not only serve as valuable educational tools for the broader public and young generations but also function as robust research platforms, contributing significantly to the ongoing advancements in autonomous driving technology. This survey outlines various small-scale car platforms, categorizing them and detailing the research advancements accomplished through their usage. The conclusion provides proposals for promising future directions in the field.

cs.RO↗

arXiv2024

Dynamic tail risk forecasting: what do realized skewness and kurtosis add?

This paper compares the accuracy of tail risk forecasts with a focus on including realized skewness and kurtosis in "additive" and "multiplicative" models. Utilizing a panel of 960 US stocks, we conduct diagnostic tests, employ scoring functions, and implement rolling window forecasting to evaluate the performance of Value at Risk (VaR) and Expected Shortfall (ES) forecasts. Additionally, we examine the impact of the window length on forecast accuracy. We propose model specifications that incorporate realized skewness and kurtosis for enhanced precision. Our findings provide insights into the importance of considering skewness and kurtosis in tail risk modeling, contributing to the existing literature and offering practical implications for risk practitioners and researchers.

econ.EM↗

arXiv2024

A Platform-Agnostic Deep Reinforcement Learning Framework for Effective Sim2Real Transfer towards Autonomous Driving

Deep Reinforcement Learning (DRL) has shown remarkable success in solving complex tasks across various research fields. However, transferring DRL agents to the real world is still challenging due to the significant discrepancies between simulation and reality. To address this issue, we propose a robust DRL framework that leverages platform-dependent perception modules to extract task-relevant information and train a lane-following and overtaking agent in simulation. This framework facilitates the seamless transfer of the DRL agent to new simulated environments and the real world with minimal effort. We evaluate the performance of the agent in various driving scenarios in both simulation and the real world, and compare it to human players and the PID baseline in simulation. Our proposed framework significantly reduces the gaps between different platforms and the Sim2Real gap, enabling the trained agent to achieve similar performance in both simulation and the real world, driving the vehicle effectively.

cs.LG↗

arXiv2024

An open-source framework for data-driven trajectory extraction from AIS data -- the $α$-method

Ship trajectories from Automatic Identification System (AIS) messages are important in maritime safety, domain awareness, and algorithmic testing. Although the specifications for transmitting and receiving AIS messages are fixed, it is well known that technical inaccuracies and lacking seafarer compliance lead to severe data quality impairment. This paper proposes an adaptable, data-driven, maneuverability-dependent, $α$-quantile-based framework for decoding, constructing, splitting, and assessing trajectories from raw AIS records to improve transparency in AIS data mining. Results indicate the proposed filtering algorithm robustly extracts clean, long, and uninterrupted trajectories for further processing. An open-source Python implementation of the framework is provided.

stat.AP↗

arXiv2024

2-Level Reinforcement Learning for Ships on Inland Waterways: Path Planning and Following

This paper proposes a realistic modularized framework for controlling autonomous surface vehicles (ASVs) on inland waterways (IWs) based on deep reinforcement learning (DRL). The framework improves operational safety and comprises two levels: a high-level local path planning (LPP) unit and a low-level path following (PF) unit, each consisting of a DRL agent. The LPP agent is responsible for planning a path under consideration of dynamic vessels, closing a gap in the current research landscape. In addition, the LPP agent adequately considers traffic rules and the geometry of the waterway. We thereby introduce a novel application of a spatial-temporal recurrent neural network architecture to continuous action spaces. The LPP agent outperforms a state-of-the-art artificial potential field (APF) method by increasing the minimum distance to other vessels by 65% on average. The PF agent performs low-level actuator control while accounting for shallow water influences and the environmental forces winds, waves, and currents. Compared with a proportional-integral-derivative (PID) controller, the PF agent yields only 61% of the mean cross-track error (MCTE) while significantly reducing control effort (CE) in terms of the required absolute rudder angle. Lastly, both agents are jointly validated in simulation, employing the lower Elbe in northern Germany as an example case and using real automatic identification system (AIS) trajectories to model the behavior of other ships.

eess.SY↗

arXiv2024

Two-step dynamic obstacle avoidance

Dynamic obstacle avoidance (DOA) is a fundamental challenge for any autonomous vehicle, independent of whether it operates in sea, air, or land. This paper proposes a two-step architecture for handling DOA tasks by combining supervised and reinforcement learning (RL). In the first step, we introduce a data-driven approach to estimate the collision risk (CR) of an obstacle using a recurrent neural network, which is trained in a supervised fashion and offers robustness to non-linear obstacle movements. In the second step, we include these CR estimates into the observation space of an RL agent to increase its situational awareness. We illustrate the power of our two-step approach by training different RL agents in a challenging environment that requires to navigate amid multiple obstacles. The non-linear movements of obstacles are exemplarily modeled based on stochastic processes and periodic patterns, although our architecture is suitable for any obstacle dynamics. The experiments reveal that integrating our CR metrics into the observation space doubles the performance in terms of reward, which is equivalent to halving the number of collisions in the considered environment. We also perform a generalization experiment to validate the proposal in an RL environment based on maritime traffic and real-world vessel trajectory data. Furthermore, we show that the architecture's performance improvement is independent of the applied RL algorithm.

cs.RO↗

arXiv2024

Addressing Maximization Bias in Reinforcement Learning with Two-Sample Testing

Value-based reinforcement-learning algorithms have shown strong results in games, robotics, and other real-world applications. Overestimation bias is a known threat to those algorithms and can sometimes lead to dramatic performance decreases or even complete algorithmic failure. We frame the bias problem statistically and consider it an instance of estimating the maximum expected value (MEV) of a set of random variables. We propose the $T$-Estimator (TE) based on two-sample testing for the mean, that flexibly interpolates between over- and underestimation by adjusting the significance level of the underlying hypothesis tests. We also introduce a generalization, termed $K$-Estimator (KE), that obeys the same bias and variance bounds as the TE and relies on a nearly arbitrary kernel function. We introduce modifications of $Q$-Learning and the Bootstrapped Deep $Q$-Network (BDQN) using the TE and the KE, and prove convergence in the tabular setting. Furthermore, we propose an adaptive variant of the TE-based BDQN that dynamically adjusts the significance level to minimize the absolute estimation bias. All proposed estimators and algorithms are thoroughly tested and validated on diverse tasks and environments, illustrating the bias control and performance potential of the TE and KE.

cs.LG↗

arXiv2024

Self-organized free-flight arrival for urban air mobility

Urban air mobility is an innovative mode of transportation in which electric vertical takeoff and landing (eVTOL) vehicles operate between nodes called vertiports. We outline a self-organized vertiport arrival system based on deep reinforcement learning. The airspace around the vertiport is assumed to be circular, and the vehicles can freely operate inside. Each aircraft is considered an individual agent and follows a shared policy, resulting in decentralized actions that are based on local information. We investigate the development of the reinforcement learning policy during training and illustrate how the algorithm moves from suboptimal local holding patterns to a safe and efficient final policy. The latter is validated in simulation-based scenarios, including robustness analyses against sensor noise and a changing distribution of inbound traffic. Lastly, we deploy the final policy on small-scale unmanned aerial vehicles to showcase its real-world usability.

cs.LG↗

arXiv2024

Leader-Follower Identification with Vehicle-Following Calibration for Non-Lane-Based Traffic

Most car-following models were originally developed for lane-based traffic. Over the past two decades, efforts have been made to calibrate car-following models for non-lane-based traffic. However, traffic conditions with varying vehicle dimensions, intermittent following, and multiple leaders often occur and make subjective Leader-Follower (LF) pair identification challenging. In this study, we analyze Vehicle Following (VF) behavior in traffic with a lack of lane discipline using high-resolution microscopic trajectory data collected in Chennai, India. The paper's main contributions are threefold. Firstly, three criteria are used to identify LF pairs from the driver's perspective, taking into account the intermittent following, lack of lane discipline due to consideration of lateral separation, and the presence of in-between vehicles. Second, the psycho-physical concept of the regime in the Wiedemann-99 model is leveraged to determine the traffic-dependent "influence zone" for LF identification. Third, a joint and consistent framework is proposed for identifying LF pairs and estimating VF parameters. The proposed methodology outperforms other heuristic-based LF identification methods from the literature in terms of quantitative and qualitative performance measures. The proposed approach can enable robust and more realistic LF identification and VF parameter calibration with practical applications such as LOS analysis, capacity, and travel time estimation.

physics.soc-ph↗

arXiv2023

Vision-based DRL Autonomous Driving Agent with Sim2Real Transfer

To achieve fully autonomous driving, vehicles must be capable of continuously performing various driving tasks, including lane keeping and car following, both of which are fundamental and well-studied driving ones. However, previous studies have mainly focused on individual tasks, and car following tasks have typically relied on complete leader-follower information to attain optimal performance. To address this limitation, we propose a vision-based deep reinforcement learning (DRL) agent that can simultaneously perform lane keeping and car following maneuvers. To evaluate the performance of our DRL agent, we compare it with a baseline controller and use various performance metrics for quantitative analysis. Furthermore, we conduct a real-world evaluation to demonstrate the Sim2Real transfer capability of the trained DRL agent. To the best of our knowledge, our vision-based car following and lane keeping agent with Sim2Real transfer capability is the first of its kind.

cs.RO↗

arXiv2023

Spatial-temporal recurrent reinforcement learning for autonomous ships

This paper proposes a spatial-temporal recurrent neural network architecture for deep $Q$-networks that can be used to steer an autonomous ship. The network design makes it possible to handle an arbitrary number of surrounding target ships while offering robustness to partial observability. Furthermore, a state-of-the-art collision risk metric is proposed to enable an easier assessment of different situations by the agent. The COLREG rules of maritime traffic are explicitly considered in the design of the reward function. The final policy is validated on a custom set of newly created single-ship encounters called `Around the Clock' problems and the commonly used Imazu (1987) problems, which include 18 multi-ship scenarios. Performance comparisons with artificial potential field and velocity obstacle methods demonstrate the potential of the proposed approach for maritime path planning. Furthermore, the new architecture exhibits robustness when it is deployed in multi-agent scenarios and it is compatible with other deep reinforcement learning algorithms, including actor-critic frameworks.

cs.LG↗

arXiv2023

Robust Path Following on Rivers Using Bootstrapped Reinforcement Learning

This paper develops a Deep Reinforcement Learning (DRL)-agent for navigation and control of autonomous surface vessels (ASV) on inland waterways. Spatial restrictions due to waterway geometry and the resulting challenges, such as high flow velocities or shallow banks, require controlled and precise movement of the ASV. A state-of-the-art bootstrapped Q-learning algorithm in combination with a versatile training environment generator leads to a robust and accurate rudder controller. To validate our results, we compare the path-following capabilities of the proposed approach to a vessel-specific PID controller on real-world river data from the lower- and middle Rhine, indicating that the DRL algorithm could effectively prove generalizability even in never-seen scenarios while simultaneously attaining high navigational accuracy.

cs.LG↗

arXiv2023

Adaptive local VAR for dynamic economic policy uncertainty spillover

The availability of data on economic uncertainty sparked a lot of interest in models that can timely quantify episodes of international spillovers of uncertainty. This challenging task involves trading off estimation accuracy for more timely quantification. This paper develops a local vector autoregressive model (VAR) that allows for adaptive estimation of the time-varying multivariate dependency. Under local, we mean that for each point in time, we simultaneously estimate the longest interval on which the model is constant with the model parameters. The simulation study shows that the model can handle one or multiple sudden breaks as well as a smooth break in the data. The empirical application is done using monthly Economic Policy Uncertainty data. The local model highlights that the empirical data primarily consists of long homogeneous episodes, interrupted by a small number of heterogeneous ones, that correspond to crises. Based on this observation, we create a crisis index, which reflects the homogeneity of the sample over time. Furthermore, the local model shows superiority against the rolling window estimation.

econ.GN↗

arXiv2022

Enhanced method for reinforcement learning based dynamic obstacle avoidance by assessment of collision risk

In the field of autonomous robots, reinforcement learning (RL) is an increasingly used method to solve the task of dynamic obstacle avoidance for mobile robots, autonomous ships, and drones. A common practice to train those agents is to use a training environment with random initialization of agent and obstacles. Such approaches might suffer from a low coverage of high-risk scenarios in training, leading to impaired final performance of obstacle avoidance. This paper proposes a general training environment where we gain control over the difficulty of the obstacle avoidance task by using short training episodes and assessing the difficulty by two metrics: The number of obstacles and a collision risk metric. We found that shifting the training towards a greater task difficulty can massively increase the final performance. A baseline agent, using a traditional training environment based on random initialization of agent and obstacles and longer training episodes, leads to a significantly weaker performance. To prove the generalizability of the proposed approach, we designed two realistic use cases: A mobile robot and a maritime ship under the threat of approaching obstacles. In both applications, the previous results can be confirmed, which emphasizes the general usability of the proposed approach, detached from a specific application context and independent of the agent's dynamics. We further added Gaussian noise to the sensor signals, resulting in only a marginal degradation of performance and thus indicating solid robustness of the trained agent.

cs.RO↗