Searcharxiv⌕ Search

arXiv subjects

Fulong Yao

Publications and source records attributed to Fulong Yao.

9 recordsLinked to original sources

Prospective Action-Resource Graph Multi-Agent Reinforcement Learning for Electric HGV Fleets under Shared Charging Infrastructure

Electric heavy goods vehicle (eHGV) fleet coordination requires each vehicle to choose driving, charging and resting actions whose consequences depend on shared infrastructure. Existing multi-agent reinforcement learning (MARL) methods mainly represent current states, leaving action-specific futures and resource interactions to be learned indirectly. We develop Horizon-Preview and Graph-Coordinated Proximal Policy Optimisation (HGC-PPO), which integrates horizon preview and candidate-resource graph coordination into the PPO decision architecture. Horizon preview represents each legal action through its short-horizon operational consequences. Graph message passing then updates these candidates through station, charger-type and time-slot resources shared across the fleet. On the 794km Folkestone-Glasgow task, HGC-PPO achieved the lowest mean charging cost among five RL methods at £504.32 per vehicle, with a fleet completion time of 24.920h. It also completed the task 1.763h faster and cost £36.51 less per vehicle than a replay-verified finite-budget MILP benchmark. Across six unseen routes and fleet sizes from 5 to 20 vehicles, HGC-PPO delivered the lowest macro-average completion time, charging cost and charging frequency among the RL methods. A 2 by 2 ablation showed that the full preview-graph architecture performed best across all four operational metrics. These results show that prospective action information and fleet-wide resource coordination improve eHGV decisions under charging and driver-hours constraints.

eess.SY↗

Dual-Agent Reinforcement Learning for Vehicle Scheduling and Multi-Energy Dispatch in Electric-Hydrogen-Transport Systems

Electric-hydrogen-transport systems (EHTSs) require coordinated scheduling of electric and hydrogen vehicles together with multi-energy dispatch. These operations form a sequential decision process: upstream vehicle scheduling (VS) determines service allocation and order, thereby endogenously shaping the electricity and hydrogen demands faced by downstream energy dispatch (ED). This paper develops a dual-agent reinforcement learning framework tailored to the heterogeneous decision structures. The vehicle-scheduling agent combines proximal policy optimisation (PPO) with a Plackett-Luce policy to learn vehicle-priority permutations, which are then mapped to feasible charger and refueller assignments through a greedy allocation. The energy-dispatch agent employs twin delayed deep deterministic policy gradient (TD3) for continuous storage and energy-flow control. The directional coupling between the two decision layers is analysed, revealing that updates to the upstream scheduling policy can shift the demand distribution experienced by the downstream dispatcher. A three-stage training mechanism is therefore introduced, combining layer-wise pretraining with concurrent fine-tuning to mitigate the effects of scheduling-induced distribution shifts on downstream ED learning. Year-long case studies show that the proposed framework achieves the lowest vehicle-service delay and energy cost among the 11 evaluated methods, demonstrating effective coordination between vehicle scheduling and multi-energy dispatch.

eess.SY↗

Electric Vehicle Scheduling and Carbon-Aware Energy Dispatch for Charging Stations

Electric vehicle (EV) charging stations integrating photovoltaic (PV) generation, battery storage, and bidirectional grid exchange must coordinate access to limited chargers with energy supply while considering both energy cost and carbon emissions. This paper proposes a solver-free greedy heuristic (SFGH) for dynamic vehicle scheduling (VS) and a twin delayed deep deterministic policy gradient (TD3) controller for carbon-aware energy dispatch (ED). SFGH assigns EVs according to reductions in service delays, and TD3 controls battery operation to balance energy costs and carbon emissions while meeting the resulting charging demand. The model allows both PV and battery exports, accounting for export revenue and carbon credits. Simulations on year-long case studies show that SFGH--TD3 achieves the lowest accumulated service delay, energy cost, and carbon emissions among the evaluated heuristic and multi-agent deep reinforcement learning (DRL) configurations.

eess.SY↗

A Hierarchical Optimisation Framework for Integrated Electric-Hydrogen-Transport Systems

Integrated electric-hydrogen infrastructures are becoming increasingly important with the growing deployment of electric vehicles (EVs) and hydrogen vehicles (HVs) in transport systems. However, the strong coupling between vehicle scheduling and multi-energy dispatch introduces significant operational challenges. This paper models an integrated electric-hydrogen-transport system (EHTS) and proposes a hierarchical optimisation framework that couples vehicle scheduling and downstream energy dispatch through a sequential, demand-driven two-layer structure. In the vehicle scheduling layer, a solver-free greedy heuristic (SFGH) algorithm is developed to avoid repeated optimisation solving, enabling real-time EV charging and HV refuelling under non-preemptive service and within-interval sequential assignment. The resulting charging and refuelling demands are subsequently passed to the energy dispatch layer, where a deep reinforcement learning (DRL)-based approach is designed to optimise battery operation, hydrogen-tank operation, and PV generation allocation to minimise the overall operational cost of the EHTS while satisfying the scheduled transport demand. Representative case studies, together with comparative, ablation, and generalisation analyses, demonstrate the effectiveness and robustness of the proposed framework. Furthermore, the learned dispatch policy maintains strong performance across diverse transport-demand scenarios and PV penetration levels without retraining, demonstrating robust generalisation capability for practical deployment.

eess.SY↗

Trend-Aware Multi-Task Learning for Short-Term Energy Forecasting

Short-term energy forecasting plays an important role in real-time operational decision-making, such as electricity market bidding and power system dispatch, where both numerical accuracy and correct directional signals are essential. However, most existing forecasting approaches formulate the problem purely as a regression task, limiting their ability to explicitly capture stepwise directional movements and trend consistency required for operational decisions. To address this limitation, this paper proposes a trend-aware multi-task forecasting framework that decomposes forecasting outputs into directional movements and deviation magnitudes relative to the latest observation, enabling both accurate numerical prediction and interpretable trend-aware outputs. The framework adopts a task-specific dual-stream architecture and explores key design choices for integrating trend and deviation information, including hard versus probabilistic trend representations, symmetric versus asymmetric deviation modelling, and parallel versus sequential conditioning strategies. To stabilize multi-task learning and reduce manual tuning, an uncertainty-aware task weighting scheme is incorporated to automatically balance directional classification, deviation regression, and final output prediction during training. Experimental results on real-world energy datasets demonstrate that the proposed framework achieves competitive numerical accuracy compared with state-of-the-art algorithms, while consistently improving trend prediction performance with moderate computational cost. This capability is particularly beneficial in short-term energy system management, where consistent directional forecasting can provide more reliable decision support for practical operational scenarios such as market bidding, resource scheduling, and risk-aware energy management.

cs.LG↗

A New Error Temporal Difference Algorithm for Deep Reinforcement Learning in Microgrid Optimization

Predictive control approaches based on deep reinforcement learning (DRL) have gained significant attention in microgrid energy optimization. However, existing research often overlooks the issue of uncertainty stemming from imperfect prediction models, which can lead to suboptimal control strategies. This paper presents a new error temporal difference (ETD) algorithm for DRL to address the uncertainty in predictions,aiming to improve the performance of microgrid operations. First,a microgrid system integrated with renewable energy sources (RES) and energy storage systems (ESS), along with its Markov decision process (MDP), is modelled. Second, a predictive control approach based on a deep Q network (DQN) is presented, in which a weighted average algorithm and a new ETD algorithm are designed to quantify and address the prediction uncertainty, respectively. Finally, simulations on a realworld US dataset suggest that the developed ETD effectively improves the performance of DRL in optimizing microgrid operations.

cs.LG↗

A Review of Vision-Based Assistive Systems for Visually Impaired People: Technologies, Applications, and Future Directions

Visually impaired individuals rely heavily on accurate and timely information about obstacles and their surrounding environments to achieve independent living. In recent years, significant progress has been made in the development of assistive technologies, particularly vision-based systems, that enhance mobility and facilitate interaction with the external world in both indoor and outdoor settings. This paper presents a comprehensive review of recent advances in assistive systems designed for the visually impaired, with a focus on state-of-the-art technologies in obstacle detection, navigation, and user interaction. In addition, emerging trends and future directions in visual guidance systems are discussed.

cs.CV↗

A Self-organizing Interval Type-2 Fuzzy Neural Network for Multi-Step Time Series Prediction

Data uncertainty is inherent in many real-world applications and poses significant challenges for accurate time series predictions. The interval type 2 fuzzy neural network (IT2FNN) has shown exceptional performance in uncertainty modelling for single-step prediction tasks. However, extending it for multi-step ahead predictions introduces further issues in uncertainty handling as well as model interpretability and accuracy. To address these issues, this paper proposes a new selforganizing interval type-2 fuzzy neural network with multiple outputs (SOIT2FNN-MO). Differing from the traditional six-layer IT2FNN, a nine-layer network architecture is developed. First, a new co-antecedent layer and a modified consequent layer are devised to improve the interpretability of the fuzzy model for multi-step time series prediction problems. Second, a new link layer is created to improve the accuracy by building temporal connections between multi-step predictions. Third, a new transformation layer is designed to address the problem of the vanishing rule strength caused by high-dimensional inputs. Furthermore, a two-stage, self-organizing learning mechanism is developed to automatically extract fuzzy rules from data and optimize network parameters. Experimental results on chaotic and microgrid prediction problems demonstrate that SOIT2FNN-MO outperforms state-of-the-art methods, by achieving a better accuracy ranging from 1.6% to 30% depending on the level of noises in data. Additionally, the proposed model is more interpretable, offering deeper insights into the prediction process.

cs.LG↗

A Holistic Power Optimization Approach for Microgrid Control Based on Deep Reinforcement Learning

The global energy landscape is undergoing a transformation towards decarbonization, sustainability, and cost-efficiency. In this transition, microgrid systems integrated with renewable energy sources (RES) and energy storage systems (ESS) have emerged as a crucial component. However, optimizing the operational control of such an integrated energy system lacks a holistic view of multiple environmental, infrastructural and economic considerations, not to mention the need to factor in the uncertainties from both the supply and demand. This paper presents a holistic datadriven power optimization approach based on deep reinforcement learning (DRL) for microgrid control considering the multiple needs of decarbonization, sustainability and cost-efficiency. First, two data-driven control schemes, namely the prediction-based (PB) and prediction-free (PF) schemes, are devised to formulate the control problem within a Markov decision process (MDP). Second, a multivariate objective (reward) function is designed to account for the market profits, carbon emissions, peak load, and battery degradation of the microgrid system. Third, we develop a Double Dueling Deep Q Network (D3QN) architecture to optimize the power flows for real-time energy management and determine charging/discharging strategies of ESS. Finally, extensive simulations are conducted to demonstrate the effectiveness and superiority of the proposed approach through a comparative analysis. The results and analysis also suggest the respective circumstances for using the two control schemes in practical implementations with uncertainties.

eess.SY↗