arXiv subjects

Carlos S. Sepúlveda

Publications and source records attributed to Carlos S. Sepúlveda.

4 recordsLinked to original sources

arXiv2026

Critic-Free Deep Reinforcement Learning for Maritime Coverage Path Planning on Irregular Hexagonal Grids

Maritime surveillance missions, such as search and rescue and environmental monitoring, rely on the efficient allocation of sensing assets over vast and geometrically complex areas. Traditional Coverage Path Planning (CPP) approaches depend on decomposition techniques that struggle with irregular coastlines, islands, and exclusion zones, or require computationally expensive re-planning for every instance. We propose a Deep Reinforcement Learning (DRL) framework to solve CPP on hexagonal grid representations of irregular maritime areas. Unlike conventional methods, we formulate the problem as a neural combinatorial optimization task where a Transformer-based pointer policy autoregressively constructs coverage tours. To overcome the instability of value estimation in long-horizon routing problems, we implement a critic-free Group-Relative Policy Optimization (GRPO) scheme. This method estimates advantages through within-instance comparisons of sampled trajectories rather than relying on a value function. Experiments on 1,000 unseen synthetic maritime environments demonstrate that a trained policy achieves a 99.1% Hamiltonian success rate, more than double the best heuristic (46.0%), while producing paths 7% shorter and up to 24.1% fewer heading changes than the closest heuristic baseline. All three inference modes (greedy, stochastic sampling, and sampling with 2-opt refinement) operate under 50 ms per instance on a laptop GPU, confirming feasibility for real-time onboard deployment.

cs.LG↗

arXiv2026

Baseline-Free Policy Optimization for Neural Combinatorial Optimization

Neural combinatorial optimization (NCO) trains autoregressive policies to solve routing problems. The standard training algorithm, REINFORCE with a rollout baseline, requires maintaining and periodically updating a frozen copy of the policy for variance reduction. This baseline introduces a structural vulnerability: on harder instances, a poor baseline produces noisy gradient estimates that can destabilize training. We evaluate Group Relative Policy Optimization (GRPO), an algorithm from large language model alignment that eliminates the baseline entirely by normalizing advantages within groups of sampled trajectories. In a controlled comparison of five RL algorithms on TSP and CVRP benchmarks within the RL4CO framework, we find that: (i) GRPO avoids the training collapse observed with REINFORCE on TSP-100, where performance degrades from cost 9.8 to 52.1 immediately after the warmup phase and does not recover under extended training; (ii) at matched gradient updates, GRPO achieves solution quality within 2% of POMO, a strong AM-based multi-start baseline, while requiring no external baseline; and (iii) P3O, a pairwise preference algorithm also from the alignment literature, is competitive on TSP but shows higher variability on CVRP. These results identify GRPO as a promising baseline-free alternative for NCO, particularly in settings where baseline-dependent training becomes fragile.

cs.LG↗

arXiv2026

Benchmarking Classical Coverage Path Planning Heuristics on Irregular Hexagonal Grids for Maritime Coverage Scenarios

Coverage path planning on irregular hexagonal grids is relevant to maritime surveillance, search and rescue and environmental monitoring, yet classical methods are often compared on small ad hoc examples or on rectangular grids. This paper presents a reproducible benchmark of deterministic single-vehicle coverage path planning heuristics on irregular hexagonal graphs derived from synthetic but maritime-motivated areas of interest. The benchmark contains 10,000 Hamiltonian-feasible instances spanning compact, elongated, and irregular morphologies, 17 heuristics from seven families, and a common evaluation protocol covering Hamiltonian success, complete-coverage success, revisits, path length, heading changes, and CPU latency. Across the released dataset, heuristics with explicit shortest-path reconnection solve the relaxed coverage task reliably but almost never produce zero-revisit tours. Exact Depth-First Search confirms that every released instance is Hamiltonian-feasible. The strongest classical Hamiltonian baseline is a Warnsdorff variant that uses an index-based tie-break together with a terminal-inclusive residual-degree policy, reaching 79.0% Hamiltonian success. The dominant design choice is not tie-breaking alone, but how the residual degree is defined when the endpoint is reserved until the final move. This shows that underreported implementation details can materially affect performance on sparse geometric graphs with bottlenecks. The benchmark is intended as a controlled testbed for heuristic analysis rather than as a claim of operational optimality at fleet scale.

cs.RO↗

arXiv2022

Exploring the Dynamics of Fungal Cellular Automata

Cells in a fungal hyphae are separated by internal walls (septa). The septa have tiny pores that allow cytoplasm flowing between cells. Cells can close their septa blocking the flow if they are injured, preventing fluid loss from the rest of filament. This action is achieved by special organelles called Woronin bodies. Using the controllable pores as an inspiration we advance one and two-dimensional cellular automata into Elementary fungal cellular automata (EFCA) and Majority fungal automata (MFA) by adding a concept of Woronin bodies to the cell state transition rules. EFCA is a cellular automaton where the communications between neighboring cells can be blocked by the activation of the Woronin bodies (Wb), allowing or blocking the flow of information (represented by a cytoplasm and chemical elements it carries) between them. We explore a novel version of the fungal automata where the evolution of the system is only affected by the activation of the Wb. We explore two case studies: the Elementary Fungal Cellular Automata (EFCA), which is a direct application of this variant for elementary cellular automata rules, and the Majority Fungal Automata (MFA), which correspond to an application of the Wb to two dimensional automaton with majority rule with Von Neumann neighborhood. By studying the EFCA model, we analyze how the 256 elementary cellular automata rules are affected by the activation of Wb in different modes, increasing the complexity on applied rule in some cases. Also we explore how a consensus over MFA is affected when the continuous flow of information is interrupted due to the activation of Woronin bodies.

nlin.CG↗