arXiv subjects

Damien Ernst

Publications and source records attributed to Damien Ernst.

At least 19 recordsLinked to original sources

arXiv2026

Voltage Sensitivity-Based Fair Allocation of Operating Envelopes in Distribution Networks

Operating envelopes (OEs) define the admissible active and reactive power injections of distributed energy resources while respecting distribution network limits. In unbalanced low-voltage feeders, OEs based only on voltage magnitude and thermal limits may overestimate available flexibility because they neglect power quality requirements. This paper develops a three-phase unbalanced AC optimal power flow framework to compute coupled active and reactive power OEs while enforcing limits on the voltage unbalance factor (VUF). It also proposes a fairness mechanism based on voltage sensitivities to allocate available flexibility among multiple photovoltaic (PV) units. The proposed mechanism is compared with lexicographic max--min fairness as a benchmark for equitable resource allocation. Case studies on unbalanced test feeders evaluate how VUF constraints change the feasible operating region of PV units. The results show how VUF constraints affect feasible flexibility and demonstrate that the proposed voltage sensitivity-based mechanism is nearly six times faster than lexicographic max-min fairness.

eess.SY↗

arXiv2026

Reconciling European Electricity Data with Diffusion Priors Learned from Observations Alone

The ENTSO-E Transparency Platform publishes generation, load, and cross-border power flows for European bidding zones. The values carry undocumented errors and violate energy conservation, and data reconciliation consists in correcting them to satisfy that balance. In this work, we formulate data reconciliation as Bayesian inference. Classical methods implicitly take a uniform prior, which we replace with a diffusion model estimated from the observations alone by empirical-Bayes expectation-maximization. The resulting posterior is sampled with a guided process whose projection step makes every sample satisfy the balance exactly. On synthetic systems of six to thirty-five bidding zones, built from ENTSO-E resource-adequacy data, the diffusion prior gives the most accurate reconciliation, increasingly so as the system grows. Its credible intervals are calibrated or conservative, and it remains the most accurate under a misspecified noise covariance. Finally, when the generation of entire zones goes unreported, the classical methods fail while the learned prior recovers it from cross-zone structure.

eess.SY↗

arXiv2026

Future-Informed Actor-Critic

Actor-critic methods optimize a parametrized policy using a critic to estimate their policy gradient. Recent work has shown that conditioning the critic on features of future states, actions, or rewards can improve sample efficiency and even final return. However, existing methods are restricted to exogenous features (independent of actions), overlooking endogenous features (dependent on actions) that may be crucial for credit assignment. We introduce the future-informed value function, which can be conditioned on both exogenous and endogenous features of future states, actions, and rewards. Then, we express the policy gradient using this value function, yielding the future-informed policy gradient. Building on that, we propose the future-informed actor-critic algorithm. Empirical results show that this algorithm leveraging endogenous features of the future outperforms standard actor-critic algorithms in environments with noisy and sparse rewards.

cs.LG↗

arXiv2026

Parallelizable memory recurrent units

With the emergence of massively parallel processing units, parallelization has become a desirable property for new sequence models. The ability to parallelize the processing of sequences with respect to the sequence length during training is one of the main factors behind the uprising of the Transformer architecture. However, Transformers lack efficiency at sequence generation, as they need to reprocess all past timesteps at every generation step. Recently, state-space models (SSMs) emerged as a more efficient alternative. These new kinds of recurrent neural networks (RNNs) keep the efficient update of the RNNs while gaining parallelization by getting rid of nonlinear dynamics (or recurrence). SSMs can reach state-of-the-art performance through the efficient training of potentially very large networks, but still suffer from limited representation capabilities. In particular, SSMs cannot exhibit persistent memory, or the capacity of retaining information for an infinite duration, because of their monostability. In this paper, we introduce a new family of RNNs, the memory recurrent units (MRUs), that combine the persistent memory capabilities of nonlinear RNNs with the parallelizable computations of SSMs. These units leverage multistability as a source of persistent memory, while getting rid of transient dynamics for efficient computations. We then derive a specific implementation as proof-of-concept: the bistable memory recurrent unit (BMRU). This new RNN is compatible with the parallel scan algorithm. We show that BMRU achieves good results in tasks with long-term dependencies, and can be combined with SSMs to create parallelizable hybrid networks that have both transient dynamics and persistent memory.

cs.LG↗

arXiv2026

Maximum-Entropy Exploration with Future State-Action Visitation Measures

Maximum entropy reinforcement learning motivates agents to explore states and actions to maximize the entropy of some distribution, typically by providing additional intrinsic rewards proportional to that entropy function. In this paper, we study intrinsic rewards proportional to the entropy of the discounted distribution of state-action features visited during future time steps. This approach is motivated by two results. First, we show that the expected sum of these intrinsic rewards is a lower bound on the entropy of the discounted distribution of state-action features visited in trajectories starting from the initial states, which we relate to an alternative maximum entropy objective. Second, we show that the distribution used in the intrinsic reward definition is the fixed point of a contraction operator and can therefore be estimated off-policy. Experiments highlight that the new objective often leads to improved visitation of features within individual trajectories, in exchange for slightly reduced visitation of features in expectation over different trajectories, as suggested by the lower bound. It also leads to improved convergence speed for learning exploration-only agents. Control performance remains similar across most methods on the considered benchmarks.

cs.LG↗

arXiv2026

Memory-State Critic for Asymmetric Actor-Critic with Application to Vision-Based Pursuit-Evasion

In partially observable Markov decision processes, the optimal policy generally depends on the history of observations and past actions. Asymmetric actor-critic methods have become popular to learn such policies when additional information, such as the true state of the environment, is available during training. The critic, which is not needed at execution, is given access to the state. A critic conditioned on the state alone is generally ill-defined and yields biased policy gradients. Conditioning on the state and the history, the history-state critic restores both. In this paper, we show that conditioning the critic on the state and the policy's own memory, i.e., the internal representation of the history through which the policy selects its actions, is already well-defined and gives unbiased policy gradients, removing the need for a second recurrent approximator of the history. We call it the memory state critic. It follows that a critic based on the policy's memory need not backpropagate its loss into that memory, even though the memory is a lossy encoding of the history. We evaluate the memory-state critic in a vision-based pursuit-evasion environment between two quadrotors across two arena types. The pursuer is the learning agent, and the evader is sampled per episode from a fixed pool of heuristic behaviours. The results show that the memory-state critic outperforms the history-state critic and converges faster. In addition to being unbiased compared to the state-only critic, it maintains a slight edge in the wall arena, where the actor's history carries information that the privileged state alone does not.

cs.LG↗

arXiv2026

Length-Independent State Tracking Under a Parallel Scan

Learning robust and scalable finite-state tracking is fundamental to sequence processing. While linear recurrent neural networks (RNNs), linear attention, and state space models enable scalable parallel training through affine recurrences, their theoretical expressivity guarantees assume idealized arithmetic and do not extend to finite precision, where the parallel scan that makes them fast is itself a source of perturbation. We formalize finite-state tracking at finite precision and characterize length independence: tracking that stays correct at every sequence length, at a precision cost that does not grow with the length. We show that length-independent state tracking requires two competing dynamics within a single map: contraction to suppress numerical perturbations and separation to keep distinct states apart. We prove that affine recurrences, which offer a single rate at each step to serve both roles, realize at most definite automata at finite precision. Instead of treating scan compatibility as a restriction on the update map, we reinterpret it as a computational budget and introduce the Neural Finite-State Machine (NFSM): a nonaffine, scan-compatible RNN built for length-independent finite-state tracking. On synthetic benchmarks spanning abelian and nonabelian groups, noninvertible monoids, and textual state-tracking tasks, affine baselines fail on every nondefinite task, most of them within a few hundred steps. A single NFSM layer instead learns the exact transition tables of every algebraic task, which certifies correctness beyond the tested lengths, and a stack of NFSMs keeps perfect accuracy on the textual tasks at every tested length.

cs.LG↗

arXiv2026

Reinformed Dreamer: An Asymmetric World Model Efficiently Trained through Latent Guidance

Much like humans benefit from guidance while learning, reinforcement learning algorithms may benefit from additional supervision beyond rewards. Leveraging additional information during training to learn better representations and behaviors has been the focus of asymmetric reinforcement learning. This learning paradigm has proven effective under partial observability when additional state information is available, but also under full observability when more refined state information is available. Focusing on model-based reinforcement learning, we study the effect of asymmetric learning on observation representations and on privileged information representations. First, we identify a limitation in the privileged information representations learned by an asymmetric model-based algorithm known as the Informed Dreamer. Then, we propose a novel asymmetric representation learning objective using latent guidance, resulting in a new algorithm called the Reinformed Dreamer. Experiments across several benchmarks show a more consistent improvement over Dreamer than previous asymmetric approaches.

cs.LG↗

arXiv2026

Informed Asymmetric Actor-Critic: Leveraging Privileged Signals Beyond Full-State Access

Asymmetric reinforcement learning leverages privileged information available during training to improve learning under partial observability. Existing asymmetric actor-critic methods typically assume access to the full environment state to condition the critic during training, which is often unrealistic in practice. We introduce the informed asymmetric actor-critic framework that allows the critic to be conditioned on arbitrary state-dependent privileged signals, and show that any such signal yields unbiased policy gradient estimates. This substantially expands the set of admissible privileged information and raises the problem of selecting the most informative signals for learning. To this end, we propose two novel informativeness criteria: a dependence-based test that can be applied prior to training, and a test based on improvements in value prediction that can be applied post hoc. Experiments on partially observable benchmarks and synthetic environments demonstrate that carefully selected privileged signals can match or outperform full-state asymmetric baselines while relying on strictly less state information.

cs.LG↗

arXiv2026

Improving the Performance and Learning Stability of Parallelizable RNNs Designed for Ultra-Low Power Applications

Sequence learning is dominated by Transformers and parallelizable recurrent neural networks (RNNs) such as state-space models, yet learning long-term dependencies remains challenging, and state-of-the-art designs trade power consumption for performance. The Bistable Memory Recurrent Unit (BMRU) was introduced to enable hardware-software co-design of ultra-low power RNNs: quantized states with hysteresis provide persistent memory while mapping directly to analog primitives. However, BMRU performance lags behind parallelizable RNNs on complex sequential tasks. In this paper, we identify gradient blocking during state updates as a key limitation and propose a cumulative update formulation that restores gradient flow while preserving persistent memory, creating skip-connections through time. This leads to the Cumulative Memory Recurrent Unit (CMRU) and its relaxed variant, the $α$CMRU. Experiments show that the cumulative formulation dramatically improves convergence stability and reduces initialization sensitivity. The CMRU and $α$CMRU match or outperform Linear Recurrent Units (LRUs) and minimal Gated Recurrent Units (minGRUs) across diverse benchmarks at small model sizes, with particular advantages on tasks requiring discrete long-range retention, while the CMRU retains quantized states, persistent memory, and noise-resilient dynamics essential for analog implementation.

cs.LG↗

arXiv2026

Hardware-Software Co-Design of Scalable, Energy-Efficient Analog Recurrent Computations

Always-on AI applications, from environmental sensors to biomedical implants, require ultra-low power consumption. Analog circuits offer a path to sub-microwatt inference, yet existing analog implementations are limited to feedforward architectures: extending them to recurrent dynamics has been considered impractical due to noise accumulation through temporal feedback. We demonstrate that this barrier can be overcome through hardware-software co-design. Specifically, we identify that Bistable Memory Recurrent Units (BMRUs), a class of Recurrent Neural Networks (RNNs) with discrete-valued outputs and hysteretic dynamics, admit an ultra-low power current-mode analog implementation which we design from first principles. The resulting circuit establishes a one-to-one correspondence between each learned parameter and a circuit element. The discrete outputs suppress analog noise by at least 20-fold at each cell boundary, breaking the noise accumulation that prevents analog recurrence. We reformulate BMRUs for first-quadrant operation with fixed thresholds, enabling the direct correspondence while preserving expressivity and trainability. Transistor-level simulations in 180 nm Complementary Metal-Oxide-Semiconductor (CMOS) show near-perfect agreement between software predictions and circuit-level behavior, with the software model thereby serving as a high-fidelity simulator of the physical hardware at low computational cost. We leverage this fidelity to conduct large-scale noise immunity and power scaling analyses: the power cost of adding recurrence scales linearly with state dimension, while the feedforward layers dominating total power scale quadratically, meaning recurrence is added at linear marginal cost relative to the feedforward backbone. End-to-end keyword spotting achieves sub-microwatt inference at the RNN core.

cs.AR↗

arXiv2026

Bayesian Inference for Estimating Generation Costs in Electricity Markets

Estimating generation costs from observed electricity market data is essential for market simulation, strategic bidding, and system planning. To that end, we model the relationship between generation costs and production schedules with a latent variable model. Estimating generation costs from observed schedules is then formulated as Bayesian inference. A prior distribution encodes an initial belief on parameters, and the inference consists of updating the belief with the posterior distribution given observations. We use balanced neural posterior estimation (BNPE) to learn this posterior. Validation on the IEEE RTS-96 test system shows that marginal costs are recovered with narrow credible intervals, while start-up costs remain largely unidentifiable from schedules alone. The method is benchmarked against an inverse-optimization algorithm that exhibits larger parameter errors without uncertainty quantification.

eess.SY↗

arXiv2026

Gym-TORAX: Open-source software for integrating reinforcement learning with plasma control simulators in tokamak research

This paper presents Gym-TORAX, a Python package enabling the implementation of Reinforcement Learning (RL) environments for simulating plasma dynamics and control in tokamaks. Users define succinctly a set of control actions and observations, and a control objective from which Gym-TORAX creates a Gymnasium environment that wraps TORAX for simulating the plasma dynamics. The objective is formulated through rewards depending on the simulated state of the plasma and control action to optimize specific characteristics of the plasma, such as performance and stability. The resulting environment instance is then compatible with a wide range of RL algorithms and libraries and will facilitate RL research in plasma control. In its current version, one environment is readily available, based on a ramp-up scenario of the International Thermonuclear Experimental Reactor (ITER).

cs.LG↗

arXiv2026

Fast reconstruction of degenerate populations of conductance-based neuron models from spike times

Inferring the biophysical parameters of conductance-based models (CBMs) from experimentally accessible recordings remains a central challenge in computational neuroscience. Spike times are the most widely available data, yet they reveal little about which combinations of ion channel conductances generate the observed activity. This inverse problem is further complicated by neuronal degeneracy, where multiple distinct conductance sets yield similar spiking patterns. We introduce a method that addresses this challenge by combining deep learning with Dynamic Input Conductances (DICs), a theoretical framework that reduces complex CBMs to three interpretable feedback components governing excitability and firing patterns. Our approach first maps spike times to DIC densities at threshold using a neural network that learns a low-dimensional representation of neuronal activity. The predicted DIC values are then used to generate degenerate CBM populations via an iterative compensation algorithm, ensuring compatibility with the intermediate target DICs, and thereby reproducing the corresponding firing patterns, even in high-dimensional models. Applied to two models, this algorithmic pipeline reconstructs spiking and bursting regimes with high accuracy and robustness to variability, including spike trains generated under noisy current injection mimicking physiological stochasticity. It produces diverse degenerate populations within milliseconds on standard hardware, enabling scalable and efficient inference from spike recordings alone. Together, this work positions DICs as a practical and interpretable link between experimentally observed activity and mechanistic models. By enabling fast and scalable reconstruction of degenerate populations directly from spike times, our approach provides a powerful way to investigate how neurons exploit conductance variability to achieve reliable computation.

q-bio.NC↗

arXiv2026

Sensitivity analysis for linear changes of the constraint matrix of a (mixed-integer) linear program

Understanding how the optimal value of an optimisation problem changes when its input data is modified is an old question in mathematical optimisation. This paper investigates the computation of the optimal values of a family of (possibly mixed-integer) linear optimisation problems in which the constraint matrix is subject to linear perturbations controlled by a scalar parameter that varies within a given interval. This is a largely unresolved question with the additional burden that the resulting value function may be largely irregular. We propose several bounding techniques that provide formal guarantees on the behaviour of the objective value across the entire parameter range. The proposed bounds rely on tools from robust optimisation, Lagrangian relaxation, and ad-hoc reformulations. Each method is assessed in terms of accuracy, precision, and computational performance. Experimental results on a large benchmark set show that the proposed bounding techniques effectively address this class of problems, delivering strong guarantees and good precision. In addition, we introduce a spatial branch-and-bound algorithm that incorporates these bounds to compute an anytime approximation of the value function within a given error tolerance, and we analyse its computational performance.

math.OC↗

arXiv2026

Efficient LP warmstarting for linear modifications of the constraint matrix

We consider the problem of computing the optimal solution and objective of a linear program under linearly changing linear constraints. The problem studied is given by $\min c^t x \text{ s.t } Ax + λDx \leq b$ where $λ$ belongs to a set of predefined values $Λ$. Based on the information given by a precomputed basis, we present three efficient LP warm-starting algorithms. Each algorithm is either based on the eigenvalue decomposition, the Schur decomposition, or a tweaked eigenvalue decomposition to evaluate the optimal solution and optimal objective of these problems. The three algorithms have an overall complexity $O(pm^2+pmn)$ where $m$ (resp. $n$) is the number of constraints (resp. variables) of the original problem and $p$ the number of values in $Λ$ after an initial preprocessing step. We also provide theorems related to the optimality conditions to verify when a basis is still optimal and a local bound on the objective.

math.OC↗

arXiv2025

SecuLEx: a Secure Limit Exchange Market for Dynamic Operating Envelopes

Distributed energy resources (DERs) are transforming power networks, challenging traditional operational methods, and requiring new coordination mechanisms. To address this challenge, this paper introduces SecuLEx (Secure Limit Exchange), a new market-based paradigm to allocate power injection and withdrawal limits that guarantee network security during time periods, called dynamic operating envelopes (DOEs). Under this paradigm, distribution system operators (DSOs) assign initial DOEs to customers. These limits can be exchanged afterward through a market, allowing customers to reallocate them according to their needs while ensuring network operational constraints. We formalize SecuLEx and illustrate DOE allocation and market exchanges on a small-scale low-voltage (LV) network, demonstrating that both procedures are computationally tractable. In this example, SecuLEx reduces renewable curtailment and improves grid utilization and social welfare compared to traditional approaches.

eess.SY↗

arXiv2025

Off-Policy Maximum Entropy RL with Future State and Action Visitation Measures

Maximum entropy reinforcement learning integrates exploration into policy learning by providing additional intrinsic rewards proportional to the entropy of some distribution. In this paper, we propose a novel approach in which the intrinsic reward function is the relative entropy of the discounted distribution of states and actions (or features derived from these states and actions) visited during future time steps. This approach is motivated by two results. First, a policy maximizing the expected discounted sum of intrinsic rewards also maximizes a lower bound on the state-action value function of the decision process. Second, the distribution used in the intrinsic reward definition is the fixed point of a contraction operator. Existing algorithms can therefore be adapted to learn this fixed point off-policy and to compute the intrinsic rewards. We finally introduce an algorithm maximizing our new objective, and we show that resulting policies have good state-action space coverage and achieve high-performance control.

cs.LG↗