arXiv subjects

Martin Müller

Publications and source records attributed to Martin Müller.

At least 19 recordsLinked to original sources

arXiv2026

Soft X-Ray Detection Down to the Nitrogen K-Edge Using a Pixelated iLGAD Sensor

Hybrid pixel detectors represent the state-of-the-art technology for hard X-ray experiments at synchrotron light sources and free-electron lasers, owing to their high frame rates, extended dynamic range, and scalable detection areas. However, their application in the soft X-ray regime remains bottlenecked by the challenge of developing a sensor that can achieve high quantum efficiency and an sufficient signal-to-noise ratio at low photon energies. Here, we present inverse low-gain avalanche diode (iLGAD) sensors integrated with a thin entrance window, demonstrating single-photon resolution down to 390 eV (below the Nitrogen K-edge) with a signal-to-noise ratio of 5. Two gain layer designs, standard gain layer and shallow gain layer, were systematically investigated to evaluate their spectral response as a function of photon energy and operating temperature. A double-peak feature is observed in the energy spectrum, arising from the dependence of charge multiplication on photon absorption depth. Furthermore, a phenomenological spectral fitting model was established, enabling the quantitative extraction of the effective $\mathrm{n}^+$ layer thickness and gain layer depth. Beyond providing a fundamental understanding of iLGAD charge multiplication mechanisms and characterization results in the soft X-ray regime, these findings lay the foundation for developing large-area, high-frame-rate hybrid pixel detectors for soft X-ray. This advancement directly benefits soft X-ray photon science applications, including high-resolution ptychography, spectroscopy, and resonant scattering at synchrotron and FEL beamlines.

physics.ins-det↗

arXiv2026

Enhanced micrometre spatial resolution and usable photon flux for hybrid pixel detectors via deep-learning-based X-ray localization

Micrometre spatial resolution is achieved for X-rays with the MÖNCH detector, a charge-integrating hybrid pixel detector with a 25 μm pixel pitch, by exploiting charge sharing between neighbouring pixels. The conventional analytical interpolation method, however, relies on a global mapping derived from the charge-sharing statistics and is applicable only to isolated single-photon clusters, which limits both the achievable resolution and the usable photon flux. To overcome these limitations, we trained deep learning models to localize single-photon and pile-up events based on high-fidelity simulation data. For knife-edge measurements at 12 keV and a low occupancy of 0.74% (photons per pixel and frame), where single-photon events dominate, the proposed method reaches a of the edge spread function of 1.36 μm, compared with 1.54 μm for the conventional eta interpolation. We demonstrate experimentally that resolving up to three photons per cluster raises the reconstruction efficiency from 59% to 93% at a 2.4{\times} higher occupancy of 1.81%, while preserving a of 1.39 μm.

physics.ins-det↗

arXiv2026

Hybrid Sequence Modeling and Reinforced Verification for Controllable Target-Conditioned Decision Making

Target-conditioned sequence models provide a simple interface for controllable offline decision making, but the requested target return can be an unreliable control signal, especially when the target return lies in underrepresented regions of the dataset. This paper proposes Doctor, a hybrid sequence modeling and reinforced verification framework for controllable target-conditioned offline decision making. Doctor trains a shared masked trajectory Transformer with two complementary objectives: masked trajectory reconstruction for candidate generation and in-sample value learning for action-value verification. At inference time, the model samples multiple nearby target returns, generates candidate actions in parallel, and selects the action whose verified value is closest to the requested target return. We analyze this verifier-guided selection rule and show that its value-level alignment error is bounded by candidate-value coverage around the target return and verifier accuracy. Experiments on D4RL and EpiCare show that Doctor improves target-return alignment under reduced high-return coverage, remains competitive on standard offline return-maximization benchmarks, and enables a single policy to modulate between conservative and aggressive operating points in a simulated clinical decision-making task. These results suggest that reinforced verification can improve the controllability of target-conditioned policies.

cs.LG↗

arXiv2026

ETGL-DDPG: A Deep Deterministic Policy Gradient Algorithm for Sparse Reward Continuous Control

We consider deep deterministic policy gradient (DDPG) in the context of reinforcement learning with sparse rewards. To enhance exploration, we introduce a search procedure, \emph{$ε{t}$-greedy}, which generates exploratory options for exploring less-visited states. We prove that search using $εt$-greedy has polynomial sample complexity under mild MDP assumptions. To more efficiently use the information provided by rewarded transitions, we develop a new dual experience replay buffer framework, \emph{GDRB}, and implement \emph{longest n-step returns}. The resulting algorithm, \emph{ETGL-DDPG}, integrates all three techniques: \bm{$εt$}-greedy, \textbf{G}DRB, and \textbf{L}ongest $n$-step, into DDPG. We evaluate ETGL-DDPG on standard benchmarks and demonstrate that it outperforms DDPG, as well as other state-of-the-art methods, across all tested sparse-reward continuous environments. Ablation studies further highlight how each strategy individually enhances the performance of DDPG in this setting.

cs.LG↗

arXiv2026

Determination of $^{170,172}$Yb($α,n$)$^{173,175}$Hf reaction cross sections in a stacked-target experiment

The ytterbium isotopic chain offers multiple stable isotopes on which cross sections can be measured and insights into the evolution of the $α$ optical-model potential with the neutron-to-proton ratio can be gained. It also includes the $p$ nucleus $^{168}$Yb, the abundance of which is significantly impacted by the $^{164,166}$Yb($α,γ$) reactions. In order to study the $^{170,172}$Yb($α,n$)$^{173,175}$Hf reaction cross sections and compare them with $^{168}$Yb($α,n$)$^{171}$Hf cross sections, that have already been measured, the activation method was used. During irradiation the targets were arranged in stacks of four to reduce the required irradiation time. The average interaction energy inside each ytterbium layer was determined via Geant4 simulations. A manganese layer was used to verify the simulations by comparing the measured $^{55}$Mn($α,(2)n$)$^{57,58}$Co reaction cross sections to previous results. For irradiation the 10 MV FN tandem accelerator located at the University of Cologne was used and the activation measurement was performed utilizing the Cologne Clover Counting setup. For the $^{170}$Yb($α,n$) reaction seven cross sections at center-of-mass energies between 12.7 and 16.5 MeV were measured. For the $^{172}$Yb($α,n$) reaction six cross sections for center-of-mass energies of 13.1 to 16.5 MeV could be determined with an additional upper limit at E$_{c.m.}$ = 12.3 MeV. Comparisons to theoretical models show that state-of-the-art $α$-optical model potentials are able to reproduce the measured cross sections very well. The ratios of ($α, n$) reaction cross sections in the ytterbium isotopic chain can be accurately reproduced as well.

nucl-ex↗

arXiv2025

$β$-DQN: Improving Deep Q-Learning By Evolving the Behavior

While many sophisticated exploration methods have been proposed, their lack of generality and high computational cost often lead researchers to favor simpler methods like $ε$-greedy. Motivated by this, we introduce $β$-DQN, a simple and efficient exploration method that augments the standard DQN with a behavior function $β$. This function estimates the probability that each action has been taken at each state. By leveraging $β$, we generate a population of diverse policies that balance exploration between state-action coverage and overestimation bias correction. An adaptive meta-controller is designed to select an effective policy for each episode, enabling flexible and explainable exploration. $β$-DQN is straightforward to implement and adds minimal computational overhead to the standard DQN. Experiments on both simple and challenging exploration domains show that $β$-DQN outperforms existing baseline methods across a wide range of tasks, providing an effective solution for improving exploration in deep reinforcement learning.

cs.LG↗

arXiv2024

A Distance-based Anomaly Detection Framework for Deep Reinforcement Learning

In deep reinforcement learning (RL) systems, abnormal states pose significant risks by potentially triggering unpredictable behaviors and unsafe actions, thus impeding the deployment of RL systems in real-world scenarios. It is crucial for reliable decision-making systems to have the capability to cast an alert whenever they encounter unfamiliar observations that they are not equipped to handle. In this paper, we propose a novel Mahalanobis distance-based (MD) anomaly detection framework, called \textit{MDX}, for deep RL algorithms. MDX simultaneously addresses random, adversarial, and out-of-distribution (OOD) state outliers in both offline and online settings. It utilizes Mahalanobis distance within class-conditional distributions for each action and operates within a statistical hypothesis testing framework under the Gaussian assumption. We further extend it to robust and distribution-free versions by incorporating Robust MD and conformal inference techniques. Through extensive experiments on classical control environments, Atari games, and autonomous driving scenarios, we demonstrate the effectiveness of our MD-based detection framework. MDX offers a simple, unified, and practical anomaly detection tool for enhancing the safety and reliability of RL systems in real-world applications.

cs.LG↗

arXiv2024

Efficiently Training Neural Networks for Imperfect Information Games by Sampling Information Sets

In imperfect information games, the evaluation of a game state not only depends on the observable world but also relies on hidden parts of the environment. As accessing the obstructed information trivialises state evaluations, one approach to tackle such problems is to estimate the value of the imperfect state as a combination of all states in the information set, i.e., all possible states that are consistent with the current imperfect information. In this work, the goal is to learn a function that maps from the imperfect game information state to its expected value. However, constructing a perfect training set, i.e. an enumeration of the whole information set for numerous imperfect states, is often infeasible. To compute the expected values for an imperfect information game like \textit{Reconnaissance Blind Chess}, one would need to evaluate thousands of chess positions just to obtain the training target for a single state. Still, the expected value of a state can already be approximated with appropriate accuracy from a much smaller set of evaluations. Thus, in this paper, we empirically investigate how a budget of perfect information game evaluations should be distributed among training samples to maximise the return. Our results show that sampling a small number of states, in our experiments roughly 3, for a larger number of separate positions is preferable over repeatedly sampling a smaller quantity of states. Thus, we find that in our case, the quantity of different samples seems to be more important than higher target quality.

cs.AI↗

arXiv2024

Learning With Generalised Card Representations for "Magic: The Gathering"

A defining feature of collectable card games is the deck building process prior to actual gameplay, in which players form their decks according to some restrictions. Learning to build decks is difficult for players and models alike due to the large card variety and highly complex semantics, as well as requiring meaningful card and deck representations when aiming to utilise AI. In addition, regular releases of new card sets lead to unforeseeable fluctuations in the available card pool, thus affecting possible deck configurations and requiring continuous updates. Previous Game AI approaches to building decks have often been limited to fixed sets of possible cards, which greatly limits their utility in practice. In this work, we explore possible card representations that generalise to unseen cards, thus greatly extending the real-world utility of AI-based deck building for the game "Magic: The Gathering".We study such representations based on numerical, nominal, and text-based features of cards, card images, and meta information about card usage from third-party services. Our results show that while the particular choice of generalised input representation has little effect on learning to predict human card selections among known cards, the performance on new, unseen cards can be greatly improved. Our generalised model is able to predict 55\% of human choices on completely unseen cards, thus showing a deep understanding of card quality and strategy.

cs.AI↗

arXiv2024

Contrastive Learning of Preferences with a Contextual InfoNCE Loss

A common problem in contextual preference ranking is that a single preferred action is compared against several choices, thereby blowing up the complexity and skewing the preference distribution. In this work, we show how one can solve this problem via a suitable adaptation of the CLIP framework.This adaptation is not entirely straight-forward, because although the InfoNCE loss used by CLIP has achieved great success in computer vision and multi-modal domains, its batch-construction technique requires the ability to compare arbitrary items, and is not well-defined if one item has multiple positive associations in the same batch. We empirically demonstrate the utility of our adapted version of the InfoNCE loss in the domain of collectable card games, where we aim to learn an embedding space that captures the associations between single cards and whole card pools based on human selections. Such selection data only exists for restricted choices, thus generating concrete preferences of one item over a set of other items rather than a perfect fit between the card and the pool. Our results show that vanilla CLIP does not perform well due to the aforementioned intuitive issues. However, by adapting CLIP to the problem, we receive a model outperforming previous work trained with the triplet loss, while also alleviating problems associated with mining triplets.

cs.AI↗

arXiv2024

Neural Network-based Information Set Weighting for Playing Reconnaissance Blind Chess

In imperfect information games, the game state is generally not fully observable to players. Therefore, good gameplay requires policies that deal with the different information that is hidden from each player. To combat this, effective algorithms often reason about information sets; the sets of all possible game states that are consistent with a player's observations. While there is no way to distinguish between the states within an information set, this property does not imply that all states are equally likely to occur in play. We extend previous research on assigning weights to the states in an information set in order to facilitate better gameplay in the imperfect information game of Reconnaissance Blind Chess. For this, we train two different neural networks which estimate the likelihood of each state in an information set from historical game data. Experimentally, we find that a Siamese neural network is able to achieve higher accuracy and is more efficient than a classical convolutional neural network for the given domain. Finally, we evaluate an RBC-playing agent that is based on the generated weightings and compare different parameter settings that influence how strongly it should rely on them. The resulting best player is ranked 5th on the public leaderboard.

cs.AI↗

arXiv2024

Expected Work Search: Combining Win Rate and Proof Size Estimation

We propose Expected Work Search (EWS), a new game solving algorithm. EWS combines win rate estimation, as used in Monte Carlo Tree Search, with proof size estimation, as used in Proof Number Search. The search efficiency of EWS stems from minimizing a novel notion of Expected Work, which predicts the expected computation required to solve a position. EWS outperforms traditional solving algorithms on the games of Go and Hex. For Go, we present the first solution to the empty 5x5 board with the commonly used positional superko ruleset. For Hex, our algorithm solves the empty 8x8 board in under 4 minutes. Experiments show that EWS succeeds both with and without extensive domain-specific knowledge.

cs.AI↗

arXiv2023

Monte Carlo Tree Search in the Presence of Transition Uncertainty

Monte Carlo Tree Search (MCTS) is an immensely popular search-based framework used for decision making. It is traditionally applied to domains where a perfect simulation model of the environment is available. We study and improve MCTS in the context where the environment model is given but imperfect. We show that the discrepancy between the model and the actual environment can lead to significant performance degradation with standard MCTS. We therefore develop Uncertainty Adapted MCTS (UA-MCTS), a more robust algorithm within the MCTS framework. We estimate the transition uncertainty in the given model, and direct the search towards more certain transitions in the state space. We modify all four MCTS phases to improve the search behavior by considering these estimates. We prove, in the corrupted bandit case, that adding uncertainty information to adapt UCB leads to tighter regret bound than standard UCB. Empirically, we evaluate UA-MCTS and its individual components on the deterministic domains from the MinAtar test suite. Our results demonstrate that UA-MCTS strongly improves MCTS in the presence of model transition errors.

cs.AI↗

arXiv2022

Efficient Sampling-Based Bayesian Active Learning for synaptic characterization

Bayesian Active Learning (BAL) is an efficient framework for learning the parameters of a model, in which input stimuli are selected to maximize the mutual information between the observations and the unknown parameters. However, the applicability of BAL to experiments is limited as it requires performing high-dimensional integrations and optimizations in real time: current methods are either too time consuming, or only applicable to specific models. Here, we propose an Efficient Sampling-Based Bayesian Active Learning (ESB-BAL) framework, which is efficient enough to be used in real-time biological experiments. We apply our method to the problem of estimating the parameters of a chemical synapse from the postsynaptic responses to evoked presynaptic action potentials. Using synthetic data and synaptic whole-cell patch-clamp recordings, we show that our method can improve the precision of model-based inferences, thereby paving the way towards more systematic and efficient experimental designs in physiology.

q-bio.QM↗

arXiv2022

Supervised and Reinforcement Learning from Observations in Reconnaissance Blind Chess

In this work, we adapt a training approach inspired by the original AlphaGo system to play the imperfect information game of Reconnaissance Blind Chess. Using only the observations instead of a full description of the game state, we first train a supervised agent on publicly available game records. Next, we increase the performance of the agent through self-play with the on-policy reinforcement learning algorithm Proximal Policy Optimization. We do not use any search to avoid problems caused by the partial observability of game states and only use the policy network to generate moves when playing. With this approach, we achieve an ELO of 1330 on the RBC leaderboard, which places our agent at position 27 at the time of this writing. We see that self-play significantly improves performance and that the agent plays acceptably well without search and without making assumptions about the true game state.

cs.AI↗

arXiv2022

A Comparison of Contextual and Non-Contextual Preference Ranking for Set Addition Problems

In this paper, we study the problem of evaluating the addition of elements to a set. This problem is difficult, because it can, in the general case, not be reduced to unconditional preferences between the choices. Therefore, we model preferences based on the context of the decision. We discuss and compare two different Siamese network architectures for this task: a twin network that compares the two sets resulting after the addition, and a triplet network that models the contribution of each candidate to the existing set. We evaluate the two settings on a real-world task; learning human card preferences for deck building in the collectible card game Magic: The Gathering. We show that the triplet approach achieves a better result than the twin network and that both outperform previous results on this task.

cs.AI↗

arXiv2022

Quantity vs Quality: Investigating the Trade-Off between Sample Size and Label Reliability

In this paper, we study learning in probabilistic domains where the learner may receive incorrect labels but can improve the reliability of labels by repeatedly sampling them. In such a setting, one faces the problem of whether the fixed budget for obtaining training examples should rather be used for obtaining all different examples or for improving the label quality of a smaller number of examples by re-sampling their labels. We motivate this problem in an application to compare the strength of poker hands where the training signal depends on the hidden community cards, and then study it in depth in an artificial setting where we insert controlled noise levels into the MNIST database. Our results show that with increasing levels of noise, resampling previous examples becomes increasingly more important than obtaining new examples, as classifier performance deteriorates when the number of incorrect labels is too high. In addition, we propose two different validation strategies; switching from lower to higher validations over the course of training and using chi-square statistics to approximate the confidence in obtained labels.

cs.LG↗

arXiv2022

Cedille: A large autoregressive French language model

Scaling up the size and training of autoregressive language models has enabled novel ways of solving Natural Language Processing tasks using zero-shot and few-shot learning. While extreme-scale language models such as GPT-3 offer multilingual capabilities, zero-shot learning for languages other than English remain largely unexplored. Here, we introduce Cedille, a large open source auto-regressive language model, specifically trained for the French language. Our results show that Cedille outperforms existing French language models and is competitive with GPT-3 on a range of French zero-shot benchmarks. Furthermore, we provide an in-depth comparison of the toxicity exhibited by these models, showing that Cedille marks an improvement in language model safety thanks to dataset filtering.

cs.CL↗