arXiv subjects

Johann Schmidt

Publications and source records attributed to Johann Schmidt.

12 recordsLinked to original sources

arXiv2026

Geometrically Constrained and Token-Based Probabilistic Spatial Transformers

Spatial transformations such as rotation and scale obscure the morphological cues needed for accurate image classification. Careful consideration is required for reliable use in high stakes settings. A model should stay robust under such transformations, expose why a correction was applied, and signal when its input is ambiguous. While geometrically equivariant architectures provide a mathematically grounded solution, they often limit model flexibility through strict symmetry constraints and incur significant computational overhead. Spatial Transformer Networks (STNs) offer a data-driven, flexible alternative for learning pseudo-equivariances to affine transformations. However, STNs have historically been restricted to convolutional architectures and suffer from training instability. To address this, we introduce a novel STN framework. It leverages the global modeling capabilities of transformers to regress the affine transformation acting on the input. For this, we decompose affine transformations into interpretable primitives, regressed under adaptable geometric constraints, thereby preventing the training instability typically caused by degenerate transformations. By sharing weights between the localization network and the classification backbone, the framework requires minimal computational overhead. Extensive experiments on challenging insect biodiversity and medical imaging benchmarks demonstrate that our approach achieves superior predictive performance under diverse spatial transformations while maintaining high efficiency. Code is available at https://github.com/johSchm/TokenSTN.

cs.CV↗

arXiv2026

Zero-Shot Test-Time Canonicalization using Out-of-Distribution Scoring

Pretrained vision models often misclassify inputs that are rotated, scaled, or sheared, even though these affine transformations leave the object class unchanged. Robustness is usually restored either by building equivariance into the architecture or by retraining with augmentation, both of which require changing or retraining the model. Test-time canonicalization instead leaves the classifier untouched. It undoes the transformation of each input, mapping it to a canonical form near the training distribution before classification. Existing canonicalizers, however, rely on a narrow set of logit-based energy scores and bespoke search procedures, leaving the design space of scoring functions and optimizers unexplored. We reframe canonicalization as out-of-distribution (OOD) detection, which lets any OOD score serve as the energy minimized over transformations. Across benchmarks ranging from handwritten characters and sketches to natural images and 3D point clouds, we systematically evaluate around twenty OOD scores and nine search algorithms, finding that distance-based scores paired with random search and local refinement perform best overall. Because canonicalizing an already-aligned input can hurt accuracy, we add a gated mechanism that transforms an input only when its OOD score indicates this is needed, preserving most in-distribution accuracy while retaining the robustness gains on transformed inputs. Code is available at github.com/johschm/its.

cs.CV↗

arXiv2025

Robust Canonicalization through Bootstrapped Data Re-Alignment

Fine-grained visual classification (FGVC) tasks, such as insect and bird identification, demand sensitivity to subtle visual cues while remaining robust to spatial transformations. A key challenge is handling geometric biases and noise, such as different orientations and scales of objects. Existing remedies rely on heavy data augmentation, which demands powerful models, or on equivariant architectures, which constrain expressivity and add cost. Canonicalization offers an alternative by shielding such biases from the downstream model. In practice, such functions are often obtained using canonicalization priors, which assume aligned training data. Unfortunately, real-world datasets never fulfill this assumption, causing the obtained canonicalizer to be brittle. We propose a bootstrapping algorithm that iteratively re-aligns training samples by progressively reducing variance and recovering the alignment assumption. We establish convergence guarantees under mild conditions for arbitrary compact groups, and show on four FGVC benchmarks that our method consistently outperforms equivariant, and canonicalization baselines while performing on par with augmentation.

cs.CV↗

arXiv2025

Saccadic Vision for Fine-Grained Visual Classification

Fine-grained visual classification (FGVC) requires distinguishing between visually similar categories through subtle, localized features - a task that remains challenging due to high intra-class variability and limited inter-class differences. Existing part-based methods often rely on complex localization networks that learn mappings from pixel to sample space, requiring a deep understanding of image content while limiting feature utility for downstream tasks. In addition, sampled points frequently suffer from high spatial redundancy, making it difficult to quantify the optimal number of required parts. Inspired by human saccadic vision, we propose a two-stage process that first extracts peripheral features (coarse view) and generates a sample map, from which fixation patches are sampled and encoded in parallel using a weight-shared encoder. We employ contextualized selective attention to weigh the impact of each fixation patch before fusing peripheral and focus representations. To prevent spatial collapse - a common issue in part-based methods - we utilize non-maximum suppression during fixation sampling to eliminate redundancy. Comprehensive evaluation on standard FGVC benchmarks (CUB-200-2011, NABirds, Food-101 and Stanford-Dogs) and challenging insect datasets (EU-Moths, Ecuador-Moths and AMI-Moths) demonstrates that our method achieves comparable performance to state-of-the-art approaches while consistently outperforming our baseline encoder.

cs.CV↗

arXiv2024

TransferLight: Zero-Shot Traffic Signal Control on any Road-Network

Traffic signal control plays a crucial role in urban mobility. However, existing methods often struggle to generalize beyond their training environments to unseen scenarios with varying traffic dynamics. We present TransferLight, a novel framework designed for robust generalization across road-networks, diverse traffic conditions and intersection geometries. At its core, we propose a log-distance reward function, offering spatially-aware signal prioritization while remaining adaptable to varied lane configurations - overcoming the limitations of traditional pressure-based rewards. Our hierarchical, heterogeneous, and directed graph neural network architecture effectively captures granular traffic dynamics, enabling transferability to arbitrary intersection layouts. Using a decentralized multi-agent approach, global rewards, and novel state transition priors, we develop a single, weight-tied policy that scales zero-shot to any road network without re-training. Through domain randomization during training, we additionally enhance generalization capabilities. Experimental results validate TransferLight's superior performance in unseen scenarios, advancing practical, generalizable intelligent transportation systems to meet evolving urban traffic demands.

cs.AI↗

arXiv2024

Tilt your Head: Activating the Hidden Spatial-Invariance of Classifiers

Deep neural networks are applied in more and more areas of everyday life. However, they still lack essential abilities, such as robustly dealing with spatially transformed input signals. Approaches to mitigate this severe robustness issue are limited to two pathways: Either models are implicitly regularised by increased sample variability (data augmentation) or explicitly constrained by hard-coded inductive biases. The limiting factor of the former is the size of the data space, which renders sufficient sample coverage intractable. The latter is limited by the engineering effort required to develop such inductive biases for every possible scenario. Instead, we take inspiration from human behaviour, where percepts are modified by mental or physical actions during inference. We propose a novel technique to emulate such an inference process for neural nets. This is achieved by traversing a sparsified inverse transformation tree during inference using parallel energy-based evaluations. Our proposed inference algorithm, called Inverse Transformation Search (ITS), is model-agnostic and equips the model with zero-shot pseudo-invariance to spatially transformed inputs. We evaluated our method on several benchmark datasets, including a synthesised ImageNet test set. ITS outperforms the utilised baselines on all zero-shot test scenarios.

cs.LG↗

arXiv2023

Learning Continuous Rotation Canonicalization with Radial Beam Sampling

Nearly all state of the art vision models are sensitive to image rotations. Existing methods often compensate for missing inductive biases by using augmented training data to learn pseudo-invariances. Alongside the resource demanding data inflation process, predictions often poorly generalize. The inductive biases inherent to convolutional neural networks allow for translation equivariance through kernels acting parallely to the horizontal and vertical axes of the pixel grid. This inductive bias, however, does not allow for rotation equivariance. We propose a radial beam sampling strategy along with radial kernels operating on these beams to inherently incorporate center-rotation covariance. Together with an angle distance loss, we present a radial beam-based image canonicalization model, short BIC. Our model allows for maximal continuous angle regression and canonicalizes arbitrary center-rotated input images. As a pre-processing model, this enables rotation-invariant vision pipelines with model-agnostic rotation-sensitive downstream predictions. We show that our end-to-end trained angle regressor is able to predict continuous rotation angles on several vision datasets, i.e. FashionMNIST, CIFAR10, COIL100, and LFW.

cs.CV↗

arXiv2022

Nematic superconductivity in magic-angle twisted bilayer graphene from atomistic modeling

Twisted bilayer graphene (TBG) develops large moiré patterns at small twist angles with flat energy bands hosting domes of superconductivity. The large system size and intricate band structure have however hampered investigations into the superconducting state. Here, using full-scale atomistic modelling with local electronic interactions, we find at and above experimentally relevant temperatures a highly inhomogeneous superconducting state with nematic ordering on both atomic and moiré length scales. The nematic state has a locally anisotropic real-valued d-wave pairing, with a nematic vector winding throughout the moiré pattern, and is three-fold degenerate. Although d-wave symmetric, the superconducting state has a full energy gap, which we tie to a π-phase interlayer coupling. The superconducting nematicity is further directly detectable in the local density of states. Our results show that atomistic modeling is essential and also that very similar local interactions produce very different superconducting states in TBG and the high-temperature cuprate superconductors.

cond-mat.supr-con↗

arXiv2020

Odd-frequency superconductivity and Meissner effect in the doped topological insulator Bi$_2$Se$_3$

Doped Bi$_2$Se$_3$ is proposed to be a nematic superconductor originating from unusual inter-orbital pairing. We classify all induced superconducting pair correlations in Bi$_2$Se$_3$ and discover that intra-orbital odd-frequency pairing dominates over a significant range of frequencies. Moreover, we explore the contributions of even- and odd-frequency pairing to the Meissner effect, including separating intra- and inter-band processes in the response function. Contrary to expectations, and due to inter-band contributions, we find a diamagnetic odd-frequency Meissner effect.

cond-mat.supr-con↗

arXiv2018

Topological nodal superconducting phases and topological phase transition in the hyperhoneycomb lattice

We establish the topology of the spin-singlet superconducting states in the bare hyperhoneycomb lattice and derive analytically the full phase diagram using only symmetry and topology in combination with simple energy arguments. The phase diagram is dominated by two states preserving time-reversal symmetry. We find that the line-nodal state dominating at low doping levels is topologically nontrivial and exhibits surface Majorana flat bands, which we show perfectly match the bulk-boundary correspondence using Berry phase approach. At higher doping levels we find a fully gapped state with trivial topology. By analytically calculating the topological invariant of the line nodes, we derive the critical point between the line-nodal and fully gapped states as a function of both pairing parameters and doping. We find that the line-nodal state is favored not only at lower doping levels but also if symmetry-allowed deformations of the lattice is present. Adding simple energy arguments we establish that a fully gapped state with broken time-reversal symmetry likely appears covering the actual phase transition. We find this time-reversal symmetry broken state to be topologically trivial, while we find an additional point nodal state at very low doping levels to have nontrivial topology with associated Fermi surface arcs. We eventually address the robustness of the phase diagram to generalized models also including adiabatic spin-orbit coupling, and show how all but the point nodal state are reasonably stable.

cond-mat.supr-con↗

arXiv2017

Topological superconductivity in the extended Kitaev-Heisenberg model

We study superconducting pairing in the doped Kitaev-Heisenberg model by taking into account the recently proposed symmetric off-diagonal exchange $Γ$. By performing a mean-field analysis, we classify all possible superconducting phases in terms of symmetry, explicitly taking into account effects of spin-orbit coupling. Solving the resulting gap equations self-consistently, we map out a phase diagram that involves several topologically nontrivial states. For $Γ<0$, we find a competition between a time-reversal symmetry breaking chiral phase with Chern number $\pm1$ and a time-reversal symmetric nematic phase that breaks the rotational symmetry of the lattice. On the other hand, for $Γ\geq 0$ we find a time-reversal symmetric phase that preserves all the lattice symmetries, thus yielding clearly distinguishable experimental signatures for all superconducting phases. Both of the time-reversal symmetric phases display a transition to a $\mathbb{Z}_2$ non-trivial phase at high doping levels. Finally, we also include a symmetry-allowed spin-orbit coupling kinetic energy and show that it destroys a tentative symmetry protected topological order at lower doping levels. However, it can be used to tune the time-reversal symmetric phases into a $\mathbb{Z}_2$ non-trivial phase even at lower doping.

cond-mat.supr-con↗

arXiv2016

From chiral d-wave to nodal line superconductivity in the harmonic honeycomb lattices

Motivated by the recent realization of the three-dimensional hyperhoneycomb and stripyhoneycomb lattices in lithium iridate (Li$_2$IrO$_3$), we study the possible spin-singlet superconducting states on the whole series of harmonic honeycomb lattices. Beginning with an isolated out-of-plane twist making the honeycomb lattice three-dimensional, we find that the chiral d$\pm$id state, well-known from the honeycomb lattice, is realized in the largest members of the series at low to intermediate doping. Along the twist, four chiral edge states form a two-dimensional dispersive band. Reducing the distance between the twists to form the smaller members of the harmonic honeycomb lattices, the degeneracy between the d-wave states is lifted, which finally destroys the chiral state. By analyzing the hyper- and stripyhoneycomb lattices and generalizing using the $D_{2h}$ point group of all the harmonic honeycomb lattices, we show that the superconducting state often belongs to the trivial irreducible representation. This state has nodal lines at low to intermediate doping, which is possible because the full lattice symmetry allows sign changes between different sets of bonds. We also find time-reversal symmetry broken states, which are either fully gapped or feature nodal points, in certain parts of the phase diagram. Finally, we draw a comparison between the states classified in terms of the $D_{2h}$ symmetries and those observed on the $D_{6h}$ honeycomb lattice.

cond-mat.supr-con↗