arXiv subjects

John Bowen

Publications and source records attributed to John Bowen.

4 recordsLinked to original sources

arXiv2025

Token Masking Improves Transformer-Based Text Classification

While transformer-based models achieve strong performance on text classification, we explore whether masking input tokens can further enhance their effectiveness. We propose token masking regularization, a simple yet theoretically motivated method that randomly replaces input tokens with a special [MASK] token at probability p. This introduces stochastic perturbations during training, leading to implicit gradient averaging that encourages the model to capture deeper inter-token dependencies. Experiments on language identification and sentiment analysis -- across diverse models (mBERT, Qwen2.5-0.5B, TinyLlama-1.1B) -- show consistent improvements over standard regularization techniques. We identify task-specific optimal masking rates, with p = 0.1 as a strong general default. We attribute the gains to two key effects: (1) input perturbation reduces overfitting, and (2) gradient-level smoothing acts as implicit ensembling.

cs.CL↗

arXiv2018

Quantum ground state isoperimetric inequalities for the energy spectrum of local Hamiltonians

We investigate the relationship between the energy spectrum of a local Hamiltonian and the geometric properties of its ground state. By generalizing a standard framework from the analysis of Markov chains to arbitrary (non-stoquastic) Hamiltonians we are naturally led to see that the spectral gap can always be upper bounded by an isoperimetric ratio that depends only on the ground state probability distribution and the range of the terms in the Hamiltonian, but not on any other details of the interaction couplings. This means that for a given probability distribution the inequality constrains the spectral gap of any local Hamiltonian with this distribution as its ground state probability distribution in some basis (Eldar and Harrow derived a similar result in order to characterize the output of low-depth quantum circuits). Going further, we relate the Hilbert space localization properties of the ground state to higher energy eigenvalues by showing that the presence of k strongly localized ground state modes (i.e. clusters of probability, or subsets with small expansion) in Hilbert space implies the presence of k energy eigenvalues that are close to the ground state energy. Our results suggest that quantum adiabatic optimization using local Hamiltonians will inevitably encounter small spectral gaps when attempting to prepare ground states corresponding to multi-modal probability distributions with strongly localized modes, and this problem cannot necessarily be alleviated with the inclusion of non-stoquastic couplings.

quant-ph↗

arXiv2018

Quantum Error Correcting Codes in Eigenstates of Translation-Invariant Spin Chains

Quantum error correction was invented to allow for fault-tolerant quantum computation. Systems with topological order turned out to give a natural physical realization of quantum error correcting codes (QECC) in their groundspaces. More recently, in the context of the AdS/CFT correspondence, it has been argued that eigenstates of CFTs with a holographic dual should also form QECCs. These two examples raise the question of how generally eigenstates of many-body models form quantum codes. In this work we establish new connections between quantum chaos and translation-invariance in many-body spin systems, on one hand, and approximate quantum error correcting codes (AQECC), on the other hand. We first observe that quantum chaotic systems exhibiting the Eigenstate Thermalization Hypothesis (ETH) have eigenstates forming approximate quantum error-correcting codes. Then we show that AQECC can be obtained probabilistically from translation-invariant energy eigenstates of every translation-invariant spin chain, including integrable models. Applying this result to 1D classical systems, we describe a method for using local symmetries to construct parent Hamiltonians that embed these codes into the low-energy subspace of gapless 1D quantum spin chains. As explicit examples we obtain local AQECC in the ground space of the 1D ferromagnetic Heisenberg model and the Motzkin spin chain model with periodic boundary conditions, thereby yielding non-stabilizer codes in the ground space and low energy subspace of physically plausible 1D gapless models.

quant-ph↗

arXiv2016

The Australian Space Eye: studying the history of galaxy formation with a CubeSat

The Australian Space Eye is a proposed astronomical telescope based on a 6U CubeSat platform. The Space Eye will exploit the low level of systematic errors achievable with a small space based telescope to enable high accuracy measurements of the optical extragalactic background light and low surface brightness emission around nearby galaxies. This project is also a demonstrator for several technologies with general applicability to astronomical observations from nanosatellites. Space Eye is based around a 90 mm aperture clear aperture all refractive telescope for broadband wide field imaging in the i and z bands.

astro-ph.IM↗