arXiv subjects

Tim Palmer

Publications and source records attributed to Tim Palmer.

17 recordsLinked to original sources

arXiv2026

Solving the Mysteries of Quantum Mechanics: Why Nature Abhors a Continuum

Feynman famously asserted that interference is the only real mystery in quantum mechanics (QM). It is concluded that the reason for this mystery, and thereby the related mysteries of complementarity, non-commutativity of observables, the uncertainty principle and violation of Bell's equality, is that the axioms of QM depend vitally on the continuum nature of Hilbert Space, deemed unphysical. We develop a theory of quantum physics - Rational Quantum Mechanics (RaQM) - in which Hilbert Space is gravitationally discretised. The key to solving the mysteries of QM in RaQM is a number-theoretic property of the cosine function, concealed in QM when angles range over the continuum. This number-theoretic property describes mathematically the utter indivisibility of the quantum world and implies that the laws of physics are profoundly holistic. We contrast holism with nonlocality. In theories which embrace the continuum, the violation of Bell's inequality requires the laws of physics to be either nonlocal or not realistic; both incomprehensible concepts. By contrast, holism, as embodied in Mach's Principle or in the fractal geometry of a chaotic attractor, is neither incomprehensible nor unphysical. As part of this, we solve the deepest mystery of all; why nature makes use of complex numbers.

quant-ph↗

arXiv2026

Impossible Counterfactuals, Discrete Hilbert Space and Bell's Theorem

Negating the Measurement Independence assumption (MI) is often referred to as the `third way' to account for the experimental violation of Bell's inequality. However, this route is generally viewed as ludicrously contrived, implying some implausible conspiracy where experimenters are denied the freedom to choose measurement settings as they like. Here, a locally realistic model of quantum physics is developed (Rational Mechanics - RaQM - based on a gravitational discretisation of Hilbert Space) which violates MI without denying free will. Crucially, RaQM distinguishes experimenters' ability to freely choose measurement settings to some nominal accuracy, from an inability to choose exact settings, which were never under their control anyway. In RaQM, Hilbert states are necessarily undefined in bases where squared amplitudes and/or complex phases are irrational numbers. Such `irrational' bases correspond to conceivable but necessarily impossible counterfactual measurements, and are shown to play a ubiquitous role in the analysis of both single- and entangled-particle quantum physics. It is concluded that violation of Bell inequalities can be understood with none of the strange processes historically associated with it. Instead, using concepts from (non-classical) $p$-adic number theory, we relate RaQM to Bohm and Hiley's concept of a holistic Machian-like Undivided Universe. If this interpretation of Bell's Theorem is correct, building more and more energetic particle accelerators to probe smaller and smaller scales, in the search for a theory which synthesises quantum and gravitational physics and hence a Theory of Everything, may be a fruitless exercise.

quant-ph↗

arXiv2025

Improved rainfall forecasts in daily use over East Africa

Ensemble forecasting has proven to be a vital tool for predicting extreme, life-threatening or only partially predictable weather events. As well as providing probabilistic products, individual ensemble members provide context and realisations of possible extreme weather. However, many National Meteorological Services in East Africa do not have the computing resources to enable them to run their local area models in ensemble mode over the full period of the two-week medium range. In this paper we test the performance of a forecast system, comprising the global ECMWF ensemble forecast, post-processed using cGAN, a neural network model, and forecasts calibrated using IDR, a recent statistical method, against the probabilistic climatology. cGAN provides comparable levels of probabilistic skill as IDR applied to the ECMWF ensemble or IDR applied to the machine-learning models FuXi and GraphCast. These methods improve the raw ECMWF ensemble forecast substantially, which is itself an improvement on deterministic forecasts. The ability of cGAN to produce individual realisations of future rainfall was important when assessed in an operational context by the National Meteorological and Hydrological Services of Kenya and Ethiopia. Moreover, in common with IDR, cGAN is cheap to train/run and requires no additional post-processing. It is run on laptops to generate many thousands of ensemble members, making it suitable for Meteorological Services with limited computational facilities.

physics.ao-ph↗

arXiv2025

Rational Quantum Mechanics: Testing Quantum Theory with Quantum Computers

Motivated in part by John Wheeler's assertion that the continuum nature of Hilbert Space conceals the `it-from-bit' information-theoretic character of the quantum wavefunction, a theory of quantum physics (Rational Quantum Mechanics - RaQM) is proposed based on a specific discretisation of complex Hilbert Space. The Schr\"{o}dinger equation is not modified in RaQM, even during measurement. However, the bases in which the quantum state is defined must satisfy certain rational-number constraints. These constraints lead to the notion of finite qubit information capacity $N_{\mathrm{max}}$: for any $N > N_{\mathrm{max}}$ qubit state, there is insufficient information in the $N$ qubits (linearly growing in $N$) to allocate even one bit to each of all $2^{N+1}-2$ continuum degrees of freedom (exponentially growing in $N$) associated with quantum mechanics/theory (QM, where $N_{\mathrm{max}}=\infty$). It is proposed that the discretisation of Hilbert Space in RaQM is due to gravity, hence QM is the (singular) continuum limit of RaQM at $G=0$. On this basis, it is estimated that $N_{\mathrm{max}}$ lies between about 200 and 400 for current qubit technologies, and will never exceed 1,000. Whilst QM and RaQM are experimentally indistinguishable for small numbers of qubits, RaQM predicts that the exponential advantage of quantum algorithms which, like Shor's, require bases with maximal $N$-qubit superposition/entanglement, will have saturated at 1,000 perfect qubits. Hence, insofar as a classical computer will never factor a 2048-bit RSA integer, RaQM predicts that a quantum computer won't either. This predicted breakdown of QM could be testable in less than 5 years.

quant-ph↗

arXiv2025

A Deterministic Model of Free Will

The issue of whether we make decisions freely has vexed philosophers for millennia, Resolving this is vital for solving a diverse range of problems, from the physiology of how the brain makes decisions (and how we assign moral responsibility to those decisions) to the interpretation of experiments on entangled quantum particles. A deterministic model of free will is developed, based on two concepts. The first generalises the notion of initialisation of nonlinear systems where information cascades upscale from the Planck scale, exemplified by the chaology of colliding billiard balls, and featured in the author's Rational Quantum Mechanics. With `just-in-time' initialisation, such Planck-scale information is only initialised when it is needed to describe super-Planck scale evolution, and not e.g., at the time of the Big Bang. In this way determinism does not imply predestination and a system with finitely many degrees of freedom can shadow a system with infinitely many, over arbitrarily long timescales. The second concept describes the upscale control of such Planck-scale information on super-Planck scales and is illustrated by reference to stochastic rounding in numerical analysis. Using these concepts, a deterministic model is proposed whereby freely-made decisions are made by using past experiences to control the impact of noise in the low-energy brain. It is claimed that such a model has evolutionary advantages, not least preventing paralysis by analysis and encouraging rational risk taking. It is concluded that humans have free will, determinism notwithstanding. The model is applied to study the foundational issue of free choice in quantum physics experiments: it is shown that violating the Measurement Independence assumption does not invalidate the free-will conclusion above.

physics.hist-ph↗

arXiv2023

Superdeterminism Without Conspiracy

Superdeterminism - where the Measurement Independence assumption in Bell's Theorem is violated - is frequently assumed to imply implausibly conspiratorial correlations between properties $\lambda$ of particles being measured and measurement settings $x$ and $y$. But it doesn't have to be: a superdeterministic but non-conspiratorial locally causal model is developed where each pair of entangled particles has unique $\lambda$. The model is based on a specific but arbitrarily fine discretisation of complex Hilbert space, where $\lambda$ defines the information, over and above the freely chosen nominal settings $x$ and $y$, which fixes the exact measurement settings $X$ and $Y$ of a run of a Bell experiment. Pearlean interventions, needed to assess whether $x$ and $y$ are Bell-type free variables, are shown to be inconsistent with rational-number constraints on the discretised Hilbert states. These constraints limit the post-hoc freedom to vary $x$ keeping $\lambda$ and $y$ fixed but disappear with any coarse-graining of $\lambda$, $X$ and $Y$, rendering so-called drug-trial conspiracies irrelevant. Points in the discretised space can be realised as ensembles of symbolically labelled deterministic trajectories on an `all-at-once' fractal attractor. It is shown how quantum mechanics might be `gloriously explained and derived' as the singular continuum limit of the discretisation of Hilbert space; It is argued that the real message behind Bell's Theorem has less to do with locality, realism or freedom to choose, and more to do with the need to develop more explicitly holistic theories when attempting to synthesise quantum and gravitational physics.

quant-ph↗

arXiv2022

Quantum Computers for Weather and Climate Prediction: The Good, the Bad and the Noisy

Over the past few years, quantum computers and quantum algorithms have attracted considerable interest and attention from numerous scientific disciplines. In this article, we aim to provide a non-technical, yet informative introduction to key aspects of quantum computing. We discuss whether quantum computers one day might become useful tools for numerical weather and climate prediction. Using a recently developed quantum algorithm for solving non-linear differential equations, we integrate a simple non-linear model. In addition to considering the advantages that quantum computers have to offer, we shall also discuss the challenges one faces when trying to use quantum computers for real-world problems involving ''big data'', such as weather prediction.

quant-ph↗

arXiv2022

Deep learning for quality control of surface physiographic fields using satellite Earth observations

A purposely built deep learning algorithm for the Verification of Earth-System ParametERisation (VESPER) is used to assess recent upgrades of the global physiographic datasets underpinning the quality of the Integrated Forecasting System (IFS) of the European Centre for Medium-Range Weather Forecasts (ECMWF), which is used both in numerical weather prediction and climate reanalyses. A neural network regression model is trained to learn the mapping between the surface physiographic dataset plus the meteorology from ERA5, and the MODIS satellite skin temperature observations. Once trained, this tool is applied to rapidly assess the quality of upgrades of the land-surface scheme. Upgrades which improve the prediction accuracy of the machine learning tool indicate a reduction of the errors in the surface fields used as input to the surface parametrisation schemes. Conversely, incorrect specifications of the surface fields decrease the accuracy with which VESPER can make predictions. We apply VESPER to assess the accuracy of recent upgrades of the permanent lake and glaciers covers as well as planned upgrades to represent seasonally varying water bodies (i.e. ephemeral lakes). We show that for grid-cells where the lake fields have been updated, the prediction accuracy in the land surface temperature (i.e mean absolute error difference between updated and original physiographic datasets) improves by 0.37 K on average, whilst for the subset of points where the lakes have been exchanged for bare ground (or vice versa) the improvement is 0.83 K. We also show that updates to the glacier cover improve the prediction accuracy by 0.22 K. We highlight how neural networks such as VESPER can assist the research and development of surface parameterizations and their input physiography to better represent Earth's surface couples processes in weather and climate models.

physics.ao-ph↗

arXiv2022

Quantum Physics from Number Theory

The properties which give quantum mechanics its unique character - unitarity, complementarity, non-commutativity, uncertainty, nonlocality - derive from the algebraic structure of Hermitian operators acting on the wavefunction in complex Hilbert space. Because of this, the wavefunction cannot be shown to describe an ensemble of deterministic states where uncertainty simply reflects a lack of knowledge about which ensemble member describes reality. This has led to endless debates about the ontology of quantum mechanics. Here we derive these same quantum properties from number theoretic attributes of trigonometric functions applied to an explicitly ensemble-based representation of discretised complex Hilbert states. To avoid fine-tuning, the metric on state space must be $p$-adic rather than Euclidean where $1/p$ determines the fineness of the discretisation. This hints at both the existence of an underpinning fractal state-space geometry onto which states of the world are constrained. In this model, violation of Bell inequalities is a manifestation of this geometric constraint and does not imply a breakdown of local space-time causality. Because the discretised wavefunction describes an ensemble of states, there is no collapse of the wavefunction. Instead measurement describes a nonlinear clustering of state-space trajectories on the state-space geometry. In this model, systems with mass greater than the Planck mass will not exhibit quantum properties and instead behave classically. The geometric constraint suggests that the exponential increase in the size of state space with qubit number may break down with qubit numbers as small as a few hundred. Quantum mechanics is itself a singular limit of this number-theoretic model at $p=\infty$. A modification of general relativity, consistent with this discretised model of quantum physics, is proposed.

quant-ph↗

arXiv2022

Climate Change Modelling at Reduced Float Precision with Stochastic Rounding

Reduced precision floating point arithmetic is now routinely deployed in numerical weather forecasting over short timescales. However the applicability of these reduced precision techniques to longer timescale climate simulations - especially those which seek to describe a dynamical, changing climate - remains unclear. We investigate this question by deploying a global atmospheric, coarse resolution model known as SPEEDY to simulate a changing climate system subject to increased $\text{CO}_2$ concentrations, over a 100 year timescale. Whilst double precision is typically the operational standard for climate modelling, we find that reduced precision solutions (Float32, Float16) are sufficiently accurate. Rounding the finite precision floats stochastically, rather than using the more common ``round-to-nearest" technique, notably improves the performance of the reduced precision solutions. Over 100 years the mean bias error (MBE) in the global mean surface temperature (precipitation) relative to the double precision solution is $+2 \times 10^{-4}$K ($-8 \times 10^{-5}$ mm/6hr) at single precision and $-3.5\times 10^{-2}$ K($-1 \times 10^{-2}$ mm/6hr) at half precision, whilst the inclusion of stochastic rounding reduced the half precision error to +1.8 $\times 10^{-2}$ K ($-8 \times10^{-4}$ mm/6hr). By examining the resultant climatic distributions that arise after 100 years, the difference in the expected value of the global surface temperature, relative to the double precision solution is $\leq 5 \times 10^{-3}$ K and for precipitation $8 \times 10^{-4}$ mm/6h when numerically integrating at half precision with stochastic rounding. Areas of the model which notably improve due to the inclusion of stochastic over deterministic rounding are also explored and discussed. [abridged]

physics.ao-ph↗

arXiv2021

Climate Modelling in Low-Precision: Effects of both Deterministic & Stochastic Rounding

Motivated by recent advances in operational weather forecasting, we study the efficacy of low-precision arithmetic for climate simulations. We develop a framework to measure rounding error in a climate model which provides a stress-test for a low-precision version of the model, and we apply our method to a variety of models including the Lorenz system; a shallow water approximation for flow over a ridge; and a coarse resolution global atmospheric model with simplified parameterisations (SPEEDY). Although double precision (52 significant bits) is standard across operational climate models, in our experiments we find that single precision (23 sbits) is more than enough and that as low as half precision (10 sbits) is often sufficient. For example, SPEEDY can be run with 12 sbits across the entire code with negligible rounding error and this can be lowered to 10 sbits if very minor errors are accepted, amounting to less than 0.1 mm/6hr for the average grid-point precipitation, for example. Our test is based on the Wasserstein metric and this provides stringent non-parametric bounds on rounding error accounting for annual means as well as extreme weather events. In addition, by testing models using both round-to-nearest (RN) and stochastic rounding (SR) we find that SR can mitigate rounding error across a range of applications. Thus our results also provide evidence that SR could be relevant to next-generation climate models. While many studies have shown that low-precision arithmetic can be suitable on short-term weather forecasting timescales, our results give the first evidence that a similar low precision level can be suitable for climate.

physics.ao-ph↗

arXiv2021

Machine learning emulation of gravity wave drag in numerical weather forecasting

We assess the value of machine learning as an accelerator for the parameterisation schemes of operational weather forecasting systems, specifically the parameterisation of non-orographic gravity wave drag. Emulators of this scheme can be trained to produce stable and accurate results up to seasonal forecasting timescales. Generally, more complex networks produce more accurate emulators. By training on an increased complexity version of the existing parameterisation scheme we build emulators that produce more accurate forecasts. {For medium range forecasting we find evidence our emulators are more accurate} than the version of the parametrisation scheme that is used for operational predictions. Using the current operational CPU hardware our emulators have a similar computational cost to the existing scheme, but are heavily limited by data movement. On GPU hardware our emulators perform ten times faster than the existing scheme on a CPU.

physics.ao-ph↗

arXiv2020

Quantum algorithm for nonlinear differential equations

Quantum computers are known to provide an exponential advantage over classical computers for the solution of linear differential equations in high-dimensional spaces. Here, we present a quantum algorithm for the solution of nonlinear differential equations. The quantum algorithm provides an exponential advantage over classical algorithms for solving nonlinear differential equations. Potential applications include the Navier-Stokes equation, plasma hydrodynamics, epidemiology, and more.

quant-ph↗

arXiv2020

A Vision for Numerical Weather Prediction in 2030

In this essay, I outline a personal vision of how I think Numerical Weather Prediction (NWP) should evolve in the years leading up to 2030 and hence what it should look like in 2030. By NWP I mean initial-value predictions from timescales of hours to seasons ahead. Here I want to focus on how NWP can better help save lives from increasingly extreme weather in those parts of the world where society is most vulnerable. Whilst we can rightly be proud of many parts of our NWP heritage, its evolution has been influenced by national or institutional politics as well as by underpinning scientific principles. Sometimes these conflict with each other. It is important to be able to separate these issues when discussing how best meteorological science can serve society in 2030; otherwise any disruptive change - no matter how compelling the scientific case for it - becomes impossibly difficult.

physics.ao-ph↗

arXiv2020

Beyond skill scores: exploring sub-seasonal forecast value through a case study of French month-ahead energy prediction

We quantify the value of sub-seasonal forecasts for a real-world prediction problem: the forecasting of French month-ahead energy demand. Using surface temperature as a predictor, we construct a trading strategy and assess the financial value of using meteorological forecasts, based on actual energy demand and price data. We show that forecasts with lead times greater than 2 weeks can have value for this application, both on their own and in conjunction with shorter range forecasts, especially during boreal winter. We consider a cost/loss framework based on this example, and show that while it captures the performance of the short range forecasts well, it misses the marginal value present in the longer range forecasts. We also contrast our assessment of forecast value to that given by traditional skill scores, which we show could be misleading if used in isolation. We emphasise the importance of basing assessment of forecast skill on variables actually used by end-users.

physics.ao-ph↗

arXiv2018

The ECMWF Ensemble Prediction System: Looking Back (more than) 25 Years and Projecting Forward 25 Years

This paper has been written to mark 25 years of operational medium-range ensemble forecasting. The origins of the ECMWF Ensemble Prediction System are outlined, including the development of the precursor real-time Met Office monthly ensemble forecast system. In particular, the reasons for the development of singular vectors and stochastic physics - particular features of the ECMWF Ensemble Prediction System - are discussed. The author speculates about the development and use of ensemble prediction in the next 25 years.

physics.ao-ph↗

arXiv2015

Invariant Set Theory: Violating Measurement Independence without Fine Tuning, Conspiracy, Constraints on Free Will or Retrocausality

Invariant Set (IS) theory is a locally causal ontic theory of physics based on the Cosmological Invariant Set postulate that the universe $U$ can be considered a deterministic dynamical system evolving precisely on a (suitably constructed) fractal dynamically invariant set in $U$'s state space. IS theory violates the Bell inequalities by violating Measurement Independence. Despite this, IS theory is not fine tuned, is not conspiratorial, does not constrain experimenter free will and does not invoke retrocausality. The reasons behind these claims are discussed in this paper. These arise from properties not found in conventional ontic models: the invariant set has zero measure in its Euclidean embedding space, has Cantor Set structure homeomorphic to the p-adic integers ($p \ggg 0$) and is non-computable. In particular, it is shown that the p-adic metric encapulates the physics of the Cosmological Invariant Set postulate, and provides the technical means to demonstrate no fine tuning or conspiracy. Quantum theory can be viewed as the singular limit of IS theory when when $p$ is set equal to infinity. Since it is based around a top-down constraint from cosmology, IS theory suggests that gravitational and quantum physics will be unified by a gravitational theory of the quantum, rather than a quantum theory of gravity. Some implications arising from such a perspective are discussed.

quant-ph↗