arXiv subjects

Zijie Zhu

Publications and source records attributed to Zijie Zhu.

17 recordsLinked to original sources

arXiv2026

MINT: A Unified Model for World-Space Camera and Hand Motion Estimation from Scalable Egocentric Pipeline Supervision

Recovering camera and hand motion in world coordinates from egocentric video is a key capability for activity understanding, robot learning, and augmented reality. Existing systems typically decompose this problem into separate stages for camera motion, depth estimation, hand reconstruction, and trajectory refinement, resulting in substantial computational overhead and preventing the joint modeling of camera and hand motion. We introduce MINT (Minting IN-the-Wild Trajectories), a foundation model for world-space hand motion reconstruction from ego-centric RGB video. From a single shared spatiotemporal video representation, MINT jointly predicts the camera trajectory, field of view (FoV), camera-frame hand states, and per-frame hand observability, and then produces world-space hand motion via explicit coordinate transformations. Training such a model at scale is challenging, since paired world-space camera and hand annotations are scarce. We therefore develop an open-source labeling EGOPIPELINE that converts large collections of public egocentric videos into structured camera-and-hand trajectory supervision. MINT is first pretrained on these large-scale pseudo-labels and then fine-tuned on a small set of high-quality camera-and-hand annotations. Across public benchmarks MINT approaches state-of-the-art accuracy without seeing either benchmark in training, reaching 0.945 frame accuracy, 13.646 mm PA-MPJPE-p and 55.058 px EPE-p for camera-frame bimanual reconstruction on HOT3D, 4.690 mm RPE-T and 0.284 degrees RPE-R for camera trajectory, and a 3.67x end-to-end speedup over the labeling pipeline that supervises it. We release the model, training and inference code, labeling pipeline, and a curated 1,021-hour egocentric trajectory dataset.

cs.CV↗

arXiv2026

OpenWAM: An Open, Modular Exploration Towards Systematic World-Action Model Pretraining

World-Action Models inherit world knowledge from video-generative priors, and channel it into executable control signals through embodied experience. Existing systems, however, are monolithic: the generative backbone, visual representation, architecture, information flow, inference procedure, and training data are tightly coupled, obscuring which design choices matter and why. We introduce OpenWAM, an open research stack that turns world-action pretraining into a controlled experimental program. OpenWAM-Infra factorizes the WAM design space into composable modules with unified training, inference, deployment, and evaluation. On this substrate, OpenWAM-Study examines three questions through controlled experiments: what to inherit, how world and action learning interact, and how their synergy scales; and distills three principles: upstream knowledge transfers through a sufficiently capable generative backbone and a compact, information-rich latent space; world-action synergy requires dedicated action capacity, explicit world-to-action information flow, and synchronized joint denoising; and embodied pretraining principally improves out-of-domain generalization, with one-stage co-training over egocentric and robot data integrating world coverage and action grounding. Composing these principles, we build OpenWAM-α, an open WAM pretrained on roughly 6,400 hours of egocentric human and robot data and evaluated across simulation and real-world benchmarks. Across the eight simulation benchmarks and the real-robot experiments, which together span embodiments from single-arm and bimanual manipulation to dexterous hands, OpenWAM-α delivers consistently excellent performance, sustaining its top-tier standing from simulation to the physical world. We release the full stack, including infrastructure, evaluation protocols, pretrained models, and data recipes, to facilitate future research.

cs.RO↗

arXiv2026

TimeRFT: Stimulating Generalizable Time Series Forecasting for TSFMs via Reinforcement Finetuning

Time Series Foundation Models (TSFMs) have demonstrated strong generalization capability and data efficiency in time series forecasting through large-scale pretraining. However, adapting TSFMs to downstream forecasting tasks remains challenging due to temporal distribution shifts and varying data availability. Specifically, the non-stationary and uncertain nature of time series data leads to discrepancies between historical training and future forecasting distributions, making existing Supervised FineTuning (SFT)-based adaptation vulnerable to overfitting and limited generalization. Moreover, forecasting tasks often operate under varying data regimes, requiring TSFMs to extract generalizable temporal patterns from limited training samples. To address these challenges, we propose Time series Reinforcement FineTuning (TimeRFT), a reinforcement learning-based adaptation paradigm for TSFMs. TimeRFT introduces two forecasting-oriented training recipes: (i) A quality-aware temporal reward mechanism providing fine-grained credit assignment by holistically evaluating the contribution of each prediction step to overall forecasting performance. (ii) A difficulty-aware data selection strategy prioritizing informative time series samples with generalizable forecasting patterns. Extensive experiments on diverse real-world forecasting benchmarks demonstrate that TimeRFT consistently surpasses SFT-based adaptation methods across various real-world forecasting tasks with different data regimes, achieving improved prediction accuracy and enhanced generalization against unforeseen distribution shifts.

eess.SP↗

arXiv2026

TabPFN beyond Tabular Data: Calibration and Accuracy on Multimodal Embeddings

Few-shot multimodal classification commonly attaches a lightweight head, such as $k$-nearest neighbors, logistic regression, or a linear SVM, to a frozen pretrained encoder. Although computationally efficient, these heads can produce poorly calibrated confidence scores. We ask whether TabPFN can provide reliable confidence estimates on multimodal embeddings without sacrificing predictive accuracy, and under what conditions. We systematically evaluate TabPFN as a zero-gradient head for frozen image, text, and audio encoders. Across 22{,}820 evaluation episodes spanning 14 datasets, 11 encoders, and three modalities, TabPFN achieves the best mean rank among nine classification heads on both negative log-likelihood (NLL) and expected calibration error (ECE). At a representative setting, it reduces NLL by 48--62\% and ECE by 2.1--5.3$\times$ relative to the average of eight baselines while matching or exceeding their average accuracy. This calibration benefit transfers broadly, whereas the accuracy advantage is conditional: it concentrates at moderate-to-high shot counts and low-to-moderate feature dimensions ($k \ge 50$, $d \le 32$), and diminishes when labeled data are scarce, feature dimensions are high, or competing methods approach ceiling accuracy. After backbone adaptation, replacing the trained linear head with TabPFN improves calibration while preserving competitive accuracy, showing that representation adaptation and reliable head choice are complementary. Together, these results identify when TabPFN can serve as a training-free head for calibration-sensitive multimodal classification. To support transparency and reproducibility, we publicly release the source code, experiment configurations, and evaluation scripts in our GitHub repository: https://github.com/Jingxiang-Zhang/tabpfn-multimodal-embeddings.

cs.LG↗

arXiv2026

Digital programming of spin correlations in a fermionic lattice quantum simulator

Analog quantum simulation provides a highly controlled platform to study diverse quantum many-body phenomena. However, current methods for state initialisation are limited to thermal ensembles or uncorrelated product states. Here we present a hybrid approach that complements analog preparation with a digital quantum-gate protocol. This approach enables the engineering of target states with specific, long-range spin-correlations from the same initial resource state. By applying collisional gates to adiabatically prepared and filtered four-fermion singlet chains, we program diverse spin-correlation patterns, including that of a Heisenberg chain. We measure the spin correlations using a sequence of quantum gates followed by singlet-pair measurements. Our method paves the way to the targeted preparation of strongly correlated states of matter.

cond-mat.quant-gas↗

arXiv2026

Protected quantum gates using qubit doublons in dynamical optical lattices

Quantum computing represents a central challenge in modern science. Neutral atoms in optical lattices have emerged as a leading computing platform, with collisional gates offering a stable mechanism for quantum logic. However, previous experiments have treated ultracold collisions as a dynamically fine-tuned process, which obscures the underlying quantum- geometry and statistics crucial for realising intrinsically robust operations. Here, we propose and experimentally demonstrate a purely geometric two-qubit swap gate by transiently populating qubit doublon states of fermionic atoms in a dynamical optical lattice. The presence of these doublon states, together with fermionic exchange anti-symmetry, enables a two-particle quantum holonomy -- a geometric evolution where dynamical phases are absent. This yields a gate mechanism that is intrinsically protected against fluctuations and inhomogeneities of the confining potentials. The resilience of the gate is further reinforced by time-reversal and chiral symmetries of the Hamiltonian. We experimentally validate this exceptional protection, achieving a loss-corrected amplitude fidelity of $99.91(7)\%$ measured across the entire system consisting of more than $17'000$ atom pairs. When combined with recently developed topological pumping methods for atom transport, our results pave the way for large-scale, highly connected quantum processors. This work introduces a new paradigm for quantum logic, transforming fundamental symmetries and quantum statistics into a powerful resource for fault-tolerant computation.

quant-ph↗

arXiv2025

Splitting and connecting singlets in atomic quantum circuits

Gate operations composed in quantum circuits form the basis for digital quantum simulation and quantum processing. While two-qubit gates generally operate on nearest neighbours, many circuits require nonlocal connectivity and necessitate some form of quantum information transport. Yet, connecting distant nodes of a quantum processor still remains challenging, particularly for neutral atoms in optical lattices. Here, we create singlet pairs of two magnetic states of fermionic potassium-40 atoms in an optical lattice and use a bi-directional topological Thouless pump to transport, coherently split, and separate the pairs, as well as to demonstrate interaction between them via tuneable $($swap$)^α$-gate operations. We achieve pumping with a single-shift fidelity of 99.78(3)% over 50 lattice sites and split the pairs within a decoherence-free subspace. Gates are implemented by superexchange interaction, allowing us to produce interwoven atomic singlets. For read-out, we apply a magnetic field gradient, resulting in single- and multi-frequency singlet-triplet oscillations. Our work shows avenues to create complex patterns of entanglement and new approaches to quantum processing, sensing, and atom interferometry.

quant-ph↗

arXiv2025

Mitigating higher-band heating in Floquet-Hubbard lattices via two-tone driving

Multi-photon resonances to high-lying energy levels represent an unavoidable source of Floquet heating in strongly driven quantum systems. In this work, we extend the recently developed two-tone approach of 'cancelling' multi-photon resonances to shaken lattices in the Hubbard regime. Our experiments show that even for strong lattice shaking the inclusion of a weak second drive leads to cancellation of multi-photon heating resonances. Surprisingly, the optimal cancelling amplitude depends on the Hubbard interaction strength $U$, in qualitative agreement with exact diagonalisation calculations. Our results call for novel analytical approaches to capture the physics of strongly-driven-strongly-interacting many-body systems.

cond-mat.quant-gas↗

arXiv2024

Exploring mechanical and thermal properties of high-entropy ceramics via general machine learning potentials

The mechanical and thermal performance of high-entropy ceramics are critical to their use in extreme conditions. However, the vast composition space of high-entropy ceramic significantly hinders their development with desired mechanical and thermal properties. Herein, taking high-entropy carbides (HECs) as the model, we show the efficiency and effectiveness of exploring the mechanical and thermal properties via machine-learning-potential-based molecular dynamics (MD). Specifically, a general neuroevolution potential (NEP) with broad compositional applicability for HECs of ten transition metal elements from group IIIB-VIB is efficiently constructed from the small dataset comprising unary and binary carbides with an equal amount of ergodic chemical compositions. Based on this well-established NEP, MD simulations on mechanical and thermal properties of different HECs have shown good agreement with the results of first-principles calculations and experimental measurements, validating the accuracy, generalization, and reliability of using the developed general NEP in investigating mechanical and thermal performance of HECs. Our work provides an efficient solution to accelerate the search for high-entropy ceramics with desirable mechanical and thermal properties.

cond-mat.mtrl-sci↗

arXiv2024

Interactions enable Thouless pumping in a nonsliding lattice

A topological 'Thouless' pump represents the quantised motion of particles in response to a slow, cyclic modulation of external control parameters. The Thouless pump, like the quantum Hall effect, is of fundamental interest in physics because it links physically measurable quantities, such as particle currents, to geometric properties of the experimental system, which can be robust against perturbations and thus technologically useful. So far, experiments probing the interplay between topology and inter-particle interactions have remained relatively scarce. Here we observe a Thouless-type charge pump in which the particle current and its directionality inherently rely on the presence of strong interactions. Experimentally, we utilise a two-component Fermi gas in a dynamical superlattice which does not exhibit a sliding motion and remains trivial in the single-particle regime. However, when tuning interparticle interactions from zero to positive values, the system undergoes a transition from being stationary to drifting in one direction, consistent with quantised pumping in the first cycle. Remarkably, the topology of the interacting pump trajectory cannot be adiabatically connected to a non-interacting limit, highlighted by the fact that only one atom is transferred per cycle. Our experiments suggest that Thouless charge pumps are promising platforms to gain insights into interaction-driven topological transitions and topological quantum matter.

cond-mat.quant-gas↗

arXiv2024

Reversal of quantised Hall drifts at non-interacting and interacting topological boundaries

The transport properties of gapless edge modes at boundaries between topologically distinct domains are of fundamental and technological importance. Therefore, it is crucial to gain a better understanding of topological edge states and their response to interparticle interactions. Here, we experimentally study long-distance quantised Hall drifts in a harmonically confined topological pump of non-interacting and interacting ultracold fermionic atoms. We find that quantised drifts halt and reverse their direction when the atoms reach a critical slope of the confining potential, revealing the presence of a topological boundary. The drift reversal corresponds to a band transfer between a band with Chern number $C = +1$ and a band with $C = -1$ via a gapless edge mode, in agreement with the bulk-edge correspondence for non-interacting particles. We establish that a non-zero repulsive Hubbard interaction leads to the emergence of an additional edge in the system, relying on a purely interaction-induced mechanism, in which pairs of fermions are split.

cond-mat.quant-gas↗

arXiv2023

Quantisation and its breakdown in a Hubbard-Thouless pump

Geometric properties of waves and wave functions can explain the appearance of integer-valued observables throughout physics. For example, these 'topological' invariants describe the plateaux observed in the quantised Hall effect and the pumped charge in its dynamic analogon, the Thouless pump. However, the presence of interparticle interactions can profoundly affect the topology of a material, invalidating the idealised formulation in terms of Bloch waves. Despite pioneering experiments in solid state systems, photonic waveguides, and optical lattices, the study of topological insulators under variation of inter-particle interactions has proven challenging. Here, we experimentally realise a topological Thouless pump with tuneable Hubbard interactions in an optical lattice and observe regimes with robust pumping, as well as an interaction-induced breakdown. We confirm the pump's robustness against interactions that are smaller than the protecting gap, which holds true for both repulsive and attractive Hubbard $U$. Furthermore, we identify that bound pairs of fermions are responsible for quantised transport at strongly attractive $U$, supported by measurements of pair fraction and adiabaticity. For strong repulsive interactions, on the contrary, topological pumping breaks down. Yet, we can reinstate quantised pumping by modifying the pump trajectory while starting from the same initial state. Our experiments pave the way for investigating interacting topological insulators, including edge effects and interaction-induced topological phases.

cond-mat.quant-gas↗

arXiv2022

Topological pumping in a Floquet-Bloch band

Constructing new topological materials is of vital interest for the development of robust quantum applications. However, engineering such materials often causes technological overhead, such as large magnetic fields, specific lattice geometries, strong spin-orbit coupling, synthetic dimensions, or dynamical superlattice potentials. Simplifying the experimental requirements has been addressed on a conceptual level - by proposing to combine simple lattice structures with Floquet engineering - but there has been no experimental implementation. Here, we demonstrate topological pumping in a Floquet-Bloch band using a plain sinusoidal lattice potential and two-tone driving with frequencies $ω$ and $2ω$. We adiabatically prepare a near-insulating Floquet band of ultracold fermions via a frequency chirp, which avoids gap closings en route from trivial to topological bands. Subsequently, we induce topological pumping by slowly cycling the amplitude and the phase of the $2 ω$ drive. Our system is well described by an effective Shockley model, establishing a novel paradigm to engineer topological matter from simple underlying lattice geometries. This approach could enable the application of quantised pumping in metrology, following recent experimental advances on two-frequency driving in real materials.

cond-mat.quant-gas↗

arXiv2021

Floquet engineering of individual band gaps in an optical lattice using a two-tone drive

The dynamic engineering of band structures for ultracold atoms in optical lattices represents an innovative approach to understand and explore the fundamental principles of topological matter. In particular, the folded Floquet spectrum determines the associated band topology via band inversion. We experimentally and theoretically study two-frequency phase modulation to asymmetrically hybridize the lowest two bands of a one-dimensional lattice. Using quasi-degenerate perturbation theory in the extended Floquet space we derive an effective two-band model that quantitatively describes our setting. The energy gaps are experimentally probed via Landau-Zener transitions between Floquet-Bloch bands using an accelerated Bose-Einstein condensate. Separate and simultaneous control over the closing and reopening of these band gaps is demonstrated. We find good agreement between experiment and theory, establishing an analytic description for resonant Floquet-Bloch engineering that includes single- and multi-photon couplings, as well as interference effects between several commensurate drives.

cond-mat.quant-gas↗

arXiv2021

Extension of the Generalized Hydrodynamics to the Dimensional Crossover Regime

In an effort to address integrability breaking in cold gas experiments, we extend the integrable hydrodynamics of the 1d Lieb-Liniger model with two additional components representing the population of atoms in the first and second transverse excited states, thus enabling a description of quasi-1d condensates. Collisions between different components are accounted for through the inclusion of a Boltzmann-type collision integral in the hydrodynamic equation. Contrary to standard generalized hydrodynamics, our extended model captures thermalization of the condensate at a rate consistent with experimental observations from a quantum Newton's cradle setup.

cond-mat.quant-gas↗

arXiv2020

Relaxation of Bosons in One Dimension and the Onset of Dimensional Crossover

We study ultra-cold bosons out of equilibrium in a one-dimensional (1D) setting and probe the breaking of integrability and the resulting relaxation at the onset of the crossover from one to three dimensions. In a quantum Newton's cradle type experiment, we excite the atoms to oscillate and collide in an array of 1D tubes and observe the evolution for up to 4.8 seconds (400 oscillations) with minimal heating and loss. By investigating the dynamics of the longitudinal momentum distribution function and the transverse excitation, we observe and quantify a two-stage relaxation process. In the initial stage single-body dephasing reduces the 1D densities, thus rapidly drives the 1D gas out of the quantum degenerate regime. The momentum distribution function asymptotically approaches the distribution of quasimomenta (rapidities), which are conserved in an integrable system. In the subsequent long time evolution, the 1D gas slowly relaxes towards thermal equilibrium through the collisions with transversely excited atoms. Moreover, we tune the dynamics in the dimensional crossover by initializing the evolution with different imprinted longitudinal momenta (energies). The dynamical evolution towards the relaxed state is quantitatively described by a semiclassical molecular dynamics simulation.

cond-mat.quant-gas↗

arXiv2018

Observation of atom-number fluctuations in optical lattices via quantum collapse and revival dynamics

Using the quantum collapse and revival phenomenon of a Bose--Einstein condensate in three-dimensional optical lattices, the atom number statistics on each lattice site are experimentally investigated. We observe an interaction driven time evolution of on-site number fluctuations in a constant lattice potential with the collapse and revival time ratio as the figure of merit. Through a shortcut loading procedure, we prepare a three-dimensional array of coherent states with Poissonian number fluctuations. The following dynamics clearly show the interaction effect on the evolution of the number fluctuations from Poissonian to sub-Poissonian. Our method can be used to create squeezed states which are important in precision measurement.

cond-mat.quant-gas↗