arXiv subjects

Anxin Guo

Publications and source records attributed to Anxin Guo.

5 recordsLinked to original sources

arXiv2026

Hallucination is a Consequence of Space-Optimality: A Rate-Distortion Theorem for Membership Testing

Large language models often hallucinate with high confidence on "random facts" that lack inferable patterns. We formalize the memorization of such facts as a membership testing problem, unifying the discrete error metrics of Bloom filters with the continuous log-loss of LLMs. By analyzing this problem in the regime where facts are sparse in the universe of plausible claims, we establish a rate-distortion theorem: the optimal memory efficiency is characterized by the minimum KL divergence between score distributions on facts and non-facts. This theoretical framework provides a distinctive explanation for hallucination under an idealized setting: even with optimal training, perfect data, and a simplified ``closed world'' setting, the information-theoretically optimal strategy under limited capacity is not to abstain or forget, but to assign high confidence to some non-facts, resulting in hallucination. We validate this theory empirically on both synthetic and real-world data, showing that hallucinations persist as a natural consequence of lossy compression. The same theorem recovers and sharpens classical space lower bounds for Bloom-type filters, pinning down an additive constant left open for two-sided filters.

cs.LG↗

arXiv2025

Neural reconstruction of 3D ocean wave hydrodynamics from camera sensing

Precise three-dimensional (3D) reconstruction of wave free surfaces and associated velocity fields is essential for developing a comprehensive understanding of ocean physics. To address the high computational cost of dense visual reconstruction in long-term ocean wave observation tasks and the challenges introduced by persistent visual occlusions, we propose an wave free surface visual reconstruction neural network, which is designed as an attention-augmented pyramid architecture tailored to the multi-scale and temporally continuous characteristics of wave motions. Using physics-based constraints, we perform time-resolved reconstruction of nonlinear 3D velocity fields from the evolving free-surface boundary. Experiments under real-sea conditions demonstrate millimetre-level wave elevation prediction in the central region, dominant-frequency errors below 0.01 Hz, precise estimation of high-frequency spectral power laws, and high-fidelity 3D reconstruction of nonlinear velocity fields, while enabling dense reconstruction of two million points in only 1.35 s. Built on a stereo-vision dataset, the model outperforms conventional visual reconstruction approaches and maintains strong generalization in occluded conditions, owing to its global multi-scale attention and its learned encoding of wave propagation dynamics.

cs.CV↗

arXiv2025

Harmonic balance-automatic differentiation method: an out-of-the-box and efficient solver for general nonlinear dynamics simulation

The Harmonic Balance-Alternating Frequency-Time domain (HB-AFT) method is extensively employed for dynamic response analysis of nonlinear systems. However, its application to high-dimensional complex systems is constrained by the manual derivation of Jacobian matrices during Newton-Raphson iterations, which become computationally intractable or error-prone for intricate nonlinearities. The Harmonic Balance-Automatic Differentiation (HB-AD) method is proposed to address this limitation, in which AD is integrated with the harmonic balance framework. This approach eliminates all manual derivations by leveraging AD to compute exact Jacobians numerically, enabling generic and efficient analysis of high-dimensional complex nonlinear systems. The implementation utilizes advanced deep learning frameworks for native parallel computing and CUDA acceleration, and combines AD with arc-length continuation, establishing an out-of-the-box and high efficiency computational architecture. Users need only supply the system's dynamic equations, HB-AD then autonomously trace the complete panorama of periodic responses -- including stable/unstable solution branches. Computational experiments on a rotor system with squeeze-film damper (SFD) demonstrate HB-AD's capability in handling complex nonlinear expressions with automated Jacobian calculations. For a high-dimensional aero-engine rotor-bearing-casing system with complex bearing nonlinearities, HB-AD achieves 17-fold higher efficiency than traditional HB-AFT and 144-fold acceleration over the Newmark method. The HB-AD method is a synergistic merger of computational mechanics and machine learning primitives, delivers an easy to use, general-purpose, high efficiency platform for high-fidelity dynamic characterization of high-dimensional engineering systems.

cs.CE↗

arXiv2024

Agnostic Learning of Arbitrary ReLU Activation under Gaussian Marginals

We consider the problem of learning an arbitrarily-biased ReLU activation (or neuron) over Gaussian marginals with the squared loss objective. Despite the ReLU neuron being the basic building block of modern neural networks, we still do not understand the basic algorithmic question of whether one arbitrary ReLU neuron is learnable in the non-realizable setting. In particular, all existing polynomial time algorithms only provide approximation guarantees for the better-behaved unbiased setting or restricted bias setting. Our main result is a polynomial time statistical query (SQ) algorithm that gives the first constant factor approximation for arbitrary bias. It outputs a ReLU activation that achieves a loss of $O(\mathrm{OPT}) + \varepsilon$ in time $\mathrm{poly}(d,1/\varepsilon)$, where $\mathrm{OPT}$ is the loss obtained by the optimal ReLU activation. Our algorithm presents an interesting departure from existing algorithms, which are all based on gradient descent and thus fall within the class of correlational statistical query (CSQ) algorithms. We complement our algorithmic result by showing that no polynomial time CSQ algorithm can achieve a constant factor approximation. Together, these results shed light on the intrinsic limitation of gradient descent, while identifying arguably the simplest setting (a single neuron) where there is a separation between SQ and CSQ algorithms.

cs.LG↗

arXiv2024

To Store or Not to Store: a graph theoretical approach for Dataset Versioning

In this work, we study the cost efficient data versioning problem, where the goal is to optimize the storage and reconstruction (retrieval) costs of data versions, given a graph of datasets as nodes and edges capturing edit/delta information. One central variant we study is MinSum Retrieval (MSR) where the goal is to minimize the total retrieval costs, while keeping the storage costs bounded. This problem (along with its variants) was introduced by Bhattacherjee et al. [VLDB'15]. While such problems are frequently encountered in collaborative tools (e.g., version control systems and data analysis pipelines), to the best of our knowledge, no existing research studies the theoretical aspects of these problems. We establish that the currently best-known heuristic, LMG, can perform arbitrarily badly in a simple worst case. Moreover, we show that it is hard to get $o(n)$-approximation for MSR on general graphs even if we relax the storage constraints by an $O(\log n)$ factor. Similar hardness results are shown for other variants. Meanwhile, we propose poly-time approximation schemes for tree-like graphs, motivated by the fact that the graphs arising in practice from typical edit operations are often not arbitrary. As version graphs typically have low treewidth, we further develop new algorithms for bounded treewidth graphs. Furthermore, we propose two new heuristics and evaluate them empirically. First, we extend LMG by considering more potential ``moves'', to propose a new heuristic LMG-All. LMG-All consistently outperforms LMG while having comparable run time on a wide variety of datasets, i.e., version graphs. Secondly, we apply our tree algorithms on the minimum-storage arborescence of an instance, yielding algorithms that are qualitatively better than all previous heuristics for MSR, as well as for another variant BoundedMin Retrieval (BMR).

cs.DS↗