Jon Hu | Searcharxiv

arXiv2026

Joint Training on AMD and NVIDIA GPUs

As large language models continue to scale, training demands on compute and system capacity grow rapidly, making single-vendor homogeneous clusters insufficient. This paper presents a technical solution for heterogeneous mixed training in AMD-NVIDIA environments. We first adopt a compatibility-oriented approach based on CPU-Forwarding Communication, with differentiated communication back-end selection across parallel groups and multi-NIC parallel data transfer. To achieve higher performance, we further propose another Device-Direct Communication approach, integrating a CPU-offloading P2P mechanism to enable direct cross-vendor GPU data transfer without host-memory staging. Experiments on LLaMA-8B and Qwen2-7B demonstrate that the proposed Device-Direct Communication approach achieves up to 98% of the throughput of an NVIDIA homogeneous system, while preserving training stability and correctness.

cs.DC↗

arXiv2013

Spatially Adaptive Stochastic Methods for Fluid-Structure Interactions Subject to Thermal Fluctuations in Domains with Complex Geometries

We develop stochastic mixed finite element methods for spatially adaptive simulations of fluid-structure interactions when subject to thermal fluctuations. To account for thermal fluctuations, we introduce a discrete fluctuation-dissipation balance condition to develop compatible stochastic driving fields for our discretization. We perform analysis that shows our condition is sufficient to ensure results consistent with statistical mechanics. We show the Gibbs-Boltzmann distribution is invariant under the stochastic dynamics of the semi-discretization. To generate efficiently the required stochastic driving fields, we develop a Gibbs sampler based on iterative methods and multigrid to generate fields with $O(N)$ computational complexity. Our stochastic methods provide an alternative to uniform discretizations on periodic domains that rely on Fast Fourier Transforms. To demonstrate in practice our stochastic computational methods, we investigate within channel geometries having internal obstacles and no-slip walls how the mobility/diffusivity of particles depends on location. Our methods extend the applicability of fluctuating hydrodynamic approaches by allowing for spatially adaptive resolution of the mechanics and for domains that have complex geometries relevant in many applications.

cond-mat.mes-hall↗