Searcharxiv⌕ Search

arXiv subjects

John Hoffman

Publications and source records attributed to John Hoffman.

At least 19 recordsLinked to original sources
arXiv2026

Parabolic Reifenberg Parametrizations and Corona Decompositions, with Applications to Singular Integrals

For parabolic Ahlfors regular sets, we show that the bilateral weak geometric lemma (BWGL), bilateral approximation by unions of planes (BAUP), and corona decompositions by $Lip$(1,1/2) graphs (whose defining functions need not have half-order time derivatives in BMO) are equivalent. These geometric conditions do not imply the strong geometric lemma. A key tool is a parabolic version of Reifenberg's theorem for sets with holes, which we prove here. As an application of these geometric results, we prove that a parabolic Ahlfors $(n+2)$-regular measure $μ$ on $\mathbb{R}^{d+1}$, $1\leq n\leq d-1$, is parabolic uniformly rectifiable if its support satisfies any of these equivalent conditions and the parabolic Riesz transform is bounded on $L^2(μ)$.

math.CA↗
arXiv2026

Aligning Autoresearch: Co-Improvement between Agents and Humans

Autoresearch agents can now attempt long-horizon AI research end-to-end, i.e., self-improvement. However, best results are still obtained by research agents interacting with human researchers, and improving AI's ability to perform this co-research will only reinforce this effect---a process termed co-improvement (Weston and Foerster, 2025). Further, aligning with human expectations is likely a safer path for research agents, especially given recent agent reward hacking phenomena (OpenAI, 2026b). In this work we address these issues in four ways: (i) we empirically study the role of humans in directing autoresearch agents, focusing on the domain of benchmark creation, and find humans still play a pivotal role particularly in the ideation phase; (ii) we also develop a benchmark, HackBench, that shows current autoresearch agents are not aligned and are unable to achieve hack localization sufficiently; (iii) we present a method to train agents for improved human interaction by using human simulators as a proxy during reinforcement learning (RL); and finally (iv) we develop a method to improve model text generation, and hence agent communication, by RL with rubrics aligned with human experts. Together, our training and evaluation recipes provide a path toward more capable and aligned (co-)research agents.

cs.AI↗
arXiv2026

Riesz Transform Bounds Imply Parabolic Uniform Rectifiability of Graphs

We prove that if a $(k+2)$-Ahlfors--David regular measure $μ$ in parabolic space $\mathbb P^n$ has support equal to a parabolic Lipschitz graph with sufficiently small Lipschitz constant, and the parabolic Riesz transform is bounded on $L^2(μ)$, then $μ$ is parabolic uniformly rectifiable and the graph is regular parabolic Lipschitz. The result holds for every integer $0\leq k\leq n-1$.

math.CA↗
arXiv2026

SAM Audio Judge: A Unified Multimodal Framework for Perceptual Evaluation of Audio Separation

The performance evaluation remains a complex challenge in audio separation, and existing evaluation metrics are often misaligned with human perception, course-grained, relying on ground truth signals. On the other hand, subjective listening tests remain the gold standard for real-world evaluation, but they are expensive, time-consuming, and difficult to scale. This paper addresses the growing need for automated systems capable of evaluating audio separation without human intervention. The proposed evaluation metric, SAM Audio Judge (SAJ), is a multimodal fine-grained reference-free objective metric, which shows highly alignment with human perceptions. SAJ supports three audio domains (speech, music and general sound events) and three prompt inputs (text, visual and span), covering four different dimensions of evaluation (recall, percision, faithfulness, and overall). SAM Audio Judge also shows potential applications in data filtering, pseudo-labeling large datasets and reranking in audio separation models. We release our code and pre-trained models at: https://github.com/facebookresearch/sam-audio.

eess.AS↗
arXiv2025

SAM Audio: Segment Anything in Audio

General audio source separation is a key capability for multimodal AI systems that can perceive and reason about sound. Despite substantial progress in recent years, existing separation models are either domain-specific, designed for fixed categories such as speech or music, or limited in controllability, supporting only a single prompting modality such as text. In this work, we present SAM Audio, a foundation model for general audio separation that unifies text, visual, and temporal span prompting within a single framework. Built on a diffusion transformer architecture, SAM Audio is trained with flow matching on large-scale audio data spanning speech, music, and general sounds, and can flexibly separate target sources described by language, visual masks, or temporal spans. The model achieves state-of-the-art performance across a diverse suite of benchmarks, including general sound, speech, music, and musical instrument separation in both in-the-wild and professionally produced audios, substantially outperforming prior general-purpose and specialized systems. Furthermore, we introduce a new real-world separation benchmark with human-labeled multimodal prompts and a reference-free evaluation model that correlates strongly with human judgment.

eess.AS↗
arXiv2025

Parabolic Singular Integrals with Nonhomogeneous Kernels

We establish $L^2$ boundedness of all "nice" parabolic singular integrals on "Good Parabolic Graphs", aka {\em regular} Lip(1,1/2) graphs. The novelty here is that we include non-homogeneous kernels, which are relevant to the theory of parabolic uniform rectifiability. Previously, the third named author had treated the case of homogeneous kernels. The present proof combines the methods of that work (which in turn was based on methods described in Christ's CBMS lecture notes), with the techniques of Coifman-David-Meyer.

math.CA↗
arXiv2025

Square Functions Controlling Smoothness and Higher-Order Rectifiability

We provide new characterizations of the $BMO$-Sobolev space $I_α(BMO)$ for the range $0 < α<2$. When $0 < α<1$, our characterizations are in terms of square functions measuring multiscale approximation of constants, and when $1 \leq α<2$ our characterizations are in terms of square functions measuring multiscale approximation by linear functions.

math.CA↗
arXiv2025

Movie Gen: A Cast of Media Foundation Models

We present Movie Gen, a cast of foundation models that generates high-quality, 1080p HD videos with different aspect ratios and synchronized audio. We also show additional capabilities such as precise instruction-based video editing and generation of personalized videos based on a user's image. Our models set a new state-of-the-art on multiple tasks: text-to-video synthesis, video personalization, video editing, video-to-audio generation, and text-to-audio generation. Our largest video generation model is a 30B parameter transformer trained with a maximum context length of 73K video tokens, corresponding to a generated video of 16 seconds at 16 frames-per-second. We show multiple technical innovations and simplifications on the architecture, latent spaces, training objectives and recipes, data curation, evaluation protocols, parallelization techniques, and inference optimizations that allow us to reap the benefits of scaling pre-training data, model size, and training compute for training large scale media generation models. We hope this paper helps the research community to accelerate progress and innovation in media generation models. All videos from this paper are available at https://go.fb.me/MovieGenResearchVideos.

cs.CV↗
arXiv2025

Meta Audiobox Aesthetics: Unified Automatic Quality Assessment for Speech, Music, and Sound

The quantification of audio aesthetics remains a complex challenge in audio processing, primarily due to its subjective nature, which is influenced by human perception and cultural context. Traditional methods often depend on human listeners for evaluation, leading to inconsistencies and high resource demands. This paper addresses the growing need for automated systems capable of predicting audio aesthetics without human intervention. Such systems are crucial for applications like data filtering, pseudo-labeling large datasets, and evaluating generative audio models, especially as these models become more sophisticated. In this work, we introduce a novel approach to audio aesthetic evaluation by proposing new annotation guidelines that decompose human listening perspectives into four distinct axes. We develop and train no-reference, per-item prediction models that offer a more nuanced assessment of audio quality. Our models are evaluated against human mean opinion scores (MOS) and existing methods, demonstrating comparable or superior performance. This research not only advances the field of audio aesthetics but also provides open-source models and datasets to facilitate future work and benchmarking. We release our code and pre-trained model at: https://github.com/facebookresearch/audiobox-aesthetics

cs.SD↗
arXiv2025

Revisiting Reliability in Large-Scale Machine Learning Research Clusters

Reliability is a fundamental challenge in operating large-scale machine learning (ML) infrastructures, particularly as the scale of ML models and training clusters continues to grow. Despite decades of research on infrastructure failures, the impact of job failures across different scales remains unclear. This paper presents a view of managing two large, multi-tenant ML clusters, providing quantitative analysis, operational experience, and our own perspective in understanding and addressing reliability concerns at scale. Our analysis reveals that while large jobs are most vulnerable to failures, smaller jobs make up the majority of jobs in the clusters and should be incorporated into optimization objectives. We identify key workload properties, compare them across clusters, and demonstrate essential reliability requirements for pushing the boundaries of ML training at scale. We hereby introduce a taxonomy of failures and key reliability metrics, analyze 11 months of data from two state-of-the-art ML environments with 4 million jobs and over 150 million A100 GPU hours. Building on our data, we fit a failure model to project Mean Time to Failure for various GPU scales. We further propose a method to estimate a related metric, Effective Training Time Ratio, as a function of job parameters, and we use this model to gauge the efficacy of potential software mitigations at scale. Our work provides valuable insights and future research directions for improving the reliability of AI supercomputer clusters, emphasizing the need for flexible, workload-agnostic, and reliability-aware infrastructure, system software, and algorithms.

cs.DC↗
arXiv2023

Seamless: Multilingual Expressive and Streaming Speech Translation

Large-scale automatic speech translation systems today lack key features that help machine-mediated communication feel seamless when compared to human-to-human dialogue. In this work, we introduce a family of models that enable end-to-end expressive and multilingual translations in a streaming fashion. First, we contribute an improved version of the massively multilingual and multimodal SeamlessM4T model-SeamlessM4T v2. This newer model, incorporating an updated UnitY2 framework, was trained on more low-resource language data. SeamlessM4T v2 provides the foundation on which our next two models are initiated. SeamlessExpressive enables translation that preserves vocal styles and prosody. Compared to previous efforts in expressive speech research, our work addresses certain underexplored aspects of prosody, such as speech rate and pauses, while also preserving the style of one's voice. As for SeamlessStreaming, our model leverages the Efficient Monotonic Multihead Attention mechanism to generate low-latency target translations without waiting for complete source utterances. As the first of its kind, SeamlessStreaming enables simultaneous speech-to-speech/text translation for multiple source and target languages. To ensure that our models can be used safely and responsibly, we implemented the first known red-teaming effort for multimodal machine translation, a system for the detection and mitigation of added toxicity, a systematic evaluation of gender bias, and an inaudible localized watermarking mechanism designed to dampen the impact of deepfakes. Consequently, we bring major components from SeamlessExpressive and SeamlessStreaming together to form Seamless, the first publicly available system that unlocks expressive cross-lingual communication in real-time. The contributions to this work are publicly released and accessible at https://github.com/facebookresearch/seamless_communication

cs.CL↗
arXiv2023

SeamlessM4T: Massively Multilingual & Multimodal Machine Translation

What does it take to create the Babel Fish, a tool that can help individuals translate speech between any two languages? While recent breakthroughs in text-based models have pushed machine translation coverage beyond 200 languages, unified speech-to-speech translation models have yet to achieve similar strides. More specifically, conventional speech-to-speech translation systems rely on cascaded systems that perform translation progressively, putting high-performing unified systems out of reach. To address these gaps, we introduce SeamlessM4T, a single model that supports speech-to-speech translation, speech-to-text translation, text-to-speech translation, text-to-text translation, and automatic speech recognition for up to 100 languages. To build this, we used 1 million hours of open speech audio data to learn self-supervised speech representations with w2v-BERT 2.0. Subsequently, we created a multimodal corpus of automatically aligned speech translations. Filtered and combined with human-labeled and pseudo-labeled data, we developed the first multilingual system capable of translating from and into English for both speech and text. On FLEURS, SeamlessM4T sets a new standard for translations into multiple target languages, achieving an improvement of 20% BLEU over the previous SOTA in direct speech-to-text translation. Compared to strong cascaded models, SeamlessM4T improves the quality of into-English translation by 1.3 BLEU points in speech-to-text and by 2.6 ASR-BLEU points in speech-to-speech. Tested for robustness, our system performs better against background noises and speaker variations in speech-to-text tasks compared to the current SOTA model. Critically, we evaluated SeamlessM4T on gender bias and added toxicity to assess translation safety. Finally, all contributions in this work are open-sourced and accessible at https://github.com/facebookresearch/seamless_communication

cs.CL↗
arXiv2023

Corona Decompositions for Parabolic Uniformly Rectifiable Sets

We prove that parabolic uniformly rectifiable sets admit (bilateral) corona decompositions with respect to regular Lip(1,1/2) graphs. Together with our previous work, this allows us to conclude that if $Σ\subset\mathbb{R}^{n+1}$ is parabolic Ahlfors-David regular, then the following statements are equivalent. (1) $Σ$ is parabolic uniformly rectifiable. (2) $Σ$ admits a corona decomposition with respect to regular Lip(1,1/2) graphs. (3) $Σ$ admits a bilateral corona decomposition with respect to regular Lip(1,1/2) graphs. (4) $Σ$ is big pieces squared of regular Lip(1,1/2) graphs.

math.MG↗
arXiv2022

No Language Left Behind: Scaling Human-Centered Machine Translation

Driven by the goal of eradicating language barriers on a global scale, machine translation has solidified itself as a key focus of artificial intelligence research today. However, such efforts have coalesced around a small subset of languages, leaving behind the vast majority of mostly low-resource languages. What does it take to break the 200 language barrier while ensuring safe, high quality results, all while keeping ethical considerations in mind? In No Language Left Behind, we took on this challenge by first contextualizing the need for low-resource language translation support through exploratory interviews with native speakers. Then, we created datasets and models aimed at narrowing the performance gap between low and high-resource languages. More specifically, we developed a conditional compute model based on Sparsely Gated Mixture of Experts that is trained on data obtained with novel and effective data mining techniques tailored for low-resource languages. We propose multiple architectural and training improvements to counteract overfitting while training on thousands of tasks. Critically, we evaluated the performance of over 40,000 different translation directions using a human-translated benchmark, Flores-200, and combined human evaluation with a novel toxicity benchmark covering all languages in Flores-200 to assess translation safety. Our model achieves an improvement of 44% BLEU relative to the previous state-of-the-art, laying important groundwork towards realizing a universal translation system. Finally, we open source all contributions described in this work, accessible at https://github.com/facebookresearch/fairseq/tree/nllb.

cs.CL↗
arXiv2021

Techniques Toward Optimizing Viewability in RTB Ad Campaigns Using Reinforcement Learning

Reinforcement learning (RL) is an effective technique for training decision-making agents through interactions with their environment. The advent of deep learning has been associated with highly notable successes with sequential decision making problems - such as defeating some of the highest-ranked human players at Go. In digital advertising, real-time bidding (RTB) is a common method of allocating advertising inventory through real-time auctions. Bidding strategies need to incorporate logic for dynamically adjusting parameters in order to deliver pre-assigned campaign goals. Here we discuss techniques toward using RL to train bidding agents. As a campaign metric we particularly focused on viewability: the percentage of inventory which goes on to be viewed by an end user. This paper is presented as a survey of techniques and experiments which we developed through the course of this research. We discuss expanding our training data to include edge cases by training on simulated interactions. We discuss the experimental results comparing the performance of several promising RL algorithms, and an approach to hyperparameter optimization of an actor/critic training pipeline through Bayesian optimization. Finally, we present live-traffic tests of some of our RL agents against a rule-based feedback-control approach, demonstrating the potential for this method as well as areas for further improvement. This paper therefore presents an arrangement of our findings in this quickly developing field, and ways that it can be applied to an RTB use case.

cs.LG↗
arXiv2021

Carleson measure estimates for caloric functions and parabolic uniformly rectifiable sets

Let $E \subset \mathbb R^{n+1}$ be a parabolic uniformly rectifiable set. We prove that every bounded solution $u$ to $$\partial_tu- Δu=0, \quad \text{in} \quad \mathbb R^{n+1}\setminus E$$ satisfies a Carleson measure estimate condition. An important technical novelty of our work is that we develop a corona domain approximation scheme for $E$ in terms of regular Lip(1/2,1) graph domains. This approximation scheme has an analogous elliptic version which is an improvement of the known results in that setting.

math.AP↗
arXiv2021

On Big Pieces approximations of parabolic hypersurfaces

Let $Σ$ be a closed subset of $\mathbb{R}^ {n+1}$ which is parabolic Ahlfors-David regular and assume that $Σ$ satisfies a 2-sided corkscrew condition. Assume, in addition, that $Σ$ is either time-forwards Ahlfors-David regular, time-backwards Ahlfors-David regular, or parabolic uniform rectifiable. We then first prove that $Σ$ satisfies a {\it weak synchronized two cube condition}. Based on this we are able to revisit the argument in \cite{NS} and prove that $Σ$ contains {\it uniform big pieces of Lip(1,1/2) graphs}. When $Σ$ is parabolic uniformly rectifiable the construction can be refined and in this case we prove that $Σ$ contains {\it uniform big pieces of regular parabolic Lip(1,1/2) graphs}. Similar results hold if $Ω\subset\mathbb R^{n+1}$ is a connected component of $\mathbb R^{n+1}\setminusΣ$ and in this context we also give a parabolic counterpart of the main result in \cite{AHMNT} by proving that if $Ω$ is a one-sided parabolic chord arc domain, and if $Σ$ is parabolic uniformly rectifiable, then $Ω$ is in fact a parabolic chord arc domain. Our results give a flexible parabolic version of the classical (elliptic) result of G. David and D. Jerison concerning the existence of uniform big pieces of Lipschitz graphs for sets satisfying a two disc condition.

math.AP↗