arXiv subjects

Michael Chen

Publications and source records attributed to Michael Chen.

At least 19 recordsLinked to original sources

arXiv2026

Nano World Models: A Minimalist Implementation of Future Video Prediction

World models have become a central paradigm for learning predictive simulators that support generation, planning, and decision-making. Yet, despite rapid progress in industry-scale interactive video generation, the broader research community still lacks compact, reproducible, and easily extensible implementations for studying the design choices underlying modern world models. We introduce Nano World Models, a minimalist codebase for future video prediction centered around diffusion forcing. Nano World Models provides a unified interface for generative objectives, model scales, action-conditioning mechanisms, latent observation spaces, datasets, evaluation protocols, and long-horizon rollout procedures. This design enables controlled studies of world-modeling components that are often entangled across separate implementations. Through experiments across simple control environments, game simulation, and real-robot data, we examine how prediction parameterization, architecture scale, action injection, sampling budget, and domain complexity affect video prediction quality and autoregressive rollout behavior. By releasing code, configurations, evaluation scripts, and pretrained checkpoints, Nano World Models aims to provide a compact yet extensible experimental substrate for open, reproducible, and scientific world-model research.

cs.CV↗

arXiv2026

DDO-RM: Distribution-Level Policy Improvement after Reward Learning

Recent theory suggests that reward-model-first methods can be more sample-efficient than direct policy fitting when the reward function is statistically simpler than the induced policy. We propose DDO-RM, a finite-candidate decision-optimization method that converts reward scores into an explicit target distribution. Unlike PPO-based RLHF or DPO, DDO-RM performs a KL-regularized mirror-descent update to project the policy toward a reward-improved distribution over a candidate set. Preliminary experiments on Pythia-410M show that DDO-RM outperforms DPO in pair accuracy (0.52 to 0.56) and mean margin (0.13 to 0.53). Our framework provides a principled connection between reward learning and mirror-descent policy improvement.

stat.ML↗

arXiv2026

LLMs in social services: How does chatbot accuracy affect human accuracy?

Social service programs like the Supplemental Nutrition Assistance Program (SNAP, or food stamps) have eligibility rules that can be challenging to understand. For nonprofit caseworkers who often support clients in navigating a dozen or more complex programs, LLM-based chatbots may offer a means to provide better, faster help to clients whose situations may be less common. In this paper, we measure the potential effects of LLM-based chatbot suggestions on caseworkers' ability to provide accurate guidance. We first created a 770-question multiple-choice benchmark dataset of difficult, but realistic questions that a caseworker might receive. Next, using these benchmark questions and corresponding expert-verified answers, we conducted a randomized experiment with caseworkers recruited from nonprofit outreach organizations in Los Angeles. Caseworkers in the control condition did not see chatbot suggestions and had a mean accuracy of 49%. Caseworkers in the treatment condition saw chatbot suggestions that we artificially varied to range in aggregate accuracy from low (53%) to high (100%). Caseworker performance significantly improves as chatbot quality improves: high-quality chatbots (96-100% accurate) improved caseworker accuracy by 27 percentage points. At the question-level, incorrect chatbot suggestions substantially reduce caseworker accuracy, with a two-thirds reduction on easy questions where the control group performed best (without chatbot suggestions). Finally, improvements in caseworker accuracy level off as chatbot accuracy increases, a phenomenon that we call the "AI underreliance plateau," which is a concern for real-world deployment and highlights the importance of evaluating human-in-the-loop tools with their users.

cs.HC↗

arXiv2026

Frontier AI Auditing: Toward Rigorous Third-Party Assessment of Safety and Security Practices at Leading AI Companies

We outline a vision for frontier AI auditing, which we define as rigorous third-party verification of frontier AI developers' safety and security claims, and evaluation of their systems and practices against relevant standards, based on deep, secure access to non-public information. Frontier AI audits should not be limited to a company's publicly deployed products, but should instead consider the full range of organization-level safety and security risks, including internal deployment of AI systems, information security practices, and safety decision-making processes. We describe four AI Assurance Levels (AALs), the higher levels of which provide greater confidence in audit findings. We recommend AAL-1 as a baseline for frontier AI generally, and AAL-2 as a near-term goal for the most advanced subset of frontier AI developers. Achieving the vision we outline will require (1) ensuring high quality standards for frontier AI auditing, so it does not devolve into a checkbox exercise or lag behind changes in the industry; (2) growing the ecosystem of audit providers at a rapid pace without compromising quality; (3) accelerating adoption of frontier AI auditing by clarifying and strengthening incentives; and (4) achieving technical readiness for high AI Assurance Levels so they can be applied when needed.

cs.CY↗

arXiv2025

List Replicable Reinforcement Learning

Replicability is a fundamental challenge in reinforcement learning (RL), as RL algorithms are empirically observed to be unstable and sensitive to variations in training conditions. To formally address this issue, we study \emph{list replicability} in the Probably Approximately Correct (PAC) RL framework, where an algorithm must return a near-optimal policy that lies in a \emph{small list} of policies across different runs, with high probability. The size of this list defines the \emph{list complexity}. We introduce both weak and strong forms of list replicability: the weak form ensures that the final learned policy belongs to a small list, while the strong form further requires that the entire sequence of executed policies remains constrained. These objectives are challenging, as existing RL algorithms exhibit exponential list complexity due to their instability. Our main theoretical contribution is a provably efficient tabular RL algorithm that guarantees list replicability by ensuring the list complexity remains polynomial in the number of states, actions, and the horizon length. We further extend our techniques to achieve strong list replicability, bounding the number of possible policy execution traces polynomially with high probability. Our theoretical result is made possible by key innovations including (i) a novel planning strategy that selects actions based on lexicographic order among near-optimal choices within a randomly chosen tolerance threshold, and (ii) a mechanism for testing state reachability in stochastic environments while preserving replicability. Finally, we demonstrate that our theoretical investigation sheds light on resolving the \emph{instability} issue of RL algorithms used in practice. In particular, we show that empirically, our new planning strategy can be incorporated into practical RL frameworks to enhance their stability.

cs.LG↗

arXiv2025

Product Depth for Temporal Point Processes Observed Only Up to the First k Events

Temporal point processes (TPPs) model the timing of discrete events along a timeline and are widely used in fields such as neuroscience and fi- nance. Statistical depth functions are powerful tools for analyzing centrality and ranking in multivariate and functional data, yet existing depth notions for TPPs remain limited. In this paper, we propose a novel product depth specifically designed for TPPs observed only up to the first k events. Our depth function comprises two key components: a normalized marginal depth, which captures the temporal distribution of the final event, and a conditional depth, which characterizes the joint distribution of the preceding events. We establish its key theoretical properties and demonstrate its practical utility through simulation studies and real data applications.

stat.ME↗

arXiv2025

Asymptotic linear dependence and ellipse statistics for multivariate two-sample homogeneity test

Statistical depth, which measures the center-outward rank of a given sample with respect to its underlying distribution, has become a popular and powerful tool in nonparametric inference. In this paper, we investigate the use of statistical depth in multivariate two-sample problems. We propose a new depth-based nonparametric two-sample test, which has the Chi-square(1) asymptotic distribution under the null hypothesis. Simulations and real-data applications highlight the efficacy and practical value of the proposed test.

stat.ME↗

arXiv2025

An Evaluation of Representation Learning Methods in Particle Physics Foundation Models

We present a systematic evaluation of representation learning objectives for particle physics within a unified framework. Our study employs a shared transformer-based particle-cloud encoder with standardized preprocessing, matched sampling, and a consistent evaluation protocol on a jet classification dataset. We compare contrastive (supervised and self-supervised), masked particle modeling, and generative reconstruction objectives under a common training regimen. In addition, we introduce targeted supervised architectural modifications that achieve state-of-the-art performance on benchmark evaluations. This controlled comparison isolates the contributions of the learning objective, highlights their respective strengths and limitations, and provides reproducible baselines. We position this work as a reference point for the future development of foundation models in particle physics, enabling more transparent and robust progress across the community.

cs.LG↗

arXiv2025

Emergency Response Measures for Catastrophic AI Risk

Chinese authorities are extending the country's four-phase emergency response framework (prevent, warn, respond, and recover) to address risks from advanced artificial intelligence (AI). Concrete mechanisms for the proactive prevention and warning phases, however, remain under development. This paper analyzes an implementation model inspired by international AI safety practices: frontier safety policies (FSPs). These policies feature pre-deployment evaluations for dangerous capabilities and tiered, pre-planned safety measures. We observe close alignment between FSPs and the proactive phases of China's emergency response framework, suggesting that the FSP model could help operationalize AI emergency preparedness in a manner consistent with China's established governance principles.

cs.CY↗

arXiv2025

AgentCaster: Reasoning-Guided Tornado Forecasting

There is a growing need to evaluate Large Language Models (LLMs) on complex, high-impact, real-world tasks to assess their true readiness as reasoning agents. To address this gap, we introduce AgentCaster, a contamination-free framework employing multimodal LLMs end-to-end for the challenging, long-horizon task of tornado forecasting. Within AgentCaster, models interpret heterogeneous spatiotemporal data from a high-resolution convection-allowing forecast archive. We assess model performance over a 40-day period featuring diverse historical data, spanning several major tornado outbreaks and including over 500 tornado reports. Each day, models query interactively from a pool of 3,625 forecast maps and 40,125 forecast soundings for a forecast horizon of 12-36 hours. Probabilistic tornado-risk polygon predictions are verified against ground truths derived from geometric comparisons across disjoint risk bands in projected coordinate space. To quantify accuracy, we propose domain-specific TornadoBench and TornadoHallucination metrics, with TornadoBench highly challenging for both LLMs and domain expert human forecasters. Notably, human experts significantly outperform state-of-the-art models, which demonstrate a strong tendency to hallucinate and overpredict risk intensity, struggle with precise geographic placement, and exhibit poor spatiotemporal reasoning in complex, dynamically evolving systems. AgentCaster aims to advance research on improving LLM agents for challenging reasoning tasks in critical domains.

cs.LG↗

arXiv2025

Scale-Consistent Learning for Partial Differential Equations

Machine learning (ML) models have emerged as a promising approach for solving partial differential equations (PDEs) in science and engineering. Previous ML models typically cannot generalize outside the training data; for example, a trained ML model for the Navier-Stokes equations only works for a fixed Reynolds number ($Re$) on a pre-defined domain. To overcome these limitations, we propose a data augmentation scheme based on scale-consistency properties of PDEs and design a scale-informed neural operator that can model a wide range of scales. Our formulation leverages the facts: (i) PDEs can be rescaled, or more concretely, a given domain can be re-scaled to unit size, and the parameters and the boundary conditions of the PDE can be appropriately adjusted to represent the original solution, and (ii) the solution operators on a given domain are consistent on the sub-domains. We leverage these facts to create a scale-consistency loss that encourages matching the solutions evaluated on a given domain and the solution obtained on its sub-domain from the rescaled PDE. Since neural operators can fit to multiple scales and resolutions, they are the natural choice for incorporating scale-consistency loss during training of neural PDE solvers. We experiment with scale-consistency loss and the scale-informed neural operator model on the Burgers' equation, Darcy Flow, Helmholtz equation, and Navier-Stokes equations. With scale-consistency, the model trained on $Re$ of 1000 can generalize to $Re$ ranging from 250 to 10000, and reduces the error by 34% on average of all datasets compared to baselines.

cs.LG↗

arXiv2025

Third-party compliance reviews for frontier AI safety frameworks

Safety frameworks have emerged as a best practice for managing risks from frontier artificial intelligence (AI) systems. However, it may be difficult for stakeholders to know if companies are adhering to their frameworks. This paper explores a potential solution: third-party compliance reviews. During a third-party compliance review, an independent external party assesses whether a frontier AI company is complying with its safety framework. First, we discuss the main benefits and challenges of such reviews. On the one hand, they can increase compliance with safety frameworks and provide assurance to internal and external stakeholders. On the other hand, they can create information security risks, impose additional cost burdens, and cause reputational damage, but these challenges can be partially mitigated by drawing on best practices from other industries. Next, we answer practical questions about third-party compliance reviews, namely: (1) Who could conduct the review? (2) What information sources could the reviewer consider? (3) How could compliance with the safety framework be assessed? (4) What information about the review could be disclosed externally? (5) How could the findings guide development and deployment actions? (6) When could the reviews be conducted? For each question, we evaluate a set of plausible options. Finally, we suggest "minimalist", "more ambitious", and "comprehensive" approaches for each question that a frontier AI company could adopt.

cs.CY↗

arXiv2025

Mapping Industry Practices to the EU AI Act's GPAI Code of Practice Safety and Security Measures

This report provides a detailed comparison between the Safety and Security measures proposed in the EU AI Act's General-Purpose AI (GPAI) Code of Practice (Third Draft) and the current commitments and practices voluntarily adopted by leading AI companies. As the EU moves toward enforcing binding obligations for GPAI model providers, the Code of Practice will be key for bridging legal requirements with concrete technical commitments. Our analysis focuses on the draft's Safety and Security section (Commitments II.1-II.16), documenting excerpts from current public-facing documents that are relevant to each individual measure. We systematically reviewed different document types, such as companies' frontier safety frameworks and model cards, from over a dozen companies, including OpenAI, Anthropic, Google DeepMind, Microsoft, Meta, Amazon, and others. This report is not meant to be an indication of legal compliance, nor does it take any prescriptive viewpoint about the Code of Practice or companies' policies. Instead, it aims to inform the ongoing dialogue between regulators and General-Purpose AI model providers by surfacing evidence of industry precedent for various measures. Nonetheless, we were able to find relevant quotes from at least 5 companies' documents for the majority of the measures in Commitments II.1-II.16.

cs.CY↗

arXiv2025

Uncertainty-Aware Graph Self-Training with Expectation-Maximization Regularization

In this paper, we propose a novel \emph{uncertainty-aware graph self-training} approach for semi-supervised node classification. Our method introduces an Expectation-Maximization (EM) regularization scheme to incorporate an uncertainty mechanism during pseudo-label generation and model retraining. Unlike conventional graph self-training pipelines that rely on fixed pseudo-labels, our approach iteratively refines label confidences with an EM-inspired uncertainty measure. This ensures that the predictive model focuses on reliable graph regions while gradually incorporating ambiguous nodes. Inspired by prior work on uncertainty-aware self-training techniques~\cite{wang2024uncertainty}, our framework is designed to handle noisy graph structures and feature spaces more effectively. Through extensive experiments on several benchmark graph datasets, we demonstrate that our method outperforms strong baselines by a margin of up to 2.5\% in accuracy while maintaining lower variance in performance across multiple runs.

cs.LG↗

arXiv2025

Not-Just-Scaling Laws: Towards a Better Understanding of the Downstream Impact of Language Model Design Decisions

Improvements in language model capabilities are often attributed to increasing model size or training data, but in some cases smaller models trained on curated data or with different architectural decisions can outperform larger ones trained on more tokens. What accounts for this? To quantify the impact of these design choices, we meta-analyze 92 open-source pretrained models across a wide array of scales, including state-of-the-art open-weights models as well as less performant models and those with less conventional design decisions. We find that by incorporating features besides model size and number of training tokens, we can achieve a relative 3-28% increase in ability to predict downstream performance compared with using scale alone. Analysis of model design decisions reveal insights into data composition, such as the trade-off between language and code tasks at 15-25\% code, as well as the better performance of some architectural decisions such as choosing rotary over learned embeddings. Broadly, our framework lays a foundation for more systematic investigation of how model development choices shape final capabilities.

cs.CL↗

arXiv2025

The JCMT Gould Belt Survey: First results from the Corona Australis molecular cloud and evidence of variable dust emissivity indices in the Coronet region

We present 450$\mu$m and 850$\mu$m James Clerk Maxwell Telescope (JCMT) observations of the Corona Australis (CrA) molecular cloud taken as part of the JCMT Gould Belt Legacy Survey (GBLS). We present a catalogue of 39 starless and protostellar sources, for which we determine source temperatures and masses using SCUBA-2 450$\mu$m/850$\mu$m flux density ratios for sources with reliable 450$\mu$m detections, and compare these to values determined using temperatures measured by the Herschel Gould Belt Survey (HGBS). In keeping with previous studies, we find that SCUBA-2 preferentially detects high-volume-density starless cores, which are most likely to be prestellar (gravitationally bound). We do not observe any anti-correlation between temperature and volume density in the starless cores in our sample. Finally, we combine our SCUBA-2 and Herschel data to perform SED fitting from 160-850$\mu$m across the central Coronet region, thereby measuring dust temperature $T$, dust emissivity index $\beta$ and column density $N({\rm H}_2)$ across the Coronet. We find that $\beta$ varies across the Coronet, particularly measuring $\beta = 1.55 \pm 0.35$ in the colder starless SMM-6 clump to the north of the B star R CrA. This relatively low value of $\beta$ is suggestive of the presence of large dust grains in SMM-6, even when considering the effects of $T-\beta$ fitting degeneracy and $^{12}$CO contamination of SCUBA-2 850$\mu$m data on the measured $\beta$ values.

astro-ph.GA↗

arXiv2024

RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human experts

Frontier AI safety policies highlight automation of AI research and development (R&D) by AI agents as an important capability to anticipate. However, there exist few evaluations for AI R&D capabilities, and none that are highly realistic and have a direct comparison to human performance. We introduce RE-Bench (Research Engineering Benchmark, v1), which consists of 7 challenging, open-ended ML research engineering environments and data from 71 8-hour attempts by 61 distinct human experts. We confirm that our experts make progress in the environments given 8 hours, with 82% of expert attempts achieving a non-zero score and 24% matching or exceeding our strong reference solutions. We compare humans to several public frontier models through best-of-k with varying time budgets and agent designs, and find that the best AI agents achieve a score 4x higher than human experts when both are given a total time budget of 2 hours per environment. However, humans currently display better returns to increasing time budgets, narrowly exceeding the top AI agent scores given an 8-hour budget, and achieving 2x the score of the top AI agent when both are given 32 total hours (across different attempts). Qualitatively, we find that modern AI agents possess significant expertise in many ML topics -- e.g. an agent wrote a faster custom Triton kernel than any of our human experts' -- and can generate and test solutions over ten times faster than humans, at much lower cost. We open-source the evaluation environments, human expert data, analysis code and agent trajectories to facilitate future research.

cs.LG↗

arXiv2024

Probing the Physics of Star-Formation (ProPStar) III. No evidence for dissipation of turbulence down to 20 mpc (4 000 au) scale

Context. Turbulence is a key component of molecular cloud structure. It is usually described by a cascade of energy down to the dissipation scale. The power spectrum for subsonic incompressible turbulence is $k^{-5/3}$, while for supersonic turbulence it is $k^{-2}$. Aims. We aim to determine the power spectrum in an actively star-forming molecular cloud, from parsec scales down to the expected magnetohydrodynamic (MHD) wave cutoff (dissipation scale). Methods. We analyze observations of the nearby NGC 1333 star-forming region in three different tracers to cover the different scales from $\sim$10 pc down to 20 mpc. The largest scales are covered with the low density gas tracer $^{13}$CO (1-0) obtained with single dish, the intermediate scales are covered with single-dish observations of the C$^{18}$O (3-2) line, while the smallest scales are covered in H$^{13}$CO$^+$ (1-0) and HNC (1-0) with a combination of NOEMA interferometer and IRAM 30m single dish observations. The complementarity of these observations enables us to generate a combined power spectrum covering more than two orders of magnitude in spatial scale. Results. We derive the power spectrum in an active star-forming region spanning more than 2 decades of spatial scales. The power spectrum of the intensity maps shows a single power-law behavior, with an exponent of 2.9$\pm$0.1 and no evidence of dissipation. Moreover, there is evidence for the power-spectrum of the ions to have more power at smaller scales than the neutrals, which is opposite from theoretical expectations. Conclusions. We show new possibilities of studying the dissipation of energy at small scales in star-forming regions provided by interferometric observations.

astro-ph.GA↗