arXiv subjects

Jeremy Chen

Publications and source records attributed to Jeremy Chen.

6 recordsLinked to original sources

arXiv2025

Training LLMs for Honesty via Confessions

Large language models (LLMs) can be dishonest when reporting on their actions and beliefs -- for example, they may overstate their confidence in factual claims or cover up evidence of covert actions. Such dishonesty may arise due to the effects of reinforcement learning (RL), where challenges with reward shaping can result in a training process that inadvertently incentivizes the model to lie or misrepresent its actions. In this work we propose a method for eliciting an honest expression of an LLM's shortcomings via a self-reported *confession*. A confession is an output, provided upon request after a model's original answer, that is meant to serve as a full account of the model's compliance with the letter and spirit of its policies and instructions. The reward assigned to a confession during training is solely based on its honesty, and does not impact positively or negatively the main answer's reward. As long as the "path of least resistance" for maximizing confession reward is to surface misbehavior rather than covering it up, this incentivizes models to be honest in their confessions. Our findings provide some justification this empirical assumption, especially in the case of egregious model misbehavior. To demonstrate the viability of our approach, we train GPT-5-Thinking to produce confessions, and we evaluate its honesty in out-of-distribution scenarios measuring hallucination, instruction following, scheming, and reward hacking. We find that when the model lies or omits shortcomings in its "main" answer, it often confesses to these behaviors honestly, and this confession honesty modestly improves with training. Confessions can enable a number of inference-time interventions including monitoring, rejection sampling, and surfacing issues to the user.

cs.LG↗

arXiv2024

TORAX: A Fast and Differentiable Tokamak Transport Simulator in JAX

We present TORAX, a new, open-source, differentiable tokamak core transport simulator implemented in Python using the JAX framework. TORAX solves the coupled equations for ion heat transport, electron heat transport, particle transport, and current diffusion, incorporating modular physics-based and ML models. JAX's just-in-time compilation ensures fast runtimes, while its automatic differentiation capability enables gradient-based optimization workflows and simplifies the use of Jacobian-based PDE solvers. Coupling to ML-surrogates of physics models is greatly facilitated by JAX's intrinsic support for neural network development and inference. TORAX is verified against the established RAPTOR code, demonstrating agreement in simulated plasma profiles. TORAX provides a powerful and versatile tool for accelerating research in tokamak scenario modeling, pulse design, and control.

physics.plasm-ph↗

arXiv2024

Gemma: Open Models Based on Gemini Research and Technology

This work introduces Gemma, a family of lightweight, state-of-the art open models built from the research and technology used to create Gemini models. Gemma models demonstrate strong performance across academic benchmarks for language understanding, reasoning, and safety. We release two sizes of models (2 billion and 7 billion parameters), and provide both pretrained and fine-tuned checkpoints. Gemma outperforms similarly sized open models on 11 out of 18 text-based tasks, and we present comprehensive evaluations of safety and responsibility aspects of the models, alongside a detailed description of model development. We believe the responsible release of LLMs is critical for improving the safety of frontier models, and for enabling the next wave of LLM innovations.

cs.CL↗

arXiv2021

Accurate Summary-based Cardinality Estimation Through the Lens of Cardinality Estimation Graphs

We study two classes of summary-based cardinality estimators that use statistics about input relations and small-size joins in the context of graph database management systems: (i) optimistic estimators that make uniformity and conditional independence assumptions; and (ii) the recent pessimistic estimators that use information theoretic linear programs. We begin by addressing the problem of how to make accurate estimates for optimistic estimators. We model these estimators as picking bottom-to-top paths in a cardinality estimation graph (CEG), which contains sub-queries as nodes and weighted edges between sub-queries that represent average degrees. We outline a space of heuristics to make an optimistic estimate in this framework and show that effective heuristics depend on the structure of the input queries. We observe that on acyclic queries and queries with small-size cycles, using the maximum-weight path is an effective technique to address the well known underestimation problem for optimistic estimators. We show that on a large suite of datasets and workloads, the accuracy of such estimates is up to three orders of magnitude more accurate in mean q-error than some prior heuristics that have been proposed in prior work. In contrast, we show that on queries with larger cycles these estimators tend to overestimate, which can partially be addressed by using minimum weight paths and more effectively by using an alternative CEG. We then show that CEGs can also model the recent pessimistic estimators. This surprising result allows us to connect two disparate lines of work on optimistic and pessimistic estimators, adopt an optimization from pessimistic estimators to optimistic ones, and provide insights into the pessimistic estimators, such as showing that there are alternative combinatorial solutions to the linear programs that define them.

cs.DB↗

arXiv2014

Weighing the "Heaviest" Polya Urn

In the classical Polya urn problem, one begins with $d$ bins, each containing one ball. Additional balls arrive one at a time, and the probability that an arriving ball is placed in a given bin is proportional to $m^γ$, where $m$ is the number of balls in that bin. In this note, we consider the case of $γ= 1$, which corresponds to a process of "proportional preferential attachment" and is a critical point with respect to the limit distribution of the fraction of balls in each bin. It is well known that for $γ< 1$ the fraction of balls in the "heaviest" bin (the bin with the most balls) tends to $1/d$, and for $γ> 1$ the fraction of balls in the "heaviest" bin tends to $1$. To partially fill in the gap for $γ= 1$, we characterize the limit distribution of the fraction of balls in the "heaviest" bin for $γ=1$ by providing explicit analytical expressions for all its moments.

math.PR↗

arXiv2014

Social Networks and the Choices People Make

Social marketing is becoming increasingly important in contemporary business. Central to social marketing is quantifying how consumers choose between alternatives and how they influence each other. This work considers a new but simple multinomial choice model for multiple agents connected in a recommendation network based on the explicit modeling of choice adoption behavior. Efficiently computable closed-form solutions, absent from analyses of threshold/cascade models, are obtained together with insights on how the network affects aggregate decision making. A stylized "brand ambassador" selection problem is posed to model targeting in social marketing. Therein, it is shown that a greedy selection strategy leads to solutions achieving at least $1-1/e$ of the optimal value. In an extended example of imposing exogenous controls, a pricing problem is considered wherein it is shown that the single player profit optimization problem is concave, implying the existence of pure strategy equilibria for the associated pricing game.

cs.SI↗