Searcharxiv⌕ Search

arXiv subjects

Ben Krause

Publications and source records attributed to Ben Krause.

At least 37 records · Page 2Linked to original sources
arXiv2024

A Framework for Evaluating Appropriateness, Trustworthiness, and Safety in Mental Wellness AI Chatbots

Large language model (LLM) chatbots are susceptible to biases and hallucinations, but current evaluations of mental wellness technologies lack comprehensive case studies to evaluate their practical applications. Here, we address this gap by introducing the MHealth-EVAL framework, a new role-play based interactive evaluation method designed specifically for evaluating the appropriateness, trustworthiness, and safety of mental wellness chatbots. We also introduce Psyfy, a new chatbot leveraging LLMs to facilitate transdiagnostic Cognitive Behavioral Therapy (CBT). We demonstrate the MHealth-EVAL framework's utility through a comparative study of two versions of Psyfy against standard baseline chatbots. Our results showed that Psyfy chatbots outperformed the baseline chatbots in delivering appropriate responses, engaging users, and avoiding untrustworthy responses. However, both Psyfy and the baseline chatbots exhibited some limitations, such as providing predominantly US-centric resources. While Psyfy chatbots were able to identify most unsafe situations and avoid giving unsafe responses, they sometimes struggled to recognize subtle harmful intentions when prompted in role play scenarios. Our study demonstrates a practical application of the MHealth-EVAL framework and showcases Psyfy's utility in harnessing LLMs to enhance user engagement and provide flexible and appropriate responses aligned with an evidence-based CBT approach.

cs.HC↗
arXiv2024

AutoGRAMS: Autonomous Graphical Agent Modeling Software

We introduce the AutoGRAMS framework for programming multi-step interactions with language models. AutoGRAMS represents AI agents as a graph, where each node can execute either a language modeling instruction or traditional code. Likewise, transitions in the graph can be governed by either language modeling decisions or traditional branch logic. AutoGRAMS supports using variables as memory and allows nodes to call other AutoGRAMS graphs as functions. We show how AutoGRAMS can be used to design highly sophisticated agents, including self-referential agents that can modify their own graph. AutoGRAMS's graph-centric approach aids interpretability, controllability, and safety during the design, development, and deployment of AI agents. We provide our framework as open source at https://github.com/autograms/autograms .

cs.CL↗
arXiv2024

Averages over the Gaussian Primes: Goldbach's Conjecture and Improving Estimates

We prove versions of Goldbach conjectures for Gaussian primes in arbitrary sectors. Fix an interval $ω\subset \mathbb{T}$. There is an integer $N_ω$, so that every odd integer $n$ with $N(n)>N_ω$ and $\text{dist}( \text{arg}(n) , \mathbb{T}\setminus ω) > (\log N(n)) ^{-B}$, is a sum of three Gaussian primes $n=p_1+p_2+p_3$, with $\text{arg}(p_j) \in ω$, for $j=1,2,3$. A density version of the binary Goldbach conjecture in a sector is also proved.

math.NT↗
arXiv2023

XGen-7B Technical Report

Large Language Models (LLMs) have become ubiquitous across various domains, transforming the way we interact with information and conduct research. However, most high-performing LLMs remain confined behind proprietary walls, hindering scientific progress. Most open-source LLMs, on the other hand, are limited in their ability to support longer sequence lengths, which is a key requirement for many tasks that require inference over an input context. To address this, we have trained XGen, a series of 7B parameter models on up to 8K sequence length for up to 1.5T tokens. We have also finetuned the XGen models on public-domain instructional data, creating their instruction-tuned counterparts (XGen-Inst). We open-source our models for both research advancements and commercial applications. Our evaluation on standard benchmarks shows that XGen models achieve comparable or better results when compared with state-of-the-art open-source LLMs. Our targeted evaluation on long sequence modeling tasks shows the benefits of our 8K-sequence models over 2K-sequence open-source LLMs.

cs.CL↗
arXiv2022

Discrete Analogues in Harmonic Analysis: A Theorem of Stein-Wainger

For $d \geq 2, \ D \geq 1$, let $\mathscr{P}_{d,D}$ denote the set of all degree $d$ polynomials in $D$ dimensions with real coefficients without linear terms. We prove that for any Calderón-Zygmund kernel, $K$, the maximally modulated and maximally truncated discrete singular integral operator, \begin{align*} \sup_{P \in \mathscr{P}_{d,D}, \ N} \Big| \sum_{0 < |m| \leq N} f(x-m) K(m) e^{2πi P(m)} \Big|, \end{align*} is bounded on $\ell^p(\mathbb{Z}^D)$, for each $1 < p < \infty$. Our proof introduces a stopping time based off of equidistribution theory of polynomial orbits to relate the analysis to its continuous analogue, introduced and studied by Stein-Wainger: \begin{align*} \sup_{P \in \mathscr{P}_{d,D}} \Big| \int_{\mathbb{R}^D} f(x-t) K(t) e^{2πi P(t)} \ dt \Big|. \end{align*}

math.CA↗
arXiv2022

Pointwise ergodic theorems for non-conventional bilinear polynomial averages

We establish convergence in norm and pointwise almost everywhere for the non-conventional (in the sense of Furstenberg) bilinear polynomial ergodic averages \[ A_N(f,g)(x) := \frac{1}{N} \sum_{n =1}^N f(T^nx) g(T^{P(n)}x)\] as $N \to \infty$, where $T \colon X \to X$ is a measure-preserving transformation of a $σ$-finite measure space $(X,μ)$, $P(\mathrm{n}) \in \mathbb Z[\mathrm{n}]$ is a polynomial of degree $d \geq 2$, and $f \in L^{p_1}(X), \ g \in L^{p_2}(X)$ for some $p_1,p_2 > 1$ with $\frac{1}{p_1} + \frac{1}{p_2} \leq 1$. We also establish an $r$-variational inequality for these averages (at lacunary scales) in the optimal range $r > 2$. We are also able to "break duality" by handling some ranges of exponents $p_1,p_2$ with $\frac{1}{p_1}+\frac{1}{p_2} > 1$, at the cost of increasing $r$ slightly. This gives an affirmative answer to Problem 11 from Frantzikinakis' open problems survey for the Furstenberg--Weiss averages (with $P(\mathrm{n})=\mathrm{n}^2$), which is a bilinear variant of Question 9 considered by Bergelson in his survey on Ergodic Ramsey Theory from 1996. This also gives a contribution to the Furstenberg-Bergelson-Leibman conjecture. Our methods combine techniques from harmonic analysis with the recent inverse theorems of Peluse and Prendiville in additive combinatorics. At large scales, the harmonic analysis of the adelic integers $\mathbb A_{\mathbb Z}$ also plays a role.

math.DS↗
arXiv2021

Deep Extrapolation for Attribute-Enhanced Generation

Attribute extrapolation in sample generation is challenging for deep neural networks operating beyond the training distribution. We formulate a new task for extrapolation in sequence generation, focusing on natural language and proteins, and propose GENhance, a generative framework that enhances attributes through a learned latent space. Trained on movie reviews and a computed protein stability dataset, GENhance can generate strongly-positive text reviews and highly stable protein sequences without being exposed to similar data during training. We release our benchmark tasks and models to contribute to the study of generative modeling extrapolation and data-driven design in biology and chemistry.

cs.LG↗
arXiv2021

Discrete analogues of maximally modulated singular integrals of Stein-Wainger type

Consider the maximal operator $$\mathscr{C} f(x) = \sup_{λ\in\mathbb{R}}\Big|\sum_{\substack{y\in\mathbb{Z}^n\setminus\{0\}}} f(x-y) e(λ|y|^{2d}) K(y)\Big|,\quad (x\in\mathbb{Z}^n),$$ where $d$ is a positive integer, $K$ a Calderón-Zygmund kernel and $n\ge 1$. This is a discrete analogue of a real-variable operator studied by Stein and Wainger. The nonlinearity of the phase introduces a variety of new difficulties that are not present in the real-variable setting. We prove $\ell^2(\mathbb{Z}^n)$-bounds for $\mathscr{C}$, answering a question posed by Lillian Pierce.

math.CA↗
arXiv2020

GeDi: Generative Discriminator Guided Sequence Generation

While large-scale language models (LMs) are able to imitate the distribution of natural language well enough to generate realistic text, it is difficult to control which regions of the distribution they generate. This is especially problematic because datasets used for training large LMs usually contain significant toxicity, hate, bias, and negativity. We propose GeDi as an efficient method for using smaller LMs as generative discriminators to guide generation from large LMs to make them safer and more controllable. GeDi guides generation at each step by computing classification probabilities for all possible next tokens via Bayes rule by normalizing over two class-conditional distributions; one conditioned on the desired attribute, or control code, and another conditioned on the undesired attribute, or anti control code. We find that GeDi gives stronger controllability than the state of the art method while also achieving generation speeds more than 30 times faster. Additionally, training GeDi on only four topics allows us to controllably generate new topics zero-shot from just a keyword, unlocking a new capability that previous controllable generation methods do not have. Lastly, we show that GeDi can make GPT-2 (1.5B parameters) significantly less toxic without sacrificing linguistic quality, making it by far the most practical existing method for detoxifying large language models while maintaining a fast generation speed.

cs.CL↗
arXiv2020

Explaining and Improving Model Behavior with k Nearest Neighbor Representations

Interpretability techniques in NLP have mainly focused on understanding individual predictions using attention visualization or gradient-based saliency maps over tokens. We propose using k nearest neighbor (kNN) representations to identify training examples responsible for a model's predictions and obtain a corpus-level understanding of the model's behavior. Apart from interpretability, we show that kNN representations are effective at uncovering learned spurious associations, identifying mislabeled examples, and improving the fine-tuned model's performance. We focus on Natural Language Inference (NLI) as a case study and experiment with multiple datasets. Our method deploys backoff to kNN for BERT and RoBERTa on examples with low model confidence without any update to the model parameters. Our results indicate that the kNN approach makes the finetuned model more robust to adversarial inputs.

cs.CL↗
arXiv2020

On Maximal Functions With Curvature

We exhibit a class of "relatively curved" $\vecγ(t) := (γ_1(t),\dots,γ_n(t))$, so that the pertaining multi-linear maximal function satisfies the sharp range of Hölder exponents, \[ \left\| \sup_{r > 0} \ \frac{1}{r} \int_{0}^r \prod_{i=1}^n |f_i(x-γ_i(t))| \ dt \right\|_{L^p(\mathbb{R})} \leq C \cdot \prod_{i=1}^n \| f_j \|_{L^{p_j}(\mathbb{R})} \] whenever $\frac{1}{p} = \sum_{j=1}^n \frac{1}{p_j}$, where $p_j > 1$ and $p \geq p_{\vecγ}$, where $1 \geq p_{\vecγ} > 1/n$ for certain curves. For instance, $p_{\vecγ} = 1/n^+$ for the case of fractional monomials, \[ \vecγ(t) = (t^{α_1},\dots,t^{α_n}), \; \; \; α_1 < \dots < α_n.\] Two sample applications of our method are as follows: For any measurable $u_1,\dots,u_n : \mathbb{R}^{n} \to \mathbb{R}$, with $u_i$ independent of the $i$th coordinate vector, and any relatively curved $\vecγ$, \[ \lim_{r \to 0} \ \frac{1}{r} \int_0^r F\big(x_1 - u_1(x) \cdot γ_1(t),\dots,x_n - u_n(x) \cdot γ_n(t) \big) \ dt = F(x_1,\dots,x_n), \; \; \; a.e. \] for every $F \in L^p(\mathbb{R}^n), \ p > 1$. Every appropriately normalized set $A \subset [0,1]$ of sufficiently large Hausdorff dimension contains the progression, \[ \{ x, x-γ_1(t),\dots,x - γ_n(t) \} \subset A, \] for some $t \geq c_{\vecγ} > 0$ strictly bounded away from zero, depending on $\vecγ$.

math.CA↗
arXiv2020

Averages Along the Primes: Improving and Sparse Bounds

Consider averages along the prime integers $ \mathbb P $ given by \begin{equation*} \mathcal{A}_N f (x) = N ^{-1} \sum_{ p \in \mathbb P \;:\; p\leq N} (\log p) f (x-p). \end{equation*} These averages satisfy a uniform scale-free $ \ell ^{p}$-improving estimate. For all $ 1< p < 2$, there is a constant $ C_p$ so that for all integer $ N$ and functions $ f$ supported on $ [0,N]$, there holds \begin{equation*} N ^{-1/p' }\lVert \mathcal{A}_N f\rVert_{\ell^{p'}} \leq C_p N ^{- 1/p} \lVert f\rVert_{\ell^p}. \end{equation*} The maximal function $ \mathcal{A}^{\ast} f =\sup_{N} \lvert \mathcal{A}_N f \rvert$ satisfies $ (p,p)$ sparse bounds for all $ 1< p < 2$. The latter are the natural variants of the scale-free bounds. As a corollary, $ \mathcal{A}^{\ast} $ is bounded on $ \ell ^{p} (w)$, for all weights $ w$ in the Muckenhoupt $A_p$ class. No prior weighted inequalities for $ \mathcal{A}^{\ast} $ were known.

math.CA↗
arXiv2020

Taylorized Training: Towards Better Approximation of Neural Network Training at Finite Width

We propose \emph{Taylorized training} as an initiative towards better understanding neural network training at finite width. Taylorized training involves training the $k$-th order Taylor expansion of the neural network at initialization, and is a principled extension of linearized training---a recently proposed theory for understanding the success of deep learning. We experiment with Taylorized training on modern neural network architectures, and show that Taylorized training (1) agrees with full neural network training increasingly better as we increase $k$, and (2) can significantly close the performance gap between linearized and full training. Compared with linearized training, higher-order training works in more realistic settings such as standard parameterization and large (initial) learning rate. We complement our experiments with theoretical results showing that the approximation error of $k$-th order Taylorized models decay exponentially over $k$ in wide neural networks.

cs.LG↗
arXiv2019

Discrete Analogues in Harmonic Analysis: Directional Maximal Functions in $\mathbb{Z}^2$

Let $V = \{ v_1,\dots,v_N\}$ be a collection of $N$ vectors that live near a discrete sphere. We consider discrete directional maximal functions on $\mathbb{Z}^2$ where the set of directions lies in $V$, given by \[ \sup_{v \in V, k \geq C \log N} \left| \sum_{n \in \mathbb{Z}} f(x-v\cdot n ) \cdot ϕ_k(n) \right|, \ f:\mathbb{Z}^2 \to \mathbb{C}, \] where and $ϕ_k(t) := 2^{-k} ϕ(2^{-k} t)$ for some bump function $ϕ$. Interestingly, the study of these operators leads one to consider an "arithmetic version" of a Kakeya-type problem in the plane, which we approach using a combination of geometric and number-theoretic methods. Motivated by the Furstenberg problem from geometric measure theory, we also consider a discrete directional maximal operator along polynomial orbits, \[ \sup_{v \in V} \left| \sum_{n \in \mathbb{Z}} f(x-v\cdot P(n) ) \cdot ϕ_k(n) \right|, \ P \in \mathbb{Z}[-] \] for $k \geq C_d \log N$ sufficiently large.

math.CA↗
arXiv2019

Directional maximal function along the primes

We study a two-dimensional discrete directional maximal operator along the set of the prime numbers. We show existence of a set of vectors, which are lattice points in a sufficiently large annulus, for which the $\ell^2$ norm of the associated maximal operator with supremum taken over all large scales grows with an epsilon power in the number of vectors. This paper is a follow-up to a prior work on the discrete directional maximal operator along the integers by the first and third author.

math.CA↗