arXiv subjects

James M. Johnson

Publications and source records attributed to James M. Johnson.

11 recordsLinked to original sources

arXiv2025

Aligning Human and LLM Judgments: Insights from EvalAssist on Task-Specific Evaluations and AI-assisted Assessment Strategy Preferences

Evaluation of large language model (LLM) outputs requires users to make critical judgments about the best outputs across various configurations. This process is costly and takes time given the large amounts of data. LLMs are increasingly used as evaluators to filter training data, evaluate model performance or assist human evaluators with detailed assessments. To support this process, effective front-end tools are critical for evaluation. Two common approaches for using LLMs as evaluators are direct assessment and pairwise comparison. In our study with machine learning practitioners (n=15), each completing 6 tasks yielding 131 evaluations, we explore how task-related factors and assessment strategies influence criteria refinement and user perceptions. Findings show that users performed more evaluations with direct assessment by making criteria task-specific, modifying judgments, and changing the evaluator model. We conclude with recommendations for how systems can better support interactions in LLM-assisted evaluations.

cs.HC↗

arXiv2025

Emerging Reliance Behaviors in Human-AI Content Grounded Data Generation: The Role of Cognitive Forcing Functions and Hallucinations

We investigate the impact of hallucinations and Cognitive Forcing Functions in human-AI collaborative content-grounded data generation, focusing on the use of Large Language Models (LLMs) to assist in generating high quality conversational data. Through a study with 34 users who each completed 8 tasks (n=272), we found that hallucinations significantly reduce data quality. While Cognitive Forcing Functions do not always alleviate these effects, their presence influences how users integrate AI responses. Specifically, we observed emerging reliance behaviors, with users often appending AI-generated responses to their correct answers, even when the AI's suggestions conflicted. This points to a potential drawback of Cognitive Forcing Functions, particularly when AI suggestions are inaccurate. Users who overrelied on AI-generated text produced lower quality data, emphasizing the nuanced dynamics of overreliance in human-LLM collaboration compared to traditional human-AI decision-making.

cs.HC↗

arXiv2024

Black-box Uncertainty Quantification Method for LLM-as-a-Judge

LLM-as-a-Judge is a widely used method for evaluating the performance of Large Language Models (LLMs) across various tasks. We address the challenge of quantifying the uncertainty of LLM-as-a-Judge evaluations. While uncertainty quantification has been well-studied in other domains, applying it effectively to LLMs poses unique challenges due to their complex decision-making capabilities and computational demands. In this paper, we introduce a novel method for quantifying uncertainty designed to enhance the trustworthiness of LLM-as-a-Judge evaluations. The method quantifies uncertainty by analyzing the relationships between generated assessments and possible ratings. By cross-evaluating these relationships and constructing a confusion matrix based on token probabilities, the method derives labels of high or low uncertainty. We evaluate our method across multiple benchmarks, demonstrating a strong correlation between the accuracy of LLM evaluations and the derived uncertainty scores. Our findings suggest that this method can significantly improve the reliability and consistency of LLM-as-a-Judge evaluations.

cs.LG↗

arXiv1996

Relativistic two-photon and two-gluon decay rates of heavy quarkonia

The decay rates of $c\bar{c}$ and $b\bar{b}$ through two-photon or two-gluon annihilations are obtained by using totally relativistic decay amplitudes and a sophisticated quantum-chromodynamic potential model for heavy quarkonia. Our results for the photonic and gluonic widths of the 1S0, 3P0, and the 3P2 states are in excellent agreement with the available experimental data. The procedures and mathematical techniques used by us for the treatment of the fermion-antifermion bound states are also applicable to other decay processes.

hep-ph↗

arXiv1996

Gauge-boson scattering signals at the LHC

We have extended our earlier treatment of the gauge-boson scattering with radiative corrections in the standard model at supercollider energies, and computed the rates for gauge-boson scattering modes in $pp$ collisions leading to the final states $W^+W^-$, $ZZ(4l)$, $ZZ(2l2ν)$, $W^\pm Z$, and $W^\pm W^\pm $. Our results at the LHC \hbox{energy} of $\sqrt{s}=14$~TeV for $m_H=1000$~GeV are compared with those recently obtained by Bagger {\it et al.} These results will be useful in the search for the Higgs bosons at supercollider energies as well as for experimentally distinguishing the standard model from non-minimal Higgs models.

hep-ph↗

arXiv1995

Bc spectroscopy in a quantum-chromodynamic potential model

We have investigated $B_c$ spectroscopy with the use of a quantum-chromodynamic potential model which was recently used by us for the light-heavy quarkonia. We give our predictions for the energy levels and the $E$1 transition widths. We also find, rather surprisingly, that although $B_c$ is not a light-heavy system, the heavy quark effective theory with the inclusion of the $m_b^{-1}$ and $m_b^{-1}\ln m_b$ corrections is as successful for $B_c$ as it is for $B$ and $B_s$.

hep-ph↗

arXiv1994

Quantum-Chromodynamic Potential Model for Light-Heavy Quarkonia and the Heavy Quark Effective Theory

We have investigated the spectra of light-heavy quarkonia with the use of a quantum-chromodynamic potential model which is similar to that used earlier for the heavy quarkonia. An essential feature of our treatment is the inclusion of the one-loop radiative corrections to the quark-antiquark potential, which contribute significantly to the spin-splittings among the quarkonium energy levels. Unlike $c\bar{c}$ and $b\bar{b}$, the potential for a light-heavy system has a complicated dependence on the light and heavy quark masses $m$ and $M$, and it contains a spin-orbit mixing term. We have obtained excellent results for the observed energy levels of $D^0$, $D_s$, $B^0$, and $B_s$, and we are able to provide predicted results for many unobserved energy levels. Our potential parameters for different quarkonia satisfy the constraints of quantum chromodynamics. We have also used our investigation to test the accuracy of the heavy quark effective theory. We find that the heavy quark expansion yields generally good results for the $B^0$ and $B_s$ energy levels provided that $M^{-1}$ and $M^{-1}\ln M$ corrections are taken into account in the quark-antiquark interactions. It does not, however, provide equally good results for the energy levels of $D^0$ and $D_s$, which indicates that the effective theory can be applied more accurately to the $b$ quark than the $c$ quark.

hep-ph↗

arXiv1994

QCD Potential Model for Light-heavy Quarkonia and the Heavy Quark Effective Theory

We have investigated the spectra of light-heavy quarkonia with the use of a quantum-chromodynamic potential model which is similar to that used earlier for the heavy quarkonia. An essential feature of our treatment is the inclusion of the one-loop radiative corrections to the quark-antiquark potential, which contribute significantly to the spin-splittings among the quarkonium energy levels. Unlike $c\bar{c}$ and $b\bar{b}$, the potential for a light-heavy system has a complicated dependence on the light and heavy quark masses $m$ and $M$, and it contains a spin-orbit mixing term. We have obtained excellent results for the observed energy levels of $D^0$, $D_s$, $B^0$, and $B_s$, and we are able to provide predicted results for many unobserved energy levels. We have also used our investigation to test the accuracy of the heavy quark effective theory. We find that the heavy quark expansion yields generally good results for the $B^0$ and $B_s$ energy levels provided that $M^{-1}$ and $M^{-1}\ln M$ corrections are taken into account in the quark-antiquark interactions. It does not, however, provide equally good results for the energy levels of $D^0$ and $D_s$, which shows that the effective theory can be applied more accurately to the $b$ quark than the $c$ quark.

hep-ph↗

arXiv1993

Heavy Quarkonium Potential Model and the ${}^1P_1$ State of Charmonium

A theoretical explanation of the observed splittings among the P~states of charmonium is given with the use of a nonsingular potential model for heavy quarkonia. We also show that the recently observed mass difference between the center of gravity of the ${}^3P_J$ states and the ${}^1P_1$ state of $c\bar{c}$ does not provide a direct test of the color hyperfine interaction in heavy quarkonia. Our theoretical value for the mass of the ${}^1P_1$ state is in agreement with the experimental result, and its E1 transition width is 341.8~keV. The mass of the $η_c'$ state is predicted to be 3622.3~MeV.

hep-ph↗

arXiv1993

W, Z and Higgs Scattering at SSC Energies

The scattering of $W$, $Z$ and Higgs bosons in the Standard Model is investigated in the region $s,m_H^2\gg m_W^2$ with no restrictions on relative sizes of $s$ and $m_H^2$, so that our results are applicable at energies below as well as above $2m_H$. We have calculated, with the inclusion of the full one-loop corrections, the scattering matrix between the states $W_L^+W_L^-$, $Z_LZ_L$ and $HH$, and computed the S-wave amplitudes as functions of the center-of-mass energy $\sqrt{s}$ for $m_H=$500~GeV and 1000~GeV. The apparent violation of unitarity is avoided by unitarizing the amplitudes by the K-matrix and the Padé methods. For the detection of the Higgs boson through gauge boson scattering in $pp$ collisions, we have used the unitarized amplitudes to obtain the invariant-mass distributions for the final $W_L^+W_L^-$ and $Z_LZ_L$ pairs at the SSC energy of $\sqrt{s}=$40~TeV by means of the effective-W approximation.

hep-ph↗

arXiv1993

W, Z and Higgs Scattering at SSC Energies

We examine the scattering of longitudinal $W$, $Z$ and Higgs bosons in the Standard Model using the equivalent Goldstone-boson Lagrangian. Our calculations include the full one-loop scattering matrix between the states $W^+_LW^-_L$, $Z_LZ_L$ and $HH$ with no restrictions on the relative sizes of $M_H$ and $\sqrt{s}$. In addition to deriving the perturbative eigen-amplitudes, we also obtain quite striking results by unitarizing the amplitudes with the use of the K-matrix and Padé techniques. (Complete postscript file can be obtained by anonymous ftp from hal.physics.wayne.edu as dpf92g.ps in directory pub/physics )

hep-ph↗