arXiv subjects

Jiading Zhu

Publications and source records attributed to Jiading Zhu.

6 recordsLinked to original sources

arXiv2026

Evaluation of Motivational Interviewing Counsellors with Task-Aware Multi-Stage LLM-Based Simulated Clients

The development and benchmarking of Large Language Model (LLM)-based Motivational Interviewing (MI) counsellors now often rely on LLM-based simulated clients. Prior work on simulated clients, however, has not aligned with the specific tasks fundamental to the MI therapy approach. A key task is evoking, in which the counsellor first elicits the client's ambivalence and then strengthens the client's motivation for change. We present Evoke-Sim, a task-aware, multi-stage LLM-based client simulation framework for evaluating MI counsellors in smoking cessation, designed specifically for the evoking MI task. Evoke-Sim employs structured client profiles, an evoking-specific three-stage conversation flow, and a reveal policy that regulates which client profile information might be disclosed at each stage. We show that compared to existing profile-grounded simulated clients, Evoke-Sim is better at differentiating levels of MI quality using task-aware evaluation metrics, while reducing non-grounded client statements and premature disclosure of client information, setting a higher standard for the evaluation of LLM-based MI counsellors.

cs.HC↗

arXiv2026

What are They Thinking? Delineation, Probing, and Tracking of Concepts in LLMs

As the influence of LLMs expands, it is imperative to gain insight into their decisions. One way to do that is to develop probes that detect the presence or absence of a broad set of high-level abstract concepts within the embeddings computed in an LLM - which is what we might say a model is ``thinking" about. Such probes should be low-cost and easily applicable to any LLM, so that monitoring for many concepts is possible during normal operation. In this paper, we take the first steps towards developing the capability of creating many such probes by defining and executing examples of the key tasks needed: first, the careful delineation of a high-level abstract concept through the creation of a dataset with the concept both present and then absent. Then, the training and testing of a set of linear probes to detect the concept on any layer of an LLM, including an exploration of the complexity of the probe needed. Finally, we show that such probes can track concepts across larger contexts. This is done with four separate concepts and three different LLMs. When this process is scaled to many more concepts, it will create the ability to monitor new models.

cs.CL↗

arXiv2026

Contact-Dependent Ion Gating Explains Directional Asymmetry in the Bacterial Flagellar Motor

The bacterial flagellar motor (BFM) is a rotary molecular machine driven by the ion electrochemical potential across the cell membrane. Recent cryo-EM structures reveal a cogwheel-like architecture in which multiple stators engage a large rotor. A longstanding puzzle is the directional asymmetry of its torque-speed relation: concave in counterclockwise (CCW) rotation but nearly linear in clockwise (CW) rotation. Here, we develop a stochastic mechanochemical model that explicitly incorporates rotor-stator coupling and detailed ion translocation kinetics. By integrating physiological torque-speed data with recent measurements of rotor-stator relative motion, we show that under physiological conditions the motor operates in a tight engagement regime, rendering the torque-speed relation largely insensitive to the specific form of mechanical interactions. This finding rules out differences in rotor-stator mechanics as the origin of CW-CCW asymmetry. Guided by cryo-EM structures, we propose a contact-dependent gating mechanism in which the MotA-FliG interaction modulates the ion release rate of the MotB subunit proximal to the FliG ring. Molecular dynamics simulations indicate tighter MotA-FliG contact in the CW motor, implying a reduced ion release rate compared to CCW. Our model demonstrates that differential gating strength accounts for the observed asymmetry: stronger gating in CCW shortens torque-free waiting phases, enhances torque generation, and produces a concave torque-speed curve, whereas weaker gating in CW yields lower torque and a linear relation. This structure-based framework quantitatively links molecular asymmetry to motor function and identifies specific interfaces for targeted perturbation and mutational studies.

physics.bio-ph↗

arXiv2025

A Fully Generative Motivational Interviewing Counsellor Chatbot for Moving Smokers Towards the Decision to Quit

The conversational capabilities of Large Language Models (LLMs) suggest that they may be able to perform as automated talk therapists. It is crucial to know if these systems would be effective and adhere to known standards. We present a counsellor chatbot that focuses on motivating tobacco smokers to quit smoking. It uses a state-of-the-art LLM and a widely applied therapeutic approach called Motivational Interviewing (MI), and was evolved in collaboration with clinician-scientists with expertise in MI. We also describe and validate an automated assessment of both the chatbot's adherence to MI and client responses. The chatbot was tested on 106 participants, and their confidence that they could succeed in quitting smoking was measured before the conversation and one week later. Participants' confidence increased by an average of 1.7 on a 0-10 scale. The automated assessment of the chatbot showed adherence to MI standards in 98% of utterances, higher than human counsellors. The chatbot scored well on a participant-reported metric of perceived empathy but lower than typical human counsellors. Furthermore, participants' language indicated a good level of motivation to change, a key goal in MI. These results suggest that the automation of talk therapy with a modern LLM has promise.

cs.CL↗

arXiv2025

RAG-Verus: Repository-Level Program Verification with LLMs using Retrieval Augmented Generation

Scaling automated formal verification to real-world projects requires resolving cross-module dependencies and global contexts, which are challenges overlooked by existing function-centric methods. We introduce RagVerus, a framework that synergizes retrieval-augmented generation with context-aware prompting to automate proof synthesis for multi-module repositories, achieving a 27% relative improvement on our novel RepoVBench benchmark -- the first repository-level dataset for Verus with 383 proof completion tasks. RagVerus triples proof pass rates on existing benchmarks under constrained language model budgets, demonstrating a scalable and sample-efficient verification.

cs.SE↗

arXiv2024

Generation, Distillation and Evaluation of Motivational Interviewing-Style Reflections with a Foundational Language Model

Large Foundational Language Models are capable of performing many tasks at a high level but are difficult to deploy in many applications because of their size and proprietary ownership. Many will be motivated to distill specific capabilities of foundational models into smaller models that can be owned and controlled. In the development of a therapeutic chatbot, we wish to distill a capability known as reflective listening, in which a therapist produces reflections of client speech. These reflections either restate what a client has said, or connect what was said to a relevant observation, idea or guess that encourages and guides the client to continue contemplation. In this paper, we present a method for distilling the generation of reflections from a Foundational Language Model (GPT-4) into smaller models. We first show that GPT-4, using zero-shot prompting, can generate reflections at near 100% success rate, superior to all previous methods. Using reflections generated by GPT-4, we fine-tune different sizes of the GPT-2 family. The GPT-2-small model achieves 83% success on a hold-out test set and the GPT-2 XL achieves 90% success. We also show that GPT-4 can help in the labor-intensive task of evaluating the quality of the distilled models, using it as a zero-shot classifier. Using triple-human review as a guide, the classifier achieves a Cohen-Kappa of 0.66, a substantial inter-rater reliability figure.

cs.CL↗