arXiv subjects

Jingyi Qu

Publications and source records attributed to Jingyi Qu.

3 recordsLinked to original sources

arXiv2025

Flexible Agent Alignment with Goal Inference from Open-Ended Dialog

We introduce Open-Universe Assistance Games (OU-AGs), a formal framework extending assistance games to LLM-based agents. Effective assistance requires reasoning over human preferences that are unbounded, underspecified, and evolving. Current LLM agents struggle in multi-turn interactions and with maintaining accurate models of user intent in collaborative settings. Existing assistance game formulations assume fixed, predefined preferences, an assumption that breaks down in open-ended dialogue where goals are revised incrementally and expressed in natural language. Grounded in cognitive science accounts of preference construction, we represent human preferences as a dynamically updated distribution over discrete natural-language goals. To operationalize OU-AGs, we introduce GOOD (GOals from Open-ended Dialogue), a data-efficient online method that extracts and ranks candidate goals during interaction, using LLM-simulated users to perform probabilistic inference over goal hypotheses. This allows for interpretable, uncertainty-aware preference representations without large offline datasets. We evaluate GOOD across three text-based domains: grocery shopping, household robotics (AI2-THOR), and coding. Compared to baselines without explicit goal tracking, GOOD produces semantically coherent goal representations and improves alignment with user intent across domains.

cs.AI↗

arXiv2024

Goal Inference from Open-Ended Dialog

Embodied AI Agents are quickly becoming important and common tools in society. These embodied agents should be able to learn about and accomplish a wide range of user goals and preferences efficiently and robustly. Large Language Models (LLMs) are often used as they allow for opportunities for rich and open-ended dialog type interaction between the human and agent to accomplish tasks according to human preferences. In this thesis, we argue that for embodied agents that deal with open-ended dialog during task assistance: 1) AI Agents should extract goals from conversations in the form of Natural Language (NL) to be better at capturing human preferences as it is intuitive for humans to communicate their preferences on tasks to agents through natural language. 2) AI Agents should quantify/maintain uncertainty about these goals to ensure that actions are being taken according to goals that the agent is extremely certain about. We present an online method for embodied agents to learn and accomplish diverse user goals. While offline methods like RLHF can represent various goals but require large datasets, our approach achieves similar flexibility with online efficiency. We extract natural language goal representations from conversations with Large Language Models (LLMs). We prompt an LLM to role play as a human with different goals and use the corresponding likelihoods to run Bayesian inference over potential goals. As a result, our method can represent uncertainty over complex goals based on unrestricted dialog. We evaluate in a text-based grocery shopping domain and an AI2Thor robot simulation. We compare our method to ablation baselines that lack either explicit goal representation or probabilistic inference.

cs.AI↗

arXiv2021

Integrating Hydrogen in Single-Price Electricity Systems: The Effects of Spatial Economic Signals

Hydrogen can contribute substantially to the reduction of carbon emissions in industry and transportation. However, the production of hydrogen through electrolysis creates interdependencies between hydrogen supply chains and electricity systems. Therefore, as governments worldwide are planning considerable financial subsidies and new regulation to promote hydrogen infrastructure investments in the next years, energy policy research is needed to guide such policies with holistic analyses. In this study, we link a electrolytic hydrogen supply chain model with an electricity system dispatch model, for a cross-sectoral case study of Germany in 2030. We find that hydrogen infrastructure investments and their effects on the electricity system are strongly influenced by electricity prices. Given current uniform prices, hydrogen production increases congestion costs in the electricity grid by 17%. In contrast, passing spatially resolved electricity price signals leads to electrolyzers being placed at low-cost grid nodes and further away from consumption centers. This causes lower end-use costs for hydrogen. Moreover, congestion management costs decrease substantially, by up to 20% compared to the benchmark case without hydrogen. These savings could be transferred into according subsidies for hydrogen production. Thus, our study demonstrates the benefits of differentiating economic signals for hydrogen production based on spatial criteria.

econ.GN↗