arXiv subjects

Anthony Liang

Publications and source records attributed to Anthony Liang.

11 recordsLinked to original sources

arXiv2026

Robometer: Scaling General-Purpose Robotic Reward Models via Trajectory Comparisons

General-purpose robot reward models are typically trained to predict absolute task progress from expert demonstrations, providing only local, frame-level supervision. While effective for expert demonstrations, this paradigm scales poorly to large-scale robotics datasets where failed and suboptimal trajectories are abundant and assigning dense progress labels is ambiguous. We introduce Robometer, a scalable reward modeling framework that combines intra-trajectory progress supervision with inter-trajectory preference supervision. Robometer is trained with a dual objective: a frame-level progress loss that anchors reward magnitude on expert data, and a trajectory-comparison preference loss that imposes global ordering constraints across trajectories of the same task, enabling effective learning from both real and augmented failed trajectories. To support this formulation at scale, we curate RBM-1M, a reward-learning dataset comprising over one million trajectories spanning diverse robot embodiments and tasks, including substantial suboptimal and failure data. Across benchmarks and real-world evaluations, Robometer learns more generalizable reward functions than prior methods and improves robot learning performance across a diverse set of downstream applications. Code, model weights, and videos at https://robometer.github.io/.

cs.RO↗

arXiv2025

Plantain: Plan-Answer Interleaved Reasoning

Reasoning models often spend a significant amount of time thinking before they generate a visible response. In the meantime, they do not give the user any hints as to whether their reasoning is on the right track, and do not give the user any recourse to stop and correct them if their reasoning is flawed. This creates a frustrating, but unfortunately common, experience: the user's time is wasted while the model reasons from a false premise that could have easily been corrected. In contrast, human speakers typically perform lightweight, incremental grounding acts to ensure that participants in the conversation are on the same page; here we ask if language models can learn to leverage a similar type of behavior? With this motivation, we propose interleaved reasoning (IR), in which the model alternates between thinking and surfacing intermediate responses, as an alternative to the standard "think-then-answer" approach. By providing useful information to the user earlier, IR reduces perceived latency, the time a user waits for an initial output, without compromising the quality of the final response. We further introduce a specialization of interleaved reasoning, Plantain (Plan-Thought-Answer Interleaving), where the first intermediate response is an explicit, step-by-step plan for executing the task. This plan-first strategy allows for user intervention and early feedback for subsequent reasoning steps. We demonstrate that Plantain yields an ~6% improvement in pass@1 across several challenging math reasoning and coding benchmarks, while reducing time-to-first-response by over 60% relative to think-then-answer baselines.

cs.LG↗

arXiv2025

HAND Me the Data: Fast Robot Adaptation via Hand Path Retrieval

We hand the community HAND, a simple and time-efficient method for teaching robots new manipulation tasks through human hand demonstrations. Instead of relying on task-specific robot demonstrations collected via teleoperation, HAND uses easy-to-provide hand demonstrations to retrieve relevant behaviors from task-agnostic robot play data. Using a visual tracking pipeline, HAND extracts the motion of the human hand from the hand demonstration and retrieves robot sub-trajectories in two stages: first filtering by visual similarity, then retrieving trajectories with similar behaviors to the hand. Fine-tuning a policy on the retrieved data enables real-time learning of tasks in under four minutes, without requiring calibrated cameras or detailed hand pose estimation. Experiments also show that HAND outperforms retrieval baselines by over 2x in average task success rates on real robots. Videos can be found at our project website: https://liralab.usc.edu/handretrieval/.

cs.RO↗

arXiv2025

CLAM: Continuous Latent Action Models for Robot Learning from Unlabeled Demonstrations

Learning robot control policies from demonstrations typically requires action-labeled expert data, which is expensive to collect through teleoperation. We study a more practical setting in which expert demonstrations are available only as observation sequences without action labels, and only task-agnostic play data contains actions. We introduce continuous latent action models (CLAM), a framework that infers continuous latent actions between consecutive observations using self-supervised dynamics prediction. To ground these latent actions into executable motor commands, CLAM jointly trains an action decoder using a small amount of task-agnostic play data. We show that continuous latent actions combined with this joint training are essential for high-dimensional continuous control. Across DMControl locomotion, MetaWorld manipulation, and real-world WidowX robot tasks, CLAM improves average task success rates by 2-3x over prior latent-action baselines and approaches behavior cloning trained with privileged expert action labels. Our results demonstrate that effective robot policies can be learned from unlabeled demonstrations and deployed on real hardware without collecting expert action-labeled data. Videos and code are available at clamrobot.github.io.

cs.RO↗

arXiv2024

ViSaRL: Visual Reinforcement Learning Guided by Human Saliency

Training robots to perform complex control tasks from high-dimensional pixel input using reinforcement learning (RL) is sample-inefficient, because image observations are comprised primarily of task-irrelevant information. By contrast, humans are able to visually attend to task-relevant objects and areas. Based on this insight, we introduce Visual Saliency-Guided Reinforcement Learning (ViSaRL). Using ViSaRL to learn visual representations significantly improves the success rate, sample efficiency, and generalization of an RL agent on diverse tasks including DeepMind Control benchmark, robot manipulation in simulation and on a real robot. We present approaches for incorporating saliency into both CNN and Transformer-based encoders. We show that visual representations learned using ViSaRL are robust to various sources of visual perturbations including perceptual noise and scene variations. ViSaRL nearly doubles success rate on the real-robot tasks compared to the baseline which does not use saliency.

cs.RO↗

arXiv2024

DynaMITE-RL: A Dynamic Model for Improved Temporal Meta-Reinforcement Learning

We introduce DynaMITE-RL, a meta-reinforcement learning (meta-RL) approach to approximate inference in environments where the latent state evolves at varying rates. We model episode sessions - parts of the episode where the latent state is fixed - and propose three key modifications to existing meta-RL methods: consistency of latent information within sessions, session masking, and prior latent conditioning. We demonstrate the importance of these modifications in various domains, ranging from discrete Gridworld environments to continuous-control and simulated robot assistive tasks, demonstrating that DynaMITE-RL significantly outperforms state-of-the-art baselines in sample efficiency and inference returns.

cs.LG↗

arXiv2020

Reinforcement Learning for Sparse-Reward Object-Interaction Tasks in a First-person Simulated 3D Environment

First-person object-interaction tasks in high-fidelity, 3D, simulated environments such as the AI2Thor virtual home-environment pose significant sample-efficiency challenges for reinforcement learning (RL) agents learning from sparse task rewards. To alleviate these challenges, prior work has provided extensive supervision via a combination of reward-shaping, ground-truth object-information, and expert demonstrations. In this work, we show that one can learn object-interaction tasks from scratch without supervision by learning an attentive object-model as an auxiliary task during task learning with an object-centric relational RL agent. Our key insight is that learning an object-model that incorporates object-attention into forward prediction provides a dense learning signal for unsupervised representation learning of both objects and their relationships. This, in turn, enables faster policy learning for an object-centric relational RL agent. We demonstrate our agent by introducing a set of challenging object-interaction tasks in the AI2Thor environment where learning with our attentive object-model is key to strong performance. Specifically, we compare our agent and relational RL agents with alternative auxiliary tasks to a relational RL agent equipped with ground-truth object-information, and show that learning with our object-model best closes the performance gap in terms of both learning speed and maximum success rate. Additionally, we find that incorporating object-attention into an object-model's forward predictions is key to learning representations which capture object-category and object-state.

cs.LG↗

arXiv2014

A novel feature in aluminum cluster photoionization spectra and possibility of electron pairing at T>~100K

A unique property of size-resolved metal nanocluster particles is their "superatom"-like electronic shell structure. The shell levels are highly degenerate, and it has been predicted that this can enable exceptionally strong superconducting-type electron pair correlations in certain clusters composed of just tens to hundreds of atoms. Here we report on the observation of a possible spectroscopic signature of such an effect. A bulge-like feature appears in the photoionization yield curve of a free cold aluminum cluster and shows a rapid rise as the temperature approaches approximately 100 K. This is an unusual effect, not previously reported for clusters. Its characteristics are consistent with an increase in the effective density of states accompanying a pairing transition, which suggests a high-temperature superconducting state with Tc>~100 K. Our results highlight the promise of metal nanoclusters as high-Tc building blocks for materials and networks.

cond-mat.supr-con↗

arXiv2010

How Metallic are Small Sodium Clusters?

Cryogenic cluster beam experiments have provided crucial insights into the evolution of the metallic state from the atom to the bulk. Surprisingly, one of the most fundamental metallic properties, the ability of a metal to efficiently screen electric fields, is still poorly understood in small clusters. Theory has predicted that many small Na clusters are unable to screen charge inhomogeneities and thus have permanent dipole moments. High precision electric deflection experiments on cryogenically cooled Na$_N$ ($N<200$) clusters show that the electric dipole moments are at least an order of magnitude smaller than predicted, and are consistent with zero, as expected for a metal. The polarizabilities of Na clusters also show metal spheroid behavior, with fine size oscillations caused by the shell structure.

physics.atm-clus↗

arXiv2010

Bistability of Free Cobalt and Iron Clusters

The cobalt and iron clusters CoN, FeN (20 < N < 150) measured in a cryogenic molecular beam are found to be bistable with magnetic moments per atom both μN/N 2μB in the ground states and μN */N μB in the metastable excited states (for iron clusters, μN ~3NμB and μN* NμB). This energy gap between the two states vanish for large clusters, which explains the rapid convergence of the magnetic moments to the bulk value and suggests that ground state for the bulk involves a superposition of the two, in line with the fluctuating local orders in the bulk itinerant ferromagnetism.

cond-mat.str-el↗

arXiv2006

Non-classical dipoles in cold niobium clusters

Electric deflections of niobium clusters in molecular beams show that they have permanent electric dipole moments at cryogenic temperatures but not higher temperatures, indicating that they are ferroelectric. Detailed analysis shows that the deflections cannot be explained in terms of a rotating classical dipole, as claimed by Anderson et al. The shapes of the deflected beam profiles and their field and temperature dependences indicates that the clusters can exist in two states, one with a dipole and the other without. Cluster with dipoles occupy lower energy states. Excitations from the lower states to the higher states can be induced by low fluence laser excitation. This causes the dipole to vanish.

cond-mat.str-el↗