arXiv subjects

Nathan Zhao

Publications and source records attributed to Nathan Zhao.

10 recordsLinked to original sources

arXiv2026

Controlling Action-Head Co-Adaptation to VLM Representation Anisotropy

Modern learning systems increasingly combine pretrained models with downstream heads. Despite the growing importance of such modular designs, how the downstream model co-adapts to the geometry of pretrained representations remains relatively underexplored. We study this problem through vision-language-action models (VLAs), where a pretrained vision-language model (VLM) conditions an action model for control. We find that the action model progressively concentrates its sensitivity on high-variance dimensions of the VLM outputs. Through controlled experiments across different backbone-head models, we further show that counteracting this sensitivity concentration can be beneficial under representation--task mismatch settings. Motivated by these observations, we introduce \emph{Adaptive Interface Equalizer} (AdaIE), a parameter-free mechanism that reweights pretrained representations according to the evolving downstream sensitivity during training. AdaIE improves VLA success rates on simulation benchmarks, with gains of up to $+5.28$ percentage points (pp) on RoboCasa-GR1 and $+1.68$ pp on RoboTwin 2.0, and achieves an average gain of $+19$ pp across five real-world manipulation tasks. It further generalizes to diffusion policies and world models, improving downstream performance in both settings. Our results highlight interface co-adaptation as an important optimization problem when coupling pretrained representations with downstream models.

cs.AI↗

arXiv2026

IronMind: Scaling Humanoid Dexterous Manipulation via Camera-Space Ego-Centric Pretraining

Egocentric human video offers a scalable data source for dexterous manipulation, yet using it to train humanoid robots presents two challenges: (1) an embodiment gap, as human hands differ structurally from robot end-effectors and low-cost egocentric recordings lack the torso kinematics required by conventional retargeting; and (2) heterogeneous data quality, including noisy hand-pose tracking and weakly aligned text annotations. We introduce IronMind, a vision-language-action (VLA) model that uses egocentric human video and heterogeneous robot data to pretrain policies for humanoid dexterous manipulation. To bridge the embodiment gap, IronMind bypasses explicit body-retargeting by using a camera-space action representation, the native reference space of egocentric video, and semantically aligning robot and human action dimensions. Across total pretraining budgets from 250 to 10,000 hours, validation loss decreases approximately log-linearly with data scale. Larger pretraining budgets also improve out-of-distribution real-robot manipulation after post-training: across six challenging tasks with unseen objects, affordances, and reasoning prompts, the 10,000-hour model achieves a 55.0% success rate, compared with at most 11.7% for every pretraining budget up to 5,000 hours and 5.0% without pretraining. At the same pretraining budget, the camera-space action representation also outperforms the torso-frame baseline. Together, these findings support pretraining with a camera-space action representation on large-scale human egocentric data as a scalable foundation for humanoid robot manipulation.

cs.RO↗

arXiv2026

Principal Steering Subspaces for Online Adaptation of Frozen Generative Robot Policies

Generative robot policies provide expressive behavior priors, but updating a large diffusion or flow-matching model through online interaction is costly. Latent-space reinforcement learning avoids updating the pretrained generator by controlling its initial sampling noise, yet high-dimensional noise can have strongly anisotropic effects on decoded actions. We introduce Principal Steering Subspaces (PSS), a forward-query interface that constructs a fixed low-dimensional control basis from finite-difference decoder responses. Soft Actor-Critic controls the leading response directions, while the orthogonal complement is independently resampled from the Gaussian prior at each query. On three RoboMimic tasks with diffusion and flow-matching policies, response spectra reveal substantial concentration. Across five matched task-generator pairs, the training curves indicate that PSS generally converges faster and exhibits more stable late-training behavior than full-latent control, while achieving stronger final performance overall. Controlled Diffusion-Square ablations further show that leading-response directions outperform random and least-responsive subspaces of equal dimension. We further integrate PSS with a frozen, closed-source 3B-parameter vision-language-action (VLA) policy in a humanoid learning system with synchronous transition collection, reset-time optimization, and latency-aware asynchronous deployment. In an exploratory screwdriver-placement evaluation, success is observed in 2/10 trials for the frozen VLA policy and 6/10 after SAC+PSS adaptation. These results support decoder-response geometry as a practical basis for online adaptation of frozen generative robot policies.

cs.RO↗

arXiv2026

WebPII: Benchmarking Visual PII Detection for Computer-Use Agents

Computer use agents create new privacy risks: training data collected from real websites inevitably contains sensitive information, and cloud-hosted inference exposes user screenshots. Detecting personally identifiable information in web screenshots is critical for privacy-preserving deployment, but no public benchmark exists for this task. We introduce WebPII, a fine-grained synthetic benchmark of 44,865 annotated e-commerce UI images designed with three key properties: extended PII taxonomy including transaction-level identifiers that enable reidentification, anticipatory detection for partially-filled forms where users are actively entering data, and scalable generation through VLM-based UI reproduction. Experiments validate that these design choices improve layout-invariant detection across diverse interfaces and generalization to held-out page types. We train WebRedact to demonstrate practical utility, more than doubling text-extraction baseline accuracy (0.753 vs 0.357 mAP@50) at real-time CPU latency (20ms). We release the dataset and model to support privacy-preserving computer use research.

cs.CR↗

arXiv2024

Using Large Language Models for Humanitarian Frontline Negotiation: Opportunities and Considerations

Humanitarian negotiations in conflict zones, called \emph{frontline negotiation}, are often highly adversarial, complex, and high-risk. Several best-practices have emerged over the years that help negotiators extract insights from large datasets to navigate nuanced and rapidly evolving scenarios. Recent advances in large language models (LLMs) have sparked interest in the potential for AI to aid decision making in frontline negotiation. Through in-depth interviews with 13 experienced frontline negotiators, we identified their needs for AI-assisted case analysis and creativity support, as well as concerns surrounding confidentiality and model bias. We further explored the potential for AI augmentation of three standard tools used in frontline negotiation planning. We evaluated the quality and stability of our ChatGPT-based negotiation tools in the context of two real cases. Our findings highlight the potential for LLMs to enhance humanitarian negotiations and underscore the need for careful ethical and practical considerations.

cs.HC↗

arXiv2022

Effect of Choices of Boundary Conditions on the Numerical Efficiency of Direct Solutions of Finite Difference frequency Domain Systems with Perfectly Matched Layers

Direct solvers are a common method for solving finite difference frequency domain (FDFD) systems that arise in numerical solutions of Maxwell's equations. In a direct solver, one factorizes the system matrix. Since the system matrix is typically very sparse, the fill-in of these factors is the single most important computational consideration in terms of time complexity and memory requirements. As a result, it is of great interest to determine ways in which this fill-in can be systematically reduced. In this paper, we show that in the context of commonly used perfectly matched boundary layer methods, the choice of boundary condition behind the perfectly matched boundary layer can be exploited to reduce fill-in incurred during the factorization, leading to significant gains of up to 40 percent in the efficiency of the factorization procedure. We illustrate our findings by solving linear systems and eigenvalue problems associated with the FDFD method.

math.NA↗

arXiv2021

Efficient method for accelerating line searches using a combined Schur complement domain decomposition and Born series expansions in photonic-based adjoint optimization

A line search in gradient-based optimization algorithm solves the problem of determining the optimal learning rate for a given gradient or search direction in a single iteration. For most problems, this is determined by evaluating different candidate learning rates to find the optimum, which can be expensive. Recent work has provided an efficient way to perform a line search with the use of the Shanks transformation of a Born series derived from the Lippman-Schwinger formalism. In this paper we show that the cost for performing such a line search can be further reduced with the use of the method of the Schur complement domain decomposition, which can lead to a 10-fold total speed-up resulting from the reduced number of iterations to convergence and reduced wall-clock time per iteration.

physics.comp-ph↗

arXiv2020

Compressed imitation learning

In analogy to compressed sensing, which allows sample-efficient signal reconstruction given prior knowledge of its sparsity in frequency domain, we propose to utilize policy simplicity (Occam's Razor) as a prior to enable sample-efficient imitation learning. We first demonstrated the feasibility of this scheme on linear case where state-value function can be sampled directly. We also extended the scheme to scenarios where only actions are visible and scenarios where the policy is obtained from nonlinear network. The method is benchmarked against behavior cloning and results in significantly higher scores with limited expert demonstrations.

cs.LG↗

arXiv2019

Accelerating Adjoint Variable Method Based Photonic Optimization with Schur Complement Domain Decomposition

Adjoint variable method in combination with gradient descent optimization has been widely used for the inverse design of nanophotonic devices. In many of such optimizations, the design region is only a small fraction of the total computational domain. Here we show that the adjoint variable method can be combined with the Schur complement domain decomposition method. With this combination, in each optimization step, the simulation only involves the degrees of freedom that are inside the design region. Our approach should significantly improve the computational efficiency of adjoint variable method based optimization of photonic structures.

physics.comp-ph↗

arXiv2018

High reflection from a one-dimensional array of graphene nanoribbons

We show that up to 90% reflectivity can be achieved by using guided plasmonic resonances in a one-dimensional periodic array of plasmonic nanoribbon. In general, to achieve strong reflection from a guided resonance system requires one to operate in the strongly over-coupled regime where the radiative decay rate dominates over the intrinsic loss rate of the resonances. Using an argument similar to what has been previous used to derive the Chu-Harrington limit for antennas, we show theoretically that there is no intrinsic limit for the radiative decay rate even when the system has an atomic scale thickness, in contrast to the existence of such limits on antennas. We also show that the current distribution due to plasmonic resonance can be designed to achieve very high external radiative rate. Our results show that high reflectivity can be achieved in an atomically-thin graphene layer, pointing to a new opportunity for creating atomically-thin optical devices.

physics.optics↗