arXiv subjects

Jiong Lin

Publications and source records attributed to Jiong Lin.

At least 19 recordsLinked to original sources

arXiv2026

Eye, Robot: Learning Expressive Gaze Dynamics via a VLM-Guided Transformer-VAE

Gaze coordinates attention in human interaction through coupled eye, head, and eyelid motion. A humanoid robot must choose where to look while generating a suitable movement toward that target; fixed tracking rules alone do not model the temporal structure of human gaze. We separate these tasks with an asynchronous vision-language model that selects a scene referent, a reactive perception module that maintains fixation, and a Realistic Gaze Transformer (RGT) that generates target-conditioned motion. RGT is a Transformer-VAE trained on markerless human recordings retargeted to the robot's 11-DoF joint space: self-supervised pretraining encodes recent full-head history, and autoregressive fine-tuning exposes the model to its own predictions during rollout. At inference, a four-dimensional binocular-eye trajectory conditions coordinated neck and eye motion; an auxiliary blink predictor triggers eyelid templates during playback. On a two-second benchmark in which all controllers receive the same ground-truth target, RGT-Base obtains the lowest reported overall seven-joint MSE, and physical demonstrations show the integrated targeting and control pipeline. The evidence establishes robot-space kinematic fidelity and real-time RGT inference; perceived naturalness and open-vocabulary target-selection accuracy remain questions for controlled evaluation.

cs.RO↗

arXiv2026

The Smart Buildings Challenge: Learning Foundation Models for Real-World Optimization at Scale

Buildings cause nearly 40\% of carbon emissions in the US. Of total building emissions, HVAC systems are responsible for approximately 40-60\% and account for 15\% of global energy consumption. Despite advances in artificial intelligence, most buildings still operate as inefficiently as they did decades ago. Building optimization, perhaps more so than any other optimization problem, has enormous potential for Machine Learning (ML) driven impact on climate change, in both the short and long term. One of the biggest obstacles for building optimization is the lack of policy transfer. Most building control systems, whether simulation or learning based, don't generalize: they only work well for the building they were developed for, but not for other buildings. To solve this problem, we propose a challenge for the development of building foundation models that easily adapt to new buildings with minimal configuration and thereby enable building optimization to scale across the globe. Participants will build control policies on a set of simulated buildings, and have their policies evaluated on an unseen set of buildings. The buildings vary in size, layout, weather, material properties, and HVAC system. Successful outcomes from this initiative promise a massive leap forward in global sustainability efforts and demonstrate the great potential of ML to combat climate change.

cs.AI↗

arXiv2026

Physics-based Sphere Packing for Lagrangian Mesh Morphing

This paper studies tetrahedral meshes as the body representation for differentiable simulation and computational design. Fixed-connectivity meshes degrade under large morphs, while remeshing from scratch discards node correspondence. We present JamTet, a physics-based sphere-packing framework for volumetric meshing and morphing. We contribute (i) a GPU-parallel mesher combining octree-hierarchical packing with constrained Delaunay tetrahedralization, producing more uniform element volumes than TetGen and fTetWild; (ii) Lagrangian mesh morphing that preserves interior-node identities by re-equilibrating the same spheres within changing shapes and rebuilding the boundary and connectivity, remaining inversion-free where fixed-connectivity and TetSphere meshes invert; and (iii) a differentiable GPU simulator in JAX, with mass-spring edges and a volumetric Neo-Hookean term, integrated with mesh morphing in a design pipeline. In soft-robot morphology design experiments, interior-node gradients improve swimming fitness by 0.73-1.07 over a matched surface-only variant, while voxelized versions of the same designs yield 32-63% lower fitness. These results establish sphere packing as a practical volumetric mesh representation for gradient-based shape optimization. Code and media: under review.

cs.GR↗

arXiv2026

iCoder-27B: Recursive AI-Led Development of Frontier Industrial Coding Model

Recursive AI, the prospect of AI taking an increasingly complete role in building and improving AI, is a crown jewel of AI for AI. Although recursive self-development has become practical for small models, bounded tasks, and fixed time budgets, a more consequential realization of this ambition, i.e., developing a release-ready, frontier-competitive model, remains far more challenging. In this work, we ask how little human involvement is sufficient for an agent to develop a frontier model. We concentrate human input into a high-density, low-frequency interface: experts encode objectives, stage scaffolds, permission boundaries, and operating procedures as reusable research skills, while the agent instantiates these priors, selects experiments, diagnoses outcomes, and revises the training strategy. In the challenging domain of industrial coding, the agent evolves data and coordinates SFT, on-policy self-distillation, and reinforcement learning with verifiable rewards, ultimately producing iCoder, a 27B model for RTL design and GPU kernel optimization. Across seven benchmarks, iCoder leads RTLLM, outperforming GPT-5.5 and Claude-Opus-4.8; ranks second on CVDP and KernelBench L2, exceeding GPT-5.5 by 16 points; and ties Claude-Opus-4.8 for the best TritonBench result. Exploratory case studies further show iCoder's competitive iterative RTL and GPU-kernel optimization with substantially fewer tokens. These results chart an engineering path toward recursive self-improvement, in which humans distill the principles of model building, agents operationalize them through evidence-driven experimentation, and each generation of AI becomes a more capable architect of the next.

cs.AI↗

arXiv2026

Continuum Robot Modeling with Action Conditioned Flow Matching

Predicting the shape of tendon driven continuum robots (TDCRs) at steady state from actuation remains challenging due to continuous deformation, complex tendon routing, compliance, friction, and fabrication variability. In this paper, we address this problem as kinematic self modeling conditioned on action. We present a lightweight 3D printed TDCR hardware platform and an RGB-D data collection pipeline with multiple cameras, and we learn a point cloud flow matching model that maps motor actuation states to the robot's settled 3D geometry. The model is trained from randomly sampled quasi static configurations and evaluated on test motor commands within the same TDCR design family and actuation range. We compare against prior 3D deformable object and robot self modeling approaches in both MuJoCo simulation and real hardware experiments. Experiments on simulated 2-, 3-, and 5-module TDCRs and real 2- and 3-module robots show improved shape prediction accuracy under CD and EMD metrics. We further show in simulation that the same conditional formulation generalizes to tip payload as a conditioning input, enabling payload conditioned steady-state shape prediction. These results demonstrate a data driven self modeling framework for quasi static TDCR geometry prediction.

cs.RO↗

arXiv2026

From Experience to Skill: Multi-Agent Generative Engine Optimization via Reusable Strategy Learning

Generative engines (GEs) are reshaping information access by replacing ranked links with citation-grounded answers, yet current Generative Engine Optimization (GEO) methods optimize each instance in isolation, unable to accumulate or transfer effective strategies across tasks and engines. We reframe GEO as a strategy learning problem and propose MAGEO, a multi-agent framework in which coordinated planning, editing, and fidelity-aware evaluation serve as the execution layer, while validated editing patterns are progressively distilled into reusable, engine-specific optimization skills. To enable controlled assessment, we introduce a Twin Branch Evaluation Protocol for causal attribution of content edits and DSV-CF, a dual-axis metric that unifies semantic visibility with attribution accuracy. We further release MSME-GEO-Bench, a multi-scenario, multi-engine benchmark grounded in real-world queries. Experiments on three mainstream engines show that MAGEO substantially outperforms heuristic baselines in both visibility and citation fidelity, with ablations confirming that engine-specific preference modeling and strategy reuse are central to these gains, suggesting a scalable learning-driven paradigm for trustworthy GEO. Code is available at https://github.com/Wu-beining/MAGEO

cs.AI↗

arXiv2026

Weaving the (AdS) spaces with partial entanglement entropy threads

In the context of the AdS/CFT correspondence, we propose a general scheme for reconstructing bulk geometric quantities in a static pure AdS background using the partial entanglement entropy (PEE), a measure of the entanglement structure on the boundary CFT. The PEE between any two points $\mathcal{I}(\vec{x}, \vec{y})$ serves as the fundamental building block of the PEE structure. Any two-point PEE $\mathcal{I}(\vec{x}, \vec{y})$ can be geometrized by the bulk geodesic connecting two boundary points $\vec{x}$ and $\vec{y}$, which we call the PEE thread, with the density of the threads determined by the boundary PEE structure. In the AdS bulk, the set of all the PEE threads forms a continuous ``network'', which we call the PEE network. In this paper, we show that the density of the PEE threads passing through any bulk point is exactly $1/(4G)$. Based on this observation we give a reformulation of the Ryu-Takayanagi (RT) formula for a generic boundary region in general dimensional Poincaré AdS space. More explicitly, for any static boundary region $A$, the homologous surface $Σ_{A}$ that has the minimal number of intersections with the bulk PEE network is exactly the RT surface of $A$, and the minimal number of intersections reproduces the holographic entanglement entropy. The reconstruction for the area of bulk geometric quantities by counting the number of intersections with the bulk PEE network applies to generic bulk geometric quantities. Interestingly, this reconstruction indicates a pure geometric statement, which is exactly the so-called \emph{Crofton formula} in Poincaré AdS.

hep-th↗

arXiv2026

Beyond Cropping and Rotation: Automated Evolution of Powerful Task-Specific Augmentations with Generative Models

Data augmentation has long been a cornerstone for reducing overfitting in vision models, with methods like AutoAugment automating the design of task-specific augmentations. Recent advances in generative models, such as conditional diffusion and few-shot NeRFs, offer a new paradigm for data augmentation by synthesizing data with significantly greater diversity and realism. However, unlike traditional augmentations like cropping or rotation, these methods introduce substantial changes that enhance robustness but also risk degrading performance if the augmentations are poorly matched to the task. In this work, we present EvoAug, an automated augmentation learning pipeline, which leverages these generative models alongside an efficient evolutionary algorithm to learn optimal task-specific augmentations. Our pipeline introduces a novel approach to image augmentation that learns stochastic augmentation trees that hierarchically compose augmentations, enabling more structured and adaptive transformations. We demonstrate strong performance across fine-grained classification and few-shot learning tasks. Notably, our pipeline discovers augmentations that align with domain knowledge, even in low-data settings. These results highlight the potential of learned generative augmentations, unlocking new possibilities for robust model training.

cs.CV↗

arXiv2025

ArticFlow: Generative Simulation of Articulated Mechanisms

Recent advances in generative models have produced strong results for static 3D shapes, whereas articulated 3D generation remains challenging due to action-dependent deformations and limited datasets. We introduce ArticFlow, a two-stage flow matching framework that learns a controllable velocity field from noise to target point sets under explicit action control. ArticFlow couples (i) a latent flow that transports noise to a shape-prior code and (ii) a point flow that transports points conditioned on the action and the shape prior, enabling a single model to represent diverse articulated categories and generalize across actions. On MuJoCo Menagerie, ArticFlow functions both as a generative model and as a neural simulator: it predicts action-conditioned kinematics from a compact prior and synthesizes novel morphologies via latent interpolation. Compared with object-specific simulators and an action-conditioned variant of static point-cloud generators, ArticFlow achieves higher kinematic accuracy and better shape quality. Results show that action-conditioned flow matching is a practical route to controllable and high-quality articulated mechanism generation.

cs.CV↗

arXiv2025

Knolling Bot: Teaching Robots the Human Notion of Tidiness

For robots to truly collaborate and assist humans, they must understand not only logic and instructions, but also the subtle emotions, aesthetics, and feelings that define our humanity. Human art and aesthetics are among the most elusive concepts-often difficult even for people to articulate-and without grasping these fundamentals, robots will be unable to help in many spheres of daily life. Consider the long-promised robotic butler: automating domestic chores demands more than motion planning. It requires an internal model of cleanliness and tidiness-a challenge largely unexplored by AI. To bridge this gap, we propose an approach that equips domestic robots to perform simple tidying tasks via knolling, the practice of arranging scattered items into neat, space-efficient layouts. Unlike the uniformity of industrial settings, household environments feature diverse objects and highly subjective notions of tidiness. Drawing inspiration from NLP, we treat knolling as a sequential prediction problem and employ a transformer based model to forecast each object's placement. Our method learns a generalizable concept of tidiness, generates diverse solutions adaptable to varying object sets, and incorporates human preferences for personalized arrangements. This work represents a step forward in building robots that internalize human aesthetic sense and can genuinely co-create in our living spaces.

cs.RO↗

arXiv2025

Creative synthesis of kinematic mechanisms

In this paper, we formulate the problem of kinematic synthesis for planar linkages as a cross-domain image generation task. We develop a planar linkages dataset using RGB image representations, covering a range of mechanisms: from simple types such as crank-rocker and crank-slider to more complex eight-bar linkages like Jansen's mechanism. A shared-latent variational autoencoder (VAE) is employed to explore the potential of image generative models for synthesizing unseen motion curves and simulating novel kinematics. By encoding the drawing speed of trajectory points as color gradients, the same architecture also supports kinematic synthesis conditioned on both trajectory shape and velocity profiles. We validate our method on three datasets of increasing complexity: a standard four-bar linkage set, a mixed set of four-bar and crank-slider mechanisms, and a complex set including multi-loop mechanisms. Preliminary results demonstrate the effectiveness of image-based representations for generative mechanical design, showing that mechanisms with revolute and prismatic joints, and potentially cams and gears, can be represented and synthesized within a unified image generation framework.

cs.GR↗

arXiv2025

AutoURDF: Unsupervised Robot Modeling from Point Cloud Frames Using Cluster Registration

Robot description models are essential for simulation and control, yet their creation often requires significant manual effort. To streamline this modeling process, we introduce AutoURDF, an unsupervised approach for constructing description files for unseen robots from point cloud frames. Our method leverages a cluster-based point cloud registration model that tracks the 6-DoF transformations of point clusters. Through analyzing cluster movements, we hierarchically address the following challenges: (1) moving part segmentation, (2) body topology inference, and (3) joint parameter estimation. The complete pipeline produces robot description files that are fully compatible with existing simulators. We validate our method across a variety of robots, using both synthetic and real-world scan data. Results indicate that our approach outperforms previous methods in registration and body topology estimation accuracy, offering a scalable solution for automated robot modeling.

cs.RO↗

arXiv2025

Teaching Robots to Build Simulations of Themselves

The emergence of vision catalysed a pivotal evolutionary advancement, enabling organisms not only to perceive but also to interact intelligently with their environment. This transformation is mirrored by the evolution of robotic systems, where the ability to leverage vision to simulate and predict their own dynamics marks a leap towards autonomy and self-awareness. Humans utilize vision to record experiences and internally simulate potential actions. For example, we can imagine that, if we stand up and raise our arms, the body will form a T shape without physical movement. Similarly, simulation allows robots to plan and predict the outcomes of potential actions without execution. Here we introduce a self-supervised learning framework to enable robots to model and predict their morphology, kinematics and motor control using only brief raw video data, eliminating the need for extensive real-world data collection and kinematic priors. By observing their own movements, akin to humans watching their reflection in a mirror, robots learn an ability to simulate themselves and predict their spatial motion for various tasks. Our results demonstrate that this self-learned simulation not only enables accurate motion planning but also allows the robot to detect abnormalities and recover from damage.

cs.RO↗

arXiv2024

Cutoff brane vs the Karch-Randall brane: the fluctuating case

Recently, certain holographic Weyl transformed CFT$_2$ is proposed to capture the main features of the AdS$_3$/BCFT$_2$ correspondence \cite{Basu:2022crn,Basu:2023wmv}. In this paper, by adapting the Weyl transformation, we simulate a generalized AdS/BCFT set-up where the fluctuation of the Karch-Randall (KR) brane is considered. In the gravity dual of the Weyl transformed CFT, the so-called cutoff brane induced by the Weyl transformation plays the same role as the KR brane. Unlike the non-fluctuating configuration, in the $2d$ effective theory the additional twist operator is inserted at a different place, compared with the one inserted on the brane. Though this is well-understood in the Weyl transformed CFT set-up, it is confusing in the AdS/BCFT set-up where the effective theory is supposed to locate on the brane. This confusion indicates that the KR brane may be emergent from the boundary CFT$_2$ via the Weyl transformations. We also calculate the balanced partial entanglement (BPE) in the fluctuating brane configurations and find it coincide with the entanglement wedge cross-section (EWCS). This is a non-trivial test for the correspondence between the BPE and the EWCS, and a non-trivial consistency check for the Weyl transformed CFT set-up.

hep-th↗

arXiv2024

Geometrizing the Partial Entanglement Entropy: from PEE Threads to Bit Threads

We give a scheme to geometrize the partial entanglement entropy (PEE) for holographic CFT in the context of AdS/CFT. More explicitly, given a point $\textbf{x}$ we geometrize the two-point PEEs between $\textbf{x}$ and any other points in terms of the bulk geodesics connecting these two points. We refer to these geodesics as the \textit{PEE threads}, which can be naturally regarded as the integral curves of a divergenceless vector field $V_{\textbf{x}}^μ$, which we call \emph{PEE thread flow}. The norm of $V_{\textbf{x}}^μ$ that characterizes the density of the PEE threads can be determined by some physical requirements of the PEE. We show that, for any static interval or spherical region $A$, a unique bit thread configuration can be generated from the PEE thread configuration determined by the state. Hence, the non-intrinsic bit threads are emergent from the intrinsic PEE threads. For static disconnected intervals, the vector fields describing a divergenceless flow is are longer suitable to reproduce the RT formula. We weight the PEE threads with the number of times it intersects with any homologous surface. Instead the RT formula is perfectly reformulated to be the minimization of the summation of the PEE threads with all possible assignment of weights.

hep-th↗

arXiv2023

Ownerless island and partial entanglement entropy in island phases

In the context of partial entanglement entropy (PEE), we study the entanglement structure of the island phases realized in several 2-dimensional holographic set-ups. The self-encoding property of the island phase changes the way we evaluate the PEE. With the contributions from islands taken into account, we give a generalized prescription to construct PEE and balanced partial entanglement entropy (BPE). Here the ownerless island region, which lies inside the island $\text{Is}(AB)$ of $A\cup B$ but outside $\text{Is}(A)\cup \text{Is}(B)$, plays a crucial role. Remarkably, we find that under different assignments for the ownerless island, we get different BPEs, which exactly correspond to different saddles of the entanglement wedge cross-section (EWCS) in the entanglement wedge of $A\cup B$. The assignments can be settled by choosing the one that minimizes the BPE. Furthermore, under this assignment we study the PEE and give a geometric picture for the PEE in holography, which is consistent with the geometric picture in the no-island phases.

hep-th↗

arXiv2023

The Markov gap in the presence of islands

The Markov gap \cite{Hayden:2021gno}, namely the difference between reflected entropy and mutual information, is explicitly computed in the defect extremal surface model, JT gravity, and the generic 2d extremal black holes, in vacuum states. The phases that contain various island contributions are considered, and their existence is carefully checked. Moreover, we show explicitly how the Markov gap originates from the OPE coefficient of the boundary CFT. And, as a generalization of \cite{Hayden:2021gno}, the lower bound of the Markov gap is given by $\frac{c}{3}\log 2$ times the number of EWCS boundaries on minimal surfaces. We propose a boundary way of counting the lower bound for the Markov gap, which states that the lower bound is given by $\frac{c}{3}\log 2$ times the number of gaps between two boundary regions in vacuum states. We discuss the limitation and possible generalization of the boundary counting, and its relation to tripartite entanglement.

hep-th↗

arXiv2023

Primordial black holes and scalar induced gravitational waves from Higgs inflation with non-canonical kinetic term

We resolve the potential-restriction problem in K/G inflation by introducing nonminimal coupling. In this context, Higgs field successfully drives inflation satisfying CMB observations while enhancing curvature perturbations at small scales, which in turn accounts for primordial black holes (PBHs) and scalar induced gravitational waves (SIGWs). We then uncover the effect of the non-canonical kinetic coupling function in more detail and study its the observational constraint. Besides, we also give the gauge invariant expression for the integral kernel of SIGWs, which is related to terms propagating with the speed of light. Finally, the non-Gaussian effect on PBH abundance and SIGWs is studied. We find that non-Gaussianity makes PBHs form more easily, but its effect on the energy density of SIGWs is negligible.

gr-qc↗