arXiv subjects

Steve Xu

Publications and source records attributed to Steve Xu.

3 recordsLinked to original sources

arXiv2026

From LLM to Silicon: RL-Driven ASIC Architecture Exploration for On-Device AI Inference

We present an RL-driven compiler that jointly optimizes ASIC architecture, memory hierarchy, and workload partitioning for AI inference across 3nm to 28nm. The design space is formulated as a single Markov Decision Process with mixed discrete-continuous actions and a unified Power-Performance-Area (PPA) objective. Soft Actor-Critic (SAC) with Mixture-of-Experts gating explores the joint space of mesh topology, per-core microarchitecture, and operator placement. We validate on two workloads, Llama 3.1 8B FP16 (high-performance mode, 29809 tokens per second at 3nm) and SmolVLM (low-power mode, less than 13 mW at all nodes, 10 MHz). Across 7 process nodes, the RL automatically adapts mesh sizes and per-tile configurations, including heterogeneous FETCH, VLEN, and memory allocation without node-specific manual retuning.

cs.AR↗

arXiv2025

Hardware-Aware Neural Network Compilation with Learned Optimization: A RISC-V Accelerator Approach

We present XgenSilicon ML Compiler, a fully automated end-to-end compilation framework that transforms high-level machine learning models into optimized RISC-V assembly code for custom ASIC accelerators. By unifying the system's cost model across software and hardware, the compiler achieves significant improvements in Power, Performance, and Area (PPA) metrics compared to standard off-the-shelf components and hand-designed chips through five key innovations: (1) a multi-algorithm auto-tuning framework with five search strategies (Bayesian Optimization, Genetic Algorithm, Simulated Annealing, Random Search, Grid Search) combined with a learned cost model, (2) an integrated quantization framework supporting extreme precisions from FP32 to Binary with full KL divergence calibration (2048-bin histogram optimization) and momentum-based QAT gradient updates, (3) hardware-aware validation ensuring 100 percent ISA compliance and memory constraint satisfaction, (4) dynamic shape support with multi-configuration specialization, and (5) advanced cache-aware cost modeling with multi-level cache hierarchy analysis. Our evaluation demonstrates that ASICs produced by this compiler achieve 2.5-4.5x better performance, 3-6x lower power consumption, and 40-60 percent area reduction compared to baseline implementations. The compiler supports more than 100 ONNX operators across 12 categories, implements advanced RISC-V Vector optimizations, and generates hardware-validated assembly code suitable for direct ASIC synthesis. All compilation steps are fully automated, requiring zero manual intervention from model input to ASIC-ready output.

cs.AR↗

arXiv2024

AutoRT: Embodied Foundation Models for Large Scale Orchestration of Robotic Agents

Foundation models that incorporate language, vision, and more recently actions have revolutionized the ability to harness internet scale data to reason about useful tasks. However, one of the key challenges of training embodied foundation models is the lack of data grounded in the physical world. In this paper, we propose AutoRT, a system that leverages existing foundation models to scale up the deployment of operational robots in completely unseen scenarios with minimal human supervision. AutoRT leverages vision-language models (VLMs) for scene understanding and grounding, and further uses large language models (LLMs) for proposing diverse and novel instructions to be performed by a fleet of robots. Guiding data collection by tapping into the knowledge of foundation models enables AutoRT to effectively reason about autonomy tradeoffs and safety while significantly scaling up data collection for robot learning. We demonstrate AutoRT proposing instructions to over 20 robots across multiple buildings and collecting 77k real robot episodes via both teleoperation and autonomous robot policies. We experimentally show that such "in-the-wild" data collected by AutoRT is significantly more diverse, and that AutoRT's use of LLMs allows for instruction following data collection robots that can align to human preferences.

cs.RO↗