arXiv subjects

Vint Lee

Publications and source records attributed to Vint Lee.

4 recordsLinked to original sources

arXiv2026

QUEST: Training Frontier Deep Research Agents with Fully Synthetic Tasks

Deep research agents extend the role of search engines from retrieving keyword-matched pages to synthesizing knowledge, fundamentally changing how humans interact with information. However, frontier systems remain proprietary, while existing open agents often generalize poorly across different task types, leaving unclear how to train a broadly capable deep research agent. We release QUEST, a family of open models (ranging from 2B to 35B) that serve as general-purpose deep research agents designed to handle a wide range of long-horizon search tasks, with strong capabilities in fact seeking, citation grounding, and report synthesis. To build QUEST, we propose an effective training recipe combining mid-training, supervised fine-tuning, and reinforcement learning. Central to this recipe is a curated data synthesis pipeline based on unified rubric trees, which applies to different task types and enables synthesizing training data with verifiable rewards without human annotation. In addition, QUEST incorporates a built-in context management mechanism that enables effective long-horizon reasoning and knowledge synthesis. Using only 8K synthesized tasks, QUEST approaches or even surpasses frontier closed-source agents across eight deep research benchmarks spanning diverse task types, and achieves the best overall performance among recent open-weight agents. We released everything: models, data, and training scripts.

cs.CL↗

arXiv2024

Chip Placement with Diffusion Models

Macro placement is a vital step in digital circuit design that defines the physical location of large collections of components, known as macros, on a 2D chip. Because key performance metrics of the chip are determined by the placement, optimizing it is crucial. Existing learning-based methods typically fall short because of their reliance on reinforcement learning (RL), which is slow and struggles to generalize, requiring online training on each new circuit. Instead, we train a diffusion model capable of placing new circuits zero-shot, using guided sampling in lieu of RL to optimize placement quality. To enable such models to train at scale, we designed a capable yet efficient architecture for the denoising model, and propose a novel algorithm to generate large synthetic datasets for pre-training. To allow zero-shot transfer to real circuits, we empirically study the design decisions of our dataset generation algorithm, and identify several key factors enabling generalization. When trained on our synthetic data, our models generate high-quality placements on unseen, realistic circuits, achieving competitive performance on placement benchmarks compared to state-of-the-art methods.

cs.LG↗

arXiv2024

Low-cost, Lightweight Electronic Flow Regulators for Throttling Liquid Rocket Engines

For small-scale liquid rockets, pressure-fed systems are commonly favoured due to their simplicity and low weight. In such systems, accurate regulation of both tank and injector pressures over a wide range of upstream pressures is critical $-$ more accurate regulation allows for higher engine efficiency and minimal tank mass, thus improving flight performance. However, existing methods such as dome-loaded pressure regulators are inflexible, or require extensive characterization to function accurately. These methods also suffer from limited orifice size, droop, and slow reaction times, making them unsuitable for throttling by adjusting pressures in flight, which are increasingly important as propulsively landing rockets become more common. To overcome these challenges, we designed an electronic pressure regulator (eReg), a multi-input multi-output system utilising closed loop feedback to accurately control downstream pressures. Our design is simple, low-cost and robust: with a single ball valve actuated by a motor, we regulate both gaseous pressurant and cryogenic liquid propellant at high flow rates (1.14 kg/s of liquid; 0.39 kg/s of gas) and upstream pressures (310 bar). Using 2 eRegs to regulate propellant tank pressures, and 2 eRegs for regulating propellant flow to the engine, we demonstrated our system's ability, in a static fire test, to regulate pressures accurately (within 0.2 bar) while simultaneously throttling our engine. To the best of our knowledge, this is the first time any undergraduate team has successfully throttled a liquid bipropellant engine.

eess.SY↗

arXiv2023

DreamSmooth: Improving Model-based Reinforcement Learning via Reward Smoothing

Model-based reinforcement learning (MBRL) has gained much attention for its ability to learn complex behaviors in a sample-efficient way: planning actions by generating imaginary trajectories with predicted rewards. Despite its success, we found that surprisingly, reward prediction is often a bottleneck of MBRL, especially for sparse rewards that are challenging (or even ambiguous) to predict. Motivated by the intuition that humans can learn from rough reward estimates, we propose a simple yet effective reward smoothing approach, DreamSmooth, which learns to predict a temporally-smoothed reward, instead of the exact reward at the given timestep. We empirically show that DreamSmooth achieves state-of-the-art performance on long-horizon sparse-reward tasks both in sample efficiency and final performance without losing performance on common benchmarks, such as Deepmind Control Suite and Atari benchmarks.

cs.LG↗