arXiv subjects

Yao Tang

Publications and source records attributed to Yao Tang.

At least 37 records · Page 2Linked to original sources

arXiv2026

VeCoR -- Velocity Contrastive Regularization for Flow Matching

Flow Matching (FM) has recently emerged as a principled and efficient alternative to diffusion models. Standard FM encourages the learned velocity field to follow a target direction; however, it may accumulate errors along the trajectory and drive samples off the data manifold, leading to perceptual degradation, especially in lightweight or low-step configurations. To enhance stability and generalization, we extend FM into a balanced attract-repel scheme that provides explicit guidance on both "where to go" and "where not to go." To be formal, we propose \textbf{Velocity Contrastive Regularization (VeCoR)}, a complementary training scheme for flow-based generative modeling that augments the standard FM objective with contrastive, two-sided supervision. VeCoR not only aligns the predicted velocity with a stable reference direction (positive supervision) but also pushes it away from inconsistent, off-manifold directions (negative supervision). This contrastive formulation transforms FM from a purely attractive, one-sided objective into a two-sided training signal, regularizing trajectory evolution and improving perceptual fidelity across datasets and backbones. On ImageNet-1K 256$\times$256, VeCoR yields 22\% and 35\% relative FID reductions on SiT-XL/2 and REPA-SiT-XL/2 backbones, respectively, and achieves further FID gains (32\% relative) on MS-COCO text-to-image generation, demonstrating consistent improvements in stability, convergence, and image quality, particularly in low-step and lightweight settings. Project page: https://p458732.github.io/VeCoR_Project_Page/

cs.CV↗

arXiv2026

Multiplex Thinking: Reasoning via Token-wise Branch-and-Merge

Large language models often solve complex reasoning tasks more effectively with Chain-of-Thought (CoT), but at the cost of long, low-bandwidth token sequences. Humans, by contrast, often reason softly by maintaining a distribution over plausible next steps. Motivated by this, we propose Multiplex Thinking, a stochastic soft reasoning mechanism that, at each thinking step, samples K candidate tokens and aggregates their embeddings into a single continuous multiplex token. This preserves the vocabulary embedding prior and the sampling dynamics of standard discrete generation, while inducing a tractable probability distribution over multiplex rollouts. Consequently, multiplex trajectories can be directly optimized with on-policy reinforcement learning (RL). Importantly, Multiplex Thinking is self-adaptive: when the model is confident, the multiplex token is nearly discrete and behaves like standard CoT; when it is uncertain, it compactly represents multiple plausible next steps without increasing sequence length. Across challenging math reasoning benchmarks, Multiplex Thinking consistently outperforms strong discrete CoT and RL baselines from Pass@1 through Pass@1024, while producing shorter sequences. The code and checkpoints are available at https://github.com/GMLR-Penn/Multiplex-Thinking.

cs.CL↗

arXiv2025

Understanding the Origins of Super-Puff Planets: A New Mass-Loss Regime Coupled to Planetary Evolution

Super-puffs are a class of low-mass, large-radius planets that have challenged planet formation and evolution models. Their high inferred H/He mass fractions, required to explain their physical sizes, would lead to rapid atmospheric escape, raising questions about their long-term retention. Recent modeling work indicates that low-mass planets typically require 50\% less H/He mass to match their observed radius, due to significant roles of the radiative atmosphere and interior heating from the rock/iron core. Here, through a new quantitative analysis of XUV-driven escape in sub-Neptunes, we find that previous studies overestimated mass loss, as scaling laws in low-gravity regimes deviate greatly from the widely used energy-limited regime. We define a new regime, thermal-energy-mediated photoevaporation (TEMP), in which thermal energy conversion critically sets the mass-loss rate. These effects make super-puffs more resilient to mass loss than previously thought. We develop a coupled evolution model integrating this updated thermal evolution framework with a 1D hydrodynamic photoevaporation model. Applying this novel, joint model to observed super-puffs and young low-density planets, we find that their masses, radii and transit pressures align with predictions assuming either a clear or hazy atmosphere. This indicates that super-puffs have undergone a combination of boil-off and photoevaporative mass loss, with boil-off dominating the process. Our results indicate that low-density planets typically possess both a thick convective envelope and substantial radiative atmosphere, which contribute to their large radii. For this to occur, these planets must have intermediate masses of 5-10$M_\oplus$ and receive stellar insolation $\lesssim 30F_\oplus$, favoring FG-type stars over M-dwarfs.

astro-ph.EP↗

arXiv2025

Representation Entanglement for Generation: Training Diffusion Transformers Is Much Easier Than You Think

REPA and its variants effectively mitigate training challenges in diffusion models by incorporating external visual representations from pretrained models, through alignment between the noisy hidden projections of denoising networks and foundational clean image representations. We argue that the external alignment, which is absent during the entire denoising inference process, falls short of fully harnessing the potential of discriminative representations. In this work, we propose a straightforward method called Representation Entanglement for Generation (REG), which entangles low-level image latents with a single high-level class token from pretrained foundation models for denoising. REG acquires the capability to produce coherent image-class pairs directly from pure noise, substantially improving both generation quality and training efficiency. This is accomplished with negligible additional inference overhead, requiring only one single additional token for denoising (<0.5\% increase in FLOPs and latency). The inference process concurrently reconstructs both image latents and their corresponding global semantics, where the acquired semantic knowledge actively guides and enhances the image generation process. On ImageNet 256$\times$256, SiT-XL/2 + REG demonstrates remarkable convergence acceleration, achieving $\textbf{63}\times$ and $\textbf{23}\times$ faster training than SiT-XL/2 and SiT-XL/2 + REPA, respectively. More impressively, SiT-L/2 + REG trained for merely 400K iterations outperforms SiT-XL/2 + REPA trained for 4M iterations ($\textbf{10}\times$ longer). Code is available at: https://github.com/Martinser/REG.

cs.CV↗

arXiv2025

LEDiT: Your Length-Extrapolatable Diffusion Transformer without Positional Encoding

Diffusion transformers (DiTs) struggle to generate images at resolutions higher than their training resolutions. The primary obstacle is that the explicit positional encodings(PE), such as RoPE, need extrapolating to unseen positions which degrades performance when the inference resolution differs from training. In this paper, We propose a Length-Extrapolatable Diffusion Transformer~(LEDiT) to overcome this limitation. LEDiT needs no explicit PEs, thereby avoiding PE extrapolation. The key innovation of LEDiT lies in the use of causal attention. We demonstrate that causal attention can implicitly encode global positional information and show that such information facilitates extrapolation. We further introduce a locality enhancement module, which captures fine-grained local information to complement the global coarse-grained position information encoded by causal attention. Experimental results on both conditional and text-to-image generation tasks demonstrate that LEDiT supports up to 4x resolution scaling (e.g., from 256x256 to 512x512), achieving better image quality compared to the state-of-the-art length extrapolation methods. We believe that LEDiT marks a departure from the standard RoPE-based methods and offers a promising insight into length extrapolation. Project page: https://shenzhang2145.github.io/ledit/

cs.CV↗

arXiv2025

NoteBar: An AI-Assisted Note-Taking System for Personal Knowledge Management

Note-taking is a critical practice for capturing, organizing, and reflecting on information in both academic and professional settings. The recent success of large language models has accelerated the development of AI-assisted tools, yet existing solutions often struggle with efficiency. We present NoteBar, an AI-assisted note-taking tool that leverages persona information and efficient language models to automatically organize notes into multiple categories and better support user workflows. To support research and evaluation in this space, we further introduce a novel persona-conditioned dataset of 3,173 notes and 8,494 annotated concepts across 16 MBTI personas, offering both diversity and semantic richness for downstream tasks. Finally, we demonstrate that NoteBar can be deployed in a practical and cost-effective manner, enabling interactive use without reliance on heavy infrastructure. Together, NoteBar and its accompanying dataset provide a scalable and extensible foundation for advancing AI-assisted personal knowledge management.

cs.CL↗

arXiv2025

Synthesizing Near-Boundary OOD Samples for Out-of-Distribution Detection

Pre-trained vision-language models have exhibited remarkable abilities in detecting out-of-distribution (OOD) samples. However, some challenging OOD samples, which lie close to in-distribution (InD) data in image feature space, can still lead to misclassification. The emergence of foundation models like diffusion models and multimodal large language models (MLLMs) offers a potential solution to this issue. In this work, we propose SynOOD, a novel approach that harnesses foundation models to generate synthetic, challenging OOD data for fine-tuning CLIP models, thereby enhancing boundary-level discrimination between InD and OOD samples. Our method uses an iterative in-painting process guided by contextual prompts from MLLMs to produce nuanced, boundary-aligned OOD samples. These samples are refined through noise adjustments based on gradients from OOD scores like the energy score, effectively sampling from the InD/OOD boundary. With these carefully synthesized images, we fine-tune the CLIP image encoder and negative label features derived from the text encoder to strengthen connections between near-boundary OOD samples and a set of negative labels. Finally, SynOOD achieves state-of-the-art performance on the large-scale ImageNet benchmark, with minimal increases in parameters and runtime. Our approach significantly surpasses existing methods, and the code is available at https://github.com/Jarvisgivemeasuit/SynOOD.

cs.CV↗

arXiv2025

Reassessing Sub-Neptune Structure, Radii, and Thermal Evolution

We present a novel python-based 1D sub-Neptune evolution model that emphasizes the thermal evolution and potential solidification of the rock/iron core and the structure of the radiative atmosphere. This model explores planetary structure from the molten center to nbar pressure levels. Treating the radiative atmosphere is crucial for sub-Neptunes, due to the large scale height and low gravity, which contributes up to 40\% of their observed radius, especially for low-mass, highly irradiated planets. Consequently, we generically find that lower H/He mass fractions are needed to match a given planetary radius, compared to previous work. While the presence of metal-enrichment in the H/He layers (here modeled as 50$\times$ solar) does not substantially influence the size of the convective envelope, it notably reduces the transit radius by shrinking the radiative atmospheric scale height. Sub-Neptunes cool differently from terrestrial planets, with the rock/iron core's cooling rate limited by the envelope, leading to longer solidification timescales. Complete solidification of the silicate mantle by 10 Gyr is found only for planets with very low masses ($\leq 1M_\oplus$) and small H/He envelopes ($\leq$ 0.1\%). Dynamo action in sub-Neptune iron cores persists as long as the mantle surface remains molten, often exceeding 10 Gyr, and becomes sensitive to core thermal conductivity after solidification. We examine aspects of ''boil-off,'' which sets the maximum allowed H/He mass and planetary radius for subsequent evolution. The rock/iron's cooling energy moderately decreases the post-boil-off H/He mass fraction in planets with large atmospheric scale heights only.

astro-ph.EP↗

arXiv2025

Reinforcement Pre-Training

In this work, we introduce Reinforcement Pre-Training (RPT) as a new scaling paradigm for large language models and reinforcement learning (RL). Specifically, we reframe next-token prediction as a reasoning task trained using RL, where it receives verifiable rewards for correctly predicting the next token for a given context. RPT offers a scalable method to leverage vast amounts of text data for general-purpose RL, rather than relying on domain-specific annotated answers. By incentivizing the capability of next-token reasoning, RPT significantly improves the language modeling accuracy of predicting the next tokens. Moreover, RPT provides a strong pre-trained foundation for further reinforcement fine-tuning. The scaling curves show that increased training compute consistently improves the next-token prediction accuracy. The results position RPT as an effective and promising scaling paradigm to advance language model pre-training.

cs.CL↗

arXiv2025

Quantum state preparation for multivariate functions

A fundamental step of any quantum algorithm is the preparation of qubit registers in a suitable initial state. Often qubit registers represent a discretization of continuous variables and the initial state is defined by a multivariate function. We develop protocols for preparing quantum states whose amplitudes encode multivariate functions by linearly combining block-encodings of Fourier and Chebyshev basis functions. Without relying on arithmetic circuits, quantum Fourier transforms, or multivariate quantum signal processing, our algorithms are simpler and more effective than previous proposals. We analyze requirements both asymptotically and pragmatically in terms of near/medium-term resources. Numerically, we prepare bivariate Student's t-distributions, 2D Ricker wavelets and electron wavefunctions in a 3D Coulomb potential, which are initial states with potential applications in finance, physics and chemistry simulations. Finally, we prepare bivariate Gaussian distributions on the Quantinuum H2-1 trapped-ion quantum processor using 24 qubits and up to 237 two-qubit gates.

quant-ph↗

arXiv2025

Hamiltonian dynamics simulation using linear combination of unitaries on an ion trap quantum computer

The linear combination of unitaries (LCU) method has proven to scale better than existing product formulas in simulating long time Hamiltonian dynamics. However, given the number of multi-control gate operations in the standard prepare-select-unprepare architecture of LCU, it is still resource-intensive to implement on the current quantum computers. In this work, we demonstrate LCU implementations on an ion trap quantum computer for calculating squared overlaps $|\langle ψ(t=0)|ψ(t>0)\rangle|^2$ of time-evolved states. This is achieved by an optimized LCU method, based on pre-selecting relevant unitaries, coupled with a compilation strategy which makes use of quantum multiplexor gates, leading to a significant reduction in the depth and number of two-qubit gates in circuits. For $L$ Pauli strings in a Taylor series expanded $n$-qubit-mapped time evolution operator, we find a two-qubit gate count of $2^{\lceil log_2(L)\rceil}(2n+1)-n-2$. We test this approach by simulating a Rabi-Hubbard Hamiltonian.

quant-ph↗

arXiv2025

QLASS: Boosting Language Agent Inference via Q-Guided Stepwise Search

Language agents have become a promising solution to complex interactive tasks. One of the key ingredients to the success of language agents is the reward model on the trajectory of the agentic workflow, which provides valuable guidance during training or inference. However, due to the lack of annotations of intermediate interactions, most existing works use an outcome reward model to optimize policies across entire trajectories. This may lead to sub-optimal policies and hinder the overall performance. To address this, we propose QLASS (Q-guided Language Agent Stepwise Search), to automatically generate annotations by estimating Q-values in a stepwise manner for open language agents. By introducing a reasoning tree and performing process reward modeling, QLASS provides effective intermediate guidance for each step. With the stepwise guidance, we propose a Q-guided generation strategy to enable language agents to better adapt to long-term value, resulting in significant performance improvement during model inference on complex interactive agent tasks. Notably, even with almost half the annotated data, QLASS retains strong performance, demonstrating its efficiency in handling limited supervision. We also empirically demonstrate that QLASS can lead to more effective decision making through qualitative analysis. We will release our code and data.

cs.LG↗

arXiv2024

Learning Versatile Skills with Curriculum Masking

Masked prediction has emerged as a promising pretraining paradigm in offline reinforcement learning (RL) due to its versatile masking schemes, enabling flexible inference across various downstream tasks with a unified model. Despite the versatility of masked prediction, it remains unclear how to balance the learning of skills at different levels of complexity. To address this, we propose CurrMask, a curriculum masking pretraining paradigm for sequential decision making. Motivated by how humans learn by organizing knowledge in a curriculum, CurrMask adjusts its masking scheme during pretraining for learning versatile skills. Through extensive experiments, we show that CurrMask exhibits superior zero-shot performance on skill prompting tasks, goal-conditioned planning tasks, and competitive finetuning performance on offline RL tasks. Additionally, our analysis of training dynamics reveals that CurrMask gradually acquires skills of varying complexity by dynamically adjusting its masking scheme.

cs.LG↗

arXiv2024

Assessing Core-Powered Mass Loss in the Context of Early Boil-Off: Minimal Long-Lived Mass Loss for the Sub-Neptune Population

We develop a python-based state-of-the-art sub-Neptune evolution model that incorporates both the post-formation boil-off at young ages $\leq$ 1 Myr and long-lived core-powered mass loss ($\sim$ Gyrs) from interior cooling. We investigate the roles of initial H/He entropy, core luminosity, energy advection, radiative atmospheric structure, and the transition to an XUV-driven mass-loss phase, with an eye on relevant timescales for planetary mass loss and thermal evolution. With particular attention to the re-equilibration process of the H/He envelope, including the energy sources that fuel the hydrodynamic wind, and energy transport timescales, we find boil-off and core-powered escape are primarily driven by stellar bolometric radiation. We further find that both boil-off and core-powered escape are decoupled from the thermal evolution. We show that, with a boil-off phase that accounts for the initial H/He mass fraction and initial entropy, post-boil-off core-powered escape has an insignificant influence on the demographics of small planets, as it is only able to remove at most 0.1% of the H/He mass fraction. Our numerical results are directly compared to previous work on analytical core-powered mass loss modeling for individual evolutionary trajectories and populations of small planets. We examine a number of assumptions made in previous studies that cause significant differences compared to our findings. We find that boil-off, though able to completely strip the gaseous envelope from a highly irradiated ($F \geq 100 F_\oplus$) planet that has a low-mass core ($M_c \leq 4M_\oplus$), cannot by itself form a pronounced radius gap as is seen in the observed population.

astro-ph.EP↗

arXiv2024

1st Place Solution of Multiview Egocentric Hand Tracking Challenge ECCV2024

Multi-view egocentric hand tracking is a challenging task and plays a critical role in VR interaction. In this report, we present a method that uses multi-view input images and camera extrinsic parameters to estimate both hand shape and pose. To reduce overfitting to the camera layout, we apply crop jittering and extrinsic parameter noise augmentation. Additionally, we propose an offline neural smoothing post-processing method to further improve the accuracy of hand position and pose. Our method achieves 13.92mm MPJPE on the Umetrack dataset and 21.66mm MPJPE on the HOT3D dataset.

cs.CV↗

arXiv2024

Almost-Optimal Computational Basis State Transpositions

We give an explicit construction to perform any $n$-qubit computational basis state transposition using $Θ(n)$ gates. This nearly coincides with the lower bound $Ω(n/\log(nd))$ on worst-case and average-case gate complexity to perform transpositions using a $d$-element gate-set, which we also prove.

quant-ph↗

arXiv2024

A High Internal Heat Flux and Large Core in a Warm Neptune Exoplanet

Interactions between exoplanetary atmospheres and internal properties have long been hypothesized to be drivers of the inflation mechanisms of gaseous planets and apparent atmospheric chemical disequilibrium conditions. However, transmission spectra of exoplanets has been limited in its ability to observational confirm these theories due to the limited wavelength coverage of HST and inferences of single molecules, mostly H$_2$O. In this work, we present the panchromatic transmission spectrum of the approximately 750 K, low-density, Neptune-sized exoplanet WASP-107b using a combination of HST WFC3, JWST NIRCam and MIRI. From this spectrum, we detect spectroscopic features due to H$_2$O (21$σ$), CH$_4$ (5$σ$), CO (7$σ$), CO$_2$ (29$σ$), SO$_2$ (9$σ$), and NH$_3$ (6$σ$). The presence of these molecules enable constraints on the atmospheric metal enrichment (M/H is 10--18$\times$ Solar), vertical mixing strength (log$_{10}$K$_{zz}$=8.4--9.0 cm$^2$s$^{-1}$), and internal temperature ($>$345 K). The high internal temperature is suggestive of tidally-driven inflation acting upon a Neptune-like internal structure, which can naturally explain the planet's large radius and low density. These findings suggest that eccentricity driven tidal heating is a critical process governing atmospheric chemistry and interior structure inferences for a majority of the cool ($<$1,000K) super-Earth-to-Saturn mass exoplanet population.

astro-ph.EP↗

arXiv2024

HiDiffusion: Unlocking Higher-Resolution Creativity and Efficiency in Pretrained Diffusion Models

Diffusion models have become a mainstream approach for high-resolution image synthesis. However, directly generating higher-resolution images from pretrained diffusion models will encounter unreasonable object duplication and exponentially increase the generation time. In this paper, we discover that object duplication arises from feature duplication in the deep blocks of the U-Net. Concurrently, We pinpoint the extended generation times to self-attention redundancy in U-Net's top blocks. To address these issues, we propose a tuning-free higher-resolution framework named HiDiffusion. Specifically, HiDiffusion contains Resolution-Aware U-Net (RAU-Net) that dynamically adjusts the feature map size to resolve object duplication and engages Modified Shifted Window Multi-head Self-Attention (MSW-MSA) that utilizes optimized window attention to reduce computations. we can integrate HiDiffusion into various pretrained diffusion models to scale image generation resolutions even to 4096x4096 at 1.5-6x the inference speed of previous methods. Extensive experiments demonstrate that our approach can address object duplication and heavy computation issues, achieving state-of-the-art performance on higher-resolution image synthesis tasks.

cs.CV↗