arXiv subjects

Zhengping Wang

Publications and source records attributed to Zhengping Wang.

12 recordsLinked to original sources

arXiv2026

20/20 Vision Language Models: A Prescription for Better VLMs through Data Curation Alone

Data curation has shifted the quality-compute frontier for language-model and contrastive image-text pretraining, but its role for vision-language models (VLMs) is far less established. We ask how far data curation alone can take VLM performance, holding architecture, training recipe, and compute fixed and varying only the training data. Our pipeline, applied to the MAmmoTH-VL single-image subset, lifts performance by +11.7pp on average across 20 public VLM benchmarks (spanning grounding, VQA, OCR/documents, captioning, spatial/3D, counting, charts, math, brand-ID, and multi-image reasoning) and by +11.3pp on average across all nine capability axes of DatBench, our high-fidelity VLM eval suite. At 2B, our curated model surpasses InternVL3.5-2B by 9.9pp at ~17x less training compute and closes the gap to Qwen3-VL-2B to within 1.8pp at ~87x less compute, from pretraining alone. Beyond accuracy, curation delivers four further properties: (1) Reliability: per-capability std across training seeds drops by ~67% and the lift survives a 4k-to-16k context-length sweep; (2) OOD generalization: the 9-eval OOD average rises by +7.2pp, and multi-image BLINK rises by +3.09pp despite single-image-only training, with Visual Correspondence gaining +11.8pp; (3) Behavioral gains beyond benchmarks: across ~1,100 open-ended queries the curated 2B is more honest and more specific than the matched-compute baseline, and more concise and less refusal-prone than a frontier 2B reference; (4) Pareto-dominance on inference cost: at every scale (1B, 2B, 4B) the curated model raises accuracy while lowering response FLOPs vs. the matched-compute baseline, and the curated 4B matches near-frontier accuracy at 3.3x lower response FLOPs than Qwen3-VL-4B. Data curation is a high-leverage tool for building better VLMs, reaching near-frontier accuracy at up to ~150x less training compute.

cs.LG↗

arXiv2026

The Finetuner's Fallacy: When to Pretrain with Your Finetuning Data

Real-world model deployments demand strong performance on narrow domains where data is often scarce. Typically, practitioners finetune models to specialize them, but this risks overfitting to the domain and forgetting general knowledge. We study a simple strategy, specialized pretraining (SPT), where a small domain dataset, typically reserved for finetuning, is repeated starting from pretraining as a fraction of the total tokens. Across three specialized domains (ChemPile, MusicPile, and ProofPile), SPT improves domain performance and preserves general capabilities after finetuning compared to standard pretraining. In our experiments, SPT reduces the pretraining tokens needed to reach a given domain performance by up to 1.75x. These gains grow when the target domain is underrepresented in the pretraining corpus: on domains far from web text, a 1B SPT model outperforms a 3B standard pretrained model. Beyond these empirical gains, we derive overfitting scaling laws to guide practitioners in selecting the optimal domain-data repetition for a given pretraining compute budget. Our observations reveal the finetuner's fallacy: while finetuning may appear to be the cheapest path to domain adaptation, introducing specialized domain data during pretraining stretches its utility. SPT yields better specialized domain performance (via reduced overfitting across repeated exposures) and better general domain performance (via reduced forgetting during finetuning), ultimately achieving stronger results with fewer parameters and less total compute when amortized over inference. To get the most out of domain data, incorporate it as early in training as possible.

cs.LG↗

arXiv2026

\"UberWeb: Insights from Multilingual Curation for a 20-Trillion-Token Dataset

Multilinguality is a core capability for modern foundation models, yet training high-quality multilingual models remains challenging due to uneven data availability across languages. A further challenge is the performance interference that can arise from joint multilingual training, commonly referred to as the "curse of multilinguality". We study multilingual data curation across thirteen languages and find that many reported regressions are not inherent to multilingual scaling but instead stem from correctable deficiencies in data quality and composition rather than fundamental capacity limits. In controlled bilingual experiments, improving data quality for any single language benefits others: curating English improves non-English performance in 12 of 13 languages, while curating non-English yields reciprocal improvements in English. Bespoke per-language curation produces substantially larger within-language improvements. Extending these findings to large-scale general-purpose training mixtures, we show that curated multilingual allocations comprising under 8% of total tokens remain remarkably effective. We operationalize this approach within an effort that produced a 20T-token pretraining corpus derived entirely from public sources. Models with 3B and 8B parameters trained on a 1T-token random subset achieve competitive multilingual accuracy with 4-10x fewer training FLOPs than strong public baselines, establishing a new Pareto frontier in multilingual performance versus compute. Moreover, these benefits extend to frontier model scale: the 20T-token corpus served as part of the pretraining dataset for Trinity Large (400B/A13B), which exhibits strong multilingual performance relative to its training FLOPs. These results show that targeted, per-language data curation mitigates multilingual interference and enables compute-efficient multilingual scaling.

cs.LG↗

arXiv2026

DatBench: Discriminative, Faithful, and Efficient VLM Evaluations

Empirical evaluation serves as the primary compass guiding research progress in foundation models. Despite a large body of work focused on training frontier vision-language models (VLMs), approaches to their evaluation remain nascent. To guide their maturation, we propose three desiderata that evaluations should satisfy: (1) faithfulness to the modality and application, (2) discriminability between models of varying quality, and (3) efficiency in compute. Through this lens, we identify critical failure modes that violate faithfulness and discriminability, misrepresenting model capabilities: (i) multiple-choice formats reward guessing, poorly reflect downstream use cases, and saturate early as models improve; (ii) blindly solvable questions, which can be answered without images, constitute up to 70% of some evaluations; and (iii) mislabeled or ambiguous samples compromise up to 42% of examples in certain datasets. Regarding efficiency, the computational burden of evaluating frontier models has become prohibitive: by some accounts, nearly 20% of development compute is devoted to evaluation alone. Rather than discarding existing benchmarks, we curate them via transformation and filtering to maximize fidelity and discriminability. We find that converting multiple-choice questions to generative tasks reveals sharp capability drops of up to 35%. In addition, filtering blindly solvable and mislabeled samples improves discriminative power while simultaneously reducing computational cost. We release DatBench-Full, a cleaned evaluation suite of 33 datasets spanning nine VLM capabilities, and DatBench, a discriminative subset that achieves 13x average speedup (up to 50x) while closely matching the discriminative power of the original datasets. Our work outlines a path toward evaluation practices that are both rigorous and sustainable as VLMs continue to scale.

cs.LG↗

arXiv2025

Luxical: High-Speed Lexical-Dense Text Embeddings

Frontier language model quality increasingly hinges on our ability to organize web-scale text corpora for training. Today's dominant tools trade off speed and flexibility: lexical classifiers (e.g., FastText) are fast but limited to producing classification output scores, while the vector-valued outputs of transformer text embedding models flexibly support numerous workflows (e.g., clustering, classification, and retrieval) but are computationally expensive to produce. We introduce Luxical, a library for high-speed "lexical-dense" text embeddings that aims to recover the best properties of both approaches for web-scale text organization. Luxical combines sparse TF--IDF features, a small ReLU network, and a knowledge distillation training regimen to approximate large transformer embedding models at a fraction of their operational cost. In this technical report, we describe the Luxical architecture and training objective and evaluate a concrete Luxical model in two disparate applications: a targeted webcrawl document retrieval test and an end-to-end language model data curation task grounded in text classification. In these tasks we demonstrate speedups ranging from 3x to 100x over varying-sized neural baselines, and comparable to FastText model inference during the data curation task. On these evaluations, the tested Luxical model illustrates favorable compute/quality trade-offs for large-scale text organization, matching the quality of neural baselines. Luxical is available as open-source software at https://github.com/datologyai/luxical.

cs.CL↗

arXiv2025

BeyondWeb: Lessons from Scaling Synthetic Data for Trillion-scale Pretraining

Recent advances in large language model (LLM) pretraining have shown that simply scaling data quantity eventually leads to diminishing returns, hitting a data wall. In response, the use of synthetic data for pretraining has emerged as a promising paradigm for pushing the frontier of performance. Despite this, the factors affecting synthetic data quality remain poorly understood. In this work, we introduce BeyondWeb, a synthetic data generation framework that produces high-quality synthetic data for pretraining. BeyondWeb significantly extends the capabilities of traditional web-scale datasets, outperforming state-of-the-art synthetic pretraining datasets such as Cosmopedia and Nemotron-CC's high-quality synthetic subset (Nemotron-Synth) by up to 5.1 percentage points (pp) and 2.6pp, respectively, when averaged across a suite of 14 benchmark evaluations. It delivers up to 7.7x faster training than open web data and 2.7x faster than Nemotron-Synth. Remarkably, a 3B model trained for 180B tokens on BeyondWeb outperforms an 8B model trained for the same token budget on Cosmopedia. We also present several insights from BeyondWeb on synthetic data for pretraining: what drives its benefits, which data to rephrase and how, and the impact of model size and family on data quality. Overall, our work shows that there's no silver bullet for generating high-quality synthetic pretraining data. The best outcomes require jointly optimizing many factors, a challenging task that requires rigorous science and practical expertise. Naive approaches can yield modest improvements, potentially at great cost, while well-executed methods can yield transformative improvements, as exemplified by BeyondWeb.

cs.LG↗

arXiv2021

From KP-I lump solution to travelling waves of Gross-Pitaevskii equation

Let $q(x,y)$ be an nondegenerate lump solution to KP-I (Kadomtsev-Petviashvili-I) equation $$\partial_x^4q-2\sqrt{2}\partial_x^2q-3\sqrt{2}\partial_x((\partial_xq) ^2)-2\partial_y^2q=0. $$ We prove the existence of a traveling wave solution $ u_{\e} (x-ct, y)$ to GP (Gross-Pitaevskii) equation $$ i\partial_{t}Ψ+ΔΨ+(1-|Ψ|^{2})Ψ=0,\ \ \ \mbox{in} \ {\mathbb R}^2 $$ in the transonic limit $$ c=\sqrt{2}-ε^2 $$ with $$ u_ε=1 + i εq(x,y) + {\mathcal O} (ε^2). $$ This proves the existence of finite energy solutions in the so-called Jones-Roberts program in the transonic range $ c \in (\sqrt{2}-ε^2, \sqrt{2})$. The main ingredients in our proof are detailed point-wise estimates of the Green function associated to a family of fourth order hypoelliptic operators $$\partial_x^4-(2\sqrt{2}-\e^2)\partial_x^2-2\partial_y^2+\e^2\partial_x^2\partial_y^2+\e^4\partial_y^4.$$

math.AP↗

arXiv2021

Realization of electron antidoping by modulating the breathing distortion in BaBiO$_3$

The recent proposal of antidoping scheme breaks new ground in conceiving conversely functional materials and devices, yet the few available examples belong to the correlated electron systems. Here we demonstrate both theoretically and experimentally that the main group oxide BaBiO$_3$ is a model system for antidoping using oxygen vacancies. The first principles calculations show that the band gap systematically increases due to the strongly enhanced BiO breathing distortions away from the vacancies and the annihilation of Bi 6s and O 2p hybridized conduction bands near the vacancies. The spectroscopic experiments confirm the band gap increasing systematically with electron doping, with a maximal gap enhancement of 75% when the film's stoichiometry is reduced to BaBiO$_{2.75}$. The Raman and diffraction experiments show the suppression of the overall breathing distortion. The study unambiguously demonstrates the remarkable antidoping effect in a material without strong electron correlations and underscores the importance of bond disproportionation in realizing such an effect.

cond-mat.mtrl-sci↗

arXiv2020

Dual-comb femtosecond solid-state laser with inherent polarization-multiplexing

Dual-comb spectroscopy is a rapidly developing technique enabling ultraprecise broadband optical diagnostics of atoms and molecules. This powerful tool typically requires two phase-locked femtosecond lasers, yet it has been shown that it can be realized without any stabilization if the combs are generated from a single laser cavity. Still, unavoidable intrinsic relative phase-fluctuations always set a limit on the precision of any spectroscopic measurements, hitherto limiting the applicability of bulk dual-comb lasers for mode-resolved studies. Here, we demonstrate a versatile concept for low-noise dual-comb generation from a single-cavity femtosecond solid-state laser based on intrinsic polarization-multiplexing inside an optically anisotropic gain crystal. Due to intracavity spatial separation of the orthogonally-polarized beams, two sub-100 fs pulse trains are simultaneously generated from a 1.05 $μ$m Yb:CNGS oscillator with a repetition rate difference of 4.7 kHz. The laser exhibits the lowest relative noise ever demonstrated for a bulk dual-comb source, supporting free-running mode-resolved spectroscopic measurements over a second. Moreover, the developed dual-comb generation technique can be applied to any solid-state laser exploiting a birefringent active crystal, paving the way towards a new class of highly-coherent, single-cavity, dual-comb laser sources operating in various spectral regions.

physics.optics↗

arXiv2016

Stability of traveling waves of nonlinear Schrödinger equation with nonzero condition at infinity

We study the stability of traveling waves of nonlinear Schrödinger equation with nonzero condition at infinity obtained via a constrained variational approach. Two important physical models are Gross-Pitaevskii (GP) equation and cubic-quintic equation. First, under a non-degeneracy condition we prove a sharp instability criterion for 3D traveling waves of (GP), which had been conjectured in the physical literature. This result is also extended for general nonlinearity and higher dimensions, including 4D (GP) and 3D cubic-quintic equations. Second, for cubic-quintic type sub-critical or critical nonlinearity, we construct slow traveling waves and prove their nonlinear instability in any dimension. For traveling waves without vortices (i.e. nonvanishing) of general nonlinearity in any dimension, we find the sharp condition for linear instability. Third, we prove that any 2D traveling wave of (GP) is transversally unstable and find the sharp interval of unstable transversal wave numbers. Near unstable traveling waves of above cases, we construct unstable and stable invariant manifolds.

math.AP↗

arXiv2014

Multiple solutions for a nonhomogeneous Schrödinger-Maxwell system in $R^3$

The paper considers the following nonhomogeneous Schrödinger-Maxwell system -Δu + u+λϕ(x) u =|u|^{p-1}u+g(x),\ x\in \mathbb{R}^3, -Δϕ= u^2, \ x\in \mathbb{R}^3, . \leqno{(SM)} where $λ>0$, $p\in(1,5)$ and $g(x)=g(|x|)\in L^2(\mathbb{R}^3)\setminus{0}$. There seems no any results on the existence of multiple solutions to problem (SM) for $p \in (1,3]$. In this paper, we find that there is a constant$C_p>0$ such that problem (SM) has at least two solutions for all $p\in (1,5)$ provided $\|g\|_{L^2} \leq C_p$, but only for $p\in(1,2]$ we need $λ>0$ is small. Moreover, $C_p=\frac{(p-1)}{2p}[\frac{(p+1)S^{p+1}}{2p}]^{1/(p-1)}$, where $S$ is the Sobolev constant.

math.AP↗

arXiv2013

Thermally driven continuous-wave and pulsed optical vortex

We demonstrated the continuous-wave (cw) and pulsed optical vortex with topological charges driven by heat generated during the lasing process without introducing the astigmatism effect and reducing lasing efficiency. During the lasing process, the topological charges were changeable by the thermal-induced lens and selected by the mode-matching between the pump and oscillating beams. With a graphene sample as the saturable absorber, the pulsed optical vortex was achieved at the wavelength of 1.36 μm, which identified that graphene could be used as a pulse modulator for the generation of pulsed optical vortex. It could be believed that the thermally driven cw and pulsed optical vortex should have various promising applications based on the compact structure, changeable topological charges and specific wavelength

physics.optics↗