arXiv subjects

Weidong Zhou

Publications and source records attributed to Weidong Zhou.

15 recordsLinked to original sources

arXiv2026

InfoLaw: Information Scaling Laws for Large Language Models with Quality-Weighted Mixture Data and Repetition

Upweighting high-quality data in LLM pretraining often improves performance, but in datalimited regimes, especially under overtraining, stronger upweighting increases repetition and can degrade performance. However, standard scaling laws do not reliably extrapolate across mixture recipes or under repetitions, making the selection for optimal data recipes at scaling underdetermined. To solve this, we introduce InfoLaw (Information Scaling Laws), a data-aware scaling framework that predicts loss from consumed tokens, model size, data mixture weights, and repetition. The key idea is to model pretraining as information accumulation, where quality controls information density and repetition induces scaledependent diminishing returns. We first collect the model performance after training on datasets that vary in scale, quality distribution, and repetition level. Then we build up the modeling for information so that information accurately predicts those model performance. InfoLaw predicts performance on unseen data recipes and larger scale runs (up to 7B, 425B tokens) with 0.15% mean and 0.96% max absolute error in loss, and it extrapolates reliably across overtraining levels, enabling efficient data-recipe selection under varying compute budgets.

cs.CL↗

arXiv2026

Target-Oriented Pretraining Data Selection via Neuron-Activated Graph

Everyday tasks come with a target, and pretraining models around this target is what turns them into experts. In this paper, we study target-oriented language model (LM) pretraining by introducing Neuron-Activated Graph Ranking (NAG-based Ranking), a training-free and interpretable framework for target pretraining data selection. Rather than using black-box representations, our approach directly characterizes each target input by a sparse set of high-impact neurons in any off-the-shelf LLMs. Concretely, we quantify neuron impact and select the most influential neurons across layers into a compact Neuron-Activated Graph (NAG), and rank candidate data by NAG similarity to target examples. We conduct experiments across six benchmarks, where our NAG-based Ranking improves target-oriented pretraining by 4.9% on average over random sampling, and also outperforms state-of-the-art baselines by 5.3% accuracy on HellaSwag. It also remains effective under a more applicable multi-target setting, where our best setup surpasses two baselines by 1.1% and 4.1%, respectively. Furthermore, we provide a comprehensive analysis on why and how our NAG works, e.g., deactivating NAG-selected neurons (only 0.12% of all) causes a 23.5% performance collapse, and restricting NAG to the final layer incurs a 4.1% average drop, indicating that NAG captures a sparse "functional backbone" for learning target features. We release the code at https://github.com/asillycat/NAG.

cs.CL↗

arXiv2025

TiKMiX: Take Data Influence into Dynamic Mixture for Language Model Pre-training

The data mixture used in the pre-training of a language model is a cornerstone of its final performance. However, a static mixing strategy is suboptimal, as the model's learning preferences for various data domains shift dynamically throughout training. Crucially, observing these evolving preferences in a computationally efficient manner remains a significant challenge. To address this, we propose TiKMiX, a method that dynamically adjusts the data mixture according to the model's evolving preferences. TiKMiX introduces Group Influence, an efficient metric for evaluating the impact of data domains on the model. This metric enables the formulation of the data mixing problem as a search for an optimal, influence-maximizing distribution. We solve this via two approaches: TiKMiX-D for direct optimization, and TiKMiX-M, which uses a regression model to predict a superior mixture. We trained models with different numbers of parameters, on up to 1 trillion tokens. TiKMiX-D exceeds the performance of state-of-the-art methods like REGMIX while using just 20% of the computational resources. TiKMiX-M leads to an average performance gain of 2% across 9 downstream benchmarks. Our experiments reveal that a model's data preferences evolve with training progress and scale, and we demonstrate that dynamically adjusting the data mixture based on Group Influence, a direct measure of these preferences, significantly improves performance by mitigating the underdigestion of data seen with static ratios.

cs.LG↗

arXiv2025

KnowDR-REC: A Benchmark for Referring Expression Comprehension with Real-World Knowledge

Referring Expression Comprehension (REC) is a popular multimodal task that aims to accurately detect target objects within a single image based on a given textual expression. However, due to the limitations of earlier models, traditional REC benchmarks either rely solely on intra-image cues or lack sufficiently fine-grained instance annotations, making them inadequate for evaluating the reasoning capabilities of Multi-modal Large Language Models (MLLMs). To address this gap, we propose a new benchmark, KnowDR-REC, characterized by three key features: Firstly, it is built upon real-world knowledge, requiring fine-grained multimodal reasoning across text and image. Secondly, the dataset includes elaborately constructed negative samples via fine-grained expression editing, designed to evaluate a model's robustness and anti-hallucination ability. Lastly, we introduce three novel evaluation metrics to systematically explore the model's internal reasoning process. We evaluate 16 state-of-the-art multimodal models on KnowDR-REC, with experimental results showing that existing MLLMs still struggle with knowledge-driven visual grounding tasks. Furthermore, we observe a decoupling between textual understanding and visual grounding in MLLMs, where many models are significantly influenced by memorized shortcut correlations, which severely affect their behavior on our benchmark and hinder genuine multimodal reasoning. We anticipate that the proposed benchmark will inspire future research towards developing more robust, interpretable, and knowledge-intensive visual grounding frameworks, driving the development of more reliable and robust multimodal systems for complex real-world scenarios.

cs.LG↗

arXiv2025

QuaDMix: Quality-Diversity Balanced Data Selection for Efficient LLM Pretraining

Quality and diversity are two critical metrics for the training data of large language models (LLMs), positively impacting performance. Existing studies often optimize these metrics separately, typically by first applying quality filtering and then adjusting data proportions. However, these approaches overlook the inherent trade-off between quality and diversity, necessitating their joint consideration. Given a fixed training quota, it is essential to evaluate both the quality of each data point and its complementary effect on the overall dataset. In this paper, we introduce a unified data selection framework called QuaDMix, which automatically optimizes the data distribution for LLM pretraining while balancing both quality and diversity. Specifically, we first propose multiple criteria to measure data quality and employ domain classification to distinguish data points, thereby measuring overall diversity. QuaDMix then employs a unified parameterized data sampling function that determines the sampling probability of each data point based on these quality and diversity related labels. To accelerate the search for the optimal parameters involved in the QuaDMix framework, we conduct simulated experiments on smaller models and use LightGBM for parameters searching, inspired by the RegMix method. Our experiments across diverse models and datasets demonstrate that QuaDMix achieves an average performance improvement of 7.2% across multiple benchmarks. These results outperform the independent strategies for quality and diversity, highlighting the necessity and ability to balance data quality and diversity.

cs.CL↗

arXiv2021

Complete 2π Phase Control by Photonic Crystal Slabs

Photonic crystal slabs are the state of the art in studies for the light confinement, optical wave modulating and guiding, as well as nonlinear optical response. Previous studies have shown abundant real-world implementations of photonic crystals in planar optics, metamaterials, sensors, and lasers. Here, we report a novel full 2π phase control method in the reflected light beam over the interaction with a photonic crystal resonant mode, verified by the temporal coupled-mode analysis and S-parameter simulations. Enhanced by the asymmetric coupling with the output ports, the 2π phase shift can be achieved with the silicon photonics platforms such as Silicon-on-Silica and Silicon-on-Insulator heterostructures. Such photonic crystal phase control method provides a general guide in the design of phase-shift metamaterials, suggesting a wide range of applications in the field of sensing, spatial light modulation, and beam steering.

physics.optics↗

arXiv2021

Controllable finite ultra-narrow quality-factor peak in a perturbed Dirac-cone band structure of a photonic crystal slab

We show that by using a perturbed photonic Dirac-cone, one can realize ultra-narrow and finite Q-factor peak in the wavevector space, with both the peak value and the width separately tunable. We also discuss a lower bound in the minimal viable width given a peak Q-value while maintaining sufficient Q differentiation among modes. The strong angular and frequency Q-selection finds applications in optical devices where strong angle- and frequency-selection is needed.

physics.optics↗

arXiv2020

Long-term Stability of Bilayer MoS2 in Ambient Air

We report that chemical vapor deposition (CVD) grown bilayer and thicker-layered MoS2 are structurally and optically stable under ambient conditions, in comparison to CVD-grown monolayer MoS2 and other transition metal dichalcogenides (TMDs) that have been reported to degrade under the same conditions, hindering their many potential applications. We present atomic force microscopy (AFM), and Raman and photoluminescence (PL) spectroscopy measurements of as-grown and preheated multilayer MoS2 after exposure to ambient conditions for periods of up to 2 years. The AFM images show that, under ambient conditions, as-grown and preheated bilayer and thicker-layered MoS2 films do not exhibit the growth of dendrites that is characteristic of monolayer degradation. Dendrites are observed to stop at the monolayer-bilayer boundary. Raman and PL spectra of the aged bilayer and thicker-layered films are comparable to those of as-grown films. The greater stability of bilayers and thicker layers supports a previously reported mechanism for monolayer degradation involving Förster resonance energy transfer. Our results show that bilayer and thicker-layered TMDs are promising materials for applications requiring ambient stability.

cond-mat.mtrl-sci↗

arXiv2019

Scalable microcavity-coupled emitters in hexagonal boron nitride

Scalable integration of bright emitters in quantum photonic structures is an important step in the broader quest to generate and manipulate single photons via compact solid-state devices. Unfortunately, implementations relying on material platforms that also serve as the emitter host often suffer from a trade-off between the desired emitter properties and the photonic system practicality and performance. Here, we demonstrate 'pick and place' integration of a Silicon Nitride microdisk optical resonator with a bright emitter host in the form of 20nm thick hexagonal boron nitride (hBN).The film folds around the microdisk maximizing contact to ultimately form a composite hBN/Si3N4 structure. The local strain that develops in the hBN film at the resonator circumference deterministically activates a low density of SPEs within the whispering gallery mode volume of the microdisk. These conditions allow us to demonstrate cavity-mediated out-coupling and Purcell enhancement of emission from hBN color centers through the microdisk cavity modes. Our results pave the route toward the development of scalable quantum photonic circuits with independent emitter/resonator optimization for active and passive functionalities.

physics.optics↗

arXiv2019

Accurate Target Localization by using Artificial Pinnae of brown long-eared bat

Echolocating bats locate the targets by echolocation. Many theoretical frameworks have been suggested the abilities of bats are related to the shapes of bats ears, but few artificial bat-like ears have been made to mimic the abilities, the difficulty of which lies in the determination of the elevation angle of the target. In this study, we present a device with artificial bat pinnae modeling by the ears of brown long-eared bat (Plecotus auritus) which can accurately estimate the elevation angle of the aerial target by virtue of active sonar. An artificial neural-network with the labeled data obtained from echoes as the trained and tested data is used and optimized by a tenfold cross-validation technique. A decision method we named sliding window averaging algorithm is designed for getting the estimation results of elevation. At last, a right-angle pinnae construction is designed for determining direction of the target. The results show a higher accuracy for the direction determination of the single target. The results also demonstrate that for the Plecotus auritus bat, not only the binaural shapes, but the binaural relative orientations also play important roles in the target localization.

eess.SP↗

arXiv2018

Laterally confined photonic crystal surface emitting laser based on monolayer tungsten disulfide operating at room temperature

We report a photonic crystal surface emitting laser using monolayer tungsten disulfide as the gain medium. The cavity design utilizes a heterostructure in the photonic crystal lattice to provide lateral confinement for a high quality factor with a compact active region. Room temperature continuous wave lasing is realized after integrating monolayer tungsten disulfide flakes onto the silicon nitride photonic crystal on quartz substrate. Highly directional, near surface normal emission has also been experimentally demonstrated.

physics.optics↗

arXiv2018

First-Principles Threshold Calculation of Photonic Crystal Surface-Emitting Lasers Using Rigorous Coupled Wave Analysis

We show that the threshold of a photonic crystal surface-emitting laser can be calculated from first-principles by the method of rigorous coupled wave analysis (RCWA), which has been widely used to simulate the response spectra of passive periodic structures. Here, the scattering matrix (S-matrix) of a surface-emitting laser structure with added gain is calculated on the complex frequency plane using RCWA, and the lasing threshold is determined by the value of gain for which the pole of the S-matrix reaches the real axis. This approach can be used for surface emitting laser struc tures in general, and is particularly useful for the surface emitting laser systems with complex in-plane structures.

physics.optics↗

arXiv2017

229 nm UV LEDs using p-type silicon for increased hole injection

Ultraviolet (UV) light emission at 229 nm wavelength from diode structures based on AlN/Al0.77Ga0.23N quantum wells and using p-type Si to significantly increase hole injection was reported. Both electrical and optical characteristics were measured. Owing to the large concentration of holes from p-Si and efficient hole injection, no efficiency droop was observed up to a current density of 76 A/cm2 under continuous wave operation and without external thermal management. An optical output power of 160 uW was obtained with corresponding external quantum efficiency of 0.027%. This study demonstrates that by adopting p-type Si nanomembrane contacts as hole injector, practical levels of hole injection can be realized in UV light-emitting diodes with very high Al composition AlGaN quantum wells, enabling emission wavelengths and power levels that were previously inaccessible using traditional p-i-n structures with poor hole injection efficiency.

physics.app-ph↗

arXiv2016

Thermal Diffusion Boron Doping of Single-Crystal Diamond

With the best overall electronic and thermal properties, single crystal diamond (SCD) is the extreme wide bandgap material that is expected to revolutionize power electronics and radio-frequency electronics in the future. However, turning SCD into useful semiconductors requires overcoming doping challenges, as conventional substitutional doping techniques, such as thermal diffusion and ion implantation, are not easily applicable to SCD. Here we report a simple and easily accessible doping strategy demonstrating that electrically activated, substitutional doping in SCD without inducing graphitization transition or lattice damage can be readily realized with thermal diffusion at relatively low temperatures by using heavily doped Si nanomembranes as a unique dopant carrying medium. Atomistic simulations elucidate a vacancy exchange boron doping mechanism that occur at the bonded interface between Si and diamond. We further demonstrate selectively doped high voltage diodes and half-wave rectifier circuits using such doped SCD. Our new doping strategy has established a reachable path toward using SCDs for future high voltage power conversion systems and for other novel diamond based electronic devices. The novel doping mechanism may find its critical use in other wide bandgap semiconductors.

cond-mat.mtrl-sci↗

arXiv2015

Thermal Diffusion Doping of Single Crystal Diamond

With the best overall electronic and thermal properties, single-crystal diamond (SCD) is the extreme wide bandgap material that is expected to revolutionize power electronics and radio-frequency electronics in the future. However, turning SCD into useful semiconductors faces doping challenges, as conventional substitutional doping techniques, such as thermal diffusion and ion-implantation, are not easily applicable to SCD. Here we report a simple and easily accessible doping strategy demonstrating that electrically activated, substitutional boron doping in natural SCD without any phase transitions or lattice damage which can be readily realized with thermal diffusion at relatively low temperature. For the boron doping, we employ a unique dopant carrying medium: heavily doped Si nanomembranes. We further demonstrate selectively doped high-voltage diodes and half-wave rectifier circuits using such doped SCD. Our new doping strategy has established a reachable path toward using SCDs for future high-voltage power conversion systems and for other novel diamond-based electronics.

cond-mat.mes-hall↗