arXiv subjects

Ru He

Publications and source records attributed to Ru He.

14 recordsLinked to original sources

arXiv2026

Bounded-Compute Multimodal Regression for Product-Rating Prediction

Vision-language models (VLMs) are increasingly attractive for multimodal quality assessment, but their default reliance on autoregressive text generation and dynamic visual processing is poorly matched to scalar regression under strict latency budgets. We present a bounded-compute adaptation of SmolVLM2-256M-Video-Instruct for product-rating prediction in the LoViF 2026 Efficient VLM challenge. Motivated by recent multimodal engagement-prediction results showing that feature-based regression can outperform token-based score generation, we replace the language-modeling head with a lightweight two-layer MLP fed by pooled decoder states, and we enforce deterministic inputs through fixed 384x384 images and truncated metadata. Across controlled ablations, static global image processing slightly outperforms dynamic tiling, and scaling from 100K to 16M training examples substantially improves validation correlation. Under the official held-out evaluation, our 228M-parameter model achieves 0.39 PLCC and 0.40 CES, providing a strong and reproducible baseline for resource-constrained multimodal regression.

cs.CV↗

arXiv2026

Understanding the anisotropic response of $\beta$-Ga$_2$O$_3$ to ion implantation

While $\beta$-Ga$_2$O$_3$ is considered a promising wide bandgap semiconductor, the impact of ion-induced defect formation and anisotropic elasticity remains poorly understood. Here, we combine a simulation and experiment X-ray diffraction (XRD) study of the strain-stress dynamics induced by ion implantation into $\beta$-Ga$_2$O$_3$ single-crystals with different surface orientations. The strain accumulation in the out-of-plane direction is observed by XRD to occur in an anisotropic manner, with compressive strain along the [010] direction and tensile strain along the directions perpendicular to (100) and (001). An anisotropic stress/strain accumulation model is proposed and probed via Molecular Dynamics (MD), showing an excellent agreement with the experiments. For higher damage levels, pole figures obtained both experimentally and by MD via a novel reciprocal-space projection method reveal an orientation-independent $\beta$-to-$\gamma$ phase transition, with a fixed crystallographic relationship between the polymorphs. By exploring the strain-stress dynamics in anisotropic systems, this work establishes a method to directly compare macroscale diffraction experiments and atomistic simulations and opens a new path to engineer the properties of such systems utilizing their anisotropic response to ion implantation/irradiation.

cond-mat.mtrl-sci↗

arXiv2025

Microtubes and nanomembranes by ion-beam-induced exfoliation of $\beta$-Ga$_{2}$O$_{3}$

This paper reports an innovative process to fabricate $\beta$-Ga$_{2}$O$_{3}$ microtubes and nanomembranes based on ion implantation in (100)-oriented single-crystals. We show that, under specific flux and fluence conditions, the irradiation-induced strain profile promotes the detachment and rolling-up of a thin surface layer, forming a microtube. The strain-disorder interplay was investigated in detail for Cr-implanted $\beta$-Ga$_{2}$O$_{3}$ with a range of complementary methods, showing an excellent agreement between experimental and simulation data, and suggesting an exfoliation mechanism that is correlated with the anisotropic nature of the $\beta$-Ga$_{2}$O$_{3}$ monoclinic system and its easy-cleavage planes. Moreover, these microtubes can be unrolled upon a subsequent annealing step, resulting in nanomembranes with bulk-like crystalline quality that can be transferred to other substrates. The recovery of the implantation-induced damage under thermal annealing has also been studied, showing a remarkable recovery at moderate temperatures (~500 {\deg}C). This observation underscores the potential of this method for the scalable production of nanomembranes with improved reproducibility compared to conventional mechanical exfoliation techniques. Importantly, such exfoliation can be done employing different ions, providing simultaneous $\beta$-Ga$_{2}$O$_{3}$ doping, chosen to control the structural, optical, magnetic and electrical properties of the nanomembranes, thus tailoring them to fit the desired applications.

cond-mat.mtrl-sci↗

arXiv2024

Phase glides and self-organization of atomically abrupt interfaces out of stochastic disorder in $\alpha$-Ga$_{2}$O$_{3}$

Disorder-induced ordering and unprecedentedly high radiation tolerance in $\gamma$-phase of gallium oxide is a recent spectacular discovery at the intersection of the fundamental physics and electronic applications. Importantly, by far, these data were collected with initial samples in form of the thermodynamically stable $\beta$-phase of this material. Here, we investigate these phenomena starting instead from already metastable $\alpha$-phase and explain radically new trend occurring in the system. We argue that in contrast to that in $\beta$-to-$\gamma$ disorder-induced transitions, the O sublattice in $\alpha$-phase exhibits hexagonal close-packed structure, so that to activate $\alpha$-to-$\gamma$ transformation significant structural rearrangements are required in both Ga and O sublattices. Moreover, consistently with theoretical predictions, $\alpha$-to-$\gamma$ phase transformation requires accumulation of the substantial tensile strain to initiate otherwise impossible lattice glides. Thus, we explain the experimentally observed trends in term of the combination of disorder and strain governing the process. Finally, and perhaps most amazingly, we demonstrate atomically abrupt $\alpha$/$\gamma$ interfaces paradoxically self-organized out of the stochastic disorder.

cond-mat.mtrl-sci↗

arXiv2024

Ultrahigh Stability of O-Sublattice in $\beta$-Ga$_2$O$_3$

Recently reported remarkably high radiation tolerance of $\gamma$/$\beta$-Ga$_2$O$_3$ double-polymorphic structure brings this ultrawide bandgap semiconductor to the frontiers of power electronics applications that are able to operate in challenging environments. Understanding the mechanism of radiation tolerance is crucial for further material modification and tailoring of the desired properties. In this study, we employ machine-learning-enhanced atomistic simulations to assess the stability of both the gallium (Ga) and oxygen (O) sublattices under various levels of damage. Our study uncovers the remarkable resilience and stability of the O-sublattice, attributing this property to the strong tendency of recovery of the O defects, especially within the stronger disordered regions. Interestingly, we observe the opposite behavior of the Ga defects that display enhanced stability in the same regions of increased disorder. Moreover, we observe that highly defective $\beta$-Ga$_2$O$_3$ is able to transform into $\gamma$-Ga$_2$O$_3$ upon annealing due to preserved lattice organization of the O-sublattice. This result clearly manifests that the ultrahigh stability of the O-sublattice provides the backbone for the exceptional radiation tolerance of the $\gamma$/$\beta$ double-polymorphic structure. These computational insights closely align with experimental observations, opening avenues for further exploration of polymorphism in Ga$_2$O$_3$ and potentially in analogous polymorphic families spanning a broad range of diverse materials of complex polymorphic nature.

cond-mat.mtrl-sci↗

arXiv2024

Threshold displacement energy map of Frenkel pair generation in $\rm Ga_2O_3$ from machine-learning-driven molecular dynamics simulations

$\beta$ phase gallium oxide ($\beta$-$\rm Ga_2O_3$) demonstrates tremendous potential for electronics applications and offers promising prospects for integration into future space systems with the necessity of high radiation resistance. Therefore, a comprehensive understanding of the threshold displacement energy (TDE) and the radiation-induced formation of Frenkel pairs (FPs) in this material is vital but has not yet been thoroughly studied. In this work, we performed over 5,000 molecular dynamics simulations using our machine-learning potentials to determine the TDE and investigate the formation of FPs. The average TDEs for the two Ga sites, Ga1 (tetrahedral site) and Ga2 (octahedral site), are 22.9 and 20.0 eV, respectively. While the average TDEs for the three O sites are nearly uniform, ranging from 17.0 to 17.4 eV. The generated TDE maps reveal significant differences in displacement behavior between these five atomic sites. Our developed defect identification methods successfully categorize various types of FPs in this material, with more than ten types of Ga FPs being produced during our simulations. O atoms are found to form two main types of FPs and the O split interstitial site on O1 site is most common. Finally, the recombination behavior and barriers of Ga and O FPs indicate that the O FP has a higher possibility of recovery upon annealing. Our findings provide important insights into the studies of radiation damage and defects in $\rm Ga_2O_3$ and can contribute to the design and development of $\rm Ga_2O_3$-based devices

cond-mat.mtrl-sci↗

arXiv2024

Crystallization Instead of Amorphization in Collision Cascades in Gallium Oxide

Disordering of solids typically leads to amorphization, but polymorph transitions, facilitated by favorable atomic rearrangements, may temporarily help to maintain long-range periodicity in the solid state. In far-from-equilibrium situations, such as atomic collision cascades, these rearrangements may not necessarily follow a thermodynamically gainful path, but may be kinetically limited. In this Letter, we focused on such crystallization instead of amorphization in collision cascades in gallium oxide (\ce{Ga2O3}). We determined the disorder threshold for irreversible $\beta$-to-$\gamma$ polymorph transition and explained why it results in elevating energy to that of the $\gamma$-polymorph, which exhibits the highest polymorph energy in the system below the amorphous state. Specifically, we demonstrate that upon reaching the disorder transition threshold, the \ce{Ga}-sublattice kinetically favors transitioning to the $\gamma$-like configuration, requiring significantly less migration for \ce{Ga} atoms to reach the lattice sites during post-cascade processes. As such, our data provide a consistent explanation of this remarkable phenomenon and can serve as a toolbox for predictive multi-polymorph fabrication.

cond-mat.mtrl-sci↗

arXiv2023

Universal radiation tolerant semiconductor

Radiation tolerance is determined as the ability of crystalline materials to withstand the accumulation of the radiation induced disorder. Nevertheless, for sufficiently high fluences, in all by far known semiconductors it ends up with either very high disorder levels or amorphization. Here we show that gamma/beta double polymorph Ga2O3 structures exhibit remarkably high radiation tolerance. Specifically, for room temperature experiments, they tolerate a disorder equivalent to hundreds of displacements per atom, without severe degradations of crystallinity; in comparison with, e.g., Si amorphizable already with the lattice atoms displaced just once. We explain this behavior by an interesting combination of the Ga- and O- sublattice properties in gamma-Ga2O3. In particular, O-sublattice exhibits a strong recrystallization trend to recover the face-centered-cubic stacking despite the stronger displacement of O atoms compared to Ga during the active periods of cascades. Notably, we also explained the origin of the beta-to-gamma Ga2O3 transformation, as a function of the increased disorder in beta-Ga2O3 and studied the phenomena as a function of the chemical nature of the implanted atoms. As a result, we conclude that gamma/beta double polymorph Ga2O3 structures, in terms of their radiation tolerance properties, benchmark a class of universal radiation tolerant semiconductors.

cond-mat.mtrl-sci↗

arXiv2022

GUIM -- General User and Item Embedding with Mixture of Representation in E-commerce

Our goal is to build general representation (embedding) for each user and each product item across Alibaba's businesses, including Taobao and Tmall which are among the world's biggest e-commerce websites. The representation of users and items has been playing a critical role in various downstream applications, including recommendation system, search, marketing, demand forecasting and so on. Inspired from the BERT model in natural language processing (NLP) domain, we propose a GUIM (General User Item embedding with Mixture of representation) model to achieve the goal with massive, structured, multi-modal data including the interactions among hundreds of millions of users and items. We utilize mixture of representation (MoR) as a novel representation form to model the diverse interests of each user. In addition, we use the InfoNCE from contrastive learning to avoid intractable computational costs due to the numerous size of item (token) vocabulary. Finally, we propose a set of representative downstream tasks to serve as a standard benchmark to evaluate the quality of the learned user and/or item embeddings, analogous to the GLUE benchmark in NLP domain. Our experimental results in these downstream tasks clearly show the comparative value of embeddings learned from our GUIM model.

cs.AI↗

arXiv2021

SAS: Self-Augmentation Strategy for Language Model Pre-training

The core of self-supervised learning for pre-training language models includes pre-training task design as well as appropriate data augmentation. Most data augmentations in language model pre-training are context-independent. A seminal contextualized augmentation was recently proposed in ELECTRA and achieved state-of-the-art performance by introducing an auxiliary generation network (generator) to produce contextualized data augmentation for the training of a main discrimination network (discriminator). This design, however, introduces extra computation cost of the generator and a need to adjust the relative capability between the generator and the discriminator. In this paper, we propose a self-augmentation strategy (SAS) where a single network is utilized for both regular pre-training and contextualized data augmentation for the training in later epochs. Essentially, this strategy eliminates a separate generator and uses the single network to jointly conduct two pre-training tasks with MLM (Masked Language Modeling) and RTD (Replaced Token Detection) heads. It avoids the challenge to search for an appropriate size of the generator, which is critical to the performance as evidenced in ELECTRA and its subsequent variant models. In addition, SAS is a general strategy that can be seamlessly combined with many new techniques emerging recently or in the future, such as the disentangled attention mechanism from DeBERTa. Our experiments show that SAS is able to outperform ELECTRA and other state-of-the-art models in the GLUE tasks with similar or less computation cost.

cs.CL↗

arXiv2020

Progressively Stacking 2.0: A Multi-stage Layerwise Training Method for BERT Training Speedup

Pre-trained language models, such as BERT, have achieved significant accuracy gain in many natural language processing tasks. Despite its effectiveness, the huge number of parameters makes training a BERT model computationally very challenging. In this paper, we propose an efficient multi-stage layerwise training (MSLT) approach to reduce the training time of BERT. We decompose the whole training process into several stages. The training is started from a small model with only a few encoder layers and we gradually increase the depth of the model by adding new encoder layers. At each stage, we only train the top (near the output layer) few encoder layers which are newly added. The parameters of the other layers which have been trained in the previous stages will not be updated in the current stage. In BERT training, the backward computation is much more time-consuming than the forward computation, especially in the distributed training setting in which the backward computation time further includes the communication time for gradient synchronization. In the proposed training strategy, only top few layers participate in backward computation, while most layers only participate in forward computation. Hence both the computation and communication efficiencies are greatly improved. Experimental results show that the proposed method can achieve more than 110% training speedup without significant performance degradation.

cs.CL↗

arXiv2015

Structure Learning in Bayesian Networks of Moderate Size by Efficient Sampling

We study the Bayesian model averaging approach to learning Bayesian network structures (DAGs) from data. We develop new algorithms including the first algorithm that is able to efficiently sample DAGs according to the exact structure posterior. The DAG samples can then be used to construct estimators for the posterior of any feature. We theoretically prove good properties of our estimators and empirically show that our estimators considerably outperform the estimators from the previous state-of-the-art methods.

cs.AI↗

arXiv2012

Computing Posterior Probabilities of Structural Features in Bayesian Networks

We study the problem of learning Bayesian network structures from data. Koivisto and Sood (2004) and Koivisto (2006) presented algorithms that can compute the exact marginal posterior probability of a subnetwork, e.g., a single edge, in O(n2n) time and the posterior probabilities for all n(n-1) potential edges in O(n2n) total time, assuming that the number of parents per node or the indegree is bounded by a constant. One main drawback of their algorithms is the requirement of a special structure prior that is non uniform and does not respect Markov equivalence. In this paper, we develop an algorithm that can compute the exact posterior probability of a subnetwork in O(3n) time and the posterior probabilities for all n(n-1) potential edges in O(n3n) total time. Our algorithm also assumes a bounded indegree but allows general structure priors. We demonstrate the applicability of the algorithm on several data sets with up to 20 variables.

cs.LG↗

arXiv2012

Bayesian Model Averaging Using the k-best Bayesian Network Structures

We study the problem of learning Bayesian network structures from data. We develop an algorithm for finding the k-best Bayesian network structures. We propose to compute the posterior probabilities of hypotheses of interest by Bayesian model averaging over the k-best Bayesian networks. We present empirical results on structural discovery over several real and synthetic data sets and show that the method outperforms the model selection method and the state of-the-art MCMC methods.

cs.LG↗