arXiv subjects

Chenyi Yang

Publications and source records attributed to Chenyi Yang.

3 recordsLinked to original sources

arXiv2025

Beyond Productivity Gaps: Temporal Patterns of Gender Differences in Scientific Knowledge Creation

Gender inequality in scientific careers has been extensively documented through aggregate measures such as total publications and cumulative citations, yet the temporal dynamics underlying these disparities remain largely unexplored. Here we developed a multi-dimensional framework to examine gender differences in scientific knowledge creation through three complementary temporal dimensions: stability (consistency of performance over time), volatility (degree of year-to-year fluctuation), and persistence (ability to maintain high performance for extended periods). Using comprehensive bibliometric data from SciSciNet covering 62.5 million authors whose careers began between 1960-2010, we constructed knowledge creation capability measures that captured how scientists absorb knowledge from diverse sources and contribute to field advancement. We found that female scientists demonstrated significantly higher knowledge production stability (0.170 vs. 0.119 for males) while simultaneously exhibiting greater year-to-year volatility (6.606 vs. 6.228), revealing a striking paradox in career dynamics. Female scientists showed persistence advantages under moderate performance requirements but faced disadvantages under extreme criteria demanding sustained peak performance. However, these patterns varied substantially across disciplines, with female advantages strongest in humanities and social sciences while STEM fields show mixed results.

cs.DL↗

arXiv2021

You Only Compress Once: Towards Effective and Elastic BERT Compression via Exploit-Explore Stochastic Nature Gradient

Despite superior performance on various natural language processing tasks, pre-trained models such as BERT are challenged by deploying on resource-constraint devices. Most existing model compression approaches require re-compression or fine-tuning across diverse constraints to accommodate various hardware deployments. This practically limits the further application of model compression. Moreover, the ineffective training and searching process of existing elastic compression paradigms[4,27] prevents the direct migration to BERT compression. Motivated by the necessity of efficient inference across various constraints on BERT, we propose a novel approach, YOCO-BERT, to achieve compress once and deploy everywhere. Specifically, we first construct a huge search space with 10^13 architectures, which covers nearly all configurations in BERT model. Then, we propose a novel stochastic nature gradient optimization method to guide the generation of optimal candidate architecture which could keep a balanced trade-off between explorations and exploitation. When a certain resource constraint is given, a lightweight distribution optimization approach is utilized to obtain the optimal network for target deployment without fine-tuning. Compared with state-of-the-art algorithms, YOCO-BERT provides more compact models, yet achieving 2.1%-4.5% average accuracy improvement on the GLUE benchmark. Besides, YOCO-BERT is also more effective, e.g.,the training complexity is O(1)for N different devices. Code is availablehttps://github.com/MAC-AutoML/YOCO-BERT.

cs.CL↗

arXiv2019

DDPNAS: Efficient Neural Architecture Search via Dynamic Distribution Pruning

Neural Architecture Search (NAS) has demonstrated state-of-the-art performance on various computer vision tasks. Despite the superior performance achieved, the efficiency and generality of existing methods are highly valued due to their high computational complexity and low generality. In this paper, we propose an efficient and unified NAS framework termed DDPNAS via dynamic distribution pruning, facilitating a theoretical bound on accuracy and efficiency. In particular, we first sample architectures from a joint categorical distribution. Then the search space is dynamically pruned and its distribution is updated every few epochs. With the proposed efficient network generation method, we directly obtain the optimal neural architectures on given constraints, which is practical for on-device models across diverse search spaces and constraints. The architectures searched by our method achieve remarkable top-1 accuracies, 97.56 and 77.2 on CIFAR-10 and ImageNet (mobile settings), respectively, with the fastest search process, i.e., only 1.8 GPU hours on a Tesla V100. Codes for searching and network generation are available at: https://openi.pcl.ac.cn/PCL AutoML/XNAS.

cs.CV↗