arXiv subjects

Yang He

Publications and source records attributed to Yang He.

At least 91 records · Page 5Linked to original sources

arXiv2024

SimiSketch: Efficiently Estimating Similarity of streaming Multisets

The challenge of estimating similarity between sets has been a significant concern in data science, finding diverse applications across various domains. However, previous approaches, such as MinHash, have predominantly centered around hashing techniques, which are well-suited for sets but less naturally adaptable to multisets, a common occurrence in scenarios like network streams and text data. Moreover, with the increasing prevalence of data arriving in streaming patterns, many existing methods struggle to handle cases where set items are presented in a continuous stream. Consequently, our focus in this paper is on the challenging scenario of multisets with item streams. To address this, we propose SimiSketch, a sketching algorithm designed to tackle this specific problem. The paper begins by presenting two simpler versions that employ intuitive sketches for similarity estimation. Subsequently, we formally introduce SimiSketch and leverage SALSA to enhance accuracy. To validate our algorithms, we conduct extensive testing on synthetic datasets, real-world network traffic, and text articles. Our experiment shows that compared with the state-of-the-art, SimiSketch can improve the accuracy by up to 42 times, and increase the throughput by up to 360 times. The complete source code is open-sourced and available on GitHub for reference.

cs.DS↗

arXiv2024

Deciphering yield modification of hadron-triggered semi-inclusive recoil jets in heavy-ion collisions

In relativistic heavy-ion collisions, a hot and dense state of matter, called the Quark-Gluon Plasma (QGP), is produced. Semi-inclusive jets recoiling from trigger hadrons of high transverse momenta ($p_{\mathrm{T}}$) can serve as an effective probe of the QGP properties, as they are expected to experience jet quenching when traversing the QGP. Recent experimental results on the ratio of recoil jet yields normalized by the trigger counts in heavy-ion collisions to that in $p$+$p$ collisions ($I_{\mathrm{AA}}$) pose an unexpected challenge in its interpretation. It is observed that $I_{\mathrm{AA}}$ rises with the jet $p_{\mathrm{T}}$ and possibly exceeds unity at high $p_{\mathrm{T}}$, while traditionally it is expected that jet quenching would lead to $I_{\mathrm{AA}} < 1$. To address this challenge, we utilize the Linear Boltzmann Transport (LBT) model to simulate jet transport in the QGP, and study the effect of jet quenching for high-$p_{\mathrm{T}}$ triggers and recoil jets separately on $I_{\mathrm{AA}}$. We find that the quenching of the colored triggers alone is responsible for the rising trend and larger-than-unity value observed experimentally.

nucl-th↗

arXiv2024

Data-independent Module-aware Pruning for Hierarchical Vision Transformers

Hierarchical vision transformers (ViTs) have two advantages over conventional ViTs. First, hierarchical ViTs achieve linear computational complexity with respect to image size by local self-attention. Second, hierarchical ViTs create hierarchical feature maps by merging image patches in deeper layers for dense prediction. However, existing pruning methods ignore the unique properties of hierarchical ViTs and use the magnitude value as the weight importance. This approach leads to two main drawbacks. First, the "local" attention weights are compared at a "global" level, which may cause some "locally" important weights to be pruned due to their relatively small magnitude "globally". The second issue with magnitude pruning is that it fails to consider the distinct weight distributions of the network, which are essential for extracting coarse to fine-grained features at various hierarchical levels. To solve the aforementioned issues, we have developed a Data-independent Module-Aware Pruning method (DIMAP) to compress hierarchical ViTs. To ensure that "local" attention weights at different hierarchical levels are compared fairly in terms of their contribution, we treat them as a module and examine their contribution by analyzing their information distortion. Furthermore, we introduce a novel weight metric that is solely based on weights and does not require input images, thereby eliminating the dependence on the patch merging process. Our method validates its usefulness and strengths on Swin Transformers of different sizes on ImageNet-1k classification. Notably, the top-5 accuracy drop is only 0.07% when we remove 52.5% FLOPs and 52.7% parameters of Swin-B. When we reduce 33.2% FLOPs and 33.2% parameters of Swin-S, we can even achieve a 0.8% higher relative top-5 accuracy than the original model. Code is available at: https://github.com/he-y/Data-independent-Module-Aware-Pruning

cs.CV↗

arXiv2024

Multisize Dataset Condensation

While dataset condensation effectively enhances training efficiency, its application in on-device scenarios brings unique challenges. 1) Due to the fluctuating computational resources of these devices, there's a demand for a flexible dataset size that diverges from a predefined size. 2) The limited computational power on devices often prevents additional condensation operations. These two challenges connect to the "subset degradation problem" in traditional dataset condensation: a subset from a larger condensed dataset is often unrepresentative compared to directly condensing the whole dataset to that smaller size. In this paper, we propose Multisize Dataset Condensation (MDC) by compressing N condensation processes into a single condensation process to obtain datasets with multiple sizes. Specifically, we introduce an "adaptive subset loss" on top of the basic condensation loss to mitigate the "subset degradation problem". Our MDC method offers several benefits: 1) No additional condensation process is required; 2) reduced storage requirement by reusing condensed images. Experiments validate our findings on networks including ConvNet, ResNet and DenseNet, and datasets including SVHN, CIFAR-10, CIFAR-100 and ImageNet. For example, we achieved 5.22%-6.40% average accuracy gains on condensing CIFAR-10 to ten images per class. Code is available at: https://github.com/he-y/Multisize-Dataset-Condensation.

cs.CV↗

arXiv2024

Nanoscale variation of the Rashba energy in BiTeI

BiTeI is a polar semiconductor with strong spin-orbit coupling (SOC) that produces large Rashba spin splitting. Due to its potential utility in spintronics and magnetoelectrics, it is essential to understand how defects impact the spin transport in this material. Using scanning tunneling microscopy and spectroscopy, we image ring-like charging states of single-atom defects on the iodine surface of BiTeI. We observe nanoscale variations in the Rashba energy around each defect, which we correlate with the local electric field extracted from the bias dependence of each ring radius. Our data demonstrate the local impact of atomic defects on the Rashba effect, which is both a challenge and an opportunity for the development of future nanoscale spintronic devices.

cond-mat.mes-hall↗

arXiv2024

VeriEQL: Bounded Equivalence Verification for Complex SQL Queries with Integrity Constraints

The task of SQL query equivalence checking is important in various real-world applications (including query rewriting and automated grading) that involve complex queries with integrity constraints; yet, state-of-the-art techniques are very limited in their capability of reasoning about complex features (e.g., those that involve sorting, case statement, rich integrity constraints, etc.) in real-life queries. To the best of our knowledge, we propose the first SMT-based approach and its implementation, VeriEQL, capable of proving and disproving bounded equivalence of complex SQL queries. VeriEQL is based on a new logical encoding that models query semantics over symbolic tuples using the theory of integers with uninterpreted functions. It is simple yet highly practical -- our comprehensive evaluation on over 20,000 benchmarks shows that VeriEQL outperforms all state-of-the-art techniques by more than one order of magnitude in terms of the number of benchmarks that can be proved or disproved. VeriEQL can also generate counterexamples that facilitate many downstream tasks (such as finding serious bugs in systems like MySQL and Apache Calcite).

cs.PL↗

arXiv2024

Chip-scale high-performance photonic microwave oscillator

Optical frequency division based on bulk or fiber optics provides unprecedented spectral purity for microwave oscillators. To extend the applications of this approach, the big challenges are to develop miniaturized optical frequency division oscillators without trading off phase noise performance. In this paper, we report a chip-scale electro-optical frequency division microwave oscillator with ultra-low phase noise performance. Dual laser sources are co-self-injection-locked to a single silicon nitride spiral resonator to provide a record high-stability, fully on-chip optical reference. An integrated electro-optical frequency comb based on a novel thin-film lithium niobate phase modulator chip is incorporated for the first time to perform optical-to-microwave frequency division. The resulting chip-scale photonic microwave oscillator achieves a phase noise level of -129 dBc/Hz at 10 kHz offset for 37.7 GHz carrier. The results represent a major advance in high performance, integrated photonic microwave oscillators for applications including signal processing, radar, timing, and coherent communications.

physics.optics↗

arXiv2024

High-order topological pumping on a superconducting quantum processor

High-order topological phases of matter refer to the systems of $n$-dimensional bulk with the topology of $m$-th order, exhibiting $(n-m)$-dimensional boundary modes and can be characterized by topological pumping. Here, we experimentally demonstrate two types of second-order topological pumps, forming four 0-dimensional corner localized states on a 4$\times$4 square lattice array of 16 superconducting qubits. The initial ground state of the system for half-filling, as a product of four identical entangled 4-qubit states, is prepared using an adiabatic scheme. During the pumping procedure, we adiabatically modulate the superlattice Bose-Hubbard Hamiltonian by precisely controlling both the hopping strengths and on-site potentials. At the half pumping period, the system evolves to a corner-localized state in a quadrupole configuration. The robustness of the second-order topological pump is also investigated by introducing different on-site disorder. Our work studies the topological properties of high-order topological phases from the dynamical transport picture using superconducting qubits, which would inspire further research on high-order topological phases.

quant-ph↗

arXiv2024

Nanoscale Conducting and Insulating Domains on YbB$_6$

YbB$_6$ is a predicted topological insulator, with experimental evidence for conducting surface states of yet-unproven origin. However, its lack of a natural cleavage plane, and resultant surface-dependent polarity, has obscured its study. We use scanning tunneling microscopy to image the cleaved surface of YbB$_6$, exhibiting several coexisting terminations with distinct atomic structures. Our spectroscopic measurements show band-bending between the terminations, resulting in both conducting and fully-gapped regions. In the conductive regions, we observe spectral peaks that are suggestive of van Hove singularities arising from Rashba spin-split quantum well states. The insulating regions rule out the possibility that YbB$_6$ is a strong topological insulator, while the spin-polarized conducting regions suggest possible utility for spintronic devices.

cond-mat.mes-hall↗

arXiv2023

Deep Learning for Code Intelligence: Survey, Benchmark and Toolkit

Code intelligence leverages machine learning techniques to extract knowledge from extensive code corpora, with the aim of developing intelligent tools to improve the quality and productivity of computer programming. Currently, there is already a thriving research community focusing on code intelligence, with efforts ranging from software engineering, machine learning, data mining, natural language processing, and programming languages. In this paper, we conduct a comprehensive literature review on deep learning for code intelligence, from the aspects of code representation learning, deep learning techniques, and application tasks. We also benchmark several state-of-the-art neural models for code intelligence, and provide an open-source toolkit tailored for the rapid prototyping of deep-learning-based code intelligence models. In particular, we inspect the existing code intelligence models under the basis of code representation learning, and provide a comprehensive overview to enhance comprehension of the present state of code intelligence. Furthermore, we publicly release the source code and data resources to provide the community with a ready-to-use benchmark, which can facilitate the evaluation and comparison of existing and future code intelligence models (https://xcodemind.github.io). At last, we also point out several challenging and promising directions for future research.

cs.SE↗

arXiv2023

Corrosion-resistant aluminum alloy design through machine learning combined with high-throughput calculations

Efficiently designing lightweight alloys with combined high corrosion resistance and mechanical properties remains an enduring topic in materials engineering. To this end, machine learning (ML) coupled ab-initio calculations is proposed within this study. Due to the inadequate accuracy of conventional stress-strain ML models caused by corrosion factors, a novel reinforcement self-learning ML algorithm (accuracy R2 >0.92) is developed. Then, a strategy that integrates ML models, calculated energetics and mechanical moduli is implemented to optimize the Al alloys. Next, this Computation Designed Corrosion-Resistant Al alloy is fabricated that verified the simulation. The performance (elongation reaches ~30%) is attributed to the H-captured Al-Sc-Cu phases (-1.44 eV H-1) and Cu-modified η/η' precipitation inside the grain boundaries (GBs). The developed Al-Mg-Zn-Cu interatomic potential (energy accuracy 6.50 meV atom-1) proves the cracking resistance of the GB region enhanced by Cu-modification. Conceptually, our strategy is of practical importance for designing new alloys exhibiting corrosion resistance and mechanical properties.

cond-mat.mtrl-sci↗

arXiv2023

Structured Pruning for Deep Convolutional Neural Networks: A survey

The remarkable performance of deep Convolutional neural networks (CNNs) is generally attributed to their deeper and wider architectures, which can come with significant computational costs. Pruning neural networks has thus gained interest since it effectively lowers storage and computational costs. In contrast to weight pruning, which results in unstructured models, structured pruning provides the benefit of realistic acceleration by producing models that are friendly to hardware implementation. The special requirements of structured pruning have led to the discovery of numerous new challenges and the development of innovative solutions. This article surveys the recent progress towards structured pruning of deep CNNs. We summarize and compare the state-of-the-art structured pruning techniques with respect to filter ranking methods, regularization methods, dynamic execution, neural architecture search, the lottery ticket hypothesis, and the applications of pruning. While discussing structured pruning algorithms, we briefly introduce the unstructured pruning counterpart to emphasize their differences. Furthermore, we provide insights into potential research opportunities in the field of structured pruning. A curated list of neural network pruning papers can be found at https://github.com/he-y/Awesome-Pruning . A dedicated website offering a more interactive comparison of structured pruning methods can be found at: https://huggingface.co/spaces/he-yang/Structured-Pruning-Survey .

cs.CV↗

arXiv2023

You Only Condense Once: Two Rules for Pruning Condensed Datasets

Dataset condensation is a crucial tool for enhancing training efficiency by reducing the size of the training dataset, particularly in on-device scenarios. However, these scenarios have two significant challenges: 1) the varying computational resources available on the devices require a dataset size different from the pre-defined condensed dataset, and 2) the limited computational resources often preclude the possibility of conducting additional condensation processes. We introduce You Only Condense Once (YOCO) to overcome these limitations. On top of one condensed dataset, YOCO produces smaller condensed datasets with two embarrassingly simple dataset pruning rules: Low LBPE Score and Balanced Construction. YOCO offers two key advantages: 1) it can flexibly resize the dataset to fit varying computational constraints, and 2) it eliminates the need for extra condensation processes, which can be computationally prohibitive. Experiments validate our findings on networks including ConvNet, ResNet and DenseNet, and datasets including CIFAR-10, CIFAR-100 and ImageNet. For example, our YOCO surpassed various dataset condensation and dataset pruning methods on CIFAR-10 with ten Images Per Class (IPC), achieving 6.98-8.89% and 6.31-23.92% accuracy gains, respectively. The code is available at: https://github.com/he-y/you-only-condense-once.

cs.CV↗

arXiv2023

Temporal Convolutional Explorer Helps Understand 1D-CNN's Learning Behavior in Time Series Classification from Frequency Domain

While one-dimensional convolutional neural networks (1D-CNNs) have been empirically proven effective in time series classification tasks, we find that there remain undesirable outcomes that could arise in their application, motivating us to further investigate and understand their underlying mechanisms. In this work, we propose a Temporal Convolutional Explorer (TCE) to empirically explore the learning behavior of 1D-CNNs from the perspective of the frequency domain. Our TCE analysis highlights that deeper 1D-CNNs tend to distract the focus from the low-frequency components leading to the accuracy degradation phenomenon, and the disturbing convolution is the driving factor. Then, we leverage our findings to the practical application and propose a regulatory framework, which can easily be integrated into existing 1D-CNNs. It aims to rectify the suboptimal learning behavior by enabling the network to selectively bypass the specified disturbing convolutions. Finally, through comprehensive experiments on widely-used UCR, UEA, and UCI benchmarks, we demonstrate that 1) TCE's insight into 1D-CNN's learning behavior; 2) our regulatory framework enables state-of-the-art 1D-CNNs to get improved performances with less consumption of memory and computational overhead.

cs.LG↗

arXiv2023

Towards Unsupervised Graph Completion Learning on Graphs with Features and Structure Missing

In recent years, graph neural networks (GNN) have achieved significant developments in a variety of graph analytical tasks. Nevertheless, GNN's superior performance will suffer from serious damage when the collected node features or structure relationships are partially missing owning to numerous unpredictable factors. Recently emerged graph completion learning (GCL) has received increasing attention, which aims to reconstruct the missing node features or structure relationships under the guidance of a specifically supervised task. Although these proposed GCL methods have made great success, they still exist the following problems: the reliance on labels, the bias of the reconstructed node features and structure relationships. Besides, the generalization ability of the existing GCL still faces a huge challenge when both collected node features and structure relationships are partially missing at the same time. To solve the above issues, we propose a more general GCL framework with the aid of self-supervised learning for improving the task performance of the existing GNN variants on graphs with features and structure missing, termed unsupervised GCL (UGCL). Specifically, to avoid the mismatch between missing node features and structure during the message-passing process of GNN, we separate the feature reconstruction and structure reconstruction and design its personalized model in turn. Then, a dual contrastive loss on the structure level and feature level is introduced to maximize the mutual information of node representations from feature reconstructing and structure reconstructing paths for providing more supervision signals. Finally, the reconstructed node features and structure can be applied to the downstream node classification task. Extensive experiments on eight datasets, three GNN variants and five missing rates demonstrate the effectiveness of our proposed method.

cs.LG↗

arXiv2023

Emergence of pseudo-time during optimal Monte Carlo sampling and temporal aspects of symmetry breaking and restoration

We argue that one can associate a pseudo-time with sequences of configurations generated in the course of classical Monte Carlo simulations for a single-minimum bound state, if the sampling is optimal. Hereby the sampling rates can be, under special circumstances, calibrated against the relaxation rate and frequency of motion of an actual physical system. The latter possibility is linked to the optimal sampling regime being a universal crossover separating two distinct suboptimal sampling regimes analogous to the physical phenomena of diffusion and effusion, respectively. Bound states break symmetry; one may thus regard the pseudo-time as a quantity emerging together with the bound state. Conversely, when transport among distinct bound states takes place -- thus restoring symmetry -- a pseudo-time can no longer be defined. One can still quantify activation barriers, if the latter barriers are smooth, but the simulation becomes impractically slow and pertains to overdamped transport only. Specially designed Monte Carlo moves that bypass activation barriers -- so as to accelerate sampling of the thermodynamics -- amount to effusive transport and lead to severe under-sampling of transition-state configurations that separate distinct bound states while destroying the said universality. Implications of the present findings for simulations of glassy liquids are discussed.

cond-mat.stat-mech↗

arXiv2023

Effects of the formation time of parton shower on jet quenching in heavy-ion collisions

Jet quenching has successfully served as a hard probe to study the properties of Quark-Gluon Plasma (QGP). As a multi-particle system, jets take time to develop from a highly virtual parton to a group of partons close to mass shells. In this study, we present a systematical study on the effects of this formation time on jet quenching in relativistic nuclear collisions. Jets from initial hard scatterings were simulated with Pythia, and their interactions with QGP were described using a Linear Boltzmann Transport (LBT) model that incorporates both elastic and inelastic scatterings between jet partons and the thermal medium. Three different estimations of the jet formation time were implemented and compared, including instantaneous formation, formation from single splitting, and formation from sequential splittings, before which no jet-medium interaction was assumed. We found that deferring the jet-medium interaction with a longer formation time not only affects the overall magnitude of the nuclear modification factor of jets, but also its dependence on the jet transverse momentum.

nucl-th↗

arXiv2022

Urban Scene Semantic Segmentation with Low-Cost Coarse Annotation

For best performance, today's semantic segmentation methods use large and carefully labeled datasets, requiring expensive annotation budgets. In this work, we show that coarse annotation is a low-cost but highly effective alternative for training semantic segmentation models. Considering the urban scene segmentation scenario, we leverage cheap coarse annotations for real-world captured data, as well as synthetic data to train our model and show competitive performance compared with finely annotated real-world data. Specifically, we propose a coarse-to-fine self-training framework that generates pseudo labels for unlabeled regions of the coarsely annotated data, using synthetic data to improve predictions around the boundaries between semantic classes, and using cross-domain data augmentation to increase diversity. Our extensive experimental results on Cityscapes and BDD100k datasets demonstrate that our method achieves a significantly better performance vs annotation cost tradeoff, yielding a comparable performance to fully annotated data with only a small fraction of the annotation budget. Also, when used as pretraining, our framework performs better compared to the standard fully supervised setting.

cs.CV↗