arXiv subjects

Zhenyu Zhang

Publications and source records attributed to Zhenyu Zhang.

At least 253 records · Page 14Linked to original sources

arXiv2022

Capacity Bounds for the Two-User IM/DD Interference Channel

This paper studies the capacity of the two-user intensity-modulation/direct-detection (IM/DD) interference channel (IC), which is relevant in the context of multi-user optical wireless communications. Despite some known single-letter capacity characterizations for general discrete-memoryless ICs, a computable capacity expression for the IM/DD IC is missing. In this paper, we provide tight and easily computable inner and outer bounds for a general two-user IM/DD IC under peak and average optical intensity constraints. The bounds enable characterizing the asymptotic sum-rate capacity in the strong and weak interference regimes, as well as the generalized degrees of freedom (GDoF) in the symmetric case. Using the obtained bounds, the GDoF of the IM/DD IC is shown to have a `W' shape similar to the Gaussian IC with power constraints. The obtained bounds are also evaluated numerically in different interference regimes to show their tightness, and used to study the performance of on-chip and indoor OWC systems.

cs.IT↗

arXiv2022

Linearity Grafting: Relaxed Neuron Pruning Helps Certifiable Robustness

Certifiable robustness is a highly desirable property for adopting deep neural networks (DNNs) in safety-critical scenarios, but often demands tedious computations to establish. The main hurdle lies in the massive amount of non-linearity in large DNNs. To trade off the DNN expressiveness (which calls for more non-linearity) and robustness certification scalability (which prefers more linearity), we propose a novel solution to strategically manipulate neurons, by "grafting" appropriate levels of linearity. The core of our proposal is to first linearize insignificant ReLU neurons, to eliminate the non-linear components that are both redundant for DNN performance and harmful to its certification. We then optimize the associated slopes and intercepts of the replaced linear activations for restoring model performance while maintaining certifiability. Hence, typical neuron pruning could be viewed as a special case of grafting a linear function of the fixed zero slopes and intercept, that might overly restrict the network flexibility and sacrifice its performance. Extensive experiments on multiple datasets and network backbones show that our linearity grafting can (1) effectively tighten certified bounds; (2) achieve competitive certifiable robustness without certified robust training (i.e., over 30% improvements on CIFAR-10 models); and (3) scale up complete verification to large adversarially trained models with 17M parameters. Codes are available at https://github.com/VITA-Group/Linearity-Grafting.

cs.LG↗

arXiv2022

Data-Efficient Double-Win Lottery Tickets from Robust Pre-training

Pre-training serves as a broadly adopted starting point for transfer learning on various downstream tasks. Recent investigations of lottery tickets hypothesis (LTH) demonstrate such enormous pre-trained models can be replaced by extremely sparse subnetworks (a.k.a. matching subnetworks) without sacrificing transferability. However, practical security-crucial applications usually pose more challenging requirements beyond standard transfer, which also demand these subnetworks to overcome adversarial vulnerability. In this paper, we formulate a more rigorous concept, Double-Win Lottery Tickets, in which a located subnetwork from a pre-trained model can be independently transferred on diverse downstream tasks, to reach BOTH the same standard and robust generalization, under BOTH standard and adversarial training regimes, as the full pre-trained model can do. We comprehensively examine various pre-training mechanisms and find that robust pre-training tends to craft sparser double-win lottery tickets with superior performance over the standard counterparts. For example, on downstream CIFAR-10/100 datasets, we identify double-win matching subnetworks with the standard, fast adversarial, and adversarial pre-training from ImageNet, at 89.26%/73.79%, 89.26%/79.03%, and 91.41%/83.22% sparsity, respectively. Furthermore, we observe the obtained double-win lottery tickets can be more data-efficient to transfer, under practical data-limited (e.g., 1% and 10%) downstream schemes. Our results show that the benefits from robust pre-training are amplified by the lottery ticket scheme, as well as the data-limited transfer setting. Codes are available at https://github.com/VITA-Group/Double-Win-LTH.

cs.LG↗

arXiv2022

Quarantine: Sparsity Can Uncover the Trojan Attack Trigger for Free

Trojan attacks threaten deep neural networks (DNNs) by poisoning them to behave normally on most samples, yet to produce manipulated results for inputs attached with a particular trigger. Several works attempt to detect whether a given DNN has been injected with a specific trigger during the training. In a parallel line of research, the lottery ticket hypothesis reveals the existence of sparse subnetworks which are capable of reaching competitive performance as the dense network after independent training. Connecting these two dots, we investigate the problem of Trojan DNN detection from the brand new lens of sparsity, even when no clean training data is available. Our crucial observation is that the Trojan features are significantly more stable to network pruning than benign features. Leveraging that, we propose a novel Trojan network detection regime: first locating a "winning Trojan lottery ticket" which preserves nearly full Trojan information yet only chance-level performance on clean inputs; then recovering the trigger embedded in this already isolated subnetwork. Extensive experiments on various datasets, i.e., CIFAR-10, CIFAR-100, and ImageNet, with different network architectures, i.e., VGG-16, ResNet-18, ResNet-20s, and DenseNet-100 demonstrate the effectiveness of our proposal. Codes are available at https://github.com/VITA-Group/Backdoor-LTH.

cs.LG↗

arXiv2022

Label Anchored Contrastive Learning for Language Understanding

Contrastive learning (CL) has achieved astonishing progress in computer vision, speech, and natural language processing fields recently with self-supervised learning. However, CL approach to the supervised setting is not fully explored, especially for the natural language understanding classification task. Intuitively, the class label itself has the intrinsic ability to perform hard positive/negative mining, which is crucial for CL. Motivated by this, we propose a novel label anchored contrastive learning approach (denoted as LaCon) for language understanding. Specifically, three contrastive objectives are devised, including a multi-head instance-centered contrastive loss (ICL), a label-centered contrastive loss (LCL), and a label embedding regularizer (LER). Our approach does not require any specialized network architecture or any extra data augmentation, thus it can be easily plugged into existing powerful pre-trained language models. Compared to the state-of-the-art baselines, LaCon obtains up to 4.1% improvement on the popular datasets of GLUE and CLUE benchmarks. Besides, LaCon also demonstrates significant advantages under the few-shot and data imbalance settings, which obtains up to 9.4% improvement on the FewGLUE and FewCLUE benchmarking tasks.

cs.CL↗

arXiv2022

Study of exotic hadrons with machine learning

We analyzed the invariant mass spectrum of near-threshold exotic states for one-channel candidates with a deep neural network. It can extract the scattering length and effective range, which would shed light on the nature of given states, from the experimental mass spectrum. As an application, the mass spectrum of the $X(3872)$ and the $T_{cc}^+$ are studied. The obtained scattering lengths, effective ranges, and most relevant thresholds are consistent with those from fitting to the experimental data. The advantage of the neural network is that it is more stable than the fitting, especially for low-statistic data. The network, which provides another way to analyze the experimental data, can also be applied to other one-channel near-threshold exotic candidates.

hep-ph↗

arXiv2022

Large Rashba Spin-Orbit Coupling and High-Temperature Quantum Anomalous Hall Effect in Re-Intercalated Graphene/CrI$_3$ Heterostructure

In 2010, quantum anomalous Hall effect (QAHE) in graphene was proposed in the presence of Rashba spin-orbit coupling and ferromagnetic exchange field. After a decade's experimental exploration, the anomalous Hall conductance can only reach about 0.25 in the units of $2e^2/h$, which was attributed to the tiny Rashba spin-orbit coupling. Here, we theoretically show that Re-intercalation in graphene/CrI$_3$ heterostructure can not only induce sizeable Rashba spin-orbit coupling ($>$ 40~meV), but also open up large band gaps at valleys $K$ (22.2 meV) and $K' $ (30.3 meV), and a global band gap over 5.5 meV (19.5 meV with random Re distribution) hosting QAHE. A low-energy continuum model is constructed to explain the underlying physical mechanism. We find that Rashba spin-orbit coupling is robust against external stress whereas a tensile strain can increase the global bulk gap. Furthermore, we also show that Re-intercalated graphene with hexagonal boron-nitride can also realize QAHE with bulk gap over 40~meV, indicating the tunability of $5d$-intercalated graphene-based heterostructure. Our finding makes a great leap towards the experimental realization of graphene-based QAHE, and will definitely accelerate the practical application of graphene-based low-power electronics.

cond-mat.mes-hall↗

arXiv2022

The Principle of Diversity: Training Stronger Vision Transformers Calls for Reducing All Levels of Redundancy

Vision transformers (ViTs) have gained increasing popularity as they are commonly believed to own higher modeling capacity and representation flexibility, than traditional convolutional networks. However, it is questionable whether such potential has been fully unleashed in practice, as the learned ViTs often suffer from over-smoothening, yielding likely redundant models. Recent works made preliminary attempts to identify and alleviate such redundancy, e.g., via regularizing embedding similarity or re-injecting convolution-like structures. However, a "head-to-toe assessment" regarding the extent of redundancy in ViTs, and how much we could gain by thoroughly mitigating such, has been absent for this field. This paper, for the first time, systematically studies the ubiquitous existence of redundancy at all three levels: patch embedding, attention map, and weight space. In view of them, we advocate a principle of diversity for training ViTs, by presenting corresponding regularizers that encourage the representation diversity and coverage at each of those levels, that enabling capturing more discriminative information. Extensive experiments on ImageNet with a number of ViT backbones validate the effectiveness of our proposals, largely eliminating the observed ViT redundancy and significantly boosting the model generalization. For example, our diversified DeiT obtains 0.70%~1.76% accuracy boosts on ImageNet with highly reduced similarity. Our codes are fully available in https://github.com/VITA-Group/Diverse-ViT.

cs.LG↗

arXiv2022

Measurements of Rayleigh Ratios in Linear Alkylbenzene

In present work an experiment has been designed to measure the Rayleigh ratio directly at $405$ nm and $432$ nm for linear alkylbenzene which is a common solvent used in liquid scintillator detectors of neutrino experiments. The perpendicularly polarized Rayleigh ratio is determined to be $(4.52\pm 0.28)\times 10^{-6}$ m$^{-1}\cdot$ sr$^{-1}$ at $405$ nm and $(3.82\pm 0.24)\times 10^{-6}$ m$^{-1}\cdot$ sr$^{-1}$ at $432$ nm, and the corresponding Rayleigh scattering length is $L_{Ray} = 22.9\pm 0.3(\mathrm{stat.})\pm 1.7(\mathrm{sys.})$ m at $405$ nm and $L_{Ray}= 27.0\pm 0.9(\mathrm{stat.})\pm 1.8(\mathrm{sys.})$ m at $432$ nm.

physics.ins-det↗

arXiv2022

Ferroelectric tuning of superconductivity and band topology in a two-dimensional heterobilayer

Realization of tunable superconductivity with concomitant nontrivial band topology is conceptually intriguing and highly desirable for superconducting devices and topological quantum computation. Based on first-principles calculations, here we present the first prediction of simultaneously tunable superconducting transition temperature (Tc) and band topology in a superconducting IrTe2 overlayer on a ferroelectric In2Se3 monolayer. We first demonstrate that the Tc is substantially enhanced from that of IrTe2 nanoflakes (Tc ~3 K) due to significant charge repartitioning around the Fermi level. More importantly, the Tc is shown to sensitively depend on the In2Se3 polarization, with the higher Tc of ~(8-10) K attributed to enhanced interlayer electron-phonon coupling when the polarization is downward. The band topology is also switched from trivial to nontrivial as the polarization is reversed from upward to downward. These findings provide physically realistic platforms for simultaneously tuning superconductivity and band topology in two-dimensional heterobilayers and related heterostructures using a reversible and nonvolatile approach.

cond-mat.supr-con↗

arXiv2022

Sparsity Winning Twice: Better Robust Generalization from More Efficient Training

Recent studies demonstrate that deep networks, even robustified by the state-of-the-art adversarial training (AT), still suffer from large robust generalization gaps, in addition to the much more expensive training costs than standard training. In this paper, we investigate this intriguing problem from a new perspective, i.e., injecting appropriate forms of sparsity during adversarial training. We introduce two alternatives for sparse adversarial training: (i) static sparsity, by leveraging recent results from the lottery ticket hypothesis to identify critical sparse subnetworks arising from the early training; (ii) dynamic sparsity, by allowing the sparse subnetwork to adaptively adjust its connectivity pattern (while sticking to the same sparsity ratio) throughout training. We find both static and dynamic sparse methods to yield win-win: substantially shrinking the robust generalization gap and alleviating the robust overfitting, meanwhile significantly saving training and inference FLOPs. Extensive experiments validate our proposals with multiple network architectures on diverse datasets, including CIFAR-10/100 and Tiny-ImageNet. For example, our methods reduce robust generalization gap and overfitting by 34.44% and 4.02%, with comparable robust/standard accuracy boosts and 87.83%/87.82% training/inference FLOPs savings on CIFAR-100 with ResNet-18. Besides, our approaches can be organically combined with existing regularizers, establishing new state-of-the-art results in AT. Codes are available in https://github.com/VITA-Group/Sparsity-Win-Robust-Generalization.

cs.CV↗

arXiv2022

A Monte Carlo Method for Rayleigh Scattering in Liquid Detectors

A new Monte Carlo method has been implemented to describe the angular and polarization distributions of anisotropic liquids, like water and linear alkylbenzene, by considering orientational fluctuations of polarizability tensors. The scattered light of anisotropic liquids is depolarized with an angular distribution of $1+(1-ρ_v)/(1+3ρ_v)\cos^2θ$, which is modified by the depolarization ratio $ρ_v$. A standalone experiment has validated the simulation results of LAB. The new method can provide more accurate knowledge on light propagation in large liquid detectors, which is beneficial to the developments of reconstruction for detectors.

physics.ins-det↗

arXiv2022

Josephson-Coulomb drag effect between graphene and LaAlO3/SrTiO3 interfacial superconductor

Coulomb drag refers to the phenomenon that a charge current in one electronic circuit induces a responsive current in a neighboring circuit solely through Coulomb interactions. For conventional interactions between fermionic particles such as electrons, the as-induced drag current in the passive layer is orders of magnitude weaker than the active current due to strong dielectric screening effect between the two. Here we propose a 'super' Coulomb drag effect between an active normal conductor and a passive superconductor of Josephson junction arrays, whereby the passive current can greatly exceed the active. The drag force originates from the interactions between the substantially enhanced dynamical quantum fluctuations of the superconducting phases in the passive layer and the normal electrons in the active layer. We demonstrate this effect in the devices composed of monolayer graphene and LaAlO3/SrTiO3 heterointerface, an inherently non-uniform superconductor described by Josephson junction arrays. Remarkable drag signal is observed in the superconducting transition regime of the LaAlO3/SrTiO3 interface, with its sign independent of the carrier type in the graphene layer. The estimated passive-to-active ratio can reach about 0.3 at the optimal gate voltage and the temperature dependence follows that of the typical Josephson energy between superconducting puddles. Strikingly, the ratio ought to be as large as 10^5 at zero temperature by theoretical extrapolation. From engineering perspective, our device may work as current or voltage transformers, and the drag mechanism lays the foundation for synchronizing Josephson-junction-array-based terahertz radiators.

cond-mat.mes-hall↗

arXiv2021

A family tree of two-dimensional magnetic materials with tunable topological properties

Two-dimensional (2D) magnetic materials empowered with nontrivial band topology may lead to the emergence of exotic quantum states with significant application potentials. Here we predict a family tree of 2D magnetic materials with tunable topological properties, starting from the parental materials of CrI3 and CrBr3. The underlying design principle is that, by substituting the alternating sites of the Cr honeycomb lattice sandwiched between the halogen layers with V or Mn, the parental materials of trivial ferromagnetic insulators are ripe to be converted into topological systems. Specifically, our first-principles calculations show that, due to the elegant interplay between bandgap narrowing and spin-orbital coupling, CrI3 branches into high-temperature quantum anomalous Hall insulators of CrVI6 and CrMnI6 with different topological invariants, while CrBr3 branches into topological half-metals of CrVBr6 and CrMnBr6. Those novel 2D magnets are also shown to be easily exfoliated from their bulk counterparts. The present study is thus geared to advance the field of 2D magnetic materials into the topologically nontrivial realm.

cond-mat.mtrl-sci↗

arXiv2021

Giant Dzyaloshinskii-Moriya interaction and ratchet motion of bimeronic excitations in two-dimensional magnets

Topological magnetic excitations rooted in Dzyaloshinskii-Moriya interaction (DMI) are promising information carriers for next-generation memory and spintronic devices. The recently discovered two-dimensional (2D) magnets provide fertile new platforms for revealing rich physical phenomena with atomic-scale precision, as exemplified by the enhanced DMI associated with an effective electric field that breaks the inversion symmetry. Here we use first-principles calculations to establish a conceptually difierent compositional engineering approach to induce giant DMI in a representative CrMnI$_6$ 2D magnet, and the underlying mechanism is rooted in the spontaneous inversion symmetry breaking in the bipartite system. Using atomistic magnetics simulations, we further reveal that bimeronic excitations can emerge upon cooling a spin random state or perturbing a ferromagnetic state. Strikingly, the bimeron can exhibit unidirectional soliton-like propagation, and such a ratchet phenomenon is highly desirable for racetrack memories. These findings may prove instrumental in developing novel devices for quantum information based on 2D magnets.

cond-mat.mtrl-sci↗

arXiv2021

You are caught stealing my winning lottery ticket! Making a lottery ticket claim its ownership

Despite tremendous success in many application scenarios, the training and inference costs of using deep learning are also rapidly increasing over time. The lottery ticket hypothesis (LTH) emerges as a promising framework to leverage a special sparse subnetwork (i.e., winning ticket) instead of a full model for both training and inference, that can lower both costs without sacrificing the performance. The main resource bottleneck of LTH is however the extraordinary cost to find the sparse mask of the winning ticket. That makes the found winning ticket become a valuable asset to the owners, highlighting the necessity of protecting its copyright. Our setting adds a new dimension to the recently soaring interest in protecting against the intellectual property (IP) infringement of deep models and verifying their ownerships, since they take owners' massive/unique resources to develop or train. While existing methods explored encrypted weights or predictions, we investigate a unique way to leverage sparse topological information to perform lottery verification, by developing several graph-based signatures that can be embedded as credentials. By further combining trigger set-based methods, our proposal can work in both white-box and black-box verification scenarios. Through extensive experiments, we demonstrate the effectiveness of lottery verification in diverse models (ResNet-20, ResNet-18, ResNet-50) on CIFAR-10 and CIFAR-100. Specifically, our verification is shown to be robust to removal attacks such as model fine-tuning and pruning, as well as several ambiguity attacks. Our codes are available at https://github.com/VITA-Group/NO-stealing-LTH.

cs.LG↗

arXiv2021

Equally Spaced Quantum States in van der Waals Epitaxy-Grown Nanoislands

Pursuing the confinement of linearly dispersive relativistic fermions is of interest in both fundamental physics and potential applications. Here, we report strong STM evidence for the equally spaced, strikingly sharp, and densely distributed quantum well states (QWSs) near Fermi energy in Pb(111) nanoislands, van-der-Waals epitaxially grown on graphitized 6H-SiC(0001). The observations can be explained as the quantized energies of confined linearly dispersive [111] electrons, which essentially 'simulate' the out-of-plane relativistic quasiparticles. The equally spaced QWSs with an origin of confined relativistic electrons are supported by phenomenological simulations and Fabry-Perot fittings based on the relativistic fermions. First-principles calculations further reveal that the spin-orbit coupling strengthens the relativistic nature of electrons near Fermi energy. Our finding uncovers the unique equally spaced quantum states in electronic systems beyond Landau levels, and may inspire future studies on confined relativistic quasiparticles in flourishing topological materials and applications in structurally simpler quantum cascade laser.

cond-mat.mes-hall↗

arXiv2021

FMFCC-A: A Challenging Mandarin Dataset for Synthetic Speech Detection

As increasing development of text-to-speech (TTS) and voice conversion (VC) technologies, the detection of synthetic speech has been suffered dramatically. In order to promote the development of synthetic speech detection model against Mandarin TTS and VC technologies, we have constructed a challenging Mandarin dataset and organized the accompanying audio track of the first fake media forensic challenge of China Society of Image and Graphics (FMFCC-A). The FMFCC-A dataset is by far the largest publicly-available Mandarin dataset for synthetic speech detection, which contains 40,000 synthesized Mandarin utterances that generated by 11 Mandarin TTS systems and two Mandarin VC systems, and 10,000 genuine Mandarin utterances collected from 58 speakers. The FMFCC-A dataset is divided into the training, development and evaluation sets, which are used for the research of detection of synthesized Mandarin speech under various previously unknown speech synthesis systems or audio post-processing operations. In addition to describing the construction of the FMFCC-A dataset, we provide a detailed analysis of two baseline methods and the top-performing submissions from the FMFCC-A, which illustrates the usefulness and challenge of FMFCC-A dataset. We hope that the FMFCC-A dataset can fill the gap of lack of Mandarin datasets for synthetic speech detection.

cs.SD↗