arXiv subjects

Lei Wang

Publications and source records attributed to Lei Wang.

At least 775 records · Page 43Linked to original sources

arXiv2023

TPU-MLIR: A Compiler For TPU Using MLIR

Multi-level intermediate representations (MLIR) show great promise for reducing the cost of building domain-specific compilers by providing a reusable and extensible compiler infrastructure. This work presents TPU-MLIR, an end-to-end compiler based on MLIR that deploys pre-trained neural network (NN) models to a custom ASIC called a Tensor Processing Unit (TPU). TPU-MLIR defines two new dialects to implement its functionality: 1. a Tensor operation (TOP) dialect that encodes the deep learning graph semantics and independent of the deep learning framework and 2. a TPU kernel dialect to provide a standard kernel computation on TPU. A NN model is translated to the TOP dialect and then lowered to the TPU dialect for different TPUs according to the chip's configuration. We demonstrate how to use the MLIR pass pipeline to organize and perform optimization on TPU to generate machine code. The paper also presents a verification procedure to ensure the correctness of each transform stage.

cs.PL↗

arXiv2023

Spontaneous CP violation electroweak baryogenesis and gravitational wave through multi-step phase transitions

In a singlet pseudoscalar extension of the two-Higgs-doublet model we discuss spontaneous CP violation electroweak baryogenesis via two different patterns of phase transitions (PTs): (i) two-step PTs whose first-step and second-step are strongly first-order; (ii) three-step PTs whose first-step is second-order and the second-step and third-step are strongly first-order. For the case of the two-step pattern, the first-step PT takes place at a high temperature, converting the origin phase into an electroweak symmetry broken phase and breaking the CP symmetry spontaneously. Thus, the baryon number is produced during the first-step PT. At the second-step PT, the phase is converted into the observed vacuum at zero temperature, and the CP-symmetry is restored. In both phases the sphaleron processes are sufficiently suppressed, which keep the baryon number unchanged. For the case of the three-step PTs, the pseudoscalar field firstly acquires a nonzero VEV, and VEVs of other fields still remain zero during the first-step PT. The following PTs and electroweak baryogenesis are similar to the case of the two-step PTs. In addition, the gravitational wave spectra can have one or two peaks through the two-step and the three-step PTs, and we discuss the detectability at the future gravitational wave detectors.

hep-ph↗

arXiv2023

Large Eddy Simulations of Fully-Developed Turbulent Flows Over Additively Manufactured Rough Surfaces

In the last decade, progresses in additive manufacturing (AM) have paved the way for optimized heat exchangers, whose disruptive design will depend on predictive numerical simulations. Typical AM rough surfaces show limited resemblance to the artificially constructed rough surfaces that have been the basis of most prior fundamental research on turbulent flow over rough walls. Therefore, a high-fidelity LES database is built to develop and assess novel wall models for AM. This article investigates the flow in rough pipes built from the surfaces created using AM techniques at Siemens based on Nickel Alloy IN939 material. We developed a code to generate the desired rough pipes from scanned planar surfaces and performed high-fidelity LES of turbulent rough pipe flows at Re = 11,700 to reveal the influence of roughness on turbulence, mainly the average roughness height and the Effective Slope. The equivalent sand-grain roughnesses, ks, of the present AM rough surfaces are predicted using the Colebrook correlation. In the present study, the existence of a logarithmic layer is marked even for high values of ks. The mean flow, the velocity fluctuations, and the Reynolds stresses show turbulence's strong dependence on the roughness topography. Profiles of turbulence statistics are compared by introducing an effective wall-normal distance. The effective distance collapses the shear stresses and the velocity fluctuations outside the roughness sublayer; thus, Townsend's similarity of the streamwise mean velocity is marked for the present roughnesses. Furthermore, a mixed scaling is introduced to improve the collapse of turbulence statistics in the roughness sublayer.

physics.flu-dyn↗

arXiv2023

Effect of temperature-dependent thermophysical properties on turbulent forced convection under constant heat flux boundary condition

In this study, we performed highly resolved large-eddy simulations (LES) to investigate the influence of variable properties on the forced turbulent convection in a channel. The constant heat flux boundary condition permits wall temperature fluctuations and thus induces variations of fluid properties. Since the effect of viscosity on the flow exhibits $Re_τ^{-1}$ scaling, we only considered $Re_τ= 180$ in the present study. Compared to the flow with constant properties, results indicate that the variable properties have trivial effects on the mean velocity and temperature profiles, Reynolds shear stress, wall-normal heat flux, as well as the small-scale turbulence characteristics. However, we also observed that the turbulence intensities, low-speed streaks, burst motions, and budgets for temperature variance and wall-normal heat flux are modified by the variable properties in a perceptible way. In addition, we showed that the classic wall scaling is a good choice for flow with small and moderate variations of fluid properties.

physics.flu-dyn↗

arXiv2023

A GOA-Based Fault-Tolerant Trajectory Tracking Control for an Underwater Vehicle of Multi-Thruster System without Actuator Saturation

This paper proposes an intelligent fault-tolerant control (FTC) strategy to tackle the trajectory tracking problem of an underwater vehicle (UV) under thruster damage (power loss) cases and meanwhile resolve the actuator saturation brought by the vehicle's physical constraints. In the proposed control strategy, the trajectory tracking component is formed by a refined backstepping algorithm that controls the velocity variation and a sliding mode control deducts the torque/force outputs; the fault-tolerant component is established based on a Grasshopper Optimization Algorithm (GOA), which provides fast convergence speed as well as satisfactory accuracy of deducting optimized reallocation of the thruster forces to compensate for the power loss in different fault cases. Simulations with or without environmental perturbations under different fault cases and comparisons to other traditional FTCs are presented, thus verifying the effectiveness and robustness of the proposed GOA-based fault-tolerant trajectory tracking design.

cs.RO↗

arXiv2022

a cognitive frequency allocation strategy for multi-carrier radar against communication interference

Modern radars often adopt multi-carrier waveform which has been widely discussed in the literature. However, with the development of civil communication, more and more spectrum resource has been occupied by communication networks. Thus, avoiding the interference from communication users is an important and challenging task for the application of multi-carrier radar. In this paper, a novel frequency allocation strategy based on the historical experiences is proposed, which is formulated as a Markov decision process (MDP). In a decision step, the multi-carrier radar needs to choose more than one frequencies, leading to a combinatorial action space. To address this challenge, we use a novel iteratively selecting technique which breaks a difficult decision task into several easy tasks. Moreover, an efficient deep reinforcement learning algorithm is adopted to handle the complicated spectrum dynamics. Numerical results show that our proposed method outperforms the existing ones.

eess.SP↗

arXiv2022

ToL: A Tensor of List-Based Unified Computation Model

Previous computation models either have equivalent abilities in representing all computations but fail to provide primitive operators for programming complex algorithms or lack generalized expression ability to represent newly-added computations. This article presents a unified computation model with generalized expression ability and a concise set of primitive operators for programming high-level algorithms. We propose a unified data abstraction -- Tensor of List, and offer a unified computation model based on Tensor of List, which we call the ToL model (in short, ToL). ToL introduces five atomic computations that can represent any elementary computation by finite composition, ensured with strict formal proof. Based on ToL, we design a pure-functional language -- ToLang. ToLang provides a concise set of primitive operators that can be used to program complex big data and AI algorithms. Our evaluations show ToL has generalized expression ability and a built-in performance indicator, born with a strictly defined computation metric -- elementary operation count (EOPs), consistent with FLOPs within a small error range.

cs.PL↗

arXiv2022

Asymmetric parametric generation of images with nonlinear dielectric metasurfaces

Subwavelength dielectric resonators assembled into metasurfaces have become a versatile tool for miniaturising optical components approaching the nanoscale. An important class of metasurface functionalities is associated with asymmetry in both generation and transmission of light with respect to reversals of the positions of emitters and receivers. Nonlinear light-matter interaction in metasurfaces offers a promising pathway towards miniaturisation of the asymmetric control of light. Here we demonstrate asymmetric parametric generation of light in nonlinear metasurfaces. We assemble dissimilar nonlinear dielectric resonators into translucent metasurfaces that produce images in the visible spectral range being illuminated by infrared radiation. By design, the metasurfaces produce different and completely independent images for the reversed direction of illumination, that is when the positions of the infrared emitter and the visible light receiver are exchanged. Nonlinearity-enabled asymmetric control of light by subwavelength resonators paves the way towards novel nanophotonic components via dense integration of large quantities of nonlinear resonators into compact metasurface designs.

physics.optics↗

arXiv2022

MutexMatch: Semi-Supervised Learning with Mutex-Based Consistency Regularization

The core issue in semi-supervised learning (SSL) lies in how to effectively leverage unlabeled data, whereas most existing methods tend to put a great emphasis on the utilization of high-confidence samples yet seldom fully explore the usage of low-confidence samples. In this paper, we aim to utilize low-confidence samples in a novel way with our proposed mutex-based consistency regularization, namely MutexMatch. Specifically, the high-confidence samples are required to exactly predict "what it is" by conventional True-Positive Classifier, while the low-confidence samples are employed to achieve a simpler goal -- to predict with ease "what it is not" by True-Negative Classifier. In this sense, we not only mitigate the pseudo-labeling errors but also make full use of the low-confidence unlabeled data by consistency of dissimilarity degree. MutexMatch achieves superior performance on multiple benchmark datasets, i.e., CIFAR-10, CIFAR-100, SVHN, STL-10, mini-ImageNet and Tiny-ImageNet. More importantly, our method further shows superiority when the amount of labeled data is scarce, e.g., 92.23% accuracy with only 20 labeled data on CIFAR-10. Our code and model weights have been released at https://github.com/NJUyued/MutexMatch4SSL.

cs.CV↗

arXiv2022

Domain Generalization by Learning and Removing Domain-specific Features

Deep Neural Networks (DNNs) suffer from domain shift when the test dataset follows a distribution different from the training dataset. Domain generalization aims to tackle this issue by learning a model that can generalize to unseen domains. In this paper, we propose a new approach that aims to explicitly remove domain-specific features for domain generalization. Following this approach, we propose a novel framework called Learning and Removing Domain-specific features for Generalization (LRDG) that learns a domain-invariant model by tactically removing domain-specific features from the input images. Specifically, we design a classifier to effectively learn the domain-specific features for each source domain, respectively. We then develop an encoder-decoder network to map each input image into a new image space where the learned domain-specific features are removed. With the images output by the encoder-decoder network, another classifier is designed to learn the domain-invariant features to conduct image classification. Extensive experiments demonstrate that our framework achieves superior performance compared with state-of-the-art methods.

cs.CV↗

arXiv2022

The uniqueness of vertex pairs in $π$-separable groups

Let $G$ be a finite $π$-separable group, where $π$ is a set of primes, and let $χ$ be an irreducible complex character that is a $π$-lift of some $π$-partial character of $G$.It was proved by Cossey and Lewis that all of the vertex pairs for $χ$ are linear and conjugate in $G$ if $2\inπ$, but the result can fail for $2\notinπ$. In this paper we introduce the notion of the twisted vertices in the case where $2\notinπ$, and establish the uniqueness for linear twisted vertices under the conditions that either $χ$ is an $\mathcal N$-lift for a $π$-chain $\mathcal N$ of $G$ or it has a linear Navarro vertex, thus answering a question proposed by them.

math.GR↗

arXiv2022

A Communication-Efficient and Privacy-Aware Distributed Algorithm for Sparse PCA

Sparse principal component analysis (PCA) improves interpretability of the classic PCA by introducing sparsity into the dimension-reduction process. Optimization models for sparse PCA, however, are generally non-convex, non-smooth and more difficult to solve, especially on large-scale datasets requiring distributed computation over a wide network. In this paper, we develop a distributed and centralized algorithm called DSSAL1 for sparse PCA that aims to achieve low communication overheads by adapting a newly proposed subspace-splitting strategy to accelerate convergence. Theoretically, convergence to stationary points is established for DSSAL1. Extensive numerical results show that DSSAL1 requires far fewer rounds of communication than state-of-the-art peer methods. In addition, we make the case that since messages exchanged in DSSAL1 are well-masked, the possibility of private-data leakage in DSSAL1 is much lower than in some other distributed algorithms.

math.OC↗

arXiv2022

Seeking Consensus on Subspaces in Federated Principal Component Analysis

In this paper, we develop an algorithm for federated principal component analysis (PCA) with emphases on both communication efficiency and data privacy. Generally speaking, federated PCA algorithms based on direct adaptations of classic iterative methods, such as simultaneous subspace iterations (SSI), are unable to preserve data privacy, while algorithms based on variable-splitting and consensus-seeking, such as alternating direction methods of multipliers (ADMM), lack in communication-efficiency. In this work, we propose a novel consensus-seeking formulation by equalizing subspaces spanned by splitting variables instead of equalizing variables themselves, thus greatly relaxing feasibility restrictions and allowing much faster convergence. Then we develop an ADMM-like algorithm with several special features to make it practically efficient, including a low-rank multiplier formula and techniques for treating subproblems. We establish that the proposed algorithm can better protect data privacy than classic methods adapted to the federated PCA setting. We derive convergence results, including a worst-case complexity estimate, for the proposed ADMM-like algorithm in the presence of the nonlinear equality constraints. Extensive empirical results are presented to show that the new algorithm, while enhancing data privacy, requires far fewer rounds of communication than existing peer algorithms for federated PCA.

math.OC↗

arXiv2022

Alignment-Enriched Tuning for Patch-Level Pre-trained Document Image Models

Alignment between image and text has shown promising improvements on patch-level pre-trained document image models. However, investigating more effective or finer-grained alignment techniques during pre-training requires a large amount of computation cost and time. Thus, a question naturally arises: Could we fine-tune the pre-trained models adaptive to downstream tasks with alignment objectives and achieve comparable or better performance? In this paper, we propose a new model architecture with alignment-enriched tuning (dubbed AETNet) upon pre-trained document image models, to adapt downstream tasks with the joint task-specific supervised and alignment-aware contrastive objective. Specifically, we introduce an extra visual transformer as the alignment-ware image encoder and an extra text transformer as the alignment-ware text encoder before multimodal fusion. We consider alignment in the following three aspects: 1) document-level alignment by leveraging the cross-modal and intra-modal contrastive loss; 2) global-local alignment for modeling localized and structural information in document images; and 3) local-level alignment for more accurate patch-level information. Experiments on various downstream tasks show that AETNet can achieve state-of-the-art performance on various downstream tasks. Notably, AETNet consistently outperforms state-of-the-art pre-trained models, such as LayoutLMv3 with fine-tuning techniques, on three different downstream tasks.

cs.CV↗

arXiv2022

Generalizing Math Word Problem Solvers via Solution Diversification

Current math word problem (MWP) solvers are usually Seq2Seq models trained by the (one-problem; one-solution) pairs, each of which is made of a problem description and a solution showing reasoning flow to get the correct answer. However, one MWP problem naturally has multiple solution equations. The training of an MWP solver with (one-problem; one-solution) pairs excludes other correct solutions, and thus limits the generalizability of the MWP solver. One feasible solution to this limitation is to augment multiple solutions to a given problem. However, it is difficult to collect diverse and accurate augment solutions through human efforts. In this paper, we design a new training framework for an MWP solver by introducing a solution buffer and a solution discriminator. The buffer includes solutions generated by an MWP solver to encourage the training data diversity. The discriminator controls the quality of buffered solutions to participate in training. Our framework is flexibly applicable to a wide setting of fully, semi-weakly and weakly supervised training for all Seq2Seq MWP solvers. We conduct extensive experiments on a benchmark dataset Math23k and a new dataset named Weak12k, and show that our framework improves the performance of various MWP solvers under different settings by generating correct and diverse solutions.

cs.CL↗

arXiv2022

Synthetic Voice Detection and Audio Splicing Detection using SE-Res2Net-Conformer Architecture

Synthetic voice and splicing audio clips have been generated to spoof Internet users and artificial intelligence (AI) technologies such as voice authentication. Existing research work treats spoofing countermeasures as a binary classification problem: bonafide vs. spoof. This paper extends the existing Res2Net by involving the recent Conformer block to further exploit the local patterns on acoustic features. Experimental results on ASVspoof 2019 database show that the proposed SE-Res2Net-Conformer architecture is able to improve the spoofing countermeasures performance for the logical access scenario. In addition, this paper also proposes to re-formulate the existing audio splicing detection problem. Instead of identifying the complete splicing segments, it is more useful to detect the boundaries of the spliced segments. Moreover, a deep learning approach can be used to solve the problem, which is different from the previous signal processing techniques.

eess.AS↗

arXiv2022

Recent Advances in RecBole: Extensions with more Practical Considerations

RecBole has recently attracted increasing attention from the research community. As the increase of the number of users, we have received a number of suggestions and update requests. This motivates us to make some significant improvements on our library, so as to meet the user requirements and contribute to the research community. In order to show the recent update in RecBole, we write this technical report to introduce our latest improvements on RecBole. In general, we focus on the flexibility and efficiency of RecBole in the past few months. More specifically, we have four development targets: (1) more flexible data processing, (2) more efficient model training, (3) more reproducible configurations, and (4) more comprehensive user documentation. Readers can download the above updates at: https://github.com/RUCAIBox/RecBole.

cs.IR↗

arXiv2022

A Unified Framework for Contrastive Learning from a Perspective of Affinity Matrix

In recent years, a variety of contrastive learning based unsupervised visual representation learning methods have been designed and achieved great success in many visual tasks. Generally, these methods can be roughly classified into four categories: (1) standard contrastive methods with an InfoNCE like loss, such as MoCo and SimCLR; (2) non-contrastive methods with only positive pairs, such as BYOL and SimSiam; (3) whitening regularization based methods, such as W-MSE and VICReg; and (4) consistency regularization based methods, such as CO2. In this study, we present a new unified contrastive learning representation framework (named UniCLR) suitable for all the above four kinds of methods from a novel perspective of basic affinity matrix. Moreover, three variants, i.e., SimAffinity, SimWhitening and SimTrace, are presented based on UniCLR. In addition, a simple symmetric loss, as a new consistency regularization term, is proposed based on this framework. By symmetrizing the affinity matrix, we can effectively accelerate the convergence of the training process. Extensive experiments have been conducted to show that (1) the proposed UniCLR framework can achieve superior results on par with and even be better than the state of the art, (2) the proposed symmetric loss can significantly accelerate the convergence of models, and (3) SimTrace can avoid the mode collapse problem by maximizing the trace of a whitened affinity matrix without relying on asymmetry designs or stop-gradients.

cs.CV↗