arXiv subjects

Xue Wang

Publications and source records attributed to Xue Wang.

At least 91 records · Page 5Linked to original sources

arXiv2023

Annealing of swift heavy ion tracks in amorphous silicon dioxide

The annealing kinetics of the high energy ion damage in amorphous silicon dioxide (a-SiO2) are still not well understood, despite the material's widespread application in material science, physics, geology, and biology. This study investigates how annealing temperature, duration, and ambient environment affect the recovery of irradiation damage produced along the trajectory of swift heavy ions in a-SiO2. The track-annealing kinetics and the changing ion track morphology were investigated using synchrotron-based small-angle X-ray scattering (SAXS) and etching methods. We found that track annealing proceeds quicker near the sample surface demonstrated by a changing track etch rate as a function of depth. Measurements of ion tracks using SAXS show only small changes in the radial density distribution profile of the ion tracks. Activation energy of the annealing process at different sample depths was determined and the effect of the capping layer during the annealing process was also studied. Combination of oxygen diffusion and stress relaxation may contribute to the observed behaviour of preferential and anisotropic healing of the ion track. The results add to the fundamental understanding of ion track damage recovery and may have direct implications for materials for radioactive waste storage and solid state nanopores.

cond-mat.mtrl-sci↗

arXiv2023

On Existence and Blowup Criterion of Strong Solutions to Cauchy Problem of 2D Full Compressible Navier-Stokes System with Vacuum

This paper investigates the Cauchy problem of two-dimensional full compressible Navier-Stokes system with density and temperature vanishing at infinity. For the strong solutions, some a priori weighted $L^2(R^2)$-norm of the gradient of velocity is obtained by the techniques of $A_p$ weights and cancellation of singularity. Based on this key weighted estimate and some basic weighted analysis of velocity and temperature, we establish the local existence and uniqueness of strong solutions with initial vacuum by means of a two-level approximation scheme. Meanwhile, for $q_1$ and $q_2$ as in (1.8), the $L^\infty_tL^{q_1}_x$-norm of velocity and the $L^2_tL^{q_2}_x$-norm of temperature are derived originally. Moreover, we obtain a blowup criterion only in terms of the temporal integral of the maximum norm of divergence of velocity, which is independent of the temperature.

math.AP↗

arXiv2023

Free Lunch for Domain Adversarial Training: Environment Label Smoothing

A fundamental challenge for machine learning models is how to generalize learned models for out-of-distribution (OOD) data. Among various approaches, exploiting invariant features by Domain Adversarial Training (DAT) received widespread attention. Despite its success, we observe training instability from DAT, mostly due to over-confident domain discriminator and environment label noise. To address this issue, we proposed Environment Label Smoothing (ELS), which encourages the discriminator to output soft probability, which thus reduces the confidence of the discriminator and alleviates the impact of noisy environment labels. We demonstrate, both experimentally and theoretically, that ELS can improve training stability, local convergence, and robustness to noisy environment labels. By incorporating ELS with DAT methods, we are able to yield state-of-art results on a wide range of domain generalization/adaptation tasks, particularly when the environment labels are highly noisy.

cs.LG↗

arXiv2022

Progressive Backdoor Erasing via connecting Backdoor and Adversarial Attacks

Deep neural networks (DNNs) are known to be vulnerable to both backdoor attacks as well as adversarial attacks. In the literature, these two types of attacks are commonly treated as distinct problems and solved separately, since they belong to training-time and inference-time attacks respectively. However, in this paper we find an intriguing connection between them: for a model planted with backdoors, we observe that its adversarial examples have similar behaviors as its triggered images, i.e., both activate the same subset of DNN neurons. It indicates that planting a backdoor into a model will significantly affect the model's adversarial examples. Based on these observations, a novel Progressive Backdoor Erasing (PBE) algorithm is proposed to progressively purify the infected model by leveraging untargeted adversarial attacks. Different from previous backdoor defense methods, one significant advantage of our approach is that it can erase backdoor even when the clean extra dataset is unavailable. We empirically show that, against 5 state-of-the-art backdoor attacks, our PBE can effectively erase the backdoor without obvious performance degradation on clean samples and significantly outperforms existing defense methods.

cs.CV↗

arXiv2022

Siamese Transition Masked Autoencoders as Uniform Unsupervised Visual Anomaly Detector

Unsupervised visual anomaly detection conveys practical significance in many scenarios and is a challenging task due to the unbounded definition of anomalies. Moreover, most previous methods are application-specific, and establishing a unified model for anomalies across application scenarios remains unsolved. This paper proposes a novel hybrid framework termed Siamese Transition Masked Autoencoders(ST-MAE) to handle various visual anomaly detection tasks uniformly via deep feature transition. Concretely, the proposed method first extracts hierarchical semantics features from a pre-trained deep convolutional neural network and then develops a feature decoupling strategy to split the deep features into two disjoint feature patch subsets. Leveraging the decoupled features, the ST-MAE is developed with the Siamese encoders that operate on each subset of feature patches and perform the latent representations transition of two subsets, along with a lightweight decoder that reconstructs the original feature from the transitioned latent representation. Finally, the anomalous attributes can be detected using the semantic deep feature residual. Our deep feature transition scheme yields a nontrivial and semantic self-supervisory task to extract prototypical normal patterns, which allows for learning uniform models that generalize well for different visual anomaly detection tasks. The extensive experiments conducted demonstrate that the proposed ST-MAE method can advance state-of-the-art performance on multiple benchmarks across application scenarios with a superior inference efficiency, which exhibits great potential to be the uniform model for unsupervised visual anomaly detection.

cs.CV↗

arXiv2022

Global Classical Solutions to Full Compressible Magnetohydrodynamic System with Large Oscillations and Vacuum in 3D Exterior Domains

The full compressible magnetohydrodynamic system in three-dimensional exterior domains is investigated. For the initial-boundary-value problem of this system with slip boundary condition for the velocity, adiabatic one for the temperature, and perfect one for the magnetic field, the global existence and uniqueness of classical solutions is established, under the condition that the initial data are of small energy but possibly large oscillations. In particular,the initial density and temperature are both allowed to vanish. Moreover, the large-time behavior of the classical solutions is also obtained.

math.AP↗

arXiv2022

Global Well-Posedness of Full Compressible Magnetohydrodynamic System in 3D Bounded Domains with Large Oscillations and Vacuum

The three-dimensional (3D) full compressible magnetohydrodynamic system is studied in a general bounded domain with slip boundary condition for the velocity filed, adiabatic condition for the temperature and perfect conduction for the magnetic field. For the regular initial data with small energy but possibly large oscillations, the global existence of classical and weak solution as well as the exponential decay rate to the initial-boundary-value problem of this system is obtained. In particular, the density and temperature of such a classical solution are both allowed to vanish initially. Moreover, it is also shown that for the classical solutions, the oscillation of the density will grow unboundedly with an exponential rate when the initial vacuum appears (even at a point). Some new observations and useful estimates are developed to overcome the difficulties caused by the slip boundary conditions.

math.AP↗

arXiv2022

Rate-Splitting Multiple Access in Multi-cell Dense Networks: A Stochastic Geometry Approach

In this paper, the potential benefits of applying the Rate-Splitting Multiple Access (RSMA) in multi-cell dense networks are explored. Using tools of stochastic geometry, the sum-rate of RSMA-enhanced multi-cell dense networks is evaluated mathematically based on a Moment Generating Function (MGF) based framework to prove that RSMA is a general and powerful strategy for multi-antenna downlink systems. Further elaboration of the systematic performance metrics is undertaken by developing analytical expressions for area spectral efficiency and sum-rate in the RSMA-enhanced multi-cell dense networks. Based on the tractable expressions, we then offer an optimization framework for energy efficiency in terms of the number of antennas. Additionally, simulation results are shown to verify the accuracy of our analytical results and provide some insightful insights into system design. Analytically, it has been shown that: 1) the sum-rate of RSMA-enhanced multi-cell dense networks is significantly influenced by the power splitting ratio, and there is a unique value that maximizes the sum-rate; 2) the RSMA-enhanced multi-cell dense networks transmission scheme has superior sum-rate performance compared with Non-Orthogonal Multiple Access (NOMA) and Space-Division Multiple Access (SDMA) in a wide range of power splitting ratio; 3) By increasing the number of antennas and BS density in an RSMA-enhanced multi-cell dense network, the area spectral efficiency can be substantially enhanced; 4) As for energy efficiency, there exists an optimal antenna number for maximizing this performance metric.

cs.IT↗

arXiv2022

KVT: k-NN Attention for Boosting Vision Transformers

Convolutional Neural Networks (CNNs) have dominated computer vision for years, due to its ability in capturing locality and translation invariance. Recently, many vision transformer architectures have been proposed and they show promising performance. A key component in vision transformers is the fully-connected self-attention which is more powerful than CNNs in modelling long range dependencies. However, since the current dense self-attention uses all image patches (tokens) to compute attention matrix, it may neglect locality of images patches and involve noisy tokens (e.g., clutter background and occlusion), leading to a slow training process and potential degradation of performance. To address these problems, we propose the $k$-NN attention for boosting vision transformers. Specifically, instead of involving all the tokens for attention matrix calculation, we only select the top-$k$ similar tokens from the keys for each query to compute the attention map. The proposed $k$-NN attention naturally inherits the local bias of CNNs without introducing convolutional operations, as nearby tokens tend to be more similar than others. In addition, the $k$-NN attention allows for the exploration of long range correlation and at the same time filters out irrelevant tokens by choosing the most similar tokens from the entire image. Despite its simplicity, we verify, both theoretically and empirically, that $k$-NN attention is powerful in speeding up training and distilling noise from input tokens. Extensive experiments are conducted by using 11 different vision transformer architectures to verify that the proposed $k$-NN attention can work with any existing transformer architectures to improve its prediction performance. The codes are available at \url{https://github.com/damo-cv/KVT}.

cs.CV↗

arXiv2022

Global Existence of Classical Solutions to Full Compressible Navier-Stokes System with Large Oscillations and Vacuum in 3D Bounded Domains

The full compressible Navier-Stokes system describing the motion of a viscous, compressible, heat-conductive, and Newtonian polytropic fluid is studied in a three-dimensional simply connected bounded domain with smooth boundary having a finite number of two-dimensional connected components. For the initial-boundary-value problem with slip boundary conditions on the velocity and Neumann boundary one on the temperature, the global existence of classical and weak solutions which are of small energy but possibly large oscillations is established. In particular, both the density and temperature are allowed to vanish initially. Finally, the exponential stability of the density, velocity, and temperature is also obtained. Moreover, it is shown that for the classical solutions, the oscillation of the density will grow unboundedly in the long run with an exponential rate provided vacuum appears (even at a point) initially. This is the first result concerning the global existence of classical solutions to the full compressible Navier-Stokes equations with vacuum in general three-dimensional bounded smooth domains.

math.AP↗

arXiv2022

TreeDRNet:A Robust Deep Model for Long Term Time Series Forecasting

Various deep learning models, especially some latest Transformer-based approaches, have greatly improved the state-of-art performance for long-term time series forecasting.However, those transformer-based models suffer a severe deterioration performance with prolonged input length, which prohibits them from using extended historical info.Moreover, these methods tend to handle complex examples in long-term forecasting with increased model complexity, which often leads to a significant increase in computation and less robustness in performance(e.g., overfitting). We propose a novel neural network architecture, called TreeDRNet, for more effective long-term forecasting. Inspired by robust regression, we introduce doubly residual link structure to make prediction more robust.Built upon Kolmogorov-Arnold representation theorem, we explicitly introduce feature selection, model ensemble, and a tree structure to further utilize the extended input sequence, which improves the robustness and representation power of TreeDRNet. Unlike previous deep models for sequential forecasting work, TreeDRNet is built entirely on multilayer perceptron and thus enjoys high computational efficiency. Our extensive empirical studies show that TreeDRNet is significantly more effective than state-of-the-art methods, reducing prediction errors by 20% to 40% for multivariate time series. In particular, TreeDRNet is over 10 times more efficient than transformer-based methods. The code will be released soon.

cs.LG↗

arXiv2022

A Feature Memory Rearrangement Network for Visual Inspection of Textured Surface Defects Toward Edge Intelligent Manufacturing

Recent advances in the industrial inspection of textured surfaces-in the form of visual inspection-have made such inspections possible for efficient, flexible manufacturing systems. We propose an unsupervised feature memory rearrangement network (FMR-Net) to accurately detect various textural defects simultaneously. Consistent with mainstream methods, we adopt the idea of background reconstruction; however, we innovatively utilize artificial synthetic defects to enable the model to recognize anomalies, while traditional wisdom relies only on defect-free samples. First, we employ an encoding module to obtain multiscale features of the textured surface. Subsequently, a contrastive-learning-based memory feature module (CMFM) is proposed to obtain discriminative representations and construct a normal feature memory bank in the latent space, which can be employed as a substitute for defects and fast anomaly scores at the patch level. Next, a novel global feature rearrangement module (GFRM) is proposed to further suppress the reconstruction of residual defects. Finally, a decoding module utilizes the restored features to reconstruct the normal texture background. In addition, to improve inspection performance, a two-phase training strategy is utilized for accurate defect restoration refinement, and we exploit a multimodal inspection method to achieve noise-robust defect localization. We verify our method through extensive experiments and test its practical deployment in collaborative edge--cloud intelligent manufacturing scenarios by means of a multilevel detection method, demonstrating that FMR-Net exhibits state-of-the-art inspection accuracy and shows great potential for use in edge-computing-enabled smart industries.

cs.CV↗

arXiv2022

FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecasting

Although Transformer-based methods have significantly improved state-of-the-art results for long-term series forecasting, they are not only computationally expensive but more importantly, are unable to capture the global view of time series (e.g. overall trend). To address these problems, we propose to combine Transformer with the seasonal-trend decomposition method, in which the decomposition method captures the global profile of time series while Transformers capture more detailed structures. To further enhance the performance of Transformer for long-term prediction, we exploit the fact that most time series tend to have a sparse representation in well-known basis such as Fourier transform, and develop a frequency enhanced Transformer. Besides being more effective, the proposed method, termed as Frequency Enhanced Decomposed Transformer ({\bf FEDformer}), is more efficient than standard Transformer with a linear complexity to the sequence length. Our empirical studies with six benchmark datasets show that compared with state-of-the-art methods, FEDformer can reduce prediction error by $14.8\%$ and $22.6\%$ for multivariate and univariate time series, respectively. Code is publicly available at https://github.com/MAZiqing/FEDformer.

cs.LG↗

arXiv2022

Hierarchical contagions in the interdependent financial network

We derive the default cascade model and the fire-sale spillover model in a unified interdependent framework. The interactions among banks include not only direct cross-holding, but also indirect dependency by holding mutual assets outside the banking system. Using data extracted from the European Banking Authority, we present the interdependency network composed of 48 banks and 21 asset classes. For the robustness, we employ three methods, called $\textit{Anan}$, $\textit{Hała}$ and $\textit{Maxe}$, to reconstruct the asset/liability cross-holding network. Then we combine the external portfolio holdings of each bank to compute the interdependency matrix. The interdependency network is much denser than the direct cross-holding network, showing the complex latent interaction among banks. Finally, we perform macroprudential stress tests for the European banking system, using the adverse scenario in EBA stress test as the initial shock. For different reconstructed networks, we illustrate the hierarchical cascades and show that the failure hierarchies are roughly the same except for a few banks, reflecting the overlapping portfolio holding accounts for the majority of defaults. We also calculate systemic vulnerability and individual vulnerability, which provide important information for supervision and relevant management actions.

q-fin.RM↗

arXiv2022

Stereo Unstructured Magnification: Multiple Homography Image for View Synthesis

This paper studies the problem of view synthesis with certain amount of rotations from a pair of images, what we called stereo unstructured magnification. While the multi-plane image representation is well suited for view synthesis with depth invariant, how to generalize it to unstructured views remains a significant challenge. This is primarily due to the depth-dependency caused by camera frontal parallel representation. Here we propose a novel multiple homography image (MHI) representation, comprising of a set of scene planes with fixed normals and distances. A two-stage network is developed for novel view synthesis. Stage-1 is an MHI reconstruction module that predicts the MHIs and composites layered multi-normal images along the normal direction. Stage-2 is a normal-blending module to find blending weights. We also derive an angle-based cost to guide the blending of multi-normal images by exploiting per-normal geometry. Compared with the state-of-the-art methods, our method achieves superior performance for view synthesis qualitatively and quantitatively, especially for cases when the cameras undergo rotations.

cs.CV↗

arXiv2022

Epipolar Focus Spectrum: A Novel Light Field Representation and Application in Dense-view Reconstruction

Existing light field representations, such as epipolar plane image (EPI) and sub-aperture images, do not consider the structural characteristics across the views, so they usually require additional disparity and spatial structure cues for follow-up tasks. Besides, they have difficulties dealing with occlusions or larger disparity scenes. To this end, this paper proposes a novel Epipolar Focus Spectrum (EFS) representation by rearranging the EPI spectrum. Different from the classical EPI representation where an EPI line corresponds to a specific depth, there is a one-to-one mapping from the EFS line to the view. Accordingly, compared to a sparsely-sampled light field, a densely-sampled one with the same field of view (FoV) leads to a more compact distribution of such linear structures in the double-cone-shaped region with the identical opening angle in its corresponding EFS. Hence the EFS representation is invariant to the scene depth. To demonstrate its effectiveness, we develop a trainable EFS-based pipeline for light field reconstruction, where a dense light field can be reconstructed by compensating the "missing EFS lines" given a sparse light field, yielding promising results with cross-view consistency, especially in the presence of severe occlusion and large disparity. Experimental results on both synthetic and real-world datasets demonstrate the validity and superiority of the proposed method over SOTA methods.

eess.IV↗

arXiv2022

Time Series Data Augmentation for Deep Learning: A Survey

Deep learning performs remarkably well on many time series analysis tasks recently. The superior performance of deep neural networks relies heavily on a large number of training data to avoid overfitting. However, the labeled data of many real-world time series applications may be limited such as classification in medical time series and anomaly detection in AIOps. As an effective way to enhance the size and quality of the training data, data augmentation is crucial to the successful application of deep learning models on time series data. In this paper, we systematically review different data augmentation methods for time series. We propose a taxonomy for the reviewed methods, and then provide a structured review for these methods by highlighting their strengths and limitations. We also empirically compare different data augmentation methods for different tasks including time series classification, anomaly detection, and forecasting. Finally, we discuss and highlight five future directions to provide useful research guidance.

cs.LG↗

arXiv2022

Geometric structure guided model and algorithms for complete deconvolution of gene expression data

Complete deconvolution analysis for bulk RNAseq data is important and helpful to distinguish whether the difference of disease-associated GEPs (gene expression profiles) in tissues of patients and normal controls are due to changes in cellular composition of tissue samples, or due to GEPs changes in specific cells. One of the major techniques to perform complete deconvolution is nonnegative matrix factorization (NMF), which also has a wide-range of applications in the machine learning community. However, the NMF is a well-known strongly ill-posed problem, so a direct application of NMF to RNAseq data will suffer severe difficulties in the interpretability of solutions. In this paper we develop an NMF-based mathematical model and corresponding computational algorithms to improve the solution identifiability of deconvoluting bulk RNAseq data. In our approach, we combine the biological concept of marker genes with the solvability conditions of the NMF theories, and develop a geometric structured guided optimization model. In this strategy, the geometric structure of bulk tissue data is first explored by the spectral clustering technique. Then, the identified information of marker genes is integrated as solvability constraints, while the overall correlation graph is used as manifold regularization. Both synthetic and biological data are used to validate the proposed model and algorithms, from which solution interpretability and accuracy are significantly improved.

math.OC↗