arXiv subjects

Xueshuang Xiang

Publications and source records attributed to Xueshuang Xiang.

At least 19 recordsLinked to original sources

arXiv2025

A Kolmogorov High Order Deep Neural Network for High Frequency Partial Differential Equations in High Dimensions

This paper proposes a Kolmogorov high order deep neural network (K-HOrderDNN) for solving high-dimensional partial differential equations (PDEs), which improves the high order deep neural networks (HOrderDNNs). HOrderDNNs have been demonstrated to outperform conventional DNNs for high frequency problems by introducing a nonlinear transformation layer consisting of $(p+1)^d$ basis functions. However, the number of basis functions grows exponentially with the dimension $d$, which results in the curse of dimensionality (CoD). Inspired by the Kolmogorov superposition theorem (KST), which expresses a multivariate function as superpositions of univariate functions and addition, K-HOrderDNN utilizes a HOrderDNN to efficiently approximate univariate inner functions instead of directly approximating the multivariate function, reducing the number of introduced basis functions to $d(p+1)$. We theoretically demonstrate that CoD is mitigated when target functions belong to a dense subset of continuous multivariate functions. Extensive numerical experiments show that: for high-dimensional problems ($d$=10, 20, 50) where HOrderDNNs($p>1$) are intractable, K-HOrderDNNs($p>1$) exhibit remarkable performance. Specifically, when $d=10$, K-HOrderDNN($p=7$) achieves an error of 4.40E-03, two orders of magnitude lower than that of HOrderDNN($p=1$) (see Table 10); for high frequency problems, K-HOrderDNNs($p>1$) can achieve higher accuracy with fewer parameters and faster convergence rates compared to HOrderDNNs (see Table 8).

math.NA↗

arXiv2024

Adaptive Basis-inspired Deep Neural Network for Solving Partial Differential Equations with Localized Features

This paper proposes an Adaptive Basis-inspired Deep Neural Network (ABI-DNN) for solving partial differential equations with localized phenomena such as sharp gradients and singularities. Like the adaptive finite element method, ABI-DNN incorporates an iteration of "solve, estimate, mark, enhancement", which automatically identifies challenging regions and adds new neurons to enhance its capability. A key challenge is to force new neurons to focus on identified regions with limited understanding of their roles in approximation. To address this, we draw inspiration from the finite element basis function and construct the novel Basis-inspired Block (BI-block), to help understand the contribution of each block. With the help of the BI-block and the famous Kolmogorov Superposition Theorem, we first develop a novel fixed network architecture named the Basis-inspired Deep Neural Network (BI-DNN), and then integrate it into the aforementioned adaptive framework to propose the ABI-DNN. Extensive numerical experiments demonstrate that both BI-DNN and ABI-DNN can effectively capture the challenging singularities in target functions. Compared to PINN, BI-DNN attains significantly lower relative errors with a similar number of trainable parameters. When a specified tolerance is set, ABI-DNN can adaptively learn an appropriate architecture that achieves an error comparable to that of BI-DNN with the same structure.

math.NA↗

arXiv2024

A high order unfitted finite element method for time-Harmonic Maxwell interface problems

We propose a high order unfitted finite element method for solving timeharmonic Maxwell interface problems. The unfitted finite element method is based on a mixed formulation in the discontinuous Galerkin framework on a Cartesian mesh with possible hanging nodes. The $H^2$ regularity of the solution to Maxwell interface problems with $C^2$ interfaces in each subdomain is proved. Practical interface-resolving mesh conditions are introduced under which the hp inverse estimates on three-dimensional curved domains are proved. Stability and hp a priori error estimate of the unfitted finite element method are proved. Numerical results are included to illustrate the performance of the method.

math.NA↗

arXiv2023

A high order explicit time finite element method for the acoustic wave equation with discontinuous coefficients

In this paper, we propose a novel high order unfitted finite element method on Cartesian meshes for solving the acoustic wave equation with discontinuous coefficients having complex interface geometry. The unfitted finite element method does not require any penalty to achieve optimal convergence. We also introduce a new explicit time discretization method for the ODE system resulting from the spatial discretization of the wave equation. The strong stability and optimal $hp$-version error estimates both in time and space are established. Numerical examples confirm our theoretical results.

math.NA↗

arXiv2022

Improving the Deconvolution of Spectrum at Finite Temperature via Neural Network

In the study of condensed matter physics, spectral information plays an important role for understand the mechanism of materials. However, it is difficult to obtain the spectrum directly through experiments or simulation. For example, the spectral information deconvoluted by scanning tunneling spectroscopy suffers from the temperature broadening effect, which is ill-posed and makes the deconvolution result unstable. To solve this problem, the core idea of existing methods, such as the maximum entropy method, tends to select appropriate regularization to suppress unstable oscillations. However, the choice of regularization is difficult, and the oscillation has not been completely eliminated. We think non-uniform sampling is the core improvement direction, combined with stochastic optimization and deep learning, we introduce a neural network based discretization scheme to solve the deconvolution problem. Due to the neural network can represent any piece-wise linear function, our method replace the target spectrum by network and can find a better approximation solution through optimization accurate and efficient. Experiments on theoretical datasets about superconductors demonstrate that the gap is estimated to be more accurate and oscillating less, plugin real experimental data, our approach can get clearer results for material analysis.

physics.comp-ph↗

arXiv2022

Adversarial Jamming for a More Effective Constellation Attack

The common jamming mode in wireless communication is band barrage jamming, which is controllable and difficult to resist. Although this method is simple to implement, it is obviously not the best jamming waveform. Therefore, based on the idea of adversarial examples, we propose the adversarial jamming waveform, which can independently optimize and find the best jamming waveform. We attack QAM with adversarial jamming and find that the optimal jamming waveform is equivalent to the amplitude and phase between the nearest constellation points. Furthermore, by verifying the jamming performance on a hardware platform, it is shown that our method significantly improves the bit error rate compared to other methods.

cs.CR↗

arXiv2022

Low-Interception Waveform: To Prevent the Recognition of Spectrum Waveform Modulation via Adversarial Examples

Deep learning is applied to many complex tasks in the field of wireless communication, such as modulation recognition of spectrum waveforms, because of its convenience and efficiency. This leads to the problem of a malicious third party using a deep learning model to easily recognize the modulation format of the transmitted waveform. Some existing works address this problem directly using the concept of adversarial examples in the image domain without fully considering the characteristics of the waveform transmission in the physical world. Therefore, we propose a low-intercept waveform~(LIW) generation method that can reduce the probability of the modulation being recognized by a third party without affecting the reliable communication of the friendly party. Our LIW exhibits significant low-interception performance even in the physical hardware experiment, decreasing the accuracy of the state of the art model to approximately $15\%$ with small perturbations.

cs.LG↗

arXiv2021

An adaptive high-order unfitted finite element method for elliptic interface problems

We design an adaptive unfitted finite element method on the Cartesian mesh with hanging nodes. We derive an hp-reliable and efficient residual type a posteriori error estimate on K-meshes. A key ingredient is a novel hp-domain inverse estimate which allows us to prove the stability of the finite element method under practical interface resolving mesh conditions and also prove the lower bound of the hp a posteriori error estimate. Numerical examples are included.

math.NA↗

arXiv2021

The QXS-SAROPT Dataset for Deep Learning in SAR-Optical Data Fusion

Deep learning techniques have made an increasing impact on the field of remote sensing. However, deep neural networks based fusion of multimodal data from different remote sensors with heterogenous characteristics has not been fully explored, due to the lack of availability of big amounts of perfectly aligned multi-sensor image data with diverse scenes of high resolutions, especially for synthetic aperture radar (SAR) data and optical imagery. To promote the development of deep learning based SAR-optical fusion approaches, we release the QXS-SAROPT dataset, which contains 20,000 pairs of SAR-optical image patches. We obtain the SAR patches from SAR satellite GaoFen-3 images and the optical patches from Google Earth images. These images cover three port cities: San Diego, Shanghai and Qingdao. Here, we present a detailed introduction of the construction of the dataset, and show its two representative exemplary applications, namely SAR-optical image matching and SAR ship detection boosted by cross-modal information from optical images. As a large open SAR-optical dataset with multiple scenes of a high resolution, we believe QXS-SAROPT will be of potential value for further research in SAR-optical data fusion technology based on deep learning.

eess.IV↗

arXiv2021

Adversarial YOLO: Defense Human Detection Patch Attacks via Detecting Adversarial Patches

The security of object detection systems has attracted increasing attention, especially when facing adversarial patch attacks. Since patch attacks change the pixels in a restricted area on objects, they are easy to implement in the physical world, especially for attacking human detection systems. The existing defenses against patch attacks are mostly applied for image classification problems and have difficulty resisting human detection attacks. Towards this critical issue, we propose an efficient and effective plug-in defense component on the YOLO detection system, which we name Ad-YOLO. The main idea is to add a patch class on the YOLO architecture, which has a negligible inference increment. Thus, Ad-YOLO is expected to directly detect both the objects of interest and adversarial patches. To the best of our knowledge, our approach is the first defense strategy against human detection attacks. We investigate Ad-YOLO's performance on the YOLOv2 baseline. To improve the ability of Ad-YOLO to detect variety patches, we first use an adversarial training process to develop a patch dataset based on the Inria dataset, which we name Inria-Patch. Then, we train Ad-YOLO by a combination of Pascal VOC, Inria, and Inria-Patch datasets. With a slight drop of $0.70\%$ mAP on VOC 2007 test set, Ad-YOLO achieves $80.31\%$ AP of persons, which highly outperforms $33.93\%$ AP for YOLOv2 when facing white-box patch attacks. Furthermore, compared with YOLOv2, the results facing a physical-world attack are also included to demonstrate Ad-YOLO's excellent generalization ability.

cs.CV↗

arXiv2021

Boosting ship detection in SAR images with complementary pretraining techniques

Deep learning methods have made significant progress in ship detection in synthetic aperture radar (SAR) images. The pretraining technique is usually adopted to support deep neural networks-based SAR ship detectors due to the scarce labeled SAR images. However, directly leveraging ImageNet pretraining is hardly to obtain a good ship detector because of different imaging perspective and geometry. In this paper, to resolve the problem of inconsistent imaging perspective between ImageNet and earth observations, we propose an optical ship detector (OSD) pretraining technique, which transfers the characteristics of ships in earth observations to SAR images from a large-scale aerial image dataset. On the other hand, to handle the problem of different imaging geometry between optical and SAR images, we propose an optical-SAR matching (OSM) pretraining technique, which transfers plentiful texture features from optical images to SAR images by common representation learning on the optical-SAR matching task. Finally, observing that the OSD pretraining based SAR ship detector has a better recall on sea area while the OSM pretraining based SAR ship detector can reduce false alarms on land area, we combine the predictions of the two detectors through weighted boxes fusion to further improve detection results. Extensive experiments on four SAR ship detection datasets and two representative CNN-based detection benchmarks are conducted to show the effectiveness and complementarity of the two proposed detectors, and the state-of-the-art performance of the combination of the two detectors. The proposed method won the sixth place of ship detection in SAR images in 2020 Gaofen challenge.

cs.CV↗

arXiv2020

A Neural Network with Plane Wave Activation for Helmholtz Equation

This paper proposes a plane wave activation based neural network (PWNN) for solving Helmholtz equation, the basic partial differential equation to represent wave propagation, e.g. acoustic wave, electromagnetic wave, and seismic wave. Unlike using traditional activation based neural network (TANN) or $sin$ activation based neural network (SIREN) for solving general partial differential equations, we instead introduce a complex activation function $e^{\mathbf{i}{x}}$, the plane wave which is the basic component of the solution of Helmholtz equation. By a simple derivation, we further find that PWNN is actually a generalization of the plane wave partition of unity method (PWPUM) by additionally imposing a learned basis with both amplitude and direction to better characterize the potential solution. We firstly investigate our performance on a problem with the solution is an integral of the plane waves with all known directions. The experiments demonstrate that: PWNN works much better than TANN and SIREN on varying architectures or the number of training samples, that means the plane wave activation indeed helps to enhance the representation ability of neural network toward the solution of Helmholtz equation; PWNN has competitive performance than PWPUM, e.g. the same convergence order but less relative error. Furthermore, we focus a more practical problem, the solution of which only integrate the plane waves with some unknown directions. We find that PWNN works much better than PWPUM at this case. Unlike using the plane wave basis with fixed directions in PWPUM, PWNN can learn a group of optimized plane wave basis which can better predict the unknown directions of the solution. The proposed approach may provide some new insights in the aspect of applying deep learning in Helmholtz equation.

math.NA↗

arXiv2020

Towards GANs' Approximation Ability

Generative adversarial networks (GANs) have attracted intense interest in the field of generative models. However, few investigations focusing either on the theoretical analysis or on algorithm design for the approximation ability of the generator of GANs have been reported. This paper will first theoretically analyze GANs' approximation property. Similar to the universal approximation property of the fully connected neural networks with one hidden layer, we prove that the generator with the input latent variable in GANs can universally approximate the potential data distribution given the increasing hidden neurons. Furthermore, we propose an approach named stochastic data generation (SDG) to enhance GANs'approximation ability. Our approach is based on the simple idea of imposing randomness through data generation in GANs by a prior distribution on the conditional probability between the layers. SDG approach can be easily implemented by using the reparameterization trick. The experimental results on synthetic dataset verify the improved approximation ability obtained by this SDG approach. In the practical dataset, four GANs using SDG can also outperform the corresponding traditional GANs when the model architectures are smaller.

cs.LG↗

arXiv2020

How Does GAN-based Semi-supervised Learning Work?

Generative adversarial networks (GANs) have been widely used and have achieved competitive results in semi-supervised learning. This paper theoretically analyzes how GAN-based semi-supervised learning (GAN-SSL) works. We first prove that, given a fixed generator, optimizing the discriminator of GAN-SSL is equivalent to optimizing that of supervised learning. Thus, the optimal discriminator in GAN-SSL is expected to be perfect on labeled data. Then, if the perfect discriminator can further cause the optimization objective to reach its theoretical maximum, the optimal generator will match the true data distribution. Since it is impossible to reach the theoretical maximum in practice, one cannot expect to obtain a perfect generator for generating data, which is apparently different from the objective of GANs. Furthermore, if the labeled data can traverse all connected subdomains of the data manifold, which is reasonable in semi-supervised classification, we additionally expect the optimal discriminator in GAN-SSL to also be perfect on unlabeled data. In conclusion, the minimax optimization in GAN-SSL will theoretically output a perfect discriminator on both labeled and unlabeled data by unexpectedly learning an imperfect generator, i.e., GAN-SSL can effectively improve the generalization ability of the discriminator by leveraging unlabeled information.

stat.ML↗

arXiv2020

Stochastic Model Pruning via Weight Dropping Away and Back

Deep neural networks have dramatically achieved great success on a variety of challenging tasks. However, most successful DNNs have an extremely complex structure, leading to extensive research on model compression.As a significant area of progress in model compression, traditional gradual pruning approaches involve an iterative prune-retrain procedure and may suffer from two critical issues: local importance judgment, where the pruned weights are merely unimportant in the current model; and an irretrievable pruning process, where the pruned weights have no chance to come back. Addressing these two issues, this paper proposes the Drop Pruning approach, which leverages stochastic optimization in the pruning process by introducing a drop strategy at each pruning step, namely, drop away, which stochastically deletes some unimportant weights, and drop back, which stochastically recovers some pruned weights. The suitable choice of drop probabilities decreases the model size during the pruning process and helps it flow to the target sparsity. Compared to the Bayesian approaches that stochastically train a compact model for pruning, we directly aim at stochastic gradual pruning. We provide a detailed analysis showing that the drop away and drop back approaches have individual contributions. Moreover, Drop Pruning can achieve competitive compression performance and accuracy on many benchmark tasks compared with state-of-the-art weights pruning and Bayesian training approaches.

cs.LG↗

arXiv2020

Deep Domain Decomposition Method: Elliptic Problems

This paper proposes a deep-learning-based domain decomposition method (DeepDDM), which leverages deep neural networks (DNN) to discretize the subproblems divided by domain decomposition methods (DDM) for solving partial differential equations (PDE). Using DNN to solve PDE is a physics-informed learning problem with the objective involving two terms, domain term and boundary term, which respectively make the desired solution satisfy the PDE and corresponding boundary conditions. DeepDDM will exchange the subproblem information across the interface in DDM by adjusting the boundary term for solving each subproblem by DNN. Benefiting from the simple implementation and mesh-free strategy of using DNN for PDE, DeepDDM will simplify the implementation of DDM and make DDM more flexible for complex PDE, e.g., those with complex interfaces in the computational domain. This paper will firstly investigate the performance of using DeepDDM for elliptic problems, including a model problem and an interface problem. The numerical examples demonstrate that DeepDDM exhibits behaviors consistent with conventional DDM: the number of iterations by DeepDDM is independent of network architecture and decreases with increasing overlapping size. The performance of DeepDDM on elliptic problems will encourage us to further investigate its performance for other kinds of PDE and may provide new insights for improving the PDE solver by deep learning.

math.NA↗

arXiv2020

Training few-shot classification via the perspective of minibatch and pretraining

Few-shot classification is a challenging task which aims to formulate the ability of humans to learn concepts from limited prior data and has drawn considerable attention in machine learning. Recent progress in few-shot classification has featured meta-learning, in which a parameterized model for a learning algorithm is defined and trained to learn the ability of handling classification tasks on extremely large or infinite episodes representing different classification task, each with a small labeled support set and its corresponding query set. In this work, we advance this few-shot classification paradigm by formulating it as a supervised classification learning problem. We further propose multi-episode and cross-way training techniques, which respectively correspond to the minibatch and pretraining in classification problems. Experimental results on a state-of-the-art few-shot classification method (prototypical networks) demonstrate that both the proposed training strategies can highly accelerate the training process without accuracy loss for varying few-shot classification problems on Omniglot and miniImageNet.

cs.LG↗

arXiv2020

Blind Adversarial Training: Balance Accuracy and Robustness

Adversarial training (AT) aims to improve the robustness of deep learning models by mixing clean data and adversarial examples (AEs). Most existing AT approaches can be grouped into restricted and unrestricted approaches. Restricted AT requires a prescribed uniform budget to constrain the magnitude of the AE perturbations during training, with the obtained results showing high sensitivity to the budget. On the other hand, unrestricted AT uses unconstrained AEs, resulting in the use of AEs located beyond the decision boundary; these overestimated AEs significantly lower the accuracy on clean data. These limitations mean that the existing AT approaches have difficulty in obtaining a comprehensively robust model with high accuracy and robustness when confronting attacks with varying strengths. Considering this problem, this paper proposes a novel AT approach named blind adversarial training (BAT) to better balance the accuracy and robustness. The main idea of this approach is to use a cutoff-scale strategy to adaptively estimate a nonuniform budget to modify the AEs used in the training, ensuring that the strengths of the AEs are dynamically located in a reasonable range and ultimately improving the overall robustness of the AT model. The experimental results obtained using BAT for training classification models on several benchmarks demonstrate the competitive performance of this method.

cs.LG↗