arXiv subjects

Liang Chen

Publications and source records attributed to Liang Chen.

At least 361 records · Page 20Linked to original sources

arXiv2022

Enhanced DeepONet for Modeling Partial Differential Operators Considering Multiple Input Functions

Machine learning, especially deep learning is gaining much attention due to the breakthrough performance in various cognitive applications. Recently, neural networks (NN) have been intensively explored to model partial differential equations as NN can be viewed as universal approximators for nonlinear functions. A deep network operator (DeepONet) architecture was proposed to model the general non-linear continuous operators for partial differential equations (PDE) due to its better generalization capabilities than existing mainstream deep neural network architectures. However, existing DeepONet can only accept one input function, which limits its application. In this work, we explore the DeepONet architecture to extend it to accept two or more input functions. We propose new Enhanced DeepONet or EDeepONet high-level neural network structure, in which two input functions are represented by two branch DNN sub-networks, which are then connected with output truck network via inner product to generate the output of the whole neural network. The proposed EDeepONet structure can be easily extended to deal with multiple input functions. Our numerical results on modeling two partial differential equation examples shows that the proposed enhanced DeepONet is about 7X-17X or about one order of magnitude more accurate than the fully connected neural network and is about 2X-3X more accurate than a simple extended DeepONet for both training and test.

cs.LG↗

arXiv2022

Faithful learning with sure data for lung nodule diagnosis

Recent evolution in deep learning has proven its value for CT-based lung nodule classification. Most current techniques are intrinsically black-box systems, suffering from two generalizability issues in clinical practice. First, benign-malignant discrimination is often assessed by human observers without pathologic diagnoses at the nodule level. We termed these data as "unsure data". Second, a classifier does not necessarily acquire reliable nodule features for stable learning and robust prediction with patch-level labels during learning. In this study, we construct a sure dataset with pathologically-confirmed labels and propose a collaborative learning framework to facilitate sure nodule classification by integrating unsure data knowledge through nodule segmentation and malignancy score regression. A loss function is designed to learn reliable features by introducing interpretability constraints regulated with nodule segmentation maps. Furthermore, based on model inference results that reflect the understanding from both machine and experts, we explore a new nodule analysis method for similar historical nodule retrieval and interpretable diagnosis. Detailed experimental results demonstrate that our approach is beneficial for achieving improved performance coupled with faithful model reasoning for lung cancer prediction. Extensive cross-evaluation results further illustrate the effect of unsure data for deep-learning-based methods in lung nodule classification.

eess.IV↗

arXiv2022

Robust Dynamic State Estimator of Integrated Energy Systems based on Natural Gas Partial Differential Equations

The reliability and precision of dynamic database are vital for the optimal operating and global control of integrated energy systems. One of the effective ways to obtain the accurate states is state estimations. A novel robust dynamic state estimation methodology for integrated natural gas and electric power systems is proposed based on Kalman filter. To take full advantage of measurement redundancies and predictions for enhancing the estimating accuracy, the dynamic state estimation model coupling gas and power systems by gas turbine units is established. The exponential smoothing technique and gas physical model are integrated in Kalman filter. Additionally, the time-varying scalar matrix is proposed to conquer bad data in Kalman filter algorithm. The proposed method is applied to an integrated gas and power systems formed by GasLib-40 and IEEE 39-bus system with five gas turbine units. The simulating results show that the method can obtain the accurate dynamic states under three different measurement error conditions, and the filtering performance are better than separate estimation methods. Additionally, the proposed method is robust when the measurements experience bad data.

eess.SY↗

arXiv2022

Neighboring Backdoor Attacks on Graph Convolutional Network

Backdoor attacks have been widely studied to hide the misclassification rules in the normal models, which are only activated when the model is aware of the specific inputs (i.e., the trigger). However, despite their success in the conventional Euclidean space, there are few studies of backdoor attacks on graph structured data. In this paper, we propose a new type of backdoor which is specific to graph data, called neighboring backdoor. Considering the discreteness of graph data, how to effectively design the triggers while retaining the model accuracy on the original task is the major challenge. To address such a challenge, we set the trigger as a single node, and the backdoor is activated when the trigger node is connected to the target node. To preserve the model accuracy, the model parameters are not allowed to be modified. Thus, when the trigger node is not connected, the model performs normally. Under these settings, in this work, we focus on generating the features of the trigger node. Two types of backdoors are proposed: (1) Linear Graph Convolution Backdoor which finds an approximation solution for the feature generation (can be viewed as an integer programming problem) by looking at the linear part of GCNs. (2) Variants of existing graph attacks. We extend current gradient-based attack methods to our backdoor attack scenario. Extensive experiments on two social networks and two citation networks datasets demonstrate that all proposed backdoors can achieve an almost 100\% attack success rate while having no impact on predictive accuracy.

cs.AI↗

arXiv2021

Carrier Phase Ranging for Indoor Positioning with 5G NR Signals

Indoor positioning is one of the core technologies of Internet of Things (IoT) and artificial intelligence (AI), and is expected to play a significant role in the upcoming era of AI. However, affected by the complexity of indoor environments, it is still highly challenging to achieve continuous and reliable indoor positioning. Currently, 5G cellular networks are being deployed worldwide, the new technologies of which have brought the approaches for improving the performance of wireless indoor positioning. In this paper, we investigate the indoor positioning under the 5G new radio (NR), which has been standardized and being commercially operated in massive markets. Specifically, a solution is proposed and a software defined receiver (SDR) is developed for indoor positioning. With our SDR indoor positioning system, the 5G NR signals are firstly sampled by universal software radio peripheral (USRP), and then, coarse synchronization is achieved via detecting the start of the synchronization signal block (SSB). Then, with the assistance of the pilots transmitted on the physical broadcasting channel (PBCH), multipath acquisition and delay tracking are sequentially carried out to estimate the time of arrival (ToA) of received signals. Furthermore, to improve the ToA ranging accuracy, the carrier phase of the first arrived path is estimated. Finally, to quantify the accuracy of our ToA estimation method, indoor field tests are carried out in an office environment, where a 5G NR base station (known as gNB) is installed for commercial use. Our test results show that, in the static test scenarios, the ToA accuracy measured by the 1-σ error interval is about 0.5 m, while in the pedestrian mobile environment, the probability of range accuracy within 0.8 m is 95%.

eess.SP↗

arXiv2021

Structural engineering from an inverse problems perspective

The field of structural engineering is vast, spanning areas from the design of new infrastructure to the assessment of existing infrastructure. From the onset, traditional entry-level university courses teach students to analyse structural response given data including external forces, geometry, member sizes, restraint, etc. -- characterising a forward problem (structural causalities $\to$ structural response). Shortly thereafter, junior engineers are introduced to structural design where they aim to, for example, select an appropriate structural form for members based on design criteria, which is the inverse of what they previously learned. Similar inverse realisations also hold true in structural health monitoring and a number of structural engineering sub-fields (response $\to$ structural causalities). In this light, we aim to demonstrate that many structural engineering sub-fields may be fundamentally or partially viewed as inverse problems and thus benefit via the rich and established methodologies from the inverse problems community. To this end, we conclude that the future of inverse problems in structural engineering is inexorably linked to engineering education and machine learning developments.

math.OC↗

arXiv2021

Crossed-Time Delay Neural Network for Speaker Recognition

Time Delay Neural Network (TDNN) is a well-performing structure for DNN-based speaker recognition systems. In this paper we introduce a novel structure Crossed-Time Delay Neural Network (CTDNN) to enhance the performance of current TDNN. Inspired by the multi-filters setting of convolution layer from convolution neural network, we set multiple time delay units each with different context size at the bottom layer and construct a multilayer parallel network. The proposed CTDNN gives significant improvements over original TDNN on both speaker verification and identification tasks. It outperforms in VoxCeleb1 dataset in verification experiment with a 2.6% absolute Equal Error Rate improvement. In few shots condition CTDNN reaches 90.4% identification accuracy, which doubles the identification accuracy of original TDNN. We also compare the proposed CTDNN with another new variant of TDNN, FTDNN, which shows that our model has a 36% absolute identification accuracy improvement under few shots condition and can better handle training of a larger batch in a shorter training time, which better utilize the calculation resources. The code of the new model is released at https://github.com/chenllliang/CTDNN

eess.AS↗

arXiv2021

REMR: A Reliability Evaluation Method for Dynamic Edge Computing Network under Time Constraints

While the concept of Artificial Intelligent Internet of Things\ (AIoT) is booming, computation and/or communication-intensive tasks accompanied by several sub-tasks are slowly moving from centralized deployment to edge-side deployment. The idea of edge computing also makes intelligent services sink locally. But in actual scenarios like dynamic edge computing networks (DECN), due to fluctuations in available computing resources of intermediate servers and changes in bandwidth during data transmission, service reliability becomes difficult to guarantee. Coupled with changes in the amount of data in a service, the above three problems all make the existing reliability evaluation methods no longer accurate. To study the effect of distributed service deployment strategies under such a background, this paper proposes a reliability evaluation method (REMR) based on lower boundary rule under time constraint to study the degree of the rationality of a service deployment plan combined with DECN. In this scenario, time delay is the main concern which would be affected by three quantitative factors: data packet storing and sending time, data transmission time and the calculation time of executing sub-tasks on the node devices, specially while the last two are in dynamic scenarios. In actual calculation, based on the idea of the minimal paths, the solution set would to be found that can meet requirements in the current deployment. Then the reliability of the service supported by the solution sets would be found out based on the principle of inclusion-exclusion combined with the distribution of available data transmission bandwidth and the distribution of node available computing resources. Besides a illustrative example was provided, to verify the calculated reliability of the designed service deployment plan, the NS3 is utilized along with Google cluster data set for simulation.

cs.PF↗

arXiv2021

Perceiving and Modeling Density is All You Need for Image Dehazing

In the real world, the degradation of images taken under haze can be quite complex, where the spatial distribution of haze is varied from image to image. Recent methods adopt deep neural networks to recover clean scenes from hazy images directly. However, due to the paradox caused by the variation of real captured haze and the fixed degradation parameters of the current networks, the generalization ability of recent dehazing methods on real-world hazy images is not ideal.To address the problem of modeling real-world haze degradation, we propose to solve this problem by perceiving and modeling density for uneven haze distribution. We propose a novel Separable Hybrid Attention (SHA) module to encode haze density by capturing features in the orthogonal directions to achieve this goal. Moreover, a density map is proposed to model the uneven distribution of the haze explicitly. The density map generates positional encoding in a semi-supervised way. Such a haze density perceiving and modeling capture the unevenly distributed degeneration at the feature level effectively. Through a suitable combination of SHA and density map, we design a novel dehazing network architecture, which achieves a good complexity-performance trade-off. The extensive experiments on two large-scale datasets demonstrate that our method surpasses all state-of-the-art approaches by a large margin both quantitatively and qualitatively, boosting the best published PSNR metric from 28.53 dB to 33.49 dB on the Haze4k test dataset and from 37.17 dB to 38.41 dB on the SOTS indoor test dataset.

cs.CV↗

arXiv2021

Peta-electron volt gamma-ray emission from the Crab Nebula

The Crab pulsar and the surrounding nebula powered by the pulsar's rotational energy through the formation and termination of a relativistic electron-positron wind is a bright source of gamma-rays carrying crucial information about this complex conglomerate. We report the detection of $γ$-rays with a spectrum showing gradual steepening over three energy decades, from $5\times 10^{-4}$ to $1.1$ petaelectronvolt (PeV). The ultra-high-energy photons exhibit the presence of a PeV electron accelerator (a pevatron) with an acceleration rate exceeding 15% of the absolute theoretical limit. Assuming that unpulsed $γ$-rays are produced at the termination of the pulsar's wind, we constrain the pevatron's size, between $0.025$ and $0.1$ pc, and the magnetic field $\approx 110 μ$G. The production rate of PeV electrons, $2.5 \times 10^{36}$ erg $\rm s^{-1}$, constitutes 0.5% of the pulsar's spin-down luminosity, although we do not exclude a non-negligible contribution of PeV protons to the production of the highest energy $γ$-rays.

astro-ph.HE↗

arXiv2021

AutoDebias: Learning to Debias for Recommendation

Recommender systems rely on user behavior data like ratings and clicks to build personalization model. However, the collected data is observational rather than experimental, causing various biases in the data which significantly affect the learned model. Most existing work for recommendation debiasing, such as the inverse propensity scoring and imputation approaches, focuses on one or two specific biases, lacking the universal capacity that can account for mixed or even unknown biases in the data. Towards this research gap, we first analyze the origin of biases from the perspective of \textit{risk discrepancy} that represents the difference between the expectation empirical risk and the true risk. Remarkably, we derive a general learning framework that well summarizes most existing debiasing strategies by specifying some parameters of the general framework. This provides a valuable opportunity to develop a universal solution for debiasing, e.g., by learning the debiasing parameters from data. However, the training data lacks important signal of how the data is biased and what the unbiased data looks like. To move this idea forward, we propose \textit{AotoDebias} that leverages another (small) set of uniform data to optimize the debiasing parameters by solving the bi-level optimization problem with meta-learning. Through theoretical analyses, we derive the generalization bound for AutoDebias and prove its ability to acquire the appropriate debiasing strategy. Extensive experiments on two real datasets and a simulated dataset demonstrated effectiveness of AutoDebias. The code is available at \url{https://github.com/DongHande/AutoDebias}.

cs.LG↗

arXiv2021

Physics-Enforced Modeling for Insertion Loss of Transmission Lines by Deep Neural Networks

In this paper, we investigate data-driven parameterized modeling of insertion loss for transmission lines with respect to design parameters. We first show that direct application of neural networks can lead to non-physics models with negative insertion loss. To mitigate this problem, we propose two deep learning solutions. One solution is to add a regulation term, which represents the passive condition, to the final loss function to enforce the negative quantity of insertion loss. In the second method, a third-order polynomial expression is defined first, which ensures positiveness, to approximate the insertion loss, then DeepONet neural network structure, which was proposed recently for function and system modeling, was employed to model the coefficients of polynomials. The resulting neural network is applied to predict the coefficients of the polynomial expression. The experimental results on an open-sourced SI/PI database of a PCB design show that both methods can ensure the positiveness for the insertion loss. Furthermore, both methods can achieve similar prediction results, while the polynomial-based DeepONet method is faster than DeepONet based method in training time.

cs.LG↗

arXiv2021

SDWNet: A Straight Dilated Network with Wavelet Transformation for Image Deblurring

Image deblurring is a classical computer vision problem that aims to recover a sharp image from a blurred image. To solve this problem, existing methods apply the Encode-Decode architecture to design the complex networks to make a good performance. However, most of these methods use repeated up-sampling and down-sampling structures to expand the receptive field, which results in texture information loss during the sampling process and some of them design the multiple stages that lead to difficulties with convergence. Therefore, our model uses dilated convolution to enable the obtainment of the large receptive field with high spatial resolution. Through making full use of the different receptive fields, our method can achieve better performance. On this basis, we reduce the number of up-sampling and down-sampling and design a simple network structure. Besides, we propose a novel module using the wavelet transform, which effectively helps the network to recover clear high-frequency texture details. Qualitative and quantitative evaluations of real and synthetic datasets show that our deblurring method is comparable to existing algorithms in terms of performance with much lower training requirements. The source code and pre-trained models are available at https://github.com/FlyEgle/SDWNet.

eess.IV↗

arXiv2021

Phonon-laser ultrasensitive force sensor

Developing nano-mechanical oscillators for ultrasensitive force detection is of great importance in exploring science. We report our achievement of ultrasensitive detection of the external force regarding the radio-frequency electric field by a nano-sensor made of a single trapped $^{40}$Ca$^{+}$ ion under injection-locking, where squeezing is additionally applied to detection of the smallest force in the ion trap. The employed ion is confined stably in a surface electrode trap and works as a phonon laser that is very sensitive to the external disturbance. The injection-locking drove the ion's oscillation with phase synchronization, yielding the force detection with sensitivity of 347 $\pm$ 50 yN/$\sqrt{Hz}$. Further with 3 dB squeezing applied on the oscillation phase variance, we achieved a successful detection of the smallest force to be 86.5 $\pm$ 70.1 yN.

quant-ph↗

arXiv2021

A Unified Convergence Analysis of a Second-Order Method of Multipliers for Nonlinear Conic Programming

In this paper, we accomplish a unified convergence analysis of a second-order method of multipliers (i.e., a second-order augmented Lagrangian method) for solving the conventional nonlinear conic optimization problems.Specifically, the algorithm that we investigated incorporates a specially designed nonsmooth (generalized) Newton step to furnish a second-order update rule for the multipliers.We first show in a unified fashion that under a few abstract assumptions, the proposed method is locally convergent and possesses a (nonasymptotic) superlinear convergence rate, even though the penalty parameter is fixed and/or the strict complementarity fails.Subsequently, we demonstrate that, for the three typical scenarios, i.e., the classic nonlinear programming, the nonlinear second-order cone programming, and the nonlinear semidefinite programming, these abstract assumptions are nothing but exactly the implications of the iconic sufficient conditions that were assumed for establishing the Q-linear convergence rates of the method of multipliers without assuming the strict complementarity.

math.OC↗

arXiv2021

Neural Architecture Search based on Cartesian Genetic Programming Coding Method

Neural architecture search (NAS) is a hot topic in the field of automated machine learning and outperforms humans in designing neural architectures on quite a few machine learning tasks. Motivated by the natural representation form of neural networks by the Cartesian genetic programming (CGP), we propose an evolutionary approach of NAS based on CGP, called CGPNAS, to solve sentence classification task. To evolve the architectures under the framework of CGP, the operations such as convolution are identified as the types of function nodes of CGP, and the evolutionary operations are designed based on Evolutionary Strategy. The experimental results show that the searched architectures are comparable with the performance of human-designed architectures. We verify the ability of domain transfer of our evolved architectures. The transfer experimental results show that the accuracy deterioration is lower than 2-5%. Finally, the ablation study identifies the Attention function as the single key function node and the linear transformations along could keep the accuracy similar with the full evolved architectures, which is worthy of investigation in the future.

cs.NE↗

arXiv2021

Understanding Structural Vulnerability in Graph Convolutional Networks

Recent studies have shown that Graph Convolutional Networks (GCNs) are vulnerable to adversarial attacks on the graph structure. Although multiple works have been proposed to improve their robustness against such structural adversarial attacks, the reasons for the success of the attacks remain unclear. In this work, we theoretically and empirically demonstrate that structural adversarial examples can be attributed to the non-robust aggregation scheme (i.e., the weighted mean) of GCNs. Specifically, our analysis takes advantage of the breakdown point which can quantitatively measure the robustness of aggregation schemes. The key insight is that weighted mean, as the basic design of GCNs, has a low breakdown point and its output can be dramatically changed by injecting a single edge. We show that adopting the aggregation scheme with a high breakdown point (e.g., median or trimmed mean) could significantly enhance the robustness of GCNs against structural attacks. Extensive experiments on four real-world datasets demonstrate that such a simple but effective method achieves the best robustness performance compared to state-of-the-art models.

cs.LG↗

arXiv2021

Statistical computation methods for microbiome compositional data network inference

Microbes can affect processes from food production to human health. Such microbes are not isolated, but rather interact with each other and establish connections with their living environments. Understanding these interactions is essential to an understanding of the organization and complex interplay of microbial communities, as well as the structure and dynamics of various ecosystems. A common and essential approach toward this objective involves the inference of microbiome interaction networks. Although network inference methods in other fields have been studied before, applying these methods to estimate microbiome associations based on compositional data will not yield valid results. On the one hand, features of microbiome data such as compositionality, sparsity and high-dimensionality challenge the data normalization and the design of computational methods. On the other hand, several issues like microbial community heterogeneity, external environmental interference and biological concerns also make it more difficult to deal with the network inference. In this paper, we provide a comprehensive review of emerging microbiome interaction network inference methods. According to various assumptions and research targets, estimated networks are divided into four main categories: correlation networks, conditional correlation networks, mixture networks and differential networks. Their scope of applications, advantages and limitations are presented in this review. Since real microbial interactions can be complex and dynamic, no unifying method has captured all the aspects of interest to date. In addition, we discuss the challenges now confronting current microbial associations study and future prospects. Finally, we highlight that the research in microbial network inference requires the joint promotion of statistical computation methods and experimental techniques.

stat.AP↗