arXiv subjects

Jun Wang

Publications and source records attributed to Jun Wang.

At least 973 records · Page 54Linked to original sources

arXiv2021

Learning the Implicit Semantic Representation on Graph-Structured Data

Existing representation learning methods in graph convolutional networks are mainly designed by describing the neighborhood of each node as a perceptual whole, while the implicit semantic associations behind highly complex interactions of graphs are largely unexploited. In this paper, we propose a Semantic Graph Convolutional Networks (SGCN) that explores the implicit semantics by learning latent semantic-paths in graphs. In previous work, there are explorations of graph semantics via meta-paths. However, these methods mainly rely on explicit heterogeneous information that is hard to be obtained in a large amount of graph-structured data. SGCN first breaks through this restriction via leveraging the semantic-paths dynamically and automatically during the node aggregating process. To evaluate our idea, we conduct sufficient experiments on several standard datasets, and the empirical results show the superior performance of our model.

cs.AI↗

arXiv2021

Efficient Semi-Implicit Variational Inference

In this paper, we propose CI-VI an efficient and scalable solver for semi-implicit variational inference (SIVI). Our method, first, maps SIVI's evidence lower bound (ELBO) to a form involving a nonlinear functional nesting of expected values and then develops a rigorous optimiser capable of correctly handling bias inherent to nonlinear nested expectations using an extrapolation-smoothing mechanism coupled with gradient sketching. Our theoretical results demonstrate convergence to a stationary point of the ELBO in general non-convex settings typically arising when using deep network models and an order of $O(t^{-\frac{4}{5}})$ gradient-bias-vanishing rate. We believe these results generalise beyond the specific nesting arising from SIVI to other forms. Finally, in a set of experiments, we demonstrate the effectiveness of our algorithm in approximating complex posteriors on various data-sets including those from natural language processing.

cs.LG↗

arXiv2021

Effective Low-Cost Time-Domain Audio Separation Using Globally Attentive Locally Recurrent Networks

Recent research on the time-domain audio separation networks (TasNets) has brought great success to speech separation. Nevertheless, conventional TasNets struggle to satisfy the memory and latency constraints in industrial applications. In this regard, we design a low-cost high-performance architecture, namely, globally attentive locally recurrent (GALR) network. Alike the dual-path RNN (DPRNN), we first split a feature sequence into 2D segments and then process the sequence along both the intra- and inter-segment dimensions. Our main innovation lies in that, on top of features recurrently processed along the inter-segment dimensions, GALR applies a self-attention mechanism to the sequence along the inter-segment dimension, which aggregates context-aware information and also enables parallelization. Our experiments suggest that GALR is a notably more effective network than the prior work. On one hand, with only 1.5M parameters, it has achieved comparable separation performance at a much lower cost with 36.1% less runtime memory and 49.4% fewer computational operations, relative to the DPRNN. On the other hand, in a comparable model size with DPRNN, GALR has consistently outperformed DPRNN in three datasets, in particular, with a substantial margin of 2.4dB absolute improvement of SI-SNRi in the benchmark WSJ0-2mix task.

eess.AS↗

arXiv2021

Multiscale Attention Guided Network for COVID-19 Diagnosis Using Chest X-ray Images

Coronavirus disease 2019 (COVID-19) is one of the most destructive pandemic after millennium, forcing the world to tackle a health crisis. Automated lung infections classification using chest X-ray (CXR) images could strengthen diagnostic capability when handling COVID-19. However, classifying COVID-19 from pneumonia cases using CXR image is a difficult task because of shared spatial characteristics, high feature variation and contrast diversity between cases. Moreover, massive data collection is impractical for a newly emerged disease, which limited the performance of data thirsty deep learning models. To address these challenges, Multiscale Attention Guided deep network with Soft Distance regularization (MAG-SD) is proposed to automatically classify COVID-19 from pneumonia CXR images. In MAG-SD, MA-Net is used to produce prediction vector and attention from multiscale feature maps. To improve the robustness of trained model and relieve the shortage of training data, attention guided augmentations along with a soft distance regularization are posed, which aims at generating meaningful augmentations and reduce noise. Our multiscale attention model achieves better classification performance on our pneumonia CXR image dataset. Plentiful experiments are proposed for MAG-SD which demonstrates its unique advantage in pneumonia classification over cutting-edge models. The code is available at https://github.com/JasonLeeGHub/MAG-SD.

eess.IV↗

arXiv2021

Facile preparation of CuCo$_2$S$_4$/Cu$_7$S$_4$ nanocomposites as high-performance cathode materials for rechargeable magnesium batteries

Searching for efficient and high-performance cathode materials for rechargeable magnesium ion batteries (RMBs) is urgent for exploring sustainable energy technologies. However, the majority of cathode materials for RMBs usually suffer from low rate capacity and inferior cycle performance caused by structure collapse. Herein, ternary CuCo$_2$S$_4$/Cu$_7$S$_4$ composites with nano-sized are first synthesized by a facile solvothermal method in this work and the magnesium ion storage behavior is discussed when applied in the cathode for RMBs. Electrochemical results demonstrate that the nanosphere-like CuCo$_2$S$_4$/Cu$_7$S$_4$ composites exhibit a high initial discharge capacity of 256 mAh/g at 10 mA/g and 123 mAh/g at 300 mA/g at room temperature. Furthermore, an outstanding long-term cyclic stability with a reversible capacity of 106 mA/g after 300 cycles and the coulombic efficiency of about 99% are achieved at 300 mA/g. Such remarkable electrochemical performance is owing to the nanosphere-like structure, ternary composition and pseudocapacitive storage behavior of CuCo$_2$S$_4$/Cu$_7$S$_4$. The results of this work indicate that the CuCo$_2$S$_4$/Cu$_7$S$_4$ nanocomposites synthesized by a facile and efficient method are promising cathodes for RMBs in energy storage/conversion application.

cond-mat.mtrl-sci↗

arXiv2020

Time irreversibility and amplitude irreversibility measures for nonequilibrium processes

Time irreversibility, which characterizes nonequilibrium processes, can be measured based on the probabilistic differences between symmetric vectors. To simplify the quantification of time irreversibility, symmetric permutations instead of symmetric vectors have been employed in some studies. However, although effective in practical applications, this approach is conceptually incorrect. Time irreversibility should be measured based on the permutations of symmetric vectors rather than symmetric permutations, whereas symmetric permutations can instead be employed to determine the quantitative amplitude irreversibility -- a novel parameter proposed in this paper for nonequilibrium calculated by means of the probabilistic difference in amplitude fluctuations. Through theoretical and experimental analyses, we highlight the strong similarities and close associations between the time irreversibility and amplitude irreversibility measures. Our paper clarifies the connections of and the differences between the two types of permutation-based parameters for quantitative nonequilibrium, and by doing so, we bridge the concepts of amplitude irreversibility and time irreversibility and broaden the selection of quantitative tools for studying nonequilibrium processes in complex systems.

eess.SP↗

arXiv2020

Causal World Models by Unsupervised Deconfounding of Physical Dynamics

The capability of imagining internally with a mental model of the world is vitally important for human cognition. If a machine intelligent agent can learn a world model to create a "dream" environment, it can then internally ask what-if questions -- simulate the alternative futures that haven't been experienced in the past yet -- and make optimal decisions accordingly. Existing world models are established typically by learning spatio-temporal regularities embedded from the past sensory signal without taking into account confounding factors that influence state transition dynamics. As such, they fail to answer the critical counterfactual questions about "what would have happened" if a certain action policy was taken. In this paper, we propose Causal World Models (CWMs) that allow unsupervised modeling of relationships between the intervened observations and the alternative futures by learning an estimator of the latent confounding factors. We empirically evaluate our method and demonstrate its effectiveness in a variety of physical reasoning environments. Specifically, we show reductions in sample complexity for reinforcement learning tasks and improvements in counterfactual physical reasoning.

cs.LG↗

arXiv2020

Learn molecular representations from large-scale unlabeled molecules for drug discovery

How to produce expressive molecular representations is a fundamental challenge in AI-driven drug discovery. Graph neural network (GNN) has emerged as a powerful technique for modeling molecular data. However, previous supervised approaches usually suffer from the scarcity of labeled data and have poor generalization capability. Here, we proposed a novel Molecular Pre-training Graph-based deep learning framework, named MPG, that leans molecular representations from large-scale unlabeled molecules. In MPG, we proposed a powerful MolGNet model and an effective self-supervised strategy for pre-training the model at both the node and graph-level. After pre-training on 11 million unlabeled molecules, we revealed that MolGNet can capture valuable chemistry insights to produce interpretable representation. The pre-trained MolGNet can be fine-tuned with just one additional output layer to create state-of-the-art models for a wide range of drug discovery tasks, including molecular properties prediction, drug-drug interaction, and drug-target interaction, involving 13 benchmark datasets. Our work demonstrates that MPG is promising to become a novel approach in the drug discovery pipeline.

cs.LG↗

arXiv2020

Learning Contextual Representations for Semantic Parsing with Generation-Augmented Pre-Training

Most recently, there has been significant interest in learning contextual representations for various NLP tasks, by leveraging large scale text corpora to train large neural language models with self-supervised learning objectives, such as Masked Language Model (MLM). However, based on a pilot study, we observe three issues of existing general-purpose language models when they are applied to text-to-SQL semantic parsers: fail to detect column mentions in the utterances, fail to infer column mentions from cell values, and fail to compose complex SQL queries. To mitigate these issues, we present a model pre-training framework, Generation-Augmented Pre-training (GAP), that jointly learns representations of natural language utterances and table schemas by leveraging generation models to generate pre-train data. GAP MODEL is trained on 2M utterance-schema pairs and 30K utterance-schema-SQL triples, whose utterances are produced by generative models. Based on experimental results, neural semantic parsers that leverage GAP MODEL as a representation encoder obtain new state-of-the-art results on both SPIDER and CRITERIA-TO-SQL benchmarks.

cs.CL↗

arXiv2020

Are we Forgetting about Compositional Optimisers in Bayesian Optimisation?

Bayesian optimisation presents a sample-efficient methodology for global optimisation. Within this framework, a crucial performance-determining subroutine is the maximisation of the acquisition function, a task complicated by the fact that acquisition functions tend to be non-convex and thus nontrivial to optimise. In this paper, we undertake a comprehensive empirical study of approaches to maximise the acquisition function. Additionally, by deriving novel, yet mathematically equivalent, compositional forms for popular acquisition functions, we recast the maximisation task as a compositional optimisation problem, allowing us to benefit from the extensive literature in this field. We highlight the empirical advantages of the compositional approach to acquisition function maximisation across 3958 individual experiments comprising synthetic optimisation tasks as well as tasks from Bayesmark. Given the generality of the acquisition function maximisation subroutine, we posit that the adoption of compositional optimisers has the potential to yield performance improvements across all domains in which Bayesian optimisation is currently being applied.

cs.LG↗

arXiv2020

Mean Field Multi-Agent Reinforcement Learning

Existing multi-agent reinforcement learning methods are limited typically to a small number of agents. When the agent number increases largely, the learning becomes intractable due to the curse of the dimensionality and the exponential growth of agent interactions. In this paper, we present \emph{Mean Field Reinforcement Learning} where the interactions within the population of agents are approximated by those between a single agent and the average effect from the overall population or neighboring agents; the interplay between the two entities is mutually reinforced: the learning of the individual agent's optimal policy depends on the dynamics of the population, while the dynamics of the population change according to the collective patterns of the individual policies. We develop practical mean field Q-learning and mean field Actor-Critic algorithms and analyze the convergence of the solution to Nash equilibrium. Experiments on Gaussian squeeze, Ising model, and battle games justify the learning effectiveness of our mean field approaches. In addition, we report the first result to solve the Ising model via model-free reinforcement learning methods.

cs.MA↗

arXiv2020

Inequalities Concerning Maximum Modulus and Zeros of Random Entire Functions

Let $f_ω(z)=\sum\limits_{j=0}^{\infty}χ_j(ω) a_j z^j$ be a random entire function, where $χ_j(ω)$ are independent and identically distributed random variables defined on a probability space $(Ω, \mathcal{F}, μ)$. In this paper, we first define a family of random entire functions, which includes Gaussian, Rademacher, Steinhaus entire functions. Then, we prove that, for almost all functions in the family and for any constant $C>1$, there exist a constant $r_0=r_0(ω)$ and a set $E\subset [e, \infty)$ of finite logarithmic measure such that, for $r>r_0$ and $r\notin E$, $$ |\log M(r, f)- N(r,0, f_ω)|\le (C/A)^{\frac1{B}}\log^{\frac1{B}}\log M(r,f) +\log\log M(r, f), \qquad a.s. $$ where $A, B$ are constants, $M(r, f)$ is the maximum modulus, and $N(r, 0, f)$ is the weighted counting-zero function of $f$. As a by-product of our main results, we prove Nevanlinna's second main theorem for random entire functions. Thus, the characteristic function of almost all functions in the family is bounded above by a weighed counting function, rather than by two weighted counting functions in the classical Nevanlinna theory. For instance, we show that, for almost all Gaussian entire functions $f_ω$ and for any $ε>0$, there is $r_0$ such that, for $r>r_0$, $$ T(r, f) \le N(r,0, f_ω)+(\frac12+ε) \log T(r, f). $$

math.CV↗

arXiv2020

Semi-supervised Active Learning for Instance Segmentation via Scoring Predictions

Active learning generally involves querying the most representative samples for human labeling, which has been widely studied in many fields such as image classification and object detection. However, its potential has not been explored in the more complex instance segmentation task that usually has relatively higher annotation cost. In this paper, we propose a novel and principled semi-supervised active learning framework for instance segmentation. Specifically, we present an uncertainty sampling strategy named Triplet Scoring Predictions (TSP) to explicitly incorporate samples ranking clues from classes, bounding boxes and masks. Moreover, we devise a progressive pseudo labeling regime using the above TSP in semi-supervised manner, it can leverage both the labeled and unlabeled data to minimize labeling effort while maximize performance of instance segmentation. Results on medical images datasets demonstrate that the proposed method results in the embodiment of knowledge from available data in a meaningful way. The extensive quantitatively and qualitatively experiments show that, our method can yield the best-performing model with notable less annotation costs, compared with state-of-the-arts.

cs.CV↗

arXiv2020

A 3D Non-Stationary Channel Model for 6G Wireless Systems Employing Intelligent Reflecting Surface

As one of the key technologies for the sixth generation (6G) mobile communications, intelligent reflecting surface IRS) has the advantages of low power consumption, low cost, and simple design methods. But channel modeling is still an open issue in this field currently. In this paper, we propose a three-dimensional (3D) geometry based stochastic model (GBSM) for a massive multiple-input multiple-output (MIMO) communication system employing IRS. The model supports the movements of the transmitter, the receiver, and clusters. The evolution of clusters on the linear array and planar array is also considered in the proposed model. In addition, the generation of reflecting coefficient is incorporated into the model and the path loss of the sub-channel assisted by IRS is also proposed. The steering vector is set up at the base station for the cooperation with IRS. Through studying statistical properties such as the temporal autocorrelation function and space correlation function, the nonstationary properties are verified. The good agreement between the simulation results and the analytical results illustrates the correctness of the proposed channel model.

eess.SP↗

arXiv2020

Automated Prostate Cancer Diagnosis Based on Gleason Grading Using Convolutional Neural Network

The Gleason grading system using histological images is the most powerful diagnostic and prognostic predictor of prostate cancer. The current standard inspection is evaluating Gleason H&E-stained histopathology images by pathologists. However, it is complicated, time-consuming, and subject to observers. Deep learning (DL) based-methods that automatically learn image features and achieve higher generalization ability have attracted significant attention. However, challenges remain especially using DL to train the whole slide image (WSI), a predominant clinical source in the current diagnostic setting, containing billions of pixels, morphological heterogeneity, and artifacts. Hence, we proposed a convolutional neural network (CNN)-based automatic classification method for accurate grading of PCa using whole slide histopathology images. In this paper, a data augmentation method named Patch-Based Image Reconstruction (PBIR) was proposed to reduce the high resolution and increase the diversity of WSIs. In addition, a distribution correction (DC) module was developed to enhance the adaption of pretrained model to the target dataset by adjusting the data distribution. Besides, a Quadratic Weighted Mean Square Error (QWMSE) function was presented to reduce the misdiagnosis caused by equal Euclidean distances. Our experiments indicated the combination of PBIR, DC, and QWMSE function was necessary for achieving superior expert-level performance, leading to the best results (0.8885 quadratic-weighted kappa coefficient).

eess.IV↗

arXiv2020

Reinforcement Learning Control of Constrained Dynamic Systems with Uniformly Ultimate Boundedness Stability Guarantee

Reinforcement learning (RL) is promising for complicated stochastic nonlinear control problems. Without using a mathematical model, an optimal controller can be learned from data evaluated by certain performance criteria through trial-and-error. However, the data-based learning approach is notorious for not guaranteeing stability, which is the most fundamental property for any control system. In this paper, the classic Lyapunov's method is explored to analyze the uniformly ultimate boundedness stability (UUB) solely based on data without using a mathematical model. It is further shown how RL with UUB guarantee can be applied to control dynamic systems with safety constraints. Based on the theoretical results, both off-policy and on-policy learning algorithms are proposed respectively. As a result, optimal controllers can be learned to guarantee UUB of the closed-loop system both at convergence and during learning. The proposed algorithms are evaluated on a series of robotic continuous control tasks with safety constraints. In comparison with the existing RL algorithms, the proposed method can achieve superior performance in terms of maintaining safety. As a qualitative evaluation of stability, our method shows impressive resilience even in the presence of external disturbances.

eess.SY↗

arXiv2020

A mirror theorem for Gromov-Witten theory without convexity

We prove a genus zero Givental-style mirror theorem for all complete intersections in proper toric Deligne-Mumford stacks, which provides an explicit slice called big $I-$function on Givental's Lagrangian cone for such targets. In particular, we remove a technical assumption called convexity needed in previous quantum Lefschetz theorem.

math.AG↗

arXiv2020

Deep Modulation Recognition with Multiple Receive Antennas: An End-to-end Feature Learning Approach

Modulation recognition using deep neural networks has shown promising advantages over conventional algorithms. However, most existing research focuses on single receive antenna. In this paper, two end-to-end feature learning deep architectures are introduced for modulation recognition with multiple receive antennas. The first is based on multi-view convolutional neural network by treating signals from different receive antennas as different views of a 3D object and designing the location and operation of view-pooling layer that are suitable for feature fusion of multi-antenna signals. Considering that the instantaneous SNRs could be different among receive antennas in wireless communications, we further propose weight-learning convolutional neural network which uses a weight-learning module to automatically learn the weights for feature combing of different receive antennas to perform end-to-end feature learning of multi-antenna signals. Results show that both end-to-end feature learning deep architectures outperform the existing algorithm, and the proposed weight-learning convolutional neural network achieves the best performance.

eess.SP↗