arXiv subjects

Kun Xu

Publications and source records attributed to Kun Xu.

At least 289 records · Page 16Linked to original sources

arXiv2019

Scalable Global Alignment Graph Kernel Using Random Features: From Node Embedding to Graph Embedding

Graph kernels are widely used for measuring the similarity between graphs. Many existing graph kernels, which focus on local patterns within graphs rather than their global properties, suffer from significant structure information loss when representing graphs. Some recent global graph kernels, which utilizes the alignment of geometric node embeddings of graphs, yield state-of-the-art performance. However, these graph kernels are not necessarily positive-definite. More importantly, computing the graph kernel matrix will have at least quadratic {time} complexity in terms of the number and the size of the graphs. In this paper, we propose a new family of global alignment graph kernels, which take into account the global properties of graphs by using geometric node embeddings and an associated node transportation based on earth mover's distance. Compared to existing global kernels, the proposed kernel is positive-definite. Our graph kernel is obtained by defining a distribution over \emph{random graphs}, which can naturally yield random feature approximations. The random feature approximations lead to our graph embeddings, which is named as "random graph embeddings" (RGE). In particular, RGE is shown to achieve \emph{(quasi-)linear scalability} with respect to the number and the size of the graphs. The experimental results on nine benchmark datasets demonstrate that RGE outperforms or matches twelve state-of-the-art graph classification algorithms.

cs.LG↗

arXiv2019

Efficient Global String Kernel with Random Features: Beyond Counting Substructures

Analysis of large-scale sequential data has been one of the most crucial tasks in areas such as bioinformatics, text, and audio mining. Existing string kernels, however, either (i) rely on local features of short substructures in the string, which hardly capture long discriminative patterns, (ii) sum over too many substructures, such as all possible subsequences, which leads to diagonal dominance of the kernel matrix, or (iii) rely on non-positive-definite similarity measures derived from the edit distance. Furthermore, while there have been works addressing the computational challenge with respect to the length of string, most of them still experience quadratic complexity in terms of the number of training samples when used in a kernel-based classifier. In this paper, we present a new class of global string kernels that aims to (i) discover global properties hidden in the strings through global alignments, (ii) maintain positive-definiteness of the kernel, without introducing a diagonal dominant kernel matrix, and (iii) have a training cost linear with respect to not only the length of the string but also the number of training string samples. To this end, the proposed kernels are explicitly defined through a series of different random feature maps, each corresponding to a distribution of random strings. We show that kernels defined this way are always positive-definite, and exhibit computational benefits as they always produce \emph{Random String Embeddings (RSE)} that can be directly used in any linear classification models. Our extensive experiments on nine benchmark datasets corroborate that RSE achieves better or comparable accuracy in comparison to state-of-the-art baselines, especially with the strings of longer lengths. In addition, we empirically show that RSE scales linearly with the increase of the number and the length of string.

cs.LG↗

arXiv2019

Unified gas-kinetic simulation for the system of multiscale radiation hydrodynamics

This paper aims at the simulation of multiple scale physics in the system of radiation hydrodynamics. The system couples the fluid dynamic evolution equations with the radiative heat transfer. The coupled system is solved by the gas-kinetic scheme (GKS) for the compressible viscous and heat conducting flow and the unified gas-kinetic scheme (UGKS) for the non-equilibrium radiative transfer, together with the momentum and energy exchange between these two phases. For the radiative transfer, due to the possible large variation of fluid opacity in different regions, the transport of photons in the flow system is simulated by the multiscale UGKS, which is capable of naturally capturing the transport process from the free streaming to the diffusive propagation.Since both GKS and UGKS are finite volume methods,all unknowns are defined inside each control volume and are discretized consistently for the hydrodynamic and radiative variables. For the coupled system, the scheme has the asymptotical preserving (AP) property, such as recovering the equilibrium diffusion limit for the radiation hydrodynamic equations in the optically thick region, where the cell size is not limited by photon's mean free path. A few test cases, such as radiative shock wave problems, are used to validate the current approach.

physics.comp-ph↗

arXiv2019

DurIAN: Duration Informed Attention Network For Multimodal Synthesis

In this paper, we present a generic and robust multimodal synthesis system that produces highly natural speech and facial expression simultaneously. The key component of this system is the Duration Informed Attention Network (DurIAN), an autoregressive model in which the alignments between the input text and the output acoustic features are inferred from a duration model. This is different from the end-to-end attention mechanism used, and accounts for various unavoidable artifacts, in existing end-to-end speech synthesis systems such as Tacotron. Furthermore, DurIAN can be used to generate high quality facial expression which can be synchronized with generated speech with/without parallel speech and face data. To improve the efficiency of speech generation, we also propose a multi-band parallel generation strategy on top of the WaveRNN model. The proposed Multi-band WaveRNN effectively reduces the total computational complexity from 9.8 to 5.5 GFLOPS, and is able to generate audio that is 6 times faster than real time on a single CPU core. We show that DurIAN could generate highly natural speech that is on par with current state of the art end-to-end systems, while at the same time avoid word skipping/repeating errors in those systems. Finally, a simple yet effective approach for fine-grained control of expressiveness of speech and facial expression is introduced.

cs.CL↗

arXiv2019

High-order gas-kinetic scheme with three-dimensional WENO reconstruction for the Euler and Navier-Stokes solutions

In this paper, a simple and efficient third-order weighted essentially non-oscillatory (WENO) reconstruction is developed for three-dimensional flows, in which the idea of two-dimensional WENO-AO scheme on unstructured meshes \cite{WENO-ao-3} is adopted. In the classical finite volume type WENO schemes, the linear weights for the candidate stencils are obtained by solving linear systems at Gaussian quadrature points of cell interface. For the three-dimensional scheme, such operations at twenty-four Gaussian quadrature points of a hexahedron would reduce the efficiency greatly, especially for the moving-mesh computation. Another drawback of classical WENO schemes is the appearance of negative weights with irregular local topology, which affect the robustness of spatial reconstruction. In such three-dimensional WENO-AO scheme, a simple strategy of selecting big stencil and sub-stencils for reconstruction is proposed. With the reconstructed quadratic polynomial from big stencil and linear polynomials from sub-stencils, the linear weights are chosen as positive numbers with the requirement that their sum equals one and be independent of local mesh topology. With such WENO reconstruction, a high-order gas-kinetic scheme (HGKS) is developed for both three-dimensional inviscid and viscous flows. Taken the grid velocity into account, the scheme is extended into the moving-mesh computation as well. Numerical results are provided to illustrate the good performance of such new finite volume WENO schemes. In the future, such WENO reconstruction will be extended to the unstructured meshes.

math.NA↗

arXiv2019

Ray Effect in Rarefied Flow Simulation

Ray effect usually appears in the radiative transfer when using discrete ordinates method (DOM) in the simulations. The cause and remedy for the ray effect have been intensively investigated in the radiation community. For rarefied gas flow, the ray effect is also associated with the discrete velocity method (DVM). However, few studies have been carried out in the rarefied community. In this paper, we take a detailed investigation of the ray effect in the rarefied flow simulations. Starting from a few commonly used benchmark tests, the root of the ray effect has been analyzed theoretically and validated numerically. At the same time, the influence of the ray effect on the quality of the numerical results of rarefied flow is estimated quantitatively. After understanding the nature of the ray effect, the strategy to minimize the ray effect through the discretization of the particle velocity space is presented and applied in the numerical simulations. An optimal velocity discretization for DVM is problem dependent and can be hardly obtained in the complex flow simulations. Due to the intrinsic self-adaptation of particle velocity, the stochastic particle methods are free from the ray effect. In rarefied regimes, the particle method seems more appropriate in the capturing of highly non-equilibrium flow behavior.

physics.comp-ph↗

arXiv2019

Efficient training and design of photonic neural network through neuroevolution

Recently, optical neural networks (ONNs) integrated in photonic chips has received extensive attention because they are expected to implement the same pattern recognition tasks in the electronic platforms with high efficiency and low power consumption. However, the current lack of various learning algorithms to train the ONNs obstructs their further development. In this article, we propose a novel learning strategy based on neuroevolution to design and train the ONNs. Two typical neuroevolution algorithms are used to determine the hyper-parameters of the ONNs and to optimize the weights (phase shifters) in the connections. In order to demonstrate the effectiveness of the training algorithms, the trained ONNs are applied in the classification tasks for iris plants dataset, wine recognition dataset and modulation formats recognition. The calculated results exhibit that the training algorithms based on neuroevolution are competitive with other traditional learning algorithms on both accuracy and stability. Compared with previous works, we introduce an efficient training method for the ONNs and demonstrate their broad application prospects in pattern recognition, reinforcement learning and so on.

cs.NE↗

arXiv2019

Cross-lingual Knowledge Graph Alignment via Graph Matching Neural Network

Previous cross-lingual knowledge graph (KG) alignment studies rely on entity embeddings derived only from monolingual KG structural information, which may fail at matching entities that have different facts in two KGs. In this paper, we introduce the topic entity graph, a local sub-graph of an entity, to represent entities with their contextual information in KG. From this view, the KB-alignment task can be formulated as a graph matching problem; and we further propose a graph-attention based solution, which first matches all entities in two topic entity graphs, and then jointly model the local matching information to derive a graph-level matching vector. Experiments show that our model outperforms previous state-of-the-art methods by a large margin.

cs.LG↗

arXiv2019

Multi-Granular Text Encoding for Self-Explaining Categorization

Self-explaining text categorization requires a classifier to make a prediction along with supporting evidence. A popular type of evidence is sub-sequences extracted from the input text which are sufficient for the classifier to make the prediction. In this work, we define multi-granular ngrams as basic units for explanation, and organize all ngrams into a hierarchical structure, so that shorter ngrams can be reused while computing longer ngrams. We leverage a tree-structured LSTM to learn a context-independent representation for each unit via parameter sharing. Experiments on medical disease classification show that our model is more accurate, efficient and compact than BiLSTM and CNN baselines. More importantly, our model can extract intuitive multi-granular evidence to support its predictions.

cs.CL↗

arXiv2019

On the fallback disk around the slowest isolated pulsar, 1E 161348$-$5055

The central compact object 1E 161348$-$5055 in the supernova remnant RCW 103 has a spin period $\sim 6.67$ hr, making it the slowest isolated pulsar. It is believed that a supernova fallback disk is required to spin down the neutron star to the current spin period within a few $10^3$ yr. The mass of the fallback disk around newborn neutron stars can provide useful information on the supernova processes and the possible detection limit with optical/infrared observations. However, it is controversial how massive the disk is in the case of 1E 161348$-$5055. In this work we simulate the spin evolution of a magnetar that is driven by the interaction between the disk and the star's magnetic field. Compared with previous studies, we take into account various critical conditions that affect the formation and evolution of the fallback disk. Our calculation shows that we can reproduce the extremely slow spin of 1E 161348$-$5055 when taking the initial disk mass $M_{\rm d} \sim 10^{-7} M_{\odot}$ and the neutron star magnetic field $B\geq 5 \times 10^{15}$ G. This implies that 1E 161348$-$5055 may be a magnetar with very special initial parameters. However, if future observations reveal more objects like 1E 161348$-$5055, then stringent constraints can be obtained on the supernova fallback.

astro-ph.HE↗

arXiv2019

Multi-objects Generation with Amortized Structural Regularization

Deep generative models (DGMs) have shown promise in image generation. However, most of the existing work learn the model by simply optimizing a divergence between the marginal distributions of the model and the data, and often fail to capture the rich structures and relations in multi-object images. Human knowledge is a critical element to the success of DGMs to infer these structures. In this paper, we propose the amortized structural regularization (ASR) framework, which adopts the posterior regularization (PR) to embed human knowledge into DGMs via a set of structural constraints. We derive a lower bound of the regularized log-likelihood, which can be jointly optimized with respect to the generative model and recognition model efficiently. Empirical results show that ASR significantly outperforms the DGM baselines in terms of inference accuracy and sample quality.

cs.LG↗

arXiv2019

Extracting Multiple-Relations in One-Pass with Pre-Trained Transformers

Most approaches to extraction multiple relations from a paragraph require multiple passes over the paragraph. In practice, multiple passes are computationally expensive and this makes difficult to scale to longer paragraphs and larger text corpora. In this work, we focus on the task of multiple relation extraction by encoding the paragraph only once (one-pass). We build our solution on the pre-trained self-attentive (Transformer) models, where we first add a structured prediction layer to handle extraction between multiple entity pairs, then enhance the paragraph embedding to capture multiple relational information associated with each entity with an entity-aware attention technique. We show that our approach is not only scalable but can also perform state-of-the-art on the standard benchmark ACE 2005.

cs.CL↗

arXiv2019

Improving Adversarial Robustness via Promoting Ensemble Diversity

Though deep neural networks have achieved significant progress on various tasks, often enhanced by model ensemble, existing high-performance models can be vulnerable to adversarial attacks. Many efforts have been devoted to enhancing the robustness of individual networks and then constructing a straightforward ensemble, e.g., by directly averaging the outputs, which ignores the interaction among networks. This paper presents a new method that explores the interaction among individual networks to improve robustness for ensemble models. Technically, we define a new notion of ensemble diversity in the adversarial setting as the diversity among non-maximal predictions of individual members, and present an adaptive diversity promoting (ADP) regularizer to encourage the diversity, which leads to globally better robustness for the ensemble by making adversarial examples difficult to transfer among individual members. Our method is computationally efficient and compatible with the defense methods acting on individual networks. Empirical results on various datasets verify that our method can improve adversarial robustness while maintaining state-of-the-art accuracy on normal examples.

cs.LG↗

arXiv2019

High-order ALE gas-kinetic scheme with unstructured WENO reconstruction

In this paper, a high-order multi-dimensional gas-kinetic scheme is presented for both inviscid and viscous flows in arbitrary Lagrangian-Eulerian (ALE) formulation. Compared with the traditional ALE method, the flow variables are updated in the finite volume framework, and the rezoning and remapping steps are not required. The two-stage fourth-order method is used for the temporal discretization, and the second-order gas-kinetic solver is applied for the flux evaluation. In the two-stage method, the spatial reconstruction is performed at both initial and intermediate stage, and the computational mesh at the corresponding stage is given by the specified mesh velocity. In the moving mesh procedure, the mesh may distort severely and the mesh quality is reduced. To achieve the accuracy and improve the robustness, the newly developed WENO method \cite{un-WENO3} on quadrilateral unstructured meshes is adopted at each stage. The Gaussian quadrature is used for flux calculation. For each Gaussian point, the reconstruction performed in the local moving coordinate, where the variation of mesh velocity is taken into account. Therefore, the accuracy and geometric conservation law can be well preserved by the current scheme even with the largely deforming mesh. Numerical examples are presented to validate the performance of current scheme, where the mesh adaptation method and cell centered Lagrangian method are used to provide mesh velocity.

physics.flu-dyn↗

arXiv2019

Performance Enhancement for High-order Gas-kinetic Scheme Based on WENO-adaptive-order Reconstruction

High-order gas-kinetic scheme (HGKS) has been well-developed in the past years. Abundant numerical tests including hypersonic flow, turbulence, and aeroacoustic problems, have been used to validate its accuracy, efficiency, and robustness. However, there are still rooms for its further improvement. Firstly, the reconstruction in the previous scheme mainly achieves a third-order accuracy for the initial non-equilibrium states. At the same time, the equilibrium state in space and time in HGKS has to be reconstructed separately. Secondly, it is complicated to get reconstructed data at Gaussian points from the WENO-type method in high dimensions. For HGKS, besides the point-wise values at the Gaussian points it also requires the slopes in both normal and tangential directions of a cell interface. Thirdly, there exists visible spurious overshoot/undershoot at weak discontinuities from the previous HGKS with the standard WENO reconstruction. In order to overcome these difficulties, in this paper we use an improved reconstruction for HGKS. The WENO with adaptive order (WENO-AO) method is implemented for reconstruction.A whole polynomial inside each cell is provided in WENO-AO reconstruction. The HGKS becomes simpler than the previous one with the direct implementation of cell interface values and their slopes from WENO-AO. The additional reconstruction of equilibrium state at the beginning of each time step can be avoided as well by dynamically merging the reconstructed non-equilibrium slopes. The new HGKS essentially releases or totally removes the above existing problems in previous HGKS. The accuracy of the scheme from 1D to 3D from the new HGKS can recover the theoretical order of accuracy of the WENO reconstruction.In the two- and three-dimensional simulations, the new HGKS shows better robustness and efficiency than the previous scheme in all test cases.

physics.comp-ph↗

arXiv2019

Three dimensional high-order gas-kinetic scheme for supersonic isotropic turbulence I: criterion for direct numerical simulation

In this paper, we intend to address the high-order gas-kinetic scheme (HGKS) in the direct numerical simulation (DNS) of compressible isotropic turbulence up to the supersonic regime. With the consideration of robustness and accuracy, the WENO-Z scheme is adopted for spatial reconstruction in the current higher-order scheme. Statistical quantities are compared with the high-order compact finite difference scheme to determine the spatial and temporal criterion for DNS. According to the grid and time convergence study, it can be concluded that the minimum spatial resolution parameter $κ_{max} η_0 \ge 2.71$ and the maximum temporal resolution parameter $Δt_{ini}/τ_{t_0} \leq 5.58/1000$ are adequate for HGKS to resolve the compressible isotropic turbulence, where $κ_{max}$ is the maximum resolved wave number, $Δt_{ini}$ is the initial time step, $η_0$ and $τ_{t_0}$ are the initial Kolmogorov length scale and the large-eddy-turnover time. Guided by such criterion, the compressible isotropic turbulence from subsonic regime $Ma_t=0.8$ to supersonic one $Ma_t=1.2$, and the Taylor microscale Reynolds number $Re_λ$ ranging from $10$ to $72$ are simulated. With the high initial turbulent Mach number, the strong random shocklets and high expansion regions are identified, as well as the wide range of probability density function over local turbulence Mach number. All those impose great challenge for high-order schemes. In order to construct compressible large eddy simulation models at high turbulent Mach number, the ensemble budget of turbulent kinetic energy is fully analyzed. The solenoidal dissipation rate decreases with the increasing of $Ma_t$ and $Re_λ$. Meanwhile, the dilational dissipation rate increases with the increasing of $Ma_t$, which cannot be neglected for constructing supersonic turbulence model.

physics.flu-dyn↗

arXiv2019

Implicit High-Order Gas Kinetic Scheme for Turbulence Simulation

In recent years, coupled with traditional turbulence models, the second-order gas-kinetic scheme (GKS) has been used in the turbulent flow simulations. At the same time, high-order GKS has been developed, such as the two-stage fourth-order scheme (S2O4) GKS, and used for laminar flow calculations. In this paper, targeting on the high-Reynolds number engineering turbulent flows, an implicit high-order GKS with Lower-Upper Symmetric Gauss-Seidel (LU-SGS) technique is developed under the S2O4 framework. Based on Vreman-type LES model and $k - ω$ SST model, a turbulent relaxation time is obtained and used for an enlarged particle collision time in the implicit high-order GKS for the high-Reynolds number turbulent flows. Numerical experiments include incompressible decaying homogeneous isotropic turbulence, incompressible high-Reynolds number flat plate turbulent flow, incompressible turbulence around NACA0012 airfoil, transonic turbulence around RAE2822 airfoil, and transonic high-Reynolds number ARA M100 wing-body turbulence. Comparisons among the numerical solutions from current implicit high-order GKS, the explicit high-order GKS, the implicit second-order GKS, and experimental measurements have been conducted. Through these examples, it is concluded that the high-order GKS has high accuracy in space and time, especially for smooth flows, obtaining more accurate turbulent flow fields on coarse grids compared with second-order GKS. In addition, significant acceleration on computational efficiency, as well as super robustness in simulating complex flows are confirmed for current implicit high-order GKS. This study also indicates that turbulence modeling plays a dominant role in the capturing physical solution, such as in the transonic three-dimensional complex RANS simulation, in comparison with numerical discretization errors.

physics.comp-ph↗

arXiv2019

Unified Gas-kinetic Wave-Particle Methods III: Multiscale Photon Transport

In this paper, we extend the unified gas-kinetic wave-particle (UGKWP) method to the multiscale photon transport. In this method, the photon free streaming and scattering processes are treated in an un-splitting way. The duality descriptions, namely the simulation particle and distribution function, are utilized to describe the photon. By accurately recovering the governing equations of the unified gas-kinetic scheme (UGKS), the UGKWP preserves the multiscale dynamics of photon transport from optically thin to optically thick regime. In the optically thin regime, the UGKWP becomes a Monte Carlo type particle tracking method, while in the optically thick regime, the UGKWP becomes a diffusion equation solver. The local photon dynamics of the UGKWP, as well as the proportion of wave-described and particle-described photons are automatically adapted according to the numerical resolution and transport regime. Compared to the $S_n$ -type UGKS, the UGKWP requires less memory cost and does not suffer ray effect. Compared to the implicit Monte Carlo (IMC) method, the statistical noise of UGKWP is greatly reduced and computational efficiency is significantly improved in the optically thick regime. Several numerical examples covering all transport regimes from the optically thin to optically thick are computed to validate the accuracy and efficiency of the UGKWP method. In comparison to the $S_n$ -type UGKS and IMC method, the UGKWP method may have several-order-of-magnitude reduction in computational cost and memory requirement in solving some multsicale transport problems.

physics.comp-ph↗