arXiv subjects

Kun He

Publications and source records attributed to Kun He.

At least 163 records · Page 9Linked to original sources

arXiv2021

Structure Amplification on Multi-layer Stochastic Block Models

Much of the complexity of social, biological, and engineered systems arises from a network of complex interactions connecting many basic components. Network analysis tools have been successful at uncovering latent structure termed communities in such networks. However, some of the most interesting structure can be difficult to uncover because it is obscured by the more dominant structure. Our previous work proposes a general structure amplification technique called HICODE that uncovers many layers of functional hidden structure in complex networks. HICODE incrementally weakens dominant structure through randomization allowing the hidden functionality to emerge, and uncovers these hidden structure in real-world networks that previous methods rarely uncover. In this work, we conduct a comprehensive and systematic theoretical analysis on the hidden community structure. In what follows, we define multi-layer stochastic block model, and provide theoretical support using the model on why the existence of hidden structure will make the detection of dominant structure harder compared with equivalent random noise. We then provide theoretical proofs that the iterative reducing methods could help promote the uncovering of hidden structure as well as boosting the detection quality of dominant structure.

cs.SI↗

arXiv2021

Perfect Sampling for (Atomic) Lovász Local Lemma

We give a Markov chain based perfect sampler for uniform sampling solutions of constraint satisfaction problems (CSP). Under some mild Lovász local lemma conditions where each constraint of the CSP has a small number of forbidden local configurations, our algorithm is accurate and efficient: it outputs a perfect uniform random solution and its expected running time is quasilinear in the number of variables. Prior to our work, perfect samplers are only shown to exist for CSPs under much more restrictive conditions (Guo, Jerrum, and Liu, JACM'19). Our algorithm has two components: 1. A simple perfect sampling algorithm using bounding chains (Huber, STOC'98; Haggstrom and Nelander, Scandinavian Journal of Statistics'99). This sampler is efficient if each variable domain is small. 2. A simple but powerful state tensorization trick to reduce large domains to smaller ones. This trick is a generalization of state compression (Feng, He, and Yin, STOC'21). The crux of our analysis is a simple information percolation argument which allows us to achieve bounds even beyond current best approximate samplers (Jain, Pham, and Vuong, ArXiv'21). Previous related works either use intricate algorithms or need sophisticated analysis or even both. Thus we view the simplicity of both our algorithm and analysis as a strength of our work.

cs.DS↗

arXiv2021

Multi-stage Optimization based Adversarial Training

In the field of adversarial robustness, there is a common practice that adopts the single-step adversarial training for quickly developing adversarially robust models. However, the single-step adversarial training is most likely to cause catastrophic overfitting, as after a few training epochs it will be hard to generate strong adversarial examples to continuously boost the adversarial robustness. In this work, we aim to avoid the catastrophic overfitting by introducing multi-step adversarial examples during the single-step adversarial training. Then, to balance the large training overhead of generating multi-step adversarial examples, we propose a Multi-stage Optimization based Adversarial Training (MOAT) method that periodically trains the model on mixed benign examples, single-step adversarial examples, and multi-step adversarial examples stage by stage. In this way, the overall training overhead is reduced significantly, meanwhile, the model could avoid catastrophic overfitting. Extensive experiments on CIFAR-10 and CIFAR-100 datasets demonstrate that under similar amount of training overhead, the proposed MOAT exhibits better robustness than either single-step or multi-step adversarial training methods.

cs.LG↗

arXiv2021

Natural Language Adversarial Defense through Synonym Encoding

In the area of natural language processing, deep learning models are recently known to be vulnerable to various types of adversarial perturbations, but relatively few works are done on the defense side. Especially, there exists few effective defense method against the successful synonym substitution based attacks that preserve the syntactic structure and semantic information of the original text while fooling the deep learning models. We contribute in this direction and propose a novel adversarial defense method called Synonym Encoding Method (SEM). Specifically, SEM inserts an encoder before the input layer of the target model to map each cluster of synonyms to a unique encoding and trains the model to eliminate possible adversarial perturbations without modifying the network architecture or adding extra data. Extensive experiments demonstrate that SEM can effectively defend the current synonym substitution based attacks and block the transferability of adversarial examples. SEM is also easy and efficient to scale to large models and big datasets.

cs.CL↗

arXiv2021

Pore-scale investigation on natural convection melting in a square cavity with gradient porous media

In this paper, a numerical study on the melting behavior of phase change material (PCM) with gradient porous media has been carried out at the pore scales. In order to solve the governing equations, a pore-scale lattice Boltzmann method with the double distribution functions is used, in which a volumetric LB scheme is employed to handle the boundary. The Monte Carlo random sampling is adopted to generate a microstructure of two-dimensional gradient foam metal which are then used to simulate the solid-liquid phase transition in the cavity. The effect of several factors, such as gradient porosity structure, gradient direction, Rayleigh number and particle diameters on the liquid fraction of PCM are systematically investigated. It is observed that the presence of gradient media affect significantly the melting rate and shortens full melting time compared to that for constant porosity by enhancing natural convection. The melting time of positive and negative gradients will change with Rayleigh number, and there is a critical value for Rayleigh number. Specifically, when Rayleigh number is below the critical value, the positive gradient is more advantageous, and when Rayleigh number exceeds the critical value, the negative gradient is more advantageous. Moreover, smaller particle diameters would lead to lower permeability and larger internal surfaces for heat transfer.

physics.flu-dyn↗

arXiv2021

Numerical study on electrohydrodynamic enhancement of PCM melting in cylindrical annulus under microgravity

Latent heat thermal energy storage (LHTES) has been recommended as an effective technology to the thermal management system of space exploration for its excellent ability of storing thermal energy. However, it is well known that the low thermal conductivity of phase change material (PCM) seriously weakens the heat charging and discharging rates of LHTES system. In present study, the electrohydrodynamic (EHD), which is a popular active heat transfer enhancement technology, is introduced to enhance the PCM melting in a shell-tube LHTES system under microgravity. In our numerical simulations, we mainly focus on the combined effects of the electric Rayleigh number $T$ and the eccentricity $Γ$ on the melting performance under microgravity. Based on the numerical results, it is found that in contrast to the case without the electric field, the presence of the electric field causes the heat transfer efficiency and melting behavior of LHTES system to be enhanced significantly. In addition, our results indicates that the EHD technique always shows a good performance in accelerating the melting process even under microgravity, and specially, the maximum time saving in some cases is more than $90\%$. Furthermore, we note that although the concentric annulus is always the optimal configuration under no-gravity condition, the optimal eccentric position of the internal tube strongly depends on the electric Rayleigh number if the gravity effects are taken into account.

physics.flu-dyn↗

arXiv2021

Null-free False Discovery Rate Control Using Decoy Permutations

The traditional approaches to false discovery rate (FDR) control in multiple hypothesis testing are usually based on the null distribution of a test statistic. However, all types of null distributions, including the theoretical, permutation-based and empirical ones, have some inherent drawbacks. For example, the theoretical null might fail because of improper assumptions on the sample distribution. Here, we propose a null distribution-free approach to FDR control for multiple hypothesis testing. This approach, named target-decoy procedure, simply builds on the ordering of tests by some statistic or score, the null distribution of which is not required to be known. Competitive decoy tests are constructed from permutations of original samples and are used to estimate the false target discoveries. We prove that this approach controls the FDR when the statistics are independent between different tests. Simulation demonstrates that it is more stable and powerful than two existing popular approaches. Evaluation is also made on a real dataset.

stat.ME↗

arXiv2021

Sampling Constraint Satisfaction Solutions in the Local Lemma Regime

We give a Markov chain based algorithm for sampling almost uniform solutions of constraint satisfaction problems (CSPs). Assuming a canonical setting for the Lovász local lemma, where each constraint is violated by a small number of forbidden local configurations, our sampling algorithm is accurate in a local lemma regime, and the running time is a fixed polynomial whose dependency on $n$ is close to linear, where $n$ is the number of variables. Our main approach is a new technique called state compression, which generalizes the "mark/unmark" paradigm of Moitra (Moitra, JACM, 2019), and can give fast local-lemma-based sampling algorithms. As concrete applications of our technique, we give the current best almost-uniform samplers for hypergraph colorings and for CNF solutions.

cs.DS↗

arXiv2021

Boosting Adversarial Transferability through Enhanced Momentum

Deep learning models are known to be vulnerable to adversarial examples crafted by adding human-imperceptible perturbations on benign images. Many existing adversarial attack methods have achieved great white-box attack performance, but exhibit low transferability when attacking other models. Various momentum iterative gradient-based methods are shown to be effective to improve the adversarial transferability. In what follows, we propose an enhanced momentum iterative gradient-based method to further enhance the adversarial transferability. Specifically, instead of only accumulating the gradient during the iterative process, we additionally accumulate the average gradient of the data points sampled in the gradient direction of the previous iteration so as to stabilize the update direction and escape from poor local maxima. Extensive experiments on the standard ImageNet dataset demonstrate that our method could improve the adversarial transferability of momentum-based methods by a large margin of 11.1% on average. Moreover, by incorporating with various input transformation methods, the adversarial transferability could be further improved significantly. We also attack several extra advanced defense models under the ensemble-model setting, and the enhancements are remarkable with at least 7.8% on average.

cs.CV↗

arXiv2021

Combining Reinforcement Learning with Lin-Kernighan-Helsgaun Algorithm for the Traveling Salesman Problem

We address the Traveling Salesman Problem (TSP), a famous NP-hard combinatorial optimization problem. And we propose a variable strategy reinforced approach, denoted as VSR-LKH, which combines three reinforcement learning methods (Q-learning, Sarsa and Monte Carlo) with the well-known TSP algorithm, called Lin-Kernighan-Helsgaun (LKH). VSR-LKH replaces the inflexible traversal operation in LKH, and lets the program learn to make choice at each search step by reinforcement learning. Experimental results on 111 TSP benchmarks from the TSPLIB with up to 85,900 cities demonstrate the excellent performance of the proposed method.

cs.AI↗

arXiv2020

Adversarial Training with Fast Gradient Projection Method against Synonym Substitution based Text Attacks

Adversarial training is the most empirically successful approach in improving the robustness of deep neural networks for image classification.For text classification, however, existing synonym substitution based adversarial attacks are effective but not efficient to be incorporated into practical text adversarial training. Gradient-based attacks, which are very efficient for images, are hard to be implemented for synonym substitution based text attacks due to the lexical, grammatical and semantic constraints and the discrete text input space. Thereby, we propose a fast text adversarial attack method called Fast Gradient Projection Method (FGPM) based on synonym substitution, which is about 20 times faster than existing text attack methods and could achieve similar attack performance. We then incorporate FGPM with adversarial training and propose a text defense method called Adversarial Training with FGPM enhanced by Logit pairing (ATFL). Experiments show that ATFL could significantly improve the model robustness and block the transferability of adversarial examples.

cs.CL↗

arXiv2020

Dynamic inference in probabilistic graphical models

Probabilistic graphical models, such as Markov random fields (MRFs), are useful for describing high-dimensional distributions in terms of local dependence structures. The probabilistic inference is a fundamental problem related to graphical models, and sampling is a main approach for the problem. In this paper, we study probabilistic inference problems when the graphical model itself is changing dynamically with time. Such dynamic inference problems arise naturally in today's application, e.g.~multivariate time-series data analysis and practical learning procedures. We give a dynamic algorithm for sampling-based probabilistic inferences in MRFs, where each dynamic update can change the underlying graph and all parameters of the MRF simultaneously, as long as the total amount of changes is bounded. More precisely, suppose that the MRF has $n$ variables and polylogarithmic-bounded maximum degree, and $N(n)$ independent samples are sufficient for the inference for a polynomial function $N(\cdot)$. Our algorithm dynamically maintains an answer to the inference problem using $\widetilde{O}(n N(n))$ space cost, and $\widetilde{O}(N(n) + n)$ incremental time cost upon each update to the MRF, as long as the well-known Dobrushin-Shlosman condition is satisfied by the MRFs. Compared to the static case, which requires $Ω(n N(n))$ time cost for redrawing all $N(n)$ samples whenever the MRF changes, our dynamic algorithm gives a $\widetildeΩ(\min\{n, N(n)\})$-factor speedup. Our approach relies on a novel dynamic sampling technique, which transforms local Markov chains (a.k.a. single-site dynamics) to dynamic sampling algorithms, and an "algorithmic Lipschitz" condition that we establish for sampling from graphical models, namely, when the MRF changes by a small difference, samples can be modified to reflect the new distribution, with cost proportional to the difference on MRF.

cs.DS↗

arXiv2020

Quadratic Optimization based Clique Expansion for Overlapping Community Detection

Community detection is crucial for analyzing social and biological networks, and comprehensive approaches have been proposed in the last two decades. Nevertheless, finding all overlapping communities in large networks that could accurately approximate the ground-truth communities remains challenging. In this work, we present the QOCE (Quadratic Optimization based Clique Expansion), an overlapping community detection algorithm that could scale to large networks with hundreds of thousands of nodes and millions of edges. QOCE follows the popular seed set expansion strategy, regarding each high-quality maximal clique as the initial seed set and applying quadratic optimization for the expansion. We extensively evaluate our algorithm on 28 synthetic LFR networks and six real-world networks of various domains and scales, and compare QOCE with four state-of-the-art overlapping community detection algorithms. Empirical results demonstrate the competitive performance of the proposed approach in terms of detection accuracy, efficiency, and scalability.

cs.SI↗

arXiv2020

Local Generalization and Bucketization Technique for Personalized Privacy Preservation

Anonymization technique has been extensively studied and widely applied for privacy-preserving data publishing. In most previous approaches, a microdata table consists of three categories of attribute: explicit-identifier, quasi-identifier (QI), and sensitive attribute. Actually, different individuals may have different view on the sensitivity of different attributes. Therefore, there is another type of attribute that contains both QI values and sensitive values, namely, semi-sensitive attribute. Based on such observation, we propose a new anonymization technique, called local generalization and bucketization, to prevent identity disclosure and protect the sensitive values on each semi-sensitive attribute and sensitive attribute. The rationale is to use local generalization and local bucketization to divide the tuples into local equivalence groups and partition the sensitive values into local buckets, respectively. The protections of local generalization and local bucketization are independent, so that they can be implemented by appropriate algorithms without weakening other protection, respectively. Besides, the protection of local bucketization for each semi-sensitive attribute and sensitive attribute is also independent. Consequently, local bucketization can comply with various principles in different attributes according to the actual requirements of anonymization. The conducted extensive experiments illustrate the effectiveness of the proposed approach.

cs.CR↗

arXiv2020

Probability Learning based Tabu Search for the Budgeted Maximum Coverage Problem

Knapsack problems are classic models that can formulate a wide range of applications. In this work, we deal with the Budgeted Maximum Coverage Problem (BMCP), which is a generalized 0-1 knapsack problem. Given a set of items with nonnegative weights and a set of elements with nonnegative profits, where each item is composed of a subset of elements, BMCP aims to pack a subset of items in a capacity-constrained knapsack such that the total weight of the selected items does not exceed the knapsack capacity, and the total profit of the associated elements is maximized. Note that each element is counted once even if it is covered multiple times. BMCP is closely related to the Set-Union Knapsack Problem (SUKP) that is well studied in recent years. As the counterpart problem of SUKP, however, BMCP was introduced early in 1999 but since then it has been rarely studied, especially there is no practical algorithm proposed. By combining the reinforcement learning technique to the local search procedure, we propose a probability learning based tabu search (PLTS) algorithm for addressing this NP-hard problem. The proposed algorithm iterates through two distinct phases, namely a tabu search phase and a probability learning based perturbation phase. As there is no benchmark instances proposed in the literature, we generate 30 benchmark instances with varied properties. Experimental results demonstrate that our PLTS algorithm significantly outperforms the general CPLEX solver for solving the challenging BMCP in terms of the solution quality.

cs.AI↗

arXiv2020

Sparse Nonnegative Matrix Factorization for Multiple Local Community Detection

Local community detection consists of finding a group of nodes closely related to the seeds, a small set of nodes of interest. Such group of nodes are densely connected or have a high probability of being connected internally than their connections to other clusters in the network. Existing local community detection methods focus on finding either one local community that all seeds are most likely to be in or finding a single community for each of the seeds. However, a seed member usually belongs to multiple local overlapping communities. In this work, we present a novel method of detecting multiple local communities to which a single seed member belongs. The proposed method consists of three key steps: (1) local sampling with Personalized PageRank (PPR); (2) using the sparseness generated by a sparse nonnegative matrix factorization (SNMF) to estimate the number of communities in the sampled subgraph; (3) using SNMF soft community membership vectors to assign nodes to communities. The proposed method shows favorable accuracy performance and a good conductance when compared to state-of-the-art community detection methods by experiments using a combination of artificial and real-world networks.

cs.SI↗

arXiv2020

Single Image Reflection Removal through Cascaded Refinement

We address the problem of removing undesirable reflections from a single image captured through a glass surface, which is an ill-posed, challenging but practically important problem for photo enhancement. Inspired by iterative structure reduction for hidden community detection in social networks, we propose an Iterative Boost Convolutional LSTM Network (IBCLN) that enables cascaded prediction for reflection removal. IBCLN is a cascaded network that iteratively refines the estimates of transmission and reflection layers in a manner that they can boost the prediction quality to each other, and information across steps of the cascade is transferred using an LSTM. The intuition is that the transmission is the strong, dominant structure while the reflection is the weak, hidden structure. They are complementary to each other in a single image and thus a better estimate and reduction on one side from the original image leads to a more accurate estimate on the other side. To facilitate training over multiple cascade steps, we employ LSTM to address the vanishing gradient problem, and propose residual reconstruction loss as further training guidance. Besides, we create a dataset of real-world images with reflection and ground-truth transmission layers to mitigate the problem of insufficient data. Comprehensive experiments demonstrate that the proposed method can effectively remove reflections in real and synthetic images compared with state-of-the-art reflection removal methods.

cs.CV↗

arXiv2020

Hidden Community Detection on Two-layer Stochastic Models: a Theoretical Perspective

Hidden community is a new graph-theoretical concept recently proposed [4], in which the authors also propose a meta-approach called HICODE (Hidden Community Detection) for detecting hidden communities. HICODE is demonstrated through experiments that it is able to uncover previously overshadowed weak layers and uncover both weak and strong layers at a higher accuracy. However, the authors provide no theoretical guarantee for the performance. In this work, we focus on the theoretical analysis of HICODE on synthetic two-layer networks, where layers are independent of each other and each layer is generated by stochastic block model. We bridge their gap through two-layer stochastic block model networks in the following aspects: 1) we show that partitions that locally optimize modularity correspond to grounded layers, indicating modularity-optimizing algorithms can detect strong layers; 2) we prove that when reducing found layers, HICODE increases absolute modularities of all unreduced layers, showing its layer reduction step makes weak layers more detectable. Our work builds a solid theoretical base for HICODE, demonstrating that it is promising in uncovering both weak and strong layers of communities in two-layer networks.

cs.SI↗