arXiv subjects

Bo Jiang

Publications and source records attributed to Bo Jiang.

At least 325 records · Page 18Linked to original sources

arXiv2018

Structured Quasi-Newton Methods for Optimization with Orthogonality Constraints

In this paper, we study structured quasi-Newton methods for optimization problems with orthogonality constraints. Note that the Riemannian Hessian of the objective function requires both the Euclidean Hessian and the Euclidean gradient. In particular, we are interested in applications that the Euclidean Hessian itself consists of a computational cheap part and a significantly expensive part. Our basic idea is to keep these parts of lower computational costs but substitute those parts of higher computational costs by the limited-memory quasi-Newton update. More specically, the part related to Euclidean gradient and the cheaper parts in the Euclidean Hessian are preserved. The initial quasi-Newton matrix is further constructed from a limited-memory Nyström approximation to the expensive part. Consequently, our subproblems approximate the original objective function in the Euclidean space and preserve the orthogonality constraints without performing the so-called vector transports. When the subproblems are solved to sufficient accuracy, both global and local q-superlinear convergence can be established under mild conditions. Preliminary numerical experiments on the linear eigenvalue problem and the electronic structure calculation show the effectiveness of our method compared with the state-of-art algorithms.

math.OC↗

arXiv2018

ContractFuzzer: Fuzzing Smart Contracts for Vulnerability Detection

Decentralized cryptocurrencies feature the use of blockchain to transfer values among peers on networks without central agency. Smart contracts are programs running on top of the blockchain consensus protocol to enable people make agreements while minimizing trusts. Millions of smart contracts have been deployed in various decentralized applications. The security vulnerabilities within those smart contracts pose significant threats to their applications. Indeed, many critical security vulnerabilities within smart contracts on Ethereum platform have caused huge financial losses to their users. In this work, we present ContractFuzzer, a novel fuzzer to test Ethereum smart contracts for security vulnerabilities. ContractFuzzer generates fuzzing inputs based on the ABI specifications of smart contracts, defines test oracles to detect security vulnerabilities, instruments the EVM to log smart contracts runtime behaviors, and analyzes these logs to report security vulnerabilities. Our fuzzing of 6991 smart contracts has flagged more than 459 vulnerabilities with high precision. In particular, our fuzzing tool successfully detects the vulnerability of the DAO contract that leads to USD 60 million loss and the vulnerabilities of Parity Wallet that have led to the loss of $30 million and the freezing of USD 150 million worth of Ether.

cs.SE↗

arXiv2018

Context-aware Data Aggregation with Localized Information Privacy

In this paper, localized information privacy (LIP) is proposed, as a new privacy definition, which allows statistical aggregation while protecting users' privacy without relying on a trusted third party. The notion of context-awareness is incorporated in LIP by the introduction of priors, which enables the design of privacy-preserving data aggregation with knowledge of priors. We show that LIP relaxes the Localized Differential Privacy (LDP) notion by explicitly modeling the adversary's knowledge. However, it is stricter than $2ε$-LDP and $ε$-mutual information privacy. The incorporation of local priors allows LIP to achieve higher utility compared to other approaches. We then present an optimization framework for privacy-preserving data aggregation, with the goal of minimizing the expected squared error while satisfying the LIP privacy constraints. Utility-privacy tradeoffs are obtained under several models in closed-form. We then validate our analysis by {numerical analysis} using both synthetic and real-world data. Results show that our LIP mechanism provides better utility-privacy tradeoffs than LDP and when the prior is not uniformly distributed, the advantage of LIP is even more significant.

cs.IT↗

arXiv2018

On a Class of Stochastic Multilayer Networks

In this paper, we introduce a new class of stochastic multilayer networks. A stochastic multilayer network is the aggregation of $M$ networks (one per layer) where each is a subgraph of a foundational network $G$. Each layer network is the result of probabilistically removing links and nodes from $G$. The resulting network includes any link that appears in at least $K$ layers. This model is an instance of a non-standard site-bond percolation model. Two sets of results are obtained: first, we derive the probability distribution that the $M$-layer network is in a given configuration for some particular graph structures (explicit results are provided for a line and an algorithm is provided for a tree), where a configuration is the collective state of all links (each either active or inactive). Next, we show that for appropriate scalings of the node and link selection processes in a layer, links are asymptotically independent as the number of layers goes to infinity, and follow Poisson distributions. Numerical results are provided to highlight the impact of having several layers on some metrics of interest (including expected size of the cluster a node belongs to in the case of the line). This model finds applications in wireless communication networks with multichannel radios, multiple social networks with overlapping memberships, transportation networks, and, more generally, in any scenario where a common set of nodes can be linked via co-existing means of connectivity.

cs.SI↗

arXiv2018

On the Convergence of the TTL Approximation for an LRU Cache under Independent Stationary Request Processes

The modeling and analysis of an LRU cache is extremely challenging as exact results for the main performance metrics (e.g. hit rate) are either lacking or cannot be used because of their high computational complexity for large caches. As a result, various approximations have been proposed. The state-of-the-art method is the so-called TTL approximation, first proposed and shown to be asymptotically exact for IRM requests by Fagin. It has been applied to various other workload models and numerically demonstrated to be accurate but without theoretical justification. In this paper we provide theoretical justification for the approximation in the case where distinct contents are described by independent stationary and ergodic processes. We show that this approximation is exact as the cache size and the number of contents go to infinity. This extends earlier results for the independent reference model. Moreover, we establish results not only for the aggregate cache hit probability but also for every individual content. Last, we obtain bounds on the rate of convergence.

cs.PF↗

arXiv2018

Temporal Coherent and Graph Optimized Manifold Ranking for Visual Tracking

Recently, weighted patch representation has been widely studied for alleviating the impact of background information included in bounding box to improve visual tracking results. However, existing weighted patch representation models generally exploit spatial structure information among patches in each frame separately which ignore (1) unary featureof each patch and (2) temporal correlation among patches in different frames. To address this problem, we propose a novel unified temporal coherence and graph optimized ranking model for weighted patch representation in visual tracking problem. There are three main contributions of this paper. First, we propose to employ a flexible graph ranking for patch weight computation which exploits both structure information among patches and unary feature of each patch simultaneously. Second, we propose a new more discriminative ranking model by further considering the temporal correlation among patches in different frames. Third, a neighborhood preserved, low-rank graph is learned and incorporated to build a unified optimized ranking model. Experiments on two benchmark datasets show the benefits of our model.

cs.CV↗

arXiv2018

On decompositions and approximations of conjugate partial-symmetric complex tensors

Conjugate partial-symmetric (CPS) tensors are the high-order generalization of Hermitian matrices. As the role played by Hermitian matrices in matrix theory and quadratic optimization, CPS tensors have shown growing interest recently in tensor theory and optimization, particularly in many application-driven complex polynomial optimization problems. In this paper, we study CPS tensors with a focus on ranks, rank-one decompositions and approximations, as well as their applications. The analysis is conducted along side with a more general class of complex tensors called partial-symmetric tensors. We prove constructively that any CPS tensor can be decomposed into a sum of rank-one CPS tensors, which provides an alternative definition of CPS tensors via linear combinations of rank-one CPS tensors. Three types of ranks for CPS tensors are defined and shown to be different in general. This leads to the invalidity of the conjugate version of Comon's conjecture. We then study rank-one approximations and matricizations of CPS tensors. By carefully unfolding CPS tensors to Hermitian matrices, rank-one equivalence can be preserved. This enables us to develop new convex optimization models and algorithms to compute best rank-one approximation of CPS tensors. Numerical experiments from various data are performed to justify the capability of our methods.

math.OC↗

arXiv2018

Highly accurate model for prediction of lung nodule malignancy with CT scans

Computed tomography (CT) examinations are commonly used to predict lung nodule malignancy in patients, which are shown to improve noninvasive early diagnosis of lung cancer. It remains challenging for computational approaches to achieve performance comparable to experienced radiologists. Here we present NoduleX, a systematic approach to predict lung nodule malignancy from CT data, based on deep learning convolutional neural networks (CNN). For training and validation, we analyze >1000 lung nodules in images from the LIDC/IDRI cohort. All nodules were identified and classified by four experienced thoracic radiologists who participated in the LIDC project. NoduleX achieves high accuracy for nodule malignancy classification, with an AUC of ~0.99. This is commensurate with the analysis of the dataset by experienced radiologists. Our approach, NoduleX, provides an effective framework for highly accurate nodule malignancy prediction with the model trained on a large patient population. Our results are replicable with software available at http://bioinformatics.astate.edu/NoduleX.

cs.CV↗

arXiv2018

Structured Nonconvex and Nonsmooth Optimization: Algorithms and Iteration Complexity Analysis

Nonconvex and nonsmooth optimization problems are frequently encountered in much of statistics, business, science and engineering, but they are not yet widely recognized as a technology in the sense of scalability. A reason for this relatively low degree of popularity is the lack of a well developed system of theory and algorithms to support the applications, as is the case for its convex counterpart. This paper aims to take one step in the direction of disciplined nonconvex and nonsmooth optimization. In particular, we consider in this paper some constrained nonconvex optimization models in block decision variables, with or without coupled affine constraints. In the case of without coupled constraints, we show a sublinear rate of convergence to an $ε$-stationary solution in the form of variational inequality for a generalized conditional gradient method, where the convergence rate is shown to be dependent on the Hölderian continuity of the gradient of the smooth part of the objective. For the model with coupled affine constraints, we introduce corresponding $ε$-stationarity conditions, and apply two proximal-type variants of the ADMM to solve such a model, assuming the proximal ADMM updates can be implemented for all the block variables except for the last block, for which either a gradient step or a majorization-minimization step is implemented. We show an iteration complexity bound of $O(1/ε^2)$ to reach an $ε$-stationary solution for both algorithms. Moreover, we show that the same iteration complexity of a proximal BCD method follows immediately. Numerical results are provided to illustrate the efficacy of the proposed algorithms for tensor robust PCA.

math.OC↗

arXiv2017

A Unified Scheme to Accelerate Adaptive Cubic Regularization and Gradient Methods for Convex Optimization

In this paper we propose a unified two-phase scheme for convex optimization to accelerate: (1) the adaptive cubic regularization methods with exact/inexact Hessian matrices, and (2) the adaptive gradient method, without any knowledge of the Lipschitz constants for the gradient or the Hessian. This is achieved by tuning the parameters used in the algorithm $\textit{adaptively}$ in its process of progression, which can be viewed as a relaxation over the existing algorithms in the literature. Under the assumption that the sub-problems can be solved approximately, we establish overall iteration complexity bounds for three newly proposed algorithms to obtain an $ε$-approximate solution. Specifically, we show that the adaptive cubic regularization methods with the exact/inexact Hessian matrix both achieve an iteration complexity in the order of $O\left( 1 / ε^{1/3} \right)$, which matches that of the original accelerated cubic regularization method presented in [Nesterov-2008-Accelerating] assuming the availability of the exact Hessian information and the Lipschitz constants, and that the sub-problems are solved to optimality. Under the same two-phase adaptive acceleration framework, the gradient method achieves an iteration complexity in the order $O\left( 1 / ε^{1/2} \right)$, which is known to be best possible (cf. Nesterov-2013-Introductory). Our numerical experiment results show a clear effect of acceleration displayed in the adaptive Newton method with cubic regularization on a set of regularized logistic regression instances.

math.OC↗

arXiv2017

The discrete moment problem with nonconvex shape constraints

The discrete moment problem is a foundational problem in distribution-free robust optimization, where the goal is to find a worst-case distribution that satisfies a given set of moments. This paper studies the discrete moment problems with additional "shape constraints" that guarantee the worst case distribution is either log-concave or has an increasing failure rate. These classes of shape constraints have not previously been studied in the literature, in part due to their inherent nonconvexities. Nonetheless, these classes of distributions are useful in practice. We characterize the structure of optimal extreme point distributions by developing new results in reverse convex optimization, a lesser-known tool previously employed in designing global optimization algorithms. We are able to show, for example, that an optimal extreme point solution to a moment problem with $m$ moments and log-concave shape constraints is piecewise geometric with at most $m$ pieces. Moreover, this structure allows us to design an exact algorithm for computing optimal solutions in a low-dimensional space of parameters. Moreover, We describe a computational approach to solving these low-dimensional problems, including numerical results for a representative set of instances.

math.OC↗

arXiv2017

L1-norm Error Function Robustness and Outlier Regularization

In many real-world applications, data come with corruptions, large errors or outliers. One popular approach is to use L1-norm function. However, the robustness of L1-norm function is not well understood so far. In this paper, we present a new outlier regularization framework to understand and analyze the robustness of L1-norm function. There are two main features for the proposed outlier regularization. (1) A key property of outlier regularization is that how far an outlier lies away from its theoretically predicted value does not affect the final regularization and analysis results. (2) Another important feature of outlier regularization is that it has an equivalent continuous representation that closely relates to L1 function. This provides a new way to understand and analyze the robustness of L1 function. We apply our outlier regularization framework to PCA and propose an outlier regularized PCA (ORPCA) model. Comparing to the trace-normbased robust PCA, ORPCA has several benefits: (1) It does not suffer singular value suppression. (2) It can retain small high rank components which help retain fine details of data. (3) ORPCA can be computed more efficiently.

cs.CV↗

arXiv2017

Vector Transport-Free SVRG with General Retraction for Riemannian Optimization: Complexity Analysis and Practical Implementation

In this paper, we propose a vector transport-free stochastic variance reduced gradient (SVRG) method with general retraction for empirical risk minimization over Riemannian manifold. Existing SVRG methods on manifold usually consider a specific retraction operation, and involve additional computational costs such as parallel transport or vector transport. The vector transport-free SVRG with general retraction we propose in this paper handles general retraction operations, and do not need additional computational costs mentioned above. As a result, we name our algorithm S-SVRG, where the first "S" means simple. We analyze the iteration complexity of S-SVRG for obtaining an $ε$-stationary point and its local linear convergence by assuming the Łojasiewicz inequality, which naturally holds for PCA and holds with high probability for matrix completion problem. We also incorporate the Barzilai-Borwein step size and design a very practical S-SVRG-BB method. Numerical results on PCA and matrix completion problems are reported to demonstrate the efficiency of our methods.

math.OC↗

arXiv2017

Multiple Images Recovery Using a Single Affine Transformation

In many real-world applications, image data often come with noises, corruptions or large errors. One approach to deal with noise image data is to use data recovery techniques which aim to recover the true uncorrupted signals from the observed noise images. In this paper, we first introduce a novel corruption recovery transformation (CRT) model which aims to recover multiple (or a collection of) corrupted images using a single affine transformation. Then, we show that the introduced CRT can be efficiently constructed through learning from training data. Once CRT is learned, we can recover the true signals from the new incoming/test corrupted images explicitly. As an application, we apply our CRT to image recognition task. Experimental results on six image datasets demonstrate that the proposed CRT model is effective in recovering noise image data and thus leads to better recognition results.

cs.CV↗

arXiv2017

On the Duration and Intensity of Competitions in Nonlinear Pólya Urn Processes with Fitness

Cumulative advantage (CA) refers to the notion that accumulated resources foster the accumulation of further resources in competitions, a phenomenon that has been empirically observed in various contexts. The oldest and arguably simplest mathematical model that embodies this general principle is the Pólya urn process, which finds applications in a myriad of problems. The original model captures the dynamics of competitions between two equally fit agents under linear CA effects, which can be readily generalized to incorporate different fitnesses and nonlinear CA effects. We study two statistics of competitions under the generalized model, namely duration (i.e., time of the last tie) and intensity (i.e., number of ties). We give rigorous mathematical characterizations of the tail distributions of both duration and intensity under the various regimes for fitness and nonlinearity, which reveal very interesting behaviors. For example, fitness superiority induces much shorter competitions in the sublinear regime while much longer competitions in the superlinear regime. Our findings can shed light on the application of Pólya urn processes in more general contexts where fitness and nonlinearity may be present.

cs.PF↗

arXiv2017

Auxiliary Problem Principle of augmented Lagrangian with Varying Core Functions for Large-Scale Structured Convex Problems

The auxiliary problem principle of augmented Lagrangian (APP-AL), proposed by Cohen and Zhu (1984), aims to find the solution of a constrained optimization problem through a sequence of auxiliary problems involving augmented Lagrangian. The merits of this approach are two folds. First, the core function is usually separable, which makes the subproblems at each step decomposable and particularly attractive for parallel computing. Second, the choice of the core function is quite flexible. Consequently, by carefully specifying this function, APP-AL may reduce to some standard optimization algorithms. In this paper, we pursue enhancing such flexibility by allowing the core function to be non-identical at each step of the algorithm, and name it varying auxiliary problem principle (VAPP-AL). Depending on the problem structure, the varying core functions in VAPP-AL can be adapted to design new flexible and suitable algorithm for parallel and distributed computing. The convergence and O(1/t) convergence rate of VAPP-AL for convex problem with coupling objective and constraints is proved. Moreover, if this function is specialized to be quadratic, an o(1/t) convergence rate can be established. Interestingly, the new VAPP framework can cover several variants of Jacobian type augmented Lagrangian decomposition methods as special cases. Furthermore, our technique works for the convex problem with nonseparable objective and multi-blocks coupled linear constraints, which usually can not be handled by ADMM.

math.OC↗

arXiv2016

Approximation Algorithms for Optimization of Real-Valued General Conjugate Complex Forms

Complex polynomial optimization has recently gained more and more attention in both theory and practice. In this paper, we study the optimization of a real-valued general conjugate complex form over various popular constraint sets including the m-th roots of complex unity, the complex unit circle, and the complex unit sphere. A real-valued general conjugate complex form is a homogenous polynomial function of complex variables as well as their conjugates, and always takes real values. General conjugate form optimization is a wide class of complex polynomial optimization models, which include many homogenous polynomial optimization in the real domain with either discrete or continuous variables, and Hermitian quadratic form optimization as well as its higher degree extensions. All the problems under consideration are NP-hard in general and we focus on polynomial-time approximation algorithms with worst-case performance ratios. These approximation ratios improve previous results when restricting our problems to some special classes of complex polynomial optimization, and improve or equate previous results when restricting our problems to some special classes of polynomial optimization in the real domain. These algorithms are based on tensor relaxation and random sampling. Our novel technical contributions are to establish the first set of probability lower bounds for random sampling over the m-th root of unity, the complex unit circle, and the complex unit sphere, and propose the first polarization formula linking general conjugate forms and complex multilinear forms.

math.OC↗

arXiv2016

Generalized R-squared for Detecting Dependence

Detecting dependence between two random variables is a fundamental problem. Although the Pearson correlation is effective for capturing linear dependency, it can be entirely powerless for detecting nonlinear and/or heteroscedastic patterns. We introduce a new measure, G-squared, to test whether two univariate random variables are independent and to measure the strength of their relationship. The G-squared is almost identical to the square of the Pearson correlation coefficient, R-squared, for linear relationships with constant error variance, and has the intuitive meaning of the piecewise R-squared between the variables. It is particularly effective in handling nonlinearity and heteroscedastic errors. We propose two estimators of G-squared and show their consistency. Simulations demonstrate that G-squared estimates are among the most powerful test statistics compared with several state-of-the-art methods.

stat.ME↗