arXiv subjects

Jiawei Li

Publications and source records attributed to Jiawei Li.

At least 199 records · Page 11Linked to original sources

arXiv2021

Towards Explainable Multi-Party Learning: A Contrastive Knowledge Sharing Framework

Multi-party learning provides solutions for training joint models with decentralized data under legal and practical constraints. However, traditional multi-party learning approaches are confronted with obstacles such as system heterogeneity, statistical heterogeneity, and incentive design. How to deal with these challenges and further improve the efficiency and performance of multi-party learning has become an urgent problem to be solved. In this paper, we propose a novel contrastive multi-party learning framework for knowledge refinement and sharing with an accountable incentive mechanism. Since the existing naive model parameter averaging method is contradictory to the learning paradigm of neural networks, we simulate the process of human cognition and communication, and analogy multi-party learning as a many-to-one knowledge sharing problem. The approach is capable of integrating the acquired explicit knowledge of each client in a transparent manner without privacy disclosure, and it reduces the dependence on data distribution and communication environments. The proposed scheme achieves significant improvement in model performance in a variety of scenarios, as we demonstrated through experiments on several real-world datasets.

cs.LG↗

arXiv2021

Analytics and Machine Learning in Vehicle Routing Research

The Vehicle Routing Problem (VRP) is one of the most intensively studied combinatorial optimisation problems for which numerous models and algorithms have been proposed. To tackle the complexities, uncertainties and dynamics involved in real-world VRP applications, Machine Learning (ML) methods have been used in combination with analytical approaches to enhance problem formulations and algorithmic performance across different problem solving scenarios. However, the relevant papers are scattered in several traditional research fields with very different, sometimes confusing, terminologies. This paper presents a first, comprehensive review of hybrid methods that combine analytical techniques with ML tools in addressing VRP problems. Specifically, we review the emerging research streams on ML-assisted VRP modelling and ML-assisted VRP optimisation. We conclude that ML can be beneficial in enhancing VRP modelling, and improving the performance of algorithms for both online and offline VRP optimisations. Finally, challenges and future opportunities of VRP research are discussed.

cs.LG↗

arXiv2021

The vortex dynamics in incompressible viscous turbulent flows

In this paper, we consider turbulence from a geometric perspective based on the vorticity equations for incompressible viscous fluid flows. We derive several quantitative statements about the statistics of turbulent flows. In particular we establish an entropy decay inequality which involves the turbulent kinetic energy and $L^{1}$-enstrophy, and we identify the small time scale of the vorticity broken down and the vorticity creation under the universality assumption of small time scales of turbulence flows.

math.AP↗

arXiv2020

MetaAugment: Sample-Aware Data Augmentation Policy Learning

Automated data augmentation has shown superior performance in image recognition. Existing works search for dataset-level augmentation policies without considering individual sample variations, which are likely to be sub-optimal. On the other hand, learning different policies for different samples naively could greatly increase the computing cost. In this paper, we learn a sample-aware data augmentation policy efficiently by formulating it as a sample reweighting problem. Specifically, an augmentation policy network takes a transformation and the corresponding augmented image as inputs, and outputs a weight to adjust the augmented image loss computed by a task network. At training stage, the task network minimizes the weighted losses of augmented training images, while the policy network minimizes the loss of the task network on a validation set via meta-learning. We theoretically prove the convergence of the training procedure and further derive the exact convergence rate. Superior performance is achieved on widely-used benchmarks including CIFAR-10/100, Omniglot, and ImageNet.

cs.LG↗

arXiv2020

Amodal Segmentation Based on Visible Region Segmentation and Shape Prior

Almost all existing amodal segmentation methods make the inferences of occluded regions by using features corresponding to the whole image. This is against the human's amodal perception, where human uses the visible part and the shape prior knowledge of the target to infer the occluded region. To mimic the behavior of human and solve the ambiguity in the learning, we propose a framework, it firstly estimates a coarse visible mask and a coarse amodal mask. Then based on the coarse prediction, our model infers the amodal mask by concentrating on the visible region and utilizing the shape prior in the memory. In this way, features corresponding to background and occlusion can be suppressed for amodal mask estimation. Consequently, the amodal mask would not be affected by what the occlusion is given the same visible regions. The leverage of shape prior makes the amodal mask estimation more robust and reasonable. Our proposed model is evaluated on three datasets. Experiments show that our proposed model outperforms existing state-of-the-art methods. The visualization of shape prior indicates that the category-specific feature in the codebook has certain interpretability.

cs.CV↗

arXiv2020

VEGA: Towards an End-to-End Configurable AutoML Pipeline

Automated Machine Learning (AutoML) is an important industrial solution for automatic discovery and deployment of the machine learning models. However, designing an integrated AutoML system faces four great challenges of configurability, scalability, integrability, and platform diversity. In this work, we present VEGA, an efficient and comprehensive AutoML framework that is compatible and optimized for multiple hardware platforms. a) The VEGA pipeline integrates various modules of AutoML, including Neural Architecture Search (NAS), Hyperparameter Optimization (HPO), Auto Data Augmentation, Model Compression, and Fully Train. b) To support a variety of search algorithms and tasks, we design a novel fine-grained search space and its description language to enable easy adaptation to different search algorithms and tasks. c) We abstract the common components of deep learning frameworks into a unified interface. VEGA can be executed with multiple back-ends and hardwares. Extensive benchmark experiments on multiple tasks demonstrate that VEGA can improve the existing AutoML algorithms and discover new high-performance models against SOTA methods, e.g. the searched DNet model zoo for Ascend 10x faster than EfficientNet-B5 and 9.2x faster than RegNetX-32GF on ImageNet. VEGA is open-sourced at https://github.com/huawei-noah/vega.

cs.CV↗

arXiv2020

JSRT: James-Stein Regression Tree

Regression tree (RT) has been widely used in machine learning and data mining community. Given a target data for prediction, a regression tree is first constructed based on a training dataset before making prediction for each leaf node. In practice, the performance of RT relies heavily on the local mean of samples from an individual node during the tree construction/prediction stage, while neglecting the global information from different nodes, which also plays an important role. To address this issue, we propose a novel regression tree, named James-Stein Regression Tree (JSRT) by considering global information from different nodes. Specifically, we incorporate the global mean information based on James-Stein estimator from different nodes during the construction/predicton stage. Besides, we analyze the generalization error of our method under the mean square error (MSE) metric. Extensive experiments on public benchmark datasets verify the effectiveness and efficiency of our method, and demonstrate the superiority of our method over other RT prediction methods.

cs.LG↗

arXiv2020

Rectified Decision Trees: Towards Interpretability, Compression and Empirical Soundness

How to obtain a model with good interpretability and performance has always been an important research topic. In this paper, we propose rectified decision trees (ReDT), a knowledge distillation based decision trees rectification with high interpretability, small model size, and empirical soundness. Specifically, we extend the impurity calculation and the pure ending condition of the classical decision tree to propose a decision tree extension that allows the use of soft labels generated by a well-trained teacher model in training and prediction process. It is worth noting that for the acquisition of soft labels, we propose a new multiple cross-validation based method to reduce the effects of randomness and overfitting. These approaches ensure that ReDT retains excellent interpretability and even achieves fewer nodes than the decision tree in the aspect of compression while having relatively good performance. Besides, in contrast to traditional knowledge distillation, back propagation of the student model is not necessarily required in ReDT, which is an attempt of a new knowledge distillation approach. Extensive experiments are conducted, which demonstrates the superiority of ReDT in interpretability, compression, and empirical soundness.

cs.LG↗

arXiv2020

Fluctuations of a nonlinear stochastic heat equation in dimensions three and higher

We study the solution to a nonlinear stochastic heat equation in $d\geq 3$. The equation is driven by a Gaussian multiplicative noise that is white in time and smooth in space. For a small coupling constant, we prove (i) the solution converges to the stationary distribution in large time; (ii) the diffusive scale fluctuations are described by the Edwards-Wilkinson equation.

math.PR↗

arXiv2020

Rectified Decision Trees: Exploring the Landscape of Interpretable and Effective Machine Learning

Interpretability and effectiveness are two essential and indispensable requirements for adopting machine learning methods in reality. In this paper, we propose a knowledge distillation based decision trees extension, dubbed rectified decision trees (ReDT), to explore the possibility of fulfilling those requirements simultaneously. Specifically, we extend the splitting criteria and the ending condition of the standard decision trees, which allows training with soft labels while preserving the deterministic splitting paths. We then train the ReDT based on the soft label distilled from a well-trained teacher model through a novel jackknife-based method. Accordingly, ReDT preserves the excellent interpretable nature of the decision trees while having a relatively good performance. The effectiveness of adopting soft labels instead of hard ones is also analyzed empirically and theoretically. Surprisingly, experiments indicate that the introduction of soft labels also reduces the model size compared with the standard decision trees from the aspect of the total nodes and rules, which is an unexpected gift from the `dark knowledge' distilled from the teacher model.

cs.LG↗

arXiv2020

HRINet: Alternative Supervision Network for High-resolution CT image Interpolation

Image interpolation in medical area is of high importance as most 3D biomedical volume images are sampled where the distance between consecutive slices significantly greater than the in-plane pixel size due to radiation dose or scanning time. Image interpolation creates a number of new slices between known slices in order to obtain an isotropic volume image. The results can be used for the higher quality of 3D reconstruction and visualization of human body structures. Semantic interpolation on the manifold has been proved to be very useful for smoothing image interpolation. Nevertheless, all previous methods focused on low-resolution image interpolation, and most of them work poorly on high-resolution image. We propose a novel network, High Resolution Interpolation Network (HRINet), aiming at producing high-resolution CT image interpolations. We combine the idea of ACAI and GANs, and propose a novel idea of alternative supervision method by applying supervised and unsupervised training alternatively to raise the accuracy of human organ structures in CT while keeping high quality. We compare an MSE based and a perceptual based loss optimizing methods for high quality interpolation, and show the tradeoff between the structural correctness and sharpness. Our experiments show the great improvement on 256 2 and 5122 images quantitatively and qualitatively.

eess.IV↗

arXiv2020

Multinomial Random Forest: Toward Consistency and Privacy-Preservation

Despite the impressive performance of random forests (RF), its theoretical properties have not been thoroughly understood. In this paper, we propose a novel RF framework, dubbed multinomial random forest (MRF), to analyze the \emph{consistency} and \emph{privacy-preservation}. Instead of deterministic greedy split rule or with simple randomness, the MRF adopts two impurity-based multinomial distributions to randomly select a split feature and a split value respectively. Theoretically, we prove the consistency of the proposed MRF and analyze its privacy-preservation within the framework of differential privacy. We also demonstrate with multiple datasets that its performance is on par with the standard RF. To the best of our knowledge, MRF is the first consistent RF variant that has comparable performance to the standard RF.

cs.LG↗

arXiv2020

RF-Rhythm: Secure and Usable Two-Factor RFID Authentication

Passive RFID technology is widely used in user authentication and access control. We propose RF-Rhythm, a secure and usable two-factor RFID authentication system with strong resilience to lost/stolen/cloned RFID cards. In RF-Rhythm, each legitimate user performs a sequence of taps on his/her RFID card according to a self-chosen secret melody. Such rhythmic taps can induce phase changes in the backscattered signals, which the RFID reader can detect to recover the user's tapping rhythm. In addition to verifying the RFID card's identification information as usual, the backend server compares the extracted tapping rhythm with what it acquires in the user enrollment phase. The user passes authentication checks if and only if both verifications succeed. We also propose a novel phase-hopping protocol in which the RFID reader emits Continuous Wave (CW) with random phases for extracting the user's secret tapping rhythm. Our protocol can prevent a capable adversary from extracting and then replaying a legitimate tapping rhythm from sniffed RFID signals. Comprehensive user experiments confirm the high security and usability of RF-Rhythm with false-positive and false-negative rates close to zero.

eess.SP↗

arXiv2020

Mesoscale investigations of fluid-solid interaction: Liquid slip flow in a parallel-plate microchannel

Liquid slip flow with a Knudsen number Kn = 0.001-0.1 plays a dominant role in confined flow channels. Its physical origin can be attributed to the intermolecular fluid-solid (F-S) interaction force. To this end, we propose herein continuous force functions (decaying either exponentially or by a power law) between fluid particles and two confined flat walls in the framework of the mesoscopic lattice Boltzmann model (LBM). The analytical solutions for density profile, velocity profile, slip length, and permeability ratio are derived and related to the mesoscale F-S interaction parameters and the size of the gap of the flow channel. Through nondimensionalization of the analytical solutions, we obtain the dimensionless numbers that indicate the key feature of slip-flow systems for each of the proposed force functions. The analytical solutions are strictly consistent with the LBM solutions. We suggest reasonable ranges for the F-S interaction parameters based on the observed range of density ratio (film fluid to bulk fluid) and the increasing permeability ratios with narrowing gap size. Within the given range of interaction parameters, simple relationships between permeability ratios and dimensionless numbers are provided by fitting. The curves for continuous F-S interaction force with two free parameters are calibrated for a hydrophobic surface by using LBM simulations, which were validated a priori by comparison with the slip velocity profile measured in a benchmark flow experiment. The mesoscopic LBM model based on the proposed F-S interaction force functions provides a robust framework to elucidate the physical process of liquid slip flow.

physics.flu-dyn↗

arXiv2019

Large deviation principle for fractional Brownian motion with respect to capacity

We show that fractional Brownian motion(fBM) defined via Volterra integral representation with Hurst parameter $H\geq\frac{1}{2}$ is a quasi-surely defined Wiener functional on classical Wiener space,and we establish the large deviation principle(LDP) for such fBM with respect to $(p,r)$-capacity on classical Wiener space in Malliavin's sense.

math.PR↗

arXiv2018

Fine properties of fractional Brownian motions on Wiener space

We study several important fine properties for the family of fractional Brownian motions with Hurst parameter $H$ under the $(p,r)$-capacity on classical Wiener space introduced by Malliavin. We regard fractional Brownian motions as Wiener functionals via the integral representation discovered by Decreusefond and Üstünel, and show non differentiability, modulus of continuity, law of iterated Logarithm(LIL) and self-avoiding properties of fractional Brownian motion sample paths using Malliavin calculus as well as the tools developed in the previous work by Fukushima, Takeda and etc. for Brownian motion case.

math.PR↗

arXiv2017

Dynamic Label Graph Matching for Unsupervised Video Re-Identification

Label estimation is an important component in an unsupervised person re-identification (re-ID) system. This paper focuses on cross-camera label estimation, which can be subsequently used in feature learning to learn robust re-ID models. Specifically, we propose to construct a graph for samples in each camera, and then graph matching scheme is introduced for cross-camera labeling association. While labels directly output from existing graph matching methods may be noisy and inaccurate due to significant cross-camera variations, this paper proposes a dynamic graph matching (DGM) method. DGM iteratively updates the image graph and the label estimation process by learning a better feature space with intermediate estimated labels. DGM is advantageous in two aspects: 1) the accuracy of estimated labels is improved significantly with the iterations; 2) DGM is robust to noisy initial training data. Extensive experiments conducted on three benchmarks including the large-scale MARS dataset show that DGM yields competitive performance to fully supervised baselines, and outperforms competing unsupervised learning methods.

cs.CV↗

arXiv2016

On Equilibria of N-seller and N-buyer Bargaining Games

A group of players which contain n sellers and n buyers bargain over the partitions of n pies. A seller(/buyer) has to reach an agreement with a buyer (/seller) on the division of a pie. The players bargain in a system like the stock market: each seller(buyer) can either offer a selling(buying) price to all buyers(sellers) or accept a price offered by another buyer(seller). The offered prices are known to all. Once a player accepts a price offered by another one, the division of a pie between them is determined. Each player has a constant discounting factor and the discounting factors of all players are common knowledge. In this article, we prove that the equilibrium of this bargaining problem is a unanimous division rate, which is equivalent to Nash bargaining equilibrium of a two-player bargaining game in which the discounting factors of two players are the average of n buyers and the average of n sellers respectively. This result shows the relevance between bargaining equilibrium and general equilibrium of markets.

cs.GT↗