arXiv subjects

Hu Wang

Publications and source records attributed to Hu Wang.

At least 73 records · Page 4Linked to original sources

arXiv2022

M^4I: Multi-modal Models Membership Inference

With the development of machine learning techniques, the attention of research has been moved from single-modal learning to multi-modal learning, as real-world data exist in the form of different modalities. However, multi-modal models often carry more information than single-modal models and they are usually applied in sensitive scenarios, such as medical report generation or disease identification. Compared with the existing membership inference against machine learning classifiers, we focus on the problem that the input and output of the multi-modal models are in different modalities, such as image captioning. This work studies the privacy leakage of multi-modal models through the lens of membership inference attack, a process of determining whether a data record involves in the model training process or not. To achieve this, we propose Multi-modal Models Membership Inference (M^4I) with two attack methods to infer the membership status, named metric-based (MB) M^4I and feature-based (FB) M^4I, respectively. More specifically, MB M^4I adopts similarity metrics while attacking to infer target data membership. FB M^4I uses a pre-trained shadow multi-modal feature extractor to achieve the purpose of data inference attack by comparing the similarities from extracted input and output features. Extensive experimental results show that both attack methods can achieve strong performances. Respectively, 72.5% and 94.83% of attack success rates on average can be obtained under unrestricted scenarios. Moreover, we evaluate multiple defense mechanisms against our attacks. The source code of M^4I attacks is publicly available at https://github.com/MultimodalMI/Multimodal-membership-inference.git.

cs.LG↗

arXiv2022

Document-aware Positional Encoding and Linguistic-guided Encoding for Abstractive Multi-document Summarization

One key challenge in multi-document summarization is to capture the relations among input documents that distinguish between single document summarization (SDS) and multi-document summarization (MDS). Few existing MDS works address this issue. One effective way is to encode document positional information to assist models in capturing cross-document relations. However, existing MDS models, such as Transformer-based models, only consider token-level positional information. Moreover, these models fail to capture sentences' linguistic structure, which inevitably causes confusions in the generated summaries. Therefore, in this paper, we propose document-aware positional encoding and linguistic-guided encoding that can be fused with Transformer architecture for MDS. For document-aware positional encoding, we introduce a general protocol to guide the selection of document encoding functions. For linguistic-guided encoding, we propose to embed syntactic dependency relations into the dependency relation mask with a simple but effective non-linear encoding learner for feature learning. Extensive experiments show the proposed model can generate summaries with high quality.

cs.CL↗

arXiv2022

Uncertainty-aware Multi-modal Learning via Cross-modal Random Network Prediction

Multi-modal learning focuses on training models by equally combining multiple input data modalities during the prediction process. However, this equal combination can be detrimental to the prediction accuracy because different modalities are usually accompanied by varying levels of uncertainty. Using such uncertainty to combine modalities has been studied by a couple of approaches, but with limited success because these approaches are either designed to deal with specific classification or segmentation problems and cannot be easily translated into other tasks, or suffer from numerical instabilities. In this paper, we propose a new Uncertainty-aware Multi-modal Learner that estimates uncertainty by measuring feature density via Cross-modal Random Network Prediction (CRNP). CRNP is designed to require little adaptation to translate between different prediction tasks, while having a stable training process. From a technical point of view, CRNP is the first approach to explore random network prediction to estimate uncertainty and to combine multi-modal data. Experiments on two 3D multi-modal medical image segmentation tasks and three 2D multi-modal computer vision classification tasks show the effectiveness, adaptability and robustness of CRNP. Also, we provide an extensive discussion on different fusion functions and visualization to validate the proposed model.

cs.CV↗

arXiv2022

Dependency Structure for News Document Summarization

In this work, we develop a neural network based model which leverages dependency parsing to capture cross-positional dependencies and grammatical structures. With the help of linguistic signals, sentence-level relations can be correctly captured, thus improving news documents summarization performance. Empirical studies demonstrate that this simple but effective method outperforms existing works on the benchmark dataset. Extensive analyses examine different settings and configurations of the proposed model which provide a good reference to the community.

cs.CL↗

arXiv2021

Multi-document Summarization via Deep Learning Techniques: A Survey

Multi-document summarization (MDS) is an effective tool for information aggregation that generates an informative and concise summary from a cluster of topic-related documents. Our survey, the first of its kind, systematically overviews the recent deep learning based MDS models. We propose a novel taxonomy to summarize the design strategies of neural networks and conduct a comprehensive summary of the state-of-the-art. We highlight the differences between various objective functions that are rarely discussed in the existing literature. Finally, we propose several future directions pertaining to this new and exciting field.

cs.CL↗

arXiv2021

Fed-EINI: An Efficient and Interpretable Inference Framework for Decision Tree Ensembles in Federated Learning

The increasing concerns about data privacy and security drive an emerging field of studying privacy-preserving machine learning from isolated data sources, i.e., federated learning. A class of federated learning, vertical federated learning, where different parties hold different features for common users, has a great potential of driving a great variety of business cooperation among enterprises in many fields. In machine learning, decision tree ensembles such as gradient boosting decision trees (GBDT) and random forest are widely applied powerful models with high interpretability and modeling efficiency. However, stateof-art vertical federated learning frameworks adapt anonymous features to avoid possible data breaches, makes the interpretability of the model compromised. To address this issue in the inference process, in this paper, we firstly make a problem analysis about the necessity of disclosure meanings of feature to Guest Party in vertical federated learning. Then we find the prediction result of a tree could be expressed as the intersection of results of sub-models of the tree held by all parties. With this key observation, we protect data privacy and allow the disclosure of feature meaning by concealing decision paths and adapt a communication-efficient secure computation method for inference outputs. The advantages of Fed-EINI will be demonstrated through both theoretical analysis and extensive numerical results. We improve the interpretability of the model by disclosing the meaning of features while ensuring efficiency and accuracy.

cs.LG↗

arXiv2021

Risk and return prediction for pricing portfolios of non-performing consumer credit

We design a system for risk-analyzing and pricing portfolios of non-performing consumer credit loans. The rapid development of credit lending business for consumers heightens the need for trading portfolios formed by overdue loans as a manner of risk transferring. However, the problem is nontrivial technically and related research is absent. We tackle the challenge by building a bottom-up architecture, in which we model the distribution of every single loan's repayment rate, followed by modeling the distribution of the portfolio's overall repayment rate. To address the technical issues encountered, we adopt the approaches of simultaneous quantile regression, R-copula, and Gaussian one-factor copula model. To our best knowledge, this is the first study that successfully adopts a bottom-up system for analyzing credit portfolio risks of consumer loans. We conduct experiments on a vast amount of data and prove that our methodology can be applied successfully in real business tasks.

q-fin.RM↗

arXiv2021

A Physics-Data-Driven Bayesian Method for Heat Conduction Problems

In this study, a novel physics-data-driven Bayesian method named Heat Conduction Equation assisted Bayesian Neural Network (HCE-BNN) is proposed. The HCE-BNN is constructed based on the Bayesian neural network, it is a physics-informed machine learning strategy. Compared with the existed pure data driven method, to acquire physical consistency and better performance of the data-driven model, the heat conduction equation is embedded into the loss function of the HCE-BNN as a regularization term. Hence, the proposed method can build a more reliable model by physical constraints with less data. The HCE-BNN can handle the forward and inverse problems consistently, that is, to infer unknown responses from known partial responses, or to identify boundary conditions or material parameters from known responses. Compared with the exact results, the test results demonstrate that the proposed method can be applied to both heat conduction forward and inverse problems successfully. In addition, the proposed method can be implemented with the noisy data and gives the corresponding uncertainty quantification for the solutions.

math.NA↗

arXiv2021

Delayed Rewards Calibration via Reward Empirical Sufficiency

Appropriate credit assignment for delay rewards is a fundamental challenge for reinforcement learning. To tackle this problem, we introduce a delay reward calibration paradigm inspired from a classification perspective. We hypothesize that well-represented state vectors share similarities with each other since they contain the same or equivalent essential information. To this end, we define an empirical sufficient distribution, where the state vectors within the distribution will lead agents to environmental reward signals in the consequent steps. Therefore, a purify-trained classifier is designed to obtain the distribution and generate the calibrated rewards. We examine the correctness of sufficient state extraction by tracking the real-time extraction and building different reward functions in environments. The results demonstrate that the classifier could generate timely and accurate calibrated rewards. Moreover, the rewards are able to make the model training process more efficient. Finally, we identify and discuss that the sufficient states extracted by our model resonate with the observations of humans.

cs.LG↗

arXiv2021

A multi-grid sampling multi-scale method for crack initiation and propagation

In this study, a multi-grid sampling multi-scale (MGSMS) method is proposed by coupling with finite element (FEM), extended finite element (XFEM) and molecular dynamics (MD) methods.Crack is studied comprehensively from microscopic initiations to macroscopic propagation by MGSMS method. In order to establish the coupling relationship between macroscopic and microscopic model, multi-grid FEM is used to transmit the macroscopic displacement boundary conditions to the atomic model and the multi-grid XFEM is used to feedback the microscopic crack initiations to the macroscopic model. Moreover, an image recognition based crack extracting method is proposed to extract the crack coordinate from the MD result files of efficiently and the Latin hypercube sampling method is used to reduce the computational cost of MD. Numerical results show that MGSMS method can be used to calculate micro-crack initiations and transmit it to the macro-crack model. The crack initiation and propagation simulation of plate under mode I loading is completed.

math.NA↗

arXiv2021

Oriole: Thwarting Privacy against Trustworthy Deep Learning Models

Deep Neural Networks have achieved unprecedented success in the field of face recognition such that any individual can crawl the data of others from the Internet without their explicit permission for the purpose of training high-precision face recognition models, creating a serious violation of privacy. Recently, a well-known system named Fawkes (published in USENIX Security 2020) claimed this privacy threat can be neutralized by uploading cloaked user images instead of their original images. In this paper, we present Oriole, a system that combines the advantages of data poisoning attacks and evasion attacks, to thwart the protection offered by Fawkes, by training the attacker face recognition model with multi-cloaked images generated by Oriole. Consequently, the face recognition accuracy of the attack model is maintained and the weaknesses of Fawkes are revealed. Experimental results show that our proposed Oriole system is able to effectively interfere with the performance of the Fawkes system to achieve promising attacking results. Our ablation study highlights multiple principal factors that affect the performance of the Oriole system, including the DSSIM perturbation budget, the ratio of leaked clean user images, and the numbers of multi-cloaks for each uncloaked image. We also identify and discuss at length the vulnerabilities of Fawkes. We hope that the new methodology presented in this paper will inform the security community of a need to design more robust privacy-preserving deep learning models.

cs.CR↗

arXiv2021

Multi-intersection Traffic Optimisation: A Benchmark Dataset and a Strong Baseline

The control of traffic signals is fundamental and critical to alleviate traffic congestion in urban areas. However, it is challenging since traffic dynamics are complicated in real-world scenarios. Because of the high complexity of the optimisation problem for modelling the traffic, experimental settings of existing works are often inconsistent. Moreover, it is not trivial to control multiple intersections properly in real complex traffic scenarios due to its vast state and action space. Failing to take intersection topology relations into account also results in inferior solutions. To address these issues, in this work we carefully design our settings and propose a new dataset including both synthetic and real traffic data in more complex scenarios. Additionally, we propose a novel baseline model with strong performance. It is based on deep reinforcement learning with an encoder-decoder structure: an edge-weighted graph convolutional encoder to excavate multi-intersection relations; and an unified structure decoder to jointly model multiple junctions in a comprehensive manner, which significantly reduces the number of the model parameters. By doing so, the proposed model is able to effectively deal with the multi-intersection traffic optimisation problem. Models are trained/tested on both synthetic and real maps and traffic data with the Simulation of Urban Mobility (SUMO) simulator. Experimental results show that the proposed model surpasses multiple competitive methods.

eess.SY↗

arXiv2021

Fully Quantized Image Super-Resolution Networks

With the rising popularity of intelligent mobile devices, it is of great practical significance to develop accurate, realtime and energy-efficient image Super-Resolution (SR) inference methods. A prevailing method for improving the inference efficiency is model quantization, which allows for replacing the expensive floating-point operations with efficient fixed-point or bitwise arithmetic. To date, it is still challenging for quantized SR frameworks to deliver feasible accuracy-efficiency trade-off. Here, we propose a Fully Quantized image Super-Resolution framework (FQSR) to jointly optimize efficiency and accuracy. In particular, we target on obtaining end-to-end quantized models for all layers, especially including skip connections, which was rarely addressed in the literature. We further identify training obstacles faced by low-bit SR networks and propose two novel methods accordingly. The two difficulites are caused by 1) activation and weight distributions being vastly distinctive in different layers; 2) the inaccurate approximation of the quantization. We apply our quantization scheme on multiple mainstream super-resolution architectures, including SRResNet, SRGAN and EDSR. Experimental results show that our FQSR using low bits quantization can achieve on par performance compared with the full-precision counterparts on five benchmark datasets and surpass state-of-the-art quantized SR methods with significantly reduced computational cost and memory consumption.

eess.IV↗

arXiv2021

Fatigue crack propagation in carbon steel using RVE based model

A representative volume element (RVE) based multi-scale method is proposed to investigate the mechanism of fatigue crack propagation by the molecular dynamics (MD) and the extended finite element methods(XFEM) in this study. An atomic model of carbon steel plate is built to study the behavior of fatigue crack at the micro scale by MD method. Then the RVE model for fatigue crack propagation should be built by fitting the data which was obtained from the MD result with the Paris law model. Moreover, the effect of micro-structural defects including interstitial atoms, vacancies have also been considered in this study. The results indicate that the micro-structural defects can deeply influence the values of Paris law constants and the life of the specimen can be evaluated by the proposed method.

math.NA↗

arXiv2020

Memory-Gated Recurrent Networks

The essence of multivariate sequential learning is all about how to extract dependencies in data. These data sets, such as hourly medical records in intensive care units and multi-frequency phonetic time series, often time exhibit not only strong serial dependencies in the individual components (the "marginal" memory) but also non-negligible memories in the cross-sectional dependencies (the "joint" memory). Because of the multivariate complexity in the evolution of the joint distribution that underlies the data generating process, we take a data-driven approach and construct a novel recurrent network architecture, termed Memory-Gated Recurrent Networks (mGRN), with gates explicitly regulating two distinct types of memories: the marginal memory and the joint memory. Through a combination of comprehensive simulation studies and empirical experiments on a range of public datasets, we show that our proposed mGRN architecture consistently outperforms state-of-the-art architectures targeting multivariate time series.

cs.LG↗

arXiv2020

Years Delayed X-ray Afterglows of TDEs Originated from Wind-Torus Interactions

Tidal disruption events (TDEs) occurred in active galactic nuclei (AGNs) are a special class of sources with outstanding scientific significance. TDEs can generate ultrafast winds, which should almost inevitably collide with the preexisting AGN dusty tori. We perform analytical calculations and simulations on the wind-torus interactions and find such a process can generate considerable X-ray afterglow radiation several years or decades later after the TDE outburst. This provides a new origin for the years delayed X-rays in TDEs. The X-ray luminosity can reach 10^{41-42} erg/s, and the light curve characteristics depend on the parameters of winds and tori. We apply the model to two TDE candidates, and provide lower limits on the masses of the disrupted stars, as well as rigorous constraints on the gas densities of tori. Our results suggest that the observations of the time delay, spectral shape, luminosity and the light curve of the X-ray afterglow can be used to constrain the physical parameters of both TDE winds and tori, including the wind velocity, wind density and torus density.

astro-ph.HE↗

arXiv2020

Molecular dynamics simulation of crack growth in mono-crystal nickel with voids and inclusions

In this study, the crack propagation of the pre-cracked mono-crystal nickel with the voids and inclusions has been investigated by molecular dynamics simulations. Different sizes of voids, inclusions and materials of inclusions are used to fully study the effect of the voids and inclusions during the crack propagation process. The dislocations evolution, stress distribution and crack length are analyzed as the associated mechanical properties. The results indicate that the voids and inclusions can change the path of crack propagation of the pre-cracked mono-crystal nickel. Moreover, the results show that the voids and inclusions can lead a better resistance to plastic deformation of the mono-crystal and the inclusions can make the system more difficult to fracture.

cs.CE↗

arXiv2020

Randomized Online CP Decomposition

CANDECOMP/PARAFAC (CP) decomposition has been widely used to deal with multi-way data. For real-time or large-scale tensors, based on the ideas of randomized-sampling CP decomposition algorithm and online CP decomposition algorithm, a novel CP decomposition algorithm called randomized online CP decomposition (ROCP) is proposed in this paper. The proposed algorithm can avoid forming full Khatri-Rao product, which leads to boost the speed largely and reduce memory usage. The experimental results on synthetic data and real-world data show the ROCP algorithm is able to cope with CP decomposition for large-scale tensors with arbitrary number of dimensions. In addition, ROCP can reduce the computing time and memory usage dramatically, especially for large-scale tensors.

cs.LG↗