arXiv subjects

Jian Liang

Publications and source records attributed to Jian Liang.

At least 199 records · Page 11Linked to original sources

arXiv2021

On Evolving Attention Towards Domain Adaptation

Towards better unsupervised domain adaptation (UDA). Recently, researchers propose various domain-conditioned attention modules and make promising progresses. However, considering that the configuration of attention, i.e., the type and the position of attention module, affects the performance significantly, it is more generalized to optimize the attention configuration automatically to be specialized for arbitrary UDA scenario. For the first time, this paper proposes EvoADA: a novel framework to evolve the attention configuration for a given UDA task without human intervention. In particular, we propose a novel search space containing diverse attention configurations. Then, to evaluate the attention configurations and make search procedure UDA-oriented (transferability + discrimination), we apply a simple and effective evaluation strategy: 1) training the network weights on two domains with off-the-shelf domain adaptation methods; 2) evolving the attention configurations under the guide of the discriminative ability on the target domain. Experiments on various kinds of cross-domain benchmarks, i.e., Office-31, Office-Home, CUB-Paintings, and Duke-Market-1510, reveal that the proposed EvoADA consistently boosts multiple state-of-the-art domain adaptation approaches, and the optimal attention configurations help them achieve better performance.

cs.CV↗

arXiv2020

Bi-Classifier Determinacy Maximization for Unsupervised Domain Adaptation

Unsupervised domain adaptation challenges the problem of transferring knowledge from a well-labelled source domain to an unlabelled target domain. Recently,adversarial learning with bi-classifier has been proven effective in pushing cross-domain distributions close. Prior approaches typically leverage the disagreement between bi-classifier to learn transferable representations, however, they often neglect the classifier determinacy in the target domain, which could result in a lack of feature discriminability. In this paper, we present a simple yet effective method, namely Bi-Classifier Determinacy Maximization(BCDM), to tackle this problem. Motivated by the observation that target samples cannot always be separated distinctly by the decision boundary, here in the proposed BCDM, we design a novel classifier determinacy disparity (CDD) metric, which formulates classifier discrepancy as the class relevance of distinct target predictions and implicitly introduces constraint on the target feature discriminability. To this end, the BCDM can generate discriminative representations by encouraging target predictive outputs to be consistent and determined, meanwhile, preserve the diversity of predictions in an adversarial manner. Furthermore, the properties of CDD as well as the theoretical guarantees of BCDM's generalization bound are both elaborated. Extensive experiments show that BCDM compares favorably against the existing state-of-the-art domain adaptation methods.

cs.CV↗

arXiv2020

Domain Agnostic Learning for Unbiased Authentication

Authentication is the task of confirming the matching relationship between a data instance and a given identity. Typical examples of authentication problems include face recognition and person re-identification. Data-driven authentication could be affected by undesired biases, i.e., the models are often trained in one domain (e.g., for people wearing spring outfits) while applied in other domains (e.g., they change the clothes to summer outfits). Previous works have made efforts to eliminate domain-difference. They typically assume domain annotations are provided, and all the domains share classes. However, for authentication, there could be a large number of domains shared by different identities/classes, and it is impossible to annotate these domains exhaustively. It could make domain-difference challenging to model and eliminate. In this paper, we propose a domain-agnostic method that eliminates domain-difference without domain labels. We alternately perform latent domain discovery and domain-difference elimination until our model no longer detects domain-difference. In our approach, the latent domains are discovered by learning the heterogeneous predictive relationships between inputs and outputs. Then domain-difference is eliminated in both class-dependent and class-independent spaces to improve robustness of elimination. We further extend our method to a meta-learning framework to pursue more thorough domain-difference elimination. Comprehensive empirical evaluation results are provided to demonstrate the effectiveness and superiority of our proposed method.

stat.ML↗

arXiv2020

A Lane-Changing Prediction Method Based on Temporal Convolution Network

Lane-changing is an important driving behavior and unreasonable lane changes can result in potentially dangerous traffic collisions. Advanced Driver Assistance System (ADAS) can assist drivers to change lanes safely and efficiently. To capture the stochastic time series of lane-changing behavior, this study proposes a temporal convolutional network (TCN) to predict the long-term lane-changing trajectory and behavior. In addition, the convolutional neural network (CNN) and recurrent neural network (RNN) methods are considered as the benchmark models to demonstrate the learning ability of the TCN. The lane-changing dataset was collected by the driving simulator. The prediction performance of TCN is demonstrated from three aspects: different input variables, different input dimensions and different driving scenarios. Prediction results show that the TCN can accurately predict the long-term lane-changing trajectory and driving behavior with shorter computational time compared with two benchmark models. The TCN can provide accurate lane-changing prediction, which is one key information for the development of accurate ADAS.

cs.LG↗

arXiv2020

Reliable Evaluations for Natural Language Inference based on a Unified Cross-dataset Benchmark

Recent studies show that crowd-sourced Natural Language Inference (NLI) datasets may suffer from significant biases like annotation artifacts. Models utilizing these superficial clues gain mirage advantages on the in-domain testing set, which makes the evaluation results over-estimated. The lack of trustworthy evaluation settings and benchmarks stalls the progress of NLI research. In this paper, we propose to assess a model's trustworthy generalization performance with cross-datasets evaluation. We present a new unified cross-datasets benchmark with 14 NLI datasets, and re-evaluate 9 widely-used neural network-based NLI models as well as 5 recently proposed debiasing methods for annotation artifacts. Our proposed evaluation scheme and experimental baselines could provide a basis to inspire future reliable NLI research.

cs.CL↗

arXiv2020

Deep Learning-Based Least Square Forward-Backward Stochastic Differential Equation Solver for High-Dimensional Derivative Pricing

We propose a new forward-backward stochastic differential equation solver for high-dimensional derivatives pricing problems by combining deep learning solver with least square regression technique widely used in the least square Monte Carlo method for the valuation of American options. Our numerical experiments demonstrate the efficiency and accuracy of our least square backward deep neural network solver and its capability to provide accurate prices for complex early exercise derivatives such as callable yield notes. Our method can serve as a generic numerical solver for pricing derivatives across various asset groups, in particular, as an efficient means for pricing high-dimensional derivatives with early exercises features.

q-fin.CP↗

arXiv2020

Robust Finite Mixture Regression for Heterogeneous Targets

Finite Mixture Regression (FMR) refers to the mixture modeling scheme which learns multiple regression models from the training data set. Each of them is in charge of a subset. FMR is an effective scheme for handling sample heterogeneity, where a single regression model is not enough for capturing the complexities of the conditional distribution of the observed samples given the features. In this paper, we propose an FMR model that 1) finds sample clusters and jointly models multiple incomplete mixed-type targets simultaneously, 2) achieves shared feature selection among tasks and cluster components, and 3) detects anomaly tasks or clustered structure among tasks, and accommodates outlier samples. We provide non-asymptotic oracle performance bounds for our model under a high-dimensional learning framework. The proposed model is evaluated on both synthetic and real-world data sets. The results show that our model can achieve state-of-the-art performance.

stat.ML↗

arXiv2020

Hybrid Differentially Private Federated Learning on Vertically Partitioned Data

We present HDP-VFL, the first hybrid differentially private (DP) framework for vertical federated learning (VFL) to demonstrate that it is possible to jointly learn a generalized linear model (GLM) from vertically partitioned data with only a negligible cost, w.r.t. training time, accuracy, etc., comparing to idealized non-private VFL. Our work builds on the recent advances in VFL-based collaborative training among different organizations which rely on protocols like Homomorphic Encryption (HE) and Secure Multi-Party Computation (MPC) to secure computation and training. In particular, we analyze how VFL's intermediate result (IR) can leak private information of the training data during communication and design a DP-based privacy-preserving algorithm to ensure the data confidentiality of VFL participants. We mathematically prove that our algorithm not only provides utility guarantees for VFL, but also offers multi-level privacy, i.e. DP w.r.t. IR and joint differential privacy (JDP) w.r.t. model weights. Experimental results demonstrate that our work, under adequate privacy budgets, is quantitatively and qualitatively similar to GLMs, learned in idealized non-private VFL setting, rather than the increased cost in memory and processing time in most prior works based on HE or MPC. Our codes will be released if this paper is accepted.

cs.LG↗

arXiv2020

Ratio of strange to $u/d$ momentum fraction in disconnected insertions

The ratio of the strange quark momentum fraction $\langle x\rangle_{s+\bar{s}}$ to that of light quark $u$ or $d$ in disconnected insertions (DI) is calculated on the lattice with overlap fermions on four domain wall fermion ensembles. These ensembles cover three lattice spacings, three volumes and several pion masses including the physical one, from which a global fitting is carried out. A complete nonperturbative renormalization and the mixing between the quark and glue operators are taken into account. We find the ratio to be $\langle x\rangle_{s+\bar{s}}/\langle x\rangle_{u+\bar{u}} ({\rm DI})=0.795(79)(77)$ at $μ= 2$ GeV in the $\overline{\rm MS}$ scheme. This ratio can be used as a constraint to better determine the strange parton distribution especially in the small $x$ region in the global fittings of PDFs when the connected and disconnected sea are fitted and evolved separately, demonstrating a new way that connects lattice calculations with global analyses.

hep-ph↗

arXiv2020

PDFs and Neutrino-Nucleon Scattering from Hadronic Tensor

We review the Euclidean path-integral formulation of the nucleon hadronic tensor and classify the gauge invariant and topologically distinct insertions in terms of connected and disconnected insertions and also in terms of leading and higher-twist contributions in the DIS region. Converting the Euclidean hadronic tensor back to the Minkowski space requires solving an inverse problem of the Laplace transform. We have investigated several inverse algorithms and studied the pros and cons of each. We show a result with a relatively large momentum transfer ($Q^2 \sim 4\, {\rm GeV^2}$) to suppress the elastic scattering and reveal the contributions from the resonance and inelastic region of the neutrino-nucleon scattering. For elastic scattering, the hadronic tensor is the the product of the elastic form factors for the two corresponding currents. We checked numerically for the case of two charge vector currents ($V_4$) with the electric form factor calculated from the three-point function and found they agree within errors.

hep-lat↗

arXiv2020

Two Sides of the Same Coin: White-box and Black-box Attacks for Transfer Learning

Transfer learning has become a common practice for training deep learning models with limited labeled data in a target domain. On the other hand, deep models are vulnerable to adversarial attacks. Though transfer learning has been widely applied, its effect on model robustness is unclear. To figure out this problem, we conduct extensive empirical evaluations to show that fine-tuning effectively enhances model robustness under white-box FGSM attacks. We also propose a black-box attack method for transfer learning models which attacks the target model with the adversarial examples produced by its source model. To systematically measure the effect of both white-box and black-box attacks, we propose a new metric to evaluate how transferable are the adversarial examples produced by a source model to a target model. Empirical results show that the adversarial examples are more transferable when fine-tuning is used than they are when the two networks are trained independently.

cs.LG↗

arXiv2020

Adversarial Infidelity Learning for Model Interpretation

Model interpretation is essential in data mining and knowledge discovery. It can help understand the intrinsic model working mechanism and check if the model has undesired characteristics. A popular way of performing model interpretation is Instance-wise Feature Selection (IFS), which provides an importance score of each feature representing the data samples to explain how the model generates the specific output. In this paper, we propose a Model-agnostic Effective Efficient Direct (MEED) IFS framework for model interpretation, mitigating concerns about sanity, combinatorial shortcuts, model identifiability, and information transmission. Also, we focus on the following setting: using selected features to directly predict the output of the given model, which serves as a primary evaluation metric for model-interpretation methods. Apart from the features, we involve the output of the given model as an additional input to learn an explainer based on more accurate information. To learn the explainer, besides fidelity, we propose an Adversarial Infidelity Learning (AIL) mechanism to boost the explanation learning by screening relatively unimportant features. Through theoretical and experimental analysis, we show that our AIL mechanism can help learn the desired conditional distribution between selected features and targets. Moreover, we extend our framework by integrating efficient interpretation methods as proper priors to provide a warm start. Comprehensive empirical evaluation results are provided by quantitative metrics and human evaluation to demonstrate the effectiveness and superiority of our proposed method. Our code is publicly available online at https://github.com/langlrsw/MEED.

stat.ML↗

arXiv2020

A Balanced and Uncertainty-aware Approach for Partial Domain Adaptation

This work addresses the unsupervised domain adaptation problem, especially in the case of class labels in the target domain being only a subset of those in the source domain. Such a partial transfer setting is realistic but challenging and existing methods always suffer from two key problems, negative transfer and uncertainty propagation. In this paper, we build on domain adversarial learning and propose a novel domain adaptation method BA$^3$US with two new techniques termed Balanced Adversarial Alignment (BAA) and Adaptive Uncertainty Suppression (AUS), respectively. On one hand, negative transfer results in misclassification of target samples to the classes only present in the source domain. To address this issue, BAA pursues the balance between label distributions across domains in a fairly simple manner. Specifically, it randomly leverages a few source samples to augment the smaller target domain during domain alignment so that classes in different domains are symmetric. On the other hand, a source sample would be denoted as uncertain if there is an incorrect class that has a relatively high prediction score, and such uncertainty easily propagates to unlabeled target data around it during alignment, which severely deteriorates adaptation performance. Thus we present AUS that emphasizes uncertain samples and exploits an adaptive weighted complement entropy objective to encourage incorrect classes to have uniform and low prediction scores. Experimental results on multiple benchmarks demonstrate our BA$^3$US surpasses state-of-the-arts for partial domain adaptation tasks. Code is available at \url{https://github.com/tim-learn/BA3US}.

cs.CV↗

arXiv2020

Relation-Guided Representation Learning

Deep auto-encoders (DAEs) have achieved great success in learning data representations via the powerful representability of neural networks. But most DAEs only focus on the most dominant structures which are able to reconstruct the data from a latent space and neglect rich latent structural information. In this work, we propose a new representation learning method that explicitly models and leverages sample relations, which in turn is used as supervision to guide the representation learning. Different from previous work, our framework well preserves the relations between samples. Since the prediction of pairwise relations themselves is a fundamental problem, our model adaptively learns them from data. This provides much flexibility to encode real data manifold. The important role of relation and representation learning is evaluated on the clustering task. Extensive experiments on benchmark data sets demonstrate the superiority of our approach. By seeking to embed samples into subspace, we further show that our method can address the large-scale and out-of-sample problem.

cs.LG↗

arXiv2020

Parton distributions and lattice QCD calculations: toward 3D structure

The strong force which binds hadrons is described by the theory of Quantum Chromodynamics (QCD). Determining the character and manifestations of QCD is one of the most important and challenging outstanding issues necessary for a comprehensive understanding of the structure of hadrons. Within the context of the QCD parton picture, the Parton Distribution Functions (PDFs) have been remarkably successful in describing a wide variety of processes. However, these PDFs have generally been confined to the description of collinear partons within the hadron. New experiments and facilities provide the opportunity to additionally explore the transverse structure of hadrons which is described by Generalized Parton Distributions (GPDs) and Transverse Momentum Dependent Parton Distribution Functions (TMD PDFs). In our previous review, we compared and contrasted the two main approaches used to determine the collinear PDFs: the first based on perturbative QCD factorization theorems, and the second based on lattice QCD calculations. In the present report, we provide an update of recent progress on the collinear PDFs, and also expand the scope to encompass the generalized PDFs (GPDs and TMD PDFs). We review the current state of the various calculations, and consider what new data might be available in the near future. We also examine how a shared effort can foster dialog between the PDF and Lattice QCD communities, and yield improvements for these generalized PDFs.

hep-ph↗

arXiv2020

General-Purpose User Embeddings based on Mobile App Usage

In this paper, we report our recent practice at Tencent for user modeling based on mobile app usage. User behaviors on mobile app usage, including retention, installation, and uninstallation, can be a good indicator for both long-term and short-term interests of users. For example, if a user installs Snapseed recently, she might have a growing interest in photographing. Such information is valuable for numerous downstream applications, including advertising, recommendations, etc. Traditionally, user modeling from mobile app usage heavily relies on handcrafted feature engineering, which requires onerous human work for different downstream applications, and could be sub-optimal without domain experts. However, automatic user modeling based on mobile app usage faces unique challenges, including (1) retention, installation, and uninstallation are heterogeneous but need to be modeled collectively, (2) user behaviors are distributed unevenly over time, and (3) many long-tailed apps suffer from serious sparsity. In this paper, we present a tailored AutoEncoder-coupled Transformer Network (AETN), by which we overcome these challenges and achieve the goals of reducing manual efforts and boosting performance. We have deployed the model at Tencent, and both online/offline experiments from multiple domains of downstream applications have demonstrated the effectiveness of the output user embeddings.

cs.LG↗

arXiv2020

Roper State from Overlap Fermions

The Roper state is extracted with valence overlap fermions on a $2+1$-flavor domain-wall fermion lattice (spacing $a = 0.114$ fm and $m_π = 330$ MeV) using both the Sequential Empirical Bayes (SEB) method and the variational method. The results are consistent, provided that a large smearing-size interpolation operator is included in the variational calculation to have better overlap with the lowest radial excitation. Similar calculations carried out for an anisotropic clover lattice with similar parameters find the Roper $\approx 280$ MeV higher than that of the overlap fermion. The fact that the prediction of the Roper state by overlap fermions is consistently lower than those of clover fermions, chirally improved fermions, and twisted-mass fermions over a wide range of pion masses has been dubbed a "Roper puzzle." To understand the origin of this difference, we study the hairpin $Z$-diagram in the isovector scalar meson ($a_0$) correlator in the quenched approximation. Comparing the $a_0$ correlators for clover and overlap fermions, at a pion mass of 290 MeV, we find that the spectral weight of the ghost state with clover fermions is smaller than that of the overlap at $a = 0.12$ fm and $0.09$ fm, whereas the whole $a_0$ correlators of clover and overlap at $a = 0.06$ fm coincide within errors. This suggests that chiral symmetry is restored for clover at $a \le 0.06$ fm and that the Roper should come down at and below this $a$. We conclude that this work supports a resolution of the "Roper puzzle" due to $Z$-graph type chiral dynamics. This entails coupling to higher components in the Fock space (e.g. $Nπ$, $Nππ$ states) to induce the effective flavor-spin interaction between quarks as prescribed in the chiral quark model, resulting in the parity-reversal pattern as observed in the experimental excited states of $N, Δ$ and $Λ$.

hep-ph↗

arXiv2020

Trace anomaly and dynamical quark mass

We investigated the origin of the RI'/MOM quark mass under the Landau gauge at the non-perturbative scale, using the chiral fermion with different quark masses and lattice spacings. Our result confirms that such a mass is non-vanishing based on the linear extrapolation to the chiral and continuum limit, and shows that such a mass comes from the spontaneous chiral symmetry breaking induced by the near zero modes with the eigenvalue $λ<{\cal O}(5m_q)$, and is proportional to the quark matrix element of the trace anomaly at least down to $\sim $1.3 GeV.

hep-lat↗