arXiv subjects

Longfei Li

Publications and source records attributed to Longfei Li.

At least 55 records · Page 3Linked to original sources

arXiv2023

Learning from Negative User Feedback and Measuring Responsiveness for Sequential Recommenders

Sequential recommenders have been widely used in industry due to their strength in modeling user preferences. While these models excel at learning a user's positive interests, less attention has been paid to learning from negative user feedback. Negative user feedback is an important lever of user control, and comes with an expectation that recommenders should respond quickly and reduce similar recommendations to the user. However, negative feedback signals are often ignored in the training objective of sequential retrieval models, which primarily aim at predicting positive user interactions. In this work, we incorporate explicit and implicit negative user feedback into the training objective of sequential recommenders in the retrieval stage using a "not-to-recommend" loss function that optimizes for the log-likelihood of not recommending items with negative feedback. We demonstrate the effectiveness of this approach using live experiments on a large-scale industrial recommender system. Furthermore, we address a challenge in measuring recommender responsiveness to negative feedback by developing a counterfactual simulation framework to compare recommender responses between different user actions, showing improved responsiveness from the modeling change.

cs.IR↗

arXiv2023

Leveraging Large Language Models for Pre-trained Recommender Systems

Recent advancements in recommendation systems have shifted towards more comprehensive and personalized recommendations by utilizing large language models (LLM). However, effectively integrating LLM's commonsense knowledge and reasoning abilities into recommendation systems remains a challenging problem. In this paper, we propose RecSysLLM, a novel pre-trained recommendation model based on LLMs. RecSysLLM retains LLM reasoning and knowledge while integrating recommendation domain knowledge through unique designs of data, training, and inference. This allows RecSysLLM to leverage LLMs' capabilities for recommendation tasks in an efficient, unified framework. We demonstrate the effectiveness of RecSysLLM on benchmarks and real-world scenarios. RecSysLLM provides a promising approach to developing unified recommendation systems by fully exploiting the power of pre-trained language models.

cs.IR↗

arXiv2023

ALT: An Automatic System for Long Tail Scenario Modeling

In this paper, we consider the problem of long tail scenario modeling with budget limitation, i.e., insufficient human resources for model training stage and limited time and computing resources for model inference stage. This problem is widely encountered in various applications, yet has received deficient attention so far. We present an automatic system named ALT to deal with this problem. Several efforts are taken to improve the algorithms used in our system, such as employing various automatic machine learning related techniques, adopting the meta learning philosophy, and proposing an essential budget-limited neural architecture search method, etc. Moreover, to build the system, many optimizations are performed from a systematic perspective, and essential modules are armed, making the system more feasible and efficient. We perform abundant experiments to validate the effectiveness of our system and demonstrate the usefulness of the critical modules in our system. Moreover, online results are provided, which fully verified the efficacy of our system.

cs.LG↗

arXiv2023

DCMT: A Direct Entire-Space Causal Multi-Task Framework for Post-Click Conversion Estimation

In recommendation scenarios, there are two long-standing challenges, i.e., selection bias and data sparsity, which lead to a significant drop in prediction accuracy for both Click-Through Rate (CTR) and post-click Conversion Rate (CVR) tasks. To cope with these issues, existing works emphasize on leveraging Multi-Task Learning (MTL) frameworks (Category 1) or causal debiasing frameworks (Category 2) to incorporate more auxiliary data in the entire exposure/inference space D or debias the selection bias in the click/training space O. However, these two kinds of solutions cannot effectively address the not-missing-at-random problem and debias the selection bias in O to fit the inference in D. To fill the research gaps, we propose a Direct entire-space Causal Multi-Task framework, namely DCMT, for post-click conversion prediction in this paper. Specifically, inspired by users' decision process of conversion, we propose a new counterfactual mechanism to debias the selection bias in D, which can predict the factual CVR and the counterfactual CVR under the soft constraint of a counterfactual prior knowledge. Extensive experiments demonstrate that our DCMT can improve the state-of-the-art methods by an average of 1.07% in terms of CVR AUC on the five offline datasets and 0.75% in terms of PV-CVR on the online A/B test (the Alipay Search). Such improvements can increase millions of conversions per week in real industrial applications, e.g., the Alipay Search.

cs.IR↗

arXiv2022

Robust Direct Learning for Causal Data Fusion

In the era of big data, the explosive growth of multi-source heterogeneous data offers many exciting challenges and opportunities for improving the inference of conditional average treatment effects. In this paper, we investigate homogeneous and heterogeneous causal data fusion problems under a general setting that allows for the presence of source-specific covariates. We provide a direct learning framework for integrating multi-source data that separates the treatment effect from other nuisance functions, and achieves double robustness against certain misspecification. To improve estimation precision and stability, we propose a causal information-aware weighting function motivated by theoretical insights from the semiparametric efficiency theory; it assigns larger weights to samples containing more causal information with high interpretability. We introduce a two-step algorithm, the weighted multi-source direct learner, based on constructing a pseudo-outcome and regressing it on covariates under a weighted least square criterion; it offers us a powerful tool for causal data fusion, enjoying the advantages of easy implementation, double robustness and model flexibility. In simulation studies, we demonstrate the effectiveness of our proposed methods in both homogeneous and heterogeneous causal data fusion scenarios.

stat.ML↗

arXiv2022

Expert Knowledge-guided Geometric Representation Learning for Magnetic Resonance Imaging-based Glioma Grading

Radiomics and deep learning have shown high popularity in automatic glioma grading. Radiomics can extract hand-crafted features that quantitatively describe the expert knowledge of glioma grades, and deep learning is powerful in extracting a large number of high-throughput features that facilitate the final classification. However, the performance of existing methods can still be improved as their complementary strengths have not been sufficiently investigated and integrated. Furthermore, lesion maps are usually needed for the final prediction at the testing phase, which is very troublesome. In this paper, we propose an expert knowledge-guided geometric representation learning (ENROL) framework . Geometric manifolds of hand-crafted features and learned features are constructed to mine the implicit relationship between deep learning and radiomics, and therefore to dig mutual consent and essential representation for the glioma grades. With a specially designed manifold discrepancy measurement, the grading model can exploit the input image data and expert knowledge more effectively in the training phase and get rid of the requirement of lesion segmentation maps at the testing phase. The proposed framework is flexible regarding deep learning architectures to be utilized. Three different architectures have been evaluated and five models have been compared, which show that our framework can always generate promising results.

eess.IV↗

arXiv2021

Radiomic biomarker extracted from PI-RADS 3 patients support more eìcient and robust prostate cancer diagnosis: a multi-center study

Prostate Imaging Reporting and Data System (PI-RADS) based on multi-parametric MRI classiêes patients into 5 categories (PI-RADS 1-5) for routine clinical diagnosis guidance. However, there is no consensus on whether PI-RADS 3 patients should go through biopsies. Mining features from these hard samples (HS) is meaningful for physicians to achieve accurate diagnoses. Currently, the mining of HS biomarkers is insuìcient, and the eéectiveness and robustness of HS biomarkers for prostate cancer diagnosis have not been explored. In this study, biomarkers from diéerent data distributions are constructed. Results show that HS biomarkers can achieve better performances in diéerent data distributions.

eess.IV↗

arXiv2021

SAIL: Self-Augmented Graph Contrastive Learning

This paper studies learning node representations with graph neural networks (GNNs) for unsupervised scenario. Specifically, we derive a theoretical analysis and provide an empirical demonstration about the non-steady performance of GNNs over different graph datasets, when the supervision signals are not appropriately defined. The performance of GNNs depends on both the node feature smoothness and the locality of graph structure. To smooth the discrepancy of node proximity measured by graph topology and node feature, we proposed SAIL - a novel \underline{S}elf-\underline{A}ugmented graph contrast\underline{i}ve \underline{L}earning framework, with two complementary self-distilling regularization modules, \emph{i.e.}, intra- and inter-graph knowledge distillation. We demonstrate the competitive performance of SAIL on a variety of graph applications. Even with a single GNN layer, SAIL has consistently competitive or even better performance on various benchmark datasets, comparing with state-of-the-art baselines.

cs.LG↗

arXiv2021

Multi-Interactive Attention Network for Fine-grained Feature Learning in CTR Prediction

In the Click-Through Rate (CTR) prediction scenario, user's sequential behaviors are well utilized to capture the user interest in the recent literature. However, despite being extensively studied, these sequential methods still suffer from three limitations. First, existing methods mostly utilize attention on the behavior of users, which is not always suitable for CTR prediction, because users often click on new products that are irrelevant to any historical behaviors. Second, in the real scenario, there exist numerous users that have operations a long time ago, but turn relatively inactive in recent times. Thus, it is hard to precisely capture user's current preferences through early behaviors. Third, multiple representations of user's historical behaviors in different feature subspaces are largely ignored. To remedy these issues, we propose a Multi-Interactive Attention Network (MIAN) to comprehensively extract the latent relationship among all kinds of fine-grained features (e.g., gender, age and occupation in user-profile). Specifically, MIAN contains a Multi-Interactive Layer (MIL) that integrates three local interaction modules to capture multiple representations of user preference through sequential behaviors and simultaneously utilize the fine-grained user-specific as well as context information. In addition, we design a Global Interaction Module (GIM) to learn the high-order interactions and balance the different impacts of multiple features. Finally, Offline experiment results from three datasets, together with an Online A/B test in a large-scale recommendation system, demonstrate the effectiveness of our proposed approach.

cs.IR↗

arXiv2021

A Unified Framework for Cross-Domain and Cross-System Recommendations

Cross-Domain Recommendation (CDR) and Cross-System Recommendation (CSR) have been proposed to improve the recommendation accuracy in a target dataset (domain/system) with the help of a source one with relatively richer information. However, most existing CDR and CSR approaches are single-target, namely, there is a single target dataset, which can only help the target dataset and thus cannot benefit the source dataset. In this paper, we focus on three new scenarios, i.e., Dual-Target CDR (DTCDR), Multi-Target CDR (MTCDR), and CDR+CSR, and aim to improve the recommendation accuracy in all datasets simultaneously for all scenarios. To do this, we propose a unified framework, called GA (based on Graph embedding and Attention techniques), for all three scenarios. In GA, we first construct separate heterogeneous graphs to generate more representative user and item embeddings. Then, we propose an element-wise attention mechanism to effectively combine the embeddings of common entities (users/items) learned from different datasets. Moreover, to avoid negative transfer, we further propose a Personalized training strategy to minimize the embedding difference of common entities between a richer dataset and a sparser dataset, deriving three new models, i.e., GA-DTCDR-P, GA-MTCDR-P, and GA-CDR+CSR-P, for the three scenarios respectively. Extensive experiments conducted on four real-world datasets demonstrate that our proposed GA models significantly outperform the state-of-the-art approaches.

cs.IR↗

arXiv2021

Stable finite difference methods for Kirchhoff-Love plates on overlapping grids

In this work, we propose and develop efficient and accurate numerical methods for solving the Kirchhoff-Love plate model in domains with complex geometries. The algorithms proposed here employ curvilinear finite-difference methods for spatial discretization of the governing PDEs on general composite overlapping grids. The coupling of different components of the composite overlapping grid is through numerical interpolations. However, interpolations introduce perturbation to the finite-difference discretization, which causes numerical instability for time-stepping schemes used to advance the resulted semi-discrete system. To address the instability, we propose to add a fourth-order hyper-dissipation to the spatially discretized system to stabilize its time integration; this additional dissipation term captures the essential upwinding effect of the original upwind scheme. The investigation of strategies for incorporating the upwind dissipation term into several time-stepping schemes (both explicit and implicit) leads to the development of four novel algorithms. For each algorithm, formulas for determining a stable time step and a sufficient dissipation coefficient on curvilinear grids are derived by performing a local Fourier analysis. Quadratic eigenvalue problems for a simplified model plate in 1D domain are considered to reveal the weak instability due to the presence of interpolating equations in the spatial discretization. This model problem is further investigated for the stabilization effects of the proposed algorithms. Carefully designed numerical experiments are carried out to validate the accuracy and stability of the proposed algorithms, followed by two benchmark problems to demonstrate the capability and efficiency of our approach for solving realistic applications. Results that concern the performance of the proposed algorithms are also presented.

math.NA↗

arXiv2021

Cross-Domain Recommendation: Challenges, Progress, and Prospects

To address the long-standing data sparsity problem in recommender systems (RSs), cross-domain recommendation (CDR) has been proposed to leverage the relatively richer information from a richer domain to improve the recommendation performance in a sparser domain. Although CDR has been extensively studied in recent years, there is a lack of a systematic review of the existing CDR approaches. To fill this gap, in this paper, we provide a comprehensive review of existing CDR approaches, including challenges, research progress, and future directions. Specifically, we first summarize existing CDR approaches into four types, including single-target CDR, multi-domain recommendation, dual-target CDR, and multi-target CDR. We then present the definitions and challenges of these CDR approaches. Next, we propose a full-view categorization and new taxonomies on these approaches and report their research progress in detail. In the end, we share several promising research directions in CDR.

cs.IR↗

arXiv2020

Stable and accurate numerical methods for generalized Kirchhoff-Love plates

Efficient and accurate numerical algorithms are developed to solve a generalized Kirchhoff-Love plate model subject to three common physical boundary conditions: (i) clamped; (ii) simply supported; and (iii) free. We solve the model equation by discretizing the spatial derivatives using second-order finite-difference schemes, and then advancing the semi-discrete problem in time with either an explicit predictor-corrector or an implicit Newmark-Beta time-stepping algorithm. Stability analysis is conducted for the schemes and the results are used to determine stable time steps in practice. A series of carefully chosen test problems are solved to demonstrate the properties and applications of our numerical approaches. The numerical results confirm the stability and 2nd-order accuracy of the algorithms, and are also comparable with experiments for similar thin plates. As an application, we illustrate a strategy to identify the natural frequencies of a plate using our numerical methods in conjunction with a fast Fourier transformation (FFT) power spectrum analysis of the computed data. Then we take advantage of one of the computed natural frequencies to simulate the interesting physical phenomena known as resonance and beat for a generalized Kirchhoff-Love plate.

math.NA↗

arXiv2020

A Time Attention based Fraud Transaction Detection Framework

With online payment platforms being ubiquitous and important, fraud transaction detection has become the key for such platforms, to ensure user account safety and platform security. In this work, we present a novel method for detecting fraud transactions by leveraging patterns from both users' static profiles and users' dynamic behaviors in a unified framework. To address and explore the information of users' behaviors in continuous time spaces, we propose to use \emph{time attention based recurrent layers} to embed the detailed information of the time interval, such as the durations of specific actions, time differences between different actions and sequential behavior patterns,etc., in the same latent space. We further combine the learned embeddings and users' static profiles altogether in a unified framework. Extensive experiments validate the effectiveness of our proposed methods over state-of-the-art methods on various evaluation metrics, especially on \emph{recall at top percent} which is an important metric for measuring the balance between service experiences and risk of potential losses.

cs.LG↗

arXiv2020

Distributed Deep Forest and its Application to Automatic Detection of Cash-out Fraud

Internet companies are facing the need for handling large-scale machine learning applications on a daily basis and distributed implementation of machine learning algorithms which can handle extra-large scale tasks with great performance is widely needed. Deep forest is a recently proposed deep learning framework which uses tree ensembles as its building blocks and it has achieved highly competitive results on various domains of tasks. However, it has not been tested on extremely large scale tasks. In this work, based on our parameter server system, we developed the distributed version of deep forest. To meet the need for real-world tasks, many improvements are introduced to the original deep forest model, including MART (Multiple Additive Regression Tree) as base learners for efficiency and effectiveness consideration, the cost-based method for handling prevalent class-imbalanced data, MART based feature selection for high dimension data and different evaluation metrics for automatically determining of the cascade level. We tested the deep forest model on an extra-large scale task, i.e., automatic detection of cash-out fraud, with more than 100 millions of training samples. Experimental results showed that the deep forest model has the best performance according to the evaluation metrics from different perspectives even with very little effort for parameter tuning. This model can block fraud transactions in a large amount of money each day. Even compared with the best-deployed model, the deep forest model can additionally bring into a significant decrease in economic loss each day.

cs.LG↗

arXiv2020

Interpretable MTL from Heterogeneous Domains using Boosted Tree

Multi-task learning (MTL) aims at improving the generalization performance of several related tasks by leveraging useful information contained in them. However, in industrial scenarios, interpretability is always demanded, and the data of different tasks may be in heterogeneous domains, making the existing methods unsuitable or unsatisfactory. In this paper, following the philosophy of boosted tree, we proposed a two-stage method. In stage one, a common model is built to learn the commonalities using the common features of all instances. Different from the training of conventional boosted tree model, we proposed a regularization strategy and an early-stopping mechanism to optimize the multi-task learning process. In stage two, started by fitting the residual error of the common model, a specific model is constructed with the task-specific instances to further boost the performance. Experiments on both benchmark and real-world datasets validate the effectiveness of the proposed method. What's more, interpretability can be naturally obtained from the tree based method, satisfying the industrial needs.

cs.LG↗

arXiv2020

SAFE: Scalable Automatic Feature Engineering Framework for Industrial Tasks

Machine learning techniques have been widely applied in Internet companies for various tasks, acting as an essential driving force, and feature engineering has been generally recognized as a crucial tache when constructing machine learning systems. Recently, a growing effort has been made to the development of automatic feature engineering methods, so that the substantial and tedious manual effort can be liberated. However, for industrial tasks, the efficiency and scalability of these methods are still far from satisfactory. In this paper, we proposed a staged method named SAFE (Scalable Automatic Feature Engineering), which can provide excellent efficiency and scalability, along with requisite interpretability and promising performance. Extensive experiments are conducted and the results show that the proposed method can provide prominent efficiency and competitive effectiveness when comparing with other methods. What's more, the adequate scalability of the proposed method ensures it to be deployed in large scale industrial tasks.

cs.LG↗

arXiv2020

A split-step finite-element method for incompressible Navier-Stokes equations with high-order accuracy up-to the boundary

An efficient and accurate finite-element algorithm is described for the numerical solution of the incompressible Navier-Stokes (INS) equations. The new algorithm that solves the INS equations in a velocity-pressure reformulation is based on a split-step scheme in conjunction with the standard finite-element method. The split-step scheme employed for the temporal discretization of our algorithm completely separates the pressure updates from the solution of velocity variables. When the pressure equation is formed explicitly, the algorithm avoids solving a saddle-point problem; therefore, our algorithm has more flexibility in choosing finite-element spaces. For efficiency and robustness, Lagrange finite elements of equal order for both velocity and pressure are used. Motivated by a post-processing technique that calculates derivatives of a finite element solution with super-convergent error estimates, an alternative numerical boundary condition is proposed for the pressure equation at the discrete level. The new numerical pressure boundary condition that can be regarded as a better implementation of the compatibility boundary condition improves the boundary-layer errors of the pressure solution. Normal-mode analysis is performed using a simplified model problem on a uniform mesh to demonstrate the numerical properties of our methods. Convergence study using $\mathbb{P}_1$ elements confirms the analytical results and demonstrates that our algorithm with the new numerical boundary condition achieves the optimal second-order accuracy for both velocity and pressure up-to the boundary. Benchmark problems are also computed and carefully compared with existing studies. Finally, as an example to illustrate that our approach can be easily adapted for higher-order finite elements, we solve the classical flow-past-a-cylinder problem using $\mathbb{P}_n$ finite elements with $n\geq 1$.

math.NA↗