arXiv subjects

Lixin Zhang

Publications and source records attributed to Lixin Zhang.

At least 37 records · Page 2Linked to original sources

arXiv2023

Graph Exploration Matters: Improving both individual-level and system-level diversity in WeChat Feed Recommender

There are roughly three stages in real industrial recommendation systems, candidates generation (retrieval), ranking and reranking. Individual-level diversity and system-level diversity are both important for industrial recommender systems. The former focus on each single user's experience, while the latter focus on the difference among users. Graph-based retrieval strategies are inevitably hijacked by heavy users and popular items, leading to the convergence of candidates for users and the lack of system-level diversity. Meanwhile, in the reranking phase, Determinantal Point Process (DPP) is deployed to increase individual-level diverisity. Heavily relying on the semantic information of items, DPP suffers from clickbait and inaccurate attributes. Besides, most studies only focus on one of the two levels of diversity, and ignore the mutual influence among different stages in real recommender systems. We argue that individual-level diversity and system-level diversity should be viewed as an integrated problem, and we provide an efficient and deployable solution for web-scale recommenders. Generally, we propose to employ the retrieval graph information in diversity-based reranking, by which to weaken the hidden similarity of items exposed to users, and consequently gain more graph explorations to improve the system-level diveristy. Besides, we argue that users' propensity for diversity changes over time in content feed recommendation. Therefore, with the explored graph, we also propose to capture the user's real-time personalized propensity to the diversity. We implement and deploy the combined system in WeChat App's Top Stories used by hundreds of millions of users. Offline simulations and online A/B tests show our solution can effectively improve both user engagement and system revenue.

cs.LG↗

arXiv2023

Small perturbations may change the sign of Lyapunov exponents for linear SDEs

In this paper, we study the existence of $n$-dimensional linear stochastic differential equations (SDEs) such that the sign of Lyapunov exponents is changed under an exponentially decaying perturbation. First, we show that the equation with all positive Lyapunov exponents will have $n-1$ linearly independent solutions with negative Lyapunov exponents under the perturbation. Meanwhile, we prove that the equation with all negative Lyapunov exponents will also have solutions with positive Lyapunov exponents under another similar perturbation. Finally, we also show that other three kinds of perturbations which appear at different positions of the equation will change the sign of Lyapunov exponents.

math.DS↗

arXiv2022

Robust parameter estimation of regression model under weakened moment assumptions

This paper provides some extended results on estimating parameter matrix of several regression models when the covariate or response possesses weaker moment condition. We study the $M$-estimator of Fan et al. (Ann Stat 49(3):1239--1266, 2021) for matrix completion model with $(1+ε)$-th moment noise. The corresponding phase transition phenomenon is observed. When $1> ε>0$, the robust estimator possesses a slower convergence rate compared with previous literature. For high dimensional multiple index coefficient model, we propose an improved estimator via applying the element-wise truncation method to handle heavy-tailed data with finite fourth moment. The extensive simulation study validates our theoretical results.

math.ST↗

arXiv2022

UFNRec: Utilizing False Negative Samples for Sequential Recommendation

Sequential recommendation models are primarily optimized to distinguish positive samples from negative ones during training in which negative sampling serves as an essential component in learning the evolving user preferences through historical records. Except for randomly sampling negative samples from a uniformly distributed subset, many delicate methods have been proposed to mine negative samples with high quality. However, due to the inherent randomness of negative sampling, false negative samples are inevitably collected in model training. Current strategies mainly focus on removing such false negative samples, which leads to overlooking potential user interests, lack of recommendation diversity, less model robustness, and suffering from exposure bias. To this end, we propose a novel method that can Utilize False Negative samples for sequential Recommendation (UFNRec) to improve model performance. We first devise a simple strategy to extract false negative samples and then transfer these samples to positive samples in the following training process. Furthermore, we construct a teacher model to provide soft labels for false negative samples and design a consistency loss to regularize the predictions of these samples from the student model and the teacher model. To the best of our knowledge, this is the first work to utilize false negative samples instead of simply removing them for the sequential recommendation. Experiments on three benchmark public datasets are conducted using three widely applied SOTA models. The experiment results demonstrate that our proposed UFNRec can effectively draw information from false negative samples and further improve the performance of SOTA models. The code is available at https://github.com/UFNRec-code/UFNRec.

cs.IR↗

arXiv2022

Single-shot Embedding Dimension Search in Recommender System

As a crucial component of most modern deep recommender systems, feature embedding maps high-dimensional sparse user/item features into low-dimensional dense embeddings. However, these embeddings are usually assigned a unified dimension, which suffers from the following issues: (1) high memory usage and computation cost. (2) sub-optimal performance due to inferior dimension assignments. In order to alleviate the above issues, some works focus on automated embedding dimension search by formulating it as hyper-parameter optimization or embedding pruning problems. However, they either require well-designed search space for hyperparameters or need time-consuming optimization procedures. In this paper, we propose a Single-Shot Embedding Dimension Search method, called SSEDS, which can efficiently assign dimensions for each feature field via a single-shot embedding pruning operation while maintaining the recommendation accuracy of the model. Specifically, it introduces a criterion for identifying the importance of each embedding dimension for each feature field. As a result, SSEDS could automatically obtain mixed-dimensional embeddings by explicitly reducing redundant embedding dimensions based on the corresponding dimension importance ranking and the predefined parameter budget. Furthermore, the proposed SSEDS is model-agnostic, meaning that it could be integrated into different base recommendation models. The extensive offline experiments are conducted on two widely used public datasets for CTR prediction tasks, and the results demonstrate that SSEDS can still achieve strong recommendation performance even if it has reduced 90\% parameters. Moreover, SSEDS has also been deployed on the WeChat Subscription platform for practical recommendation services. The 7-day online A/B test results show that SSEDS can significantly improve the performance of the online recommendation model.

cs.IR↗

arXiv2021

Robust covariance estimation for distributed principal component analysis

Fan et al. [$\mathit{Annals}$ $\mathit{of}$ $\mathit{Statistics}$ $\textbf{47}$(6) (2019) 3009-3031] constructed a distributed principal component analysis (PCA) algorithm to reduce the communication cost between multiple servers significantly. However, their algorithm's guarantee is only for sub-Gaussian data. Spurred by this deficiency, this paper enhances the effectiveness of their distributed PCA algorithm by utilizing robust covariance matrix estimators of Minsker [$\mathit{Annals}$ $\mathit{of}$ $\mathit{Statistics}$ $\textbf{46}$(6A) (2018) 2871-2903] and Ke et al. [$\mathit{Statistical}$ $\mathit{Science}$ $\textbf{34}$(3) (2019) 454-471] to tame heavy-tailed data. The theoretical results demonstrate that when the sampling distribution is symmetric innovation with the bounded fourth moment or asymmetric with the finite $6$-th moment, the statistical error rate of the final estimator produced by the robust algorithm is similar to that of sub-Gaussian tails. Extensive numerical trials support the theoretical analysis and indicate that our algorithm is robust to heavy-tailed data and outliers.

math.ST↗

arXiv2020

A Lightweight Isolation Mechanism for Secure Branch Predictors

Recently exposed vulnerabilities reveal the necessity to improve the security of branch predictors. Branch predictors record history about the execution of different programs, and such information from different processes are stored in the same structure and thus accessible to each other. This leaves the attackers with the opportunities for malicious training and malicious perception. Instead of flush-based or physical isolation of hardware resources, we want to achieve isolation of the content in these hardware tables with some lightweight processing using randomization as follows. (1) Content encoding. We propose to use hardware-based thread-private random numbers to encode the contents of the branch predictor tables (both direction and destination histories) which we call XOR-BP. Specifically, the data is encoded by XOR operation with the key before written in the table and decoded after read from the table. Such a mechanism obfuscates the information adding difficulties to cross-process or cross-privilege level analysis and perception. It achieves a similar effect of logical isolation but adds little in terms of space or time overheads. (2) Index encoding. We propose a randomized index mechanism of the branch predictor (Noisy-XOR-BP). Similar to the XOR-BP, another thread-private random number is used together with the branch instruction address as the input to compute the index of the branch predictor. This randomized indexing mechanism disrupts the correspondence between the branch instruction address and the branch predictor entry, thus increases the noise for malicious perception attacks. Our analyses using an FPGA-based RISC-V processor prototype and additional auxiliary simulations suggest that the proposed mechanisms incur a very small performance cost while providing strong protection.

cs.CR↗

arXiv2019

Interaction between Mo and intrinsic or extrinsic defects of Mo doped LiNbO$_3$ from first-principles calculations

Lithium niobate (LiNbO$_3$, LN) plays an important role in holographic storage, and molybdenum doped LiNbO$_3$ (LN:Mo) is an excellent candidate for holographic data storage. In this paper, the basic features of Mo doped LiNbO$_3$, such as the site preference, electronic structure, and the lattice distortions, have been explored from first-principles calculations. Mo substituting Nb with its highest charge state of +6 is found to be the most stable point defect form. The energy levels formed by Mo with different charge states are distributed in the band gap, which are responsible for the absorption in the visible region. The transition of Mo in different charge states implies molybdenum can serve as a photorefractive center in LN:Mo. In addition, the interactions between Mo and intrinsic or extrinsic point defects are also investigated in this work. Intrinsic defects $\tt V_{Li}^-$ could cause the movement of the $\tt Mo_{Nb}^+$ energy levels. The exploration of Mo, Mg co-doped LiNbO$_3$ reveals that although Mg ion could not shift the energy level of Mo, it can change the distribution of electrons in Mo and Mg co-doped LN (LN:Mo,Mg) which help with the photorefractive phenomenon.

physics.chem-ph↗

arXiv2019

Enabling Privacy-Preserving, Compute- and Data-Intensive Computing using Heterogeneous Trusted Execution Environment

There is an urgent demand for privacy-preserving techniques capable of supporting compute and data intensive (CDI) computing in the era of big data. However, none of existing TEEs can truly support CDI computing tasks, as CDI requires high throughput accelerators like GPU and TPU but TEEs do not offer security protection of such accelerators. This paper present HETEE (Heterogeneous TEE), the first design of TEE capable of strongly protecting heterogeneous computing with unsecure accelerators. HETEE is uniquely constructed to work with today's servers, and does not require any changes for existing commercial CPUs or accelerators. The key idea of our design runs security controller as a stand-alone computing system to dynamically adjust the boundary of between secure and insecure worlds through the PCIe switches, rendering the control of an accelerator to the host OS when it is not needed for secure computing, and shifting it back when it is. The controller is the only trust unit in the system and it runs the custom OS and accelerator runtimes, together with the encryption, authentication and remote attestation components. The host server and other computing systems communicate with controller through an in memory task queue that accommodates the computing tasks offloaded to HETEE, in the form of encrypted and signed code and data. Also, HETEE offers a generic and efficient programming model to the host CPU. We have implemented the HETEE design on a hardware prototype system, and evaluated it with large-scale Neural Networks inference and training tasks. Our evaluations show that HETEE can easily support such secure computing tasks and only incurs a 12.34% throughput overhead for inference and 9.87% overhead for training on average.

cs.CR↗

arXiv2019

XOS: An Application-Defined Operating System for Data Center Servers

Rapid growth of datacenter (DC) scale, urgency of cost control, increasing workload diversity, and huge software investment protection place unprecedented demands on the operating system (OS) efficiency, scalability, performance isolation, and backward-compatibility. The traditional OSes are not built to work with deep-hierarchy software stacks, large numbers of cores, tail latency guarantee, and increasingly rich variety of applications seen in modern DCs, and thus they struggle to meet the demands of such workloads. This paper presents XOS, an application-defined OS for modern DC servers. Our design moves resource management out of the OS kernel, supports customizable kernel subsystems in user space, and enables elastic partitioning of hardware resources. Specifically, XOS leverages modern hardware support for virtualization to move resource management functionality out of the conventional kernel and into user space, which lets applications achieve near bare-metal performance. We implement XOS on top of Linux to provide backward compatibility. XOS speeds up a set of DC workloads by up to 1.6X over our baseline Linux on a 24-core server, and outperforms the state-of-the-art Dune by up to 3.3X in terms of virtual memory management. In addition, XOS demonstrates good scalability and strong performance isolation.

cs.OS↗

arXiv2017

Three series theorem for independent random variables under sub-linear expectations with applications

In this paper, motived by the notion of independent and identically distributed random variables under the sub-linear expectation initiated by Peng, we give a theorem about the convergence of a random series and establish a three series theorem of independent random variables under the sub-linear expectations. As an application, we obtain the Marcinkiewicz's strong law of large numbers for independent and identically distributed random variables under the sub-linear expectations. The technical details are different from those for classical theorems because the sub-linear expectation and its related capacity are not additive.

math.PR↗

arXiv2017

Marcinkiewicz's strong law of large numbers for non-additive expectation

The sub-linear expectation space is a nonlinear expectation space having advantages of modelling the uncertainty of probability and distribution. In the sub-linear expectation space, we use capacity and sub-linear expectation to replace probability and expectation of classical probability theory. In this paper, the method of selecting subsequence is used to prove Marcinkiewicz type strong law of large numbers under sub-linear expectation space. This result is a natural extension of the classical Marcinkiewicz's strong law of large numbers to the case where the expectation is nonadditive. In addition, this paper also gives a theorem about convergence of a random series.

stat.ME↗

arXiv2016

Energy calculations of charged point defects on surfaces

We present a virtual ionic crystal (VIC) method to calculate energies of charged point defects on surfaces. No artificial charge but an actual zero-dimensional (0D) species is introduced to charge a defect. Effect of dielectric substrate on lattice energy is depressed through suitable configuration of the unit cell. The lattice energy approximates to Madelung energy with defect and 0D species considered as point charges in vacuum. Energy required to charge the defect is derived from charge quantity on the defect in VIC, energy of unit cell, and energy required to charge the 0D species.

cond-mat.mes-hall↗

arXiv2015

Understanding Big Data Analytic Workloads on Modern Processors

Big data analytics applications play a significant role in data centers, and hence it has become increasingly important to understand their behaviors in order to further improve the performance of data center computer systems, in which characterizing representative workloads is a key practical problem. In this paper, after investigating three most impor- tant application domains in terms of page views and daily visitors, we chose 11 repre- sentative data analytics workloads and characterized their micro-architectural behaviors by using hardware performance counters, so as to understand the impacts and implications of data analytics workloads on the systems equipped with modern superscalar out-of-order processors. Our study reveals that big data analytics applications themselves share many inherent characteristics, which place them in a different class from traditional workloads and scale-out services. To further understand the characteristics of big data analytics work- loads we performed a correlation analysis of CPI (cycles per instruction) with other micro- architecture level characteristics and an investigation of the big data software stack impacts on application behaviors. Our correlation analysis showed that even though big data ana- lytics workloads own notable pipeline front end stalls, the main factors affecting the CPI performance are long latency data accesses rather than the front end stalls. Our software stack investigation found that the typical big data software stack significantly contributes to the front end stalls and incurs bigger working set. Finally we gave several recommen- dations for architects, programmers and big data system designers with the knowledge acquired from this paper.

cs.DC↗

arXiv2013

The Implications of Diverse Applications and Scalable Data Sets in Benchmarking Big Data Systems

Now we live in an era of big data, and big data applications are becoming more and more pervasive. How to benchmark data center computer systems running big data applications (in short big data systems) is a hot topic. In this paper, we focus on measuring the performance impacts of diverse applications and scalable volumes of data sets on big data systems. For four typical data analysis applications---an important class of big data applications, we find two major results through experiments: first, the data scale has a significant impact on the performance of big data systems, so we must provide scalable volumes of data sets in big data benchmarks. Second, for the four applications, even all of them use the simple algorithms, the performance trends are different with increasing data scales, and hence we must consider not only variety of data sets but also variety of applications in benchmarking big data systems.

cs.PF↗

arXiv2013

Characterizing Data Analysis Workloads in Data Centers

As the amount of data explodes rapidly, more and more corporations are using data centers to make effective decisions and gain a competitive edge. Data analysis applications play a significant role in data centers, and hence it has became increasingly important to understand their behaviors in order to further improve the performance of data center computer systems. In this paper, after investigating three most important application domains in terms of page views and daily visitors, we choose eleven representative data analysis workloads and characterize their micro-architectural characteristics by using hardware performance counters, in order to understand the impacts and implications of data analysis workloads on the systems equipped with modern superscalar out-of-order processors. Our study on the workloads reveals that data analysis applications share many inherent characteristics, which place them in a different class from desktop (SPEC CPU2006), HPC (HPCC), and service workloads, including traditional server workloads (SPECweb2005) and scale-out service workloads (four among six benchmarks in CloudSuite), and accordingly we give several recommendations for architecture and system optimizations. On the basis of our workload characterization work, we released a benchmark suite named DCBench for typical datacenter workloads, including data analysis and service workloads, with an open-source license on our project home page on http://prof.ict.ac.cn/DCBench. We hope that DCBench is helpful for performing architecture and small-to-medium scale system researches for datacenter computing.

cs.PF↗

arXiv2013

High Volume Computing: Identifying and Characterizing Throughput Oriented Workloads in Data Centers

For the first time, this paper systematically identifies three categories of throughput oriented workloads in data centers: services, data processing applications, and interactive real-time applications, whose targets are to increase the volume of throughput in terms of processed requests or data, or supported maximum number of simultaneous subscribers, respectively, and we coin a new term high volume computing (in short HVC) to describe those workloads and data center computer systems designed for them. We characterize and compare HVC with other computing paradigms, e.g., high throughput computing, warehouse-scale computing, and cloud computing, in terms of levels, workloads, metrics, coupling degree, data scales, and number of jobs or service instances. We also preliminarily report our ongoing work on the metrics and benchmarks for HVC systems, which is the foundation of designing innovative data center computer systems for HVC workloads.

cs.DC↗

arXiv2010

Ab initio study of the formation of transparent carbon under pressure

A body-centered tetragonal carbon (bct-Carbon) allotrope has been predicted to be a transparent carbon polymorph obtained under pressure. The structural transition pathways from graphite to diamond, M-Carbon, and bct-Carbon are simulated and the lowest activation barrier is found for the graphite-bct transition. Furthermore, bct-Carbon has higher shear strength than diamond due to its perpendicular graphene-like structure. Our results provide a possible explanation for the formation of a transparent carbon allotrope via the cold compression of graphite. We also verify that this allotrope is hard enough to crack diamond.

cond-mat.mtrl-sci↗