arXiv subjects

Lu Chen

Publications and source records attributed to Lu Chen.

At least 289 records · Page 16Linked to original sources

arXiv2022

MetaKG: Meta-learning on Knowledge Graph for Cold-start Recommendation

A knowledge graph (KG) consists of a set of interconnected typed entities and their attributes. Recently, KGs are popularly used as the auxiliary information to enable more accurate, explainable, and diverse user preference recommendations. Specifically, existing KG-based recommendation methods target modeling high-order relations/dependencies from long connectivity user-item interactions hidden in KG. However, most of them ignore the cold-start problems (i.e., user cold-start and item cold-start) of recommendation analytics, which restricts their performance in scenarios when involving new users or new items. Inspired by the success of meta-learning on scarce training samples, we propose a novel meta-learning based framework called MetaKG, which encompasses a collaborative-aware meta learner and a knowledge-aware meta learner, to capture meta users' preference and entities' knowledge for cold-start recommendations. The collaborative-aware meta learner aims to locally aggregate user preferences for each user preference learning task. In contrast, the knowledge-aware meta learner is to globally generalize knowledge representation across different user preference learning tasks. Guided by two meta learners, MetaKG can effectively capture the high-order collaborative relations and semantic representations, which could be easily adapted to cold-start scenarios. Besides, we devise a novel adaptive task scheduler which can adaptively select the informative tasks for meta learning in order to prevent the model from being corrupted by noisy tasks. Extensive experiments on various cold-start scenarios using three real data sets demonstrate that our presented MetaKG outperforms all the existing state-of-the-art competitors in terms of effectiveness, efficiency, and scalability.

cs.IR↗

arXiv2022

HAKG: Hierarchy-Aware Knowledge Gated Network for Recommendation

Knowledge graph (KG) plays an increasingly important role to improve the recommendation performance and interpretability. A recent technical trend is to design end-to-end models based on information propagation schemes. However, existing propagation-based methods fail to (1) model the underlying hierarchical structures and relations, and (2) capture the high-order collaborative signals of items for learning high-quality user and item representations. In this paper, we propose a new model, called Hierarchy-Aware Knowledge Gated Network (HAKG), to tackle the aforementioned problems. Technically, we model users and items (that are captured by a user-item graph), as well as entities and relations (that are captured in a KG) in hyperbolic space, and design a hyperbolic aggregation scheme to gather relational contexts over KG. Meanwhile, we introduce a novel angle constraint to preserve characteristics of items in the embedding space. Furthermore, we propose a dual item embeddings design to represent and propagate collaborative signals and knowledge associations separately, and leverage the gated aggregation to distill discriminative information for better capturing user behavior patterns. Experimental results on three benchmark datasets show that, HAKG achieves significant improvement over the state-of-the-art methods like CKAN, Hyper-Know, and KGIN. Further analyses on the learned hyperbolic embeddings confirm that HAKG offers meaningful insights into the hierarchies of data.

cs.IR↗

arXiv2022

Deep Spatially and Temporally Aware Similarity Computation for Road Network Constrained Trajectories

Trajectory similarity computation has drawn massive attention, as it is core functionality in a wide range of applications such as ride-sharing, traffic analysis, and social recommendation. Motivated by the recent success of deep learning technologies, researchers start devoting efforts to learning-based similarity analyses to overcome the limitations (i.e., high cost and poor adaptability) of traditional methods. Specifically, deep trajectory similarity computation aims to learn a distance function that can evaluate how similar two trajectories are via neural networks. However, existing learning-based methods focus on spatial similarity but ignore the time dimension of trajectories, which is suboptimal for time-aware applications. Besides, they tend to disregard the embedding of trajectories into road networks, restricting their applicability in real scenarios. In this paper, we propose an effective learning-based framework, called ST2Vec, to perform efficient spatially and temporally aware trajectory similarity computation in road networks. Finally, extensive experimental evaluation using three real trajectory data sets shows that ST2Vec outperforms all the state-of-the-art approaches substantially.

cs.LG↗

arXiv2022

Linear Array Network for Low-light Image Enhancement

Convolution neural networks (CNNs) based methods have dominated the low-light image enhancement tasks due to their outstanding performance. However, the convolution operation is based on a local sliding window mechanism, which is difficult to construct the long-range dependencies of the feature maps. Meanwhile, the self-attention based global relationship aggregation methods have been widely used in computer vision, but these methods are difficult to handle high-resolution images because of the high computational complexity. To solve this problem, this paper proposes a Linear Array Self-attention (LASA) mechanism, which uses only two 2-D feature encodings to construct 3-D global weights and then refines feature maps generated by convolution layers. Based on LASA, Linear Array Network (LAN) is proposed, which is superior to the existing state-of-the-art (SOTA) methods in both RGB and RAW based low-light enhancement tasks with a smaller amount of parameters. The code is released in https://github.com/cuiziteng/LASA_enhancement.

cs.CV↗

arXiv2021

LargeEA: Aligning Entities for Large-scale Knowledge Graphs

Entity alignment (EA) aims to find equivalent entities in different knowledge graphs (KGs). Current EA approaches suffer from scalability issues, limiting their usage in real-world EA scenarios. To tackle this challenge, we propose LargeEA to align entities between large-scale KGs. LargeEA consists of two channels, i.e., structure channel and name channel. For the structure channel, we present METIS-CPS, a memory-saving mini-batch generation strategy, to partition large KGs into smaller mini-batches. LargeEA, designed as a general tool, can adopt any existing EA approach to learn entities' structural features within each mini-batch independently. For the name channel, we first introduce NFF, a name feature fusion method, to capture rich name features of entities without involving any complex training process. Then, we exploit a name-based data augmentation to generate seed alignment without any human intervention. Such design fits common real-world scenarios much better, as seed alignment is not always available. Finally, LargeEA derives the EA results by fusing the structural features and name features of entities. Since no widely-acknowledged benchmark is available for large-scale EA evaluation, we also develop a large-scale EA benchmark called DBP1M extracted from real-world KGs. Extensive experiments confirm the superiority of LargeEA against state-of-the-art competitors.

cs.DB↗

arXiv2021

Few-Shot NLU with Vector Projection Distance and Abstract Triangular CRF

Data sparsity problem is a key challenge of Natural Language Understanding (NLU), especially for a new target domain. By training an NLU model in source domains and applying the model to an arbitrary target domain directly (even without fine-tuning), few-shot NLU becomes crucial to mitigate the data scarcity issue. In this paper, we propose to improve prototypical networks with vector projection distance and abstract triangular Conditional Random Field (CRF) for the few-shot NLU. The vector projection distance exploits projections of contextual word embeddings on label vectors as word-label similarities, which is equivalent to a normalized linear model. The abstract triangular CRF learns domain-agnostic label transitions for joint intent classification and slot filling tasks. Extensive experiments demonstrate that our proposed methods can significantly surpass strong baselines. Specifically, our approach can achieve a new state-of-the-art on two few-shot NLU benchmarks (Few-Joint and SNIPS) in Chinese and English without fine-tuning on target domains.

cs.CL↗

arXiv2021

FastSGD: A Fast Compressed SGD Framework for Distributed Machine Learning

With the rapid increase of big data, distributed Machine Learning (ML) has been widely applied in training large-scale models. Stochastic Gradient Descent (SGD) is arguably the workhorse algorithm of ML. Distributed ML models trained by SGD involve large amounts of gradient communication, which limits the scalability of distributed ML. Thus, it is important to compress the gradients for reducing communication. In this paper, we propose FastSGD, a Fast compressed SGD framework for distributed ML. To achieve a high compression ratio at a low cost, FastSGD represents the gradients as key-value pairs, and compresses both the gradient keys and values in linear time complexity. For the gradient value compression, FastSGD first uses a reciprocal mapper to transform original values into reciprocal values, and then, it utilizes a logarithm quantization to further reduce reciprocal values to small integers. Finally, FastSGD filters reduced gradient integers by a given threshold. For the gradient key compression, FastSGD provides an adaptive fine-grained delta encoding method to store gradient keys with fewer bits. Extensive experiments on practical ML models and datasets demonstrate that FastSGD achieves the compression ratio up to 4 orders of magnitude, and accelerates the convergence time up to 8x, compared with state-of-the-art methods.

cs.LG↗

arXiv2021

Does the Reionization Model Influence the Constraints on Dark Matter Decay or Annihilation?

If dark matter decay or annihilate, a large amount of energy and particles would be released into the cosmic plasma. Therefore, they could modify the thermal and ionization history of our universe, then leave footprints on the cosmic microwave background power spectra. In this paper, we take dark matter annihilation as an example and investigate whether different reionization models influence the constraints on dark matter annihilation. We consider the ionization history including both dark matter annihilation and star formation, then put constraints on DM annihilation. Combining the latest Planck data, BAO data, SNIa measurement, $Q_\text{HII}$ constraints from observations of quasars, as well as the star formation rate density from UV and IR data, the optical depth is $τ=0.0571^{+0.0005}_{-0.0006}$ at 68$\%$C.L. and the upper limit of $ε_0 f_d$ reads $2.7765\times 10^{-24}$ at 95$\%$C.L.. By comparison, we also constrain dark matter annihilation in the instantaneous reionization model from the same data combination except the $Q_\text{HII}$ constraints and star formation rate density. We get $τ=0.0559^{+0.0069}_{-0.0076}$ at 68$\%$C.L. and the upper limit of $ε_0 f_d$ is $2.8468\times 10^{-24}$ at 95$\%$C.L.. %, which is $2.5\%$ higher than the results in the previous model. This indicates various reionization models have little influence ($\lesssim 2.5\%$) on constraining parameters of dark matter decay or annihilation.

astro-ph.CO↗

arXiv2021

Machine Learning-Based Soft Sensors for Vacuum Distillation Unit

Product quality assessment in the petroleum processing industry can be difficult and time-consuming, e.g. due to a manual collection of liquid samples from the plant and subsequent chemical laboratory analysis of the samples. The product quality is an important property that informs whether the products of the process are within the specifications. In particular, the delays caused by sample processing (collection, laboratory measurements, results analysis, reporting) can lead to detrimental economic effects. One of the strategies to deal with this problem is soft sensors. Soft sensors are a collection of models that can be used to predict and forecast some infrequently measured properties (such as laboratory measurements of petroleum products) based on more frequent measurements of quantities like temperature, pressure and flow rate provided by physical sensors. Soft sensors short-cut the pathway to obtain relevant information about the product quality, often providing measurements as frequently as every minute. One of the applications of soft sensors is for the real-time optimization of a chemical process by a targeted adaptation of operating parameters. Models used for soft sensors can have various forms, however, among the most common are those based on artificial neural networks (ANNs). While soft sensors can deal with some of the issues in the refinery processes, their development and deployment can pose other challenges that are addressed in this paper. Firstly, it is important to enhance the quality of both sets of data (laboratory measurements and physical sensors) in a data pre-processing stage (as described in Methodology section). Secondly, once the data sets are pre-processed, different models need to be tested against prediction error and the model's interpretability. In this work, we present a framework for soft sensor development from raw data to ready-to-use models.

cs.LG↗

arXiv2021

A Unified Game-Theoretic Interpretation of Adversarial Robustness

This paper provides a unified view to explain different adversarial attacks and defense methods, i.e. the view of multi-order interactions between input variables of DNNs. Based on the multi-order interaction, we discover that adversarial attacks mainly affect high-order interactions to fool the DNN. Furthermore, we find that the robustness of adversarially trained DNNs comes from category-specific low-order interactions. Our findings provide a potential method to unify adversarial perturbations and robustness, which can explain the existing defense methods in a principle way. Besides, our findings also make a revision of previous inaccurate understanding of the shape bias of adversarially learned features.

cs.LG↗

arXiv2021

WebSRC: A Dataset for Web-Based Structural Reading Comprehension

Web search is an essential way for humans to obtain information, but it's still a great challenge for machines to understand the contents of web pages. In this paper, we introduce the task of structural reading comprehension (SRC) on web. Given a web page and a question about it, the task is to find the answer from the web page. This task requires a system not only to understand the semantics of texts but also the structure of the web page. Moreover, we proposed WebSRC, a novel Web-based Structural Reading Comprehension dataset. WebSRC consists of 400K question-answer pairs, which are collected from 6.4K web pages. Along with the QA pairs, corresponding HTML source code, screenshots, and metadata are also provided in our dataset. Each question in WebSRC requires a certain structural understanding of a web page to answer, and the answer is either a text span on the web page or yes/no. We evaluate various baselines on our dataset to show the difficulty of our task. We also investigate the usefulness of structural information and visual features. Our dataset and baselines have been publicly available at https://x-lance.github.io/WebSRC/.

cs.CL↗

arXiv2021

A Unified Game-Theoretic Interpretation of Adversarial Robustness

This paper provides a unified view to explain different adversarial attacks and defense methods, \emph{i.e.} the view of multi-order interactions between input variables of DNNs. Based on the multi-order interaction, we discover that adversarial attacks mainly affect high-order interactions to fool the DNN. Furthermore, we find that the robustness of adversarially trained DNNs comes from category-specific low-order interactions. Our findings provide a potential method to unify adversarial perturbations and robustness, which can explain the existing defense methods in a principle way. Besides, our findings also make a revision of previous inaccurate understanding of the shape bias of adversarially learned features.

cs.LG↗

arXiv2021

Large Phonon Thermal Hall Conductivity in a Simple Antiferromagnetic Insulator

Phonons are known to generate a thermal Hall effect in certain insulators, including oxides with rare-earth impurities, quantum paraelectrics, multiferroic materials and cuprate Mott insulators. In each case, a special feature of the material is presumed relevant for the underlying mechanism that confers chirality to phonons in a magnetic field. The question is whether a phonon Hall effect is an unusual occurrence - linked to special characteristics such as skew scattering off rare-earth impurities, structural domains, ferroelectricity, ferromagnetism - or a much more common property of insulators than hitherto believed. To help answer this question, we have turned to a simple insulator, with none of the previously encountered special features: the cubic antiferromagnet Cu$_3$TeO$_6$. We find that it has the largest thermal Hall conductivity $κ_{\rm{xy}}$ of any insulator so far. We show that this record-high $κ_{\rm{xy}}$ signal is due to phonons and it does not require the presence of magnetic order, as it persists above the ordering temperature. We conclude that the phonon Hall effect is likely to be a fairly common property of solids.

cond-mat.str-el↗

arXiv2021

On invariant tori in some reversible systems

In the present paper, we consider the following reversible system \begin{equation*} \begin{cases} \dot{x}=ω_0+f(x,y),\\ \dot{y}=g(x,y), \end{cases} \end{equation*} where $x\in\mathbf{T}^{d}$, $y\backsim0\in \mathbf{R}^{d}$, $ω_0$ is Diophantine, $f(x,y)=O(y)$, $g(x,y)=O(y^2)$ and $f$, $g$ are reversible with respect to the involution G: $(x,y)\mapsto(-x,y)$, that is, $f(-x,y)=f(x,y)$, $g(-x,y)=-g(x,y)$. We study the accumulation of an analytic invariant torus $Γ_0$ of the reversible system with Diophantine frequency $ω_0$ by other invariant tori. We will prove that if the Birkhoff normal form around $Γ_0$ is 0-degenerate, then $Γ_0$ is accumulated by other analytic invariant tori, the Lebesgue measure of the union of these tori being positive and the density of the union of these tori at $Γ_0$ being one. We will also prove that if the Birkhoff normal form around $Γ_0$ is $j$-degenerate ($1\leq j\leq d-1$) and condition (1.6) is satisfied, then through $Γ_0$ there passes an analytic subvariety of dimension $d+j$ foliated into analytic invariant tori with frequency vector $ω_0$. If the Birkhoff normal form around $Γ_0$ is $d-1$-degenerate, we will prove a stronger result, that is, a full neighborhood of $Γ_0$ is foliated into analytic invariant tori with frequency vectors proportional to $ω_0$.

math.DS↗

arXiv2021

KAM theorem with large twist and finite smooth large perturbation

In the present paper, we will discuss the following non-degenerate Hamiltonian system \begin{equation*} H(θ,t,I)=\frac{H_0(I)}{\varepsilon^{a}}+\frac{P(θ,t,I)}{\varepsilon^{b}}, \end{equation*} where $(θ,t,I)\in\mathbf{T}^{d+1}\times[1,2]^d$ ($\mathbf{T}:=\mathbf{R}/{2π\mathbf{Z}}$), $a,b$ are given positive constants with $a>b$, $H_0: [1,2]^d\rightarrow \mathbf R$ is real analytic and $P: \mathbf T^{d+1}\times [1,2]^d\rightarrow \mathbf R$ is $C^{\ell}$ with $\ell=\frac{2(d+1)(5a-b+2ad)}{a-b}+μ$, $0<μ\ll1$. We prove that if $\varepsilon$ is sufficiently small, there is an invariant torus with given Diophantine frequency vector for the above Hamiltonian system. As for application, we prove that a finite network of Duffing oscillators with periodic exterior forces possesses Lagrangian stability for almost all initial data.

math.DS↗

arXiv2021

An Algebraic Approach to Physical Fields

According to the algebraic approach to spacetime, a thoroughgoing dynamicism, physical fields exist without an underlying manifold. This view is usually implemented by postulating an algebraic structure (e.g., commutative ring) of scalar-valued functions, which can be interpreted as representing a scalar field, and deriving other structures from it. In this work, we point out that this leads to the unjustified primacy of an undetermined scalar field. Instead, we propose to consider algebraic structures in which all (and only) physical fields are primitive. We explain how the theory of \emph{natural operations} in differential geometry -- the modern formalism behind classifying diffeomorphism-invariant constructions -- can be used to obtain concrete implementations of this idea for any given collection of fields. For concrete examples, we illustrate how our approach applies to a number of particular physical fields, including electrodynamics coupled to a Weyl spinor.

physics.hist-ph↗

arXiv2021

Evolutionary Clustering of Streaming Trajectories

The widespread deployment of smartphones and location-enabled, networked in-vehicle devices renders it increasingly feasible to collect streaming trajectory data of moving objects. The continuous clustering of such data can enable a variety of real-time services, such as identifying representative paths or common moving trends among objects in real-time. However, little attention has so far been given to the quality of clusters -- for example, it is beneficial to smooth short-term fluctuations in clusters to achieve robustness to exceptional data. We propose the notion of evolutionary clustering of streaming trajectories, abbreviated ECO, that enhances streaming-trajectory clustering quality by means of temporal smoothing that prevents abrupt changes in clusters across successive timestamps. Employing the notions of snapshot and historical trajectory costs, we formalize ECO and then formulate ECO as an optimization problem and prove that ECO can be performed approximately in linear time, thus eliminating the iterative processes employed in previous studies. Further, we propose a minimal-group structure and a seed point shifting strategy to facilitate temporal smoothing. Finally, we present all algorithms underlying ECO along with a set of optimization techniques. Extensive experiments with two real-life datasets offer insight into ECO and show that it outperforms state-of-the-art solutions in terms of both clustering quality and efficiency.

cs.DB↗

arXiv2021

CollaborER: A Self-supervised Entity Resolution Framework Using Multi-features Collaboration

Entity Resolution (ER) aims to identify whether two tuples refer to the same real-world entity and is well-known to be labor-intensive. It is a prerequisite to anomaly detection, as comparing the attribute values of two matched tuples from two different datasets provides one effective way to detect anomalies. Existing ER approaches, due to insufficient feature discovery or error-prone inherent characteristics, are not able to achieve stable performance. In this paper, we present CollaborER, a self-supervised entity resolution framework via multi-features collaboration. It is capable of (i) obtaining reliable ER results with zero human annotations and (ii) discovering adequate tuples' features in a fault-tolerant manner. CollaborER consists of two phases, i.e., automatic label generation (ALG) and collaborative ER training (CERT). In the first phase, ALG is proposed to generate a set of positive tuple pairs and a set of negative tuple pairs. ALG guarantees the high quality of the generated tuples and hence ensures the training quality of the subsequent CERT. In the second phase, CERT is introduced to learn the matching signals by discovering graph features and sentence features of tuples collaboratively. Extensive experimental results over eight real-world ER benchmarks show that CollaborER outperforms all the existing unsupervised ER approaches and is comparable or even superior to the state-of-the-art supervised ER methods.

cs.DB↗