arXiv subjects

Dong Yao

Publications and source records attributed to Dong Yao.

30 records · Page 2Linked to original sources

arXiv2023

Denoising Multi-modal Sequential Recommenders with Contrastive Learning

There is a rapidly-growing research interest in engaging users with multi-modal data for accurate user modeling on recommender systems. Existing multimedia recommenders have achieved substantial improvements by incorporating various modalities and devising delicate modules. However, when users decide to interact with items, most of them do not fully read the content of all modalities. We refer to modalities that directly cause users' behaviors as point-of-interests, which are important aspects to capture users' interests. In contrast, modalities that do not cause users' behaviors are potential noises and might mislead the learning of a recommendation model. Not surprisingly, little research in the literature has been devoted to denoising such potential noises due to the inaccessibility of users' explicit feedback on their point-of-interests. To bridge the gap, we propose a weakly-supervised framework based on contrastive learning for denoising multi-modal recommenders (dubbed Demure). In a weakly-supervised manner, Demure circumvents the requirement of users' explicit feedback and identifies the noises by analyzing the modalities of all interacted items from a given user.

cs.IR↗

arXiv2023

Determinantal point processes on spheres: multivariate linear statistics

In this paper, we will derive the first and 2nd order Wiener chaos decomposition for the multivariate linear statistics of the determinantal point processes associated with the spectral projection kernels on the unit spheres $S^d$. We will first get a graphical representation for the cumulants of multivariate linear statistics for any determinantal point process. The main results then follow from the very precise estimates and identities regarding the spectral projection kernels and the symmetry of the spheres.

math.PR↗

arXiv2022

Mean Field Behavior during the Big Bang Regime for Coalescing Random Walks

In this paper we consider coalescing random walks on a general connected graph $G=(V,E)$. We set up a unified framework to study the leading order of the decay rate of $P_t$, the expectation of the fraction of occupied sites at time $t$, particularly for the `Big Bang' regime where $t\ll t_{\text{coal}}:=\mathbb{E}[\inf\{s:\text{There is only one particle at time }s\}]$. Our results show that $P_t$ satisfies certain mean field behavior, if the graphs satisfy certain transience-like conditions. We apply this framework to two families of graphs: (1) graphs given by the configuration model with a degree distribution supported in $[3,\bar d]$ for some $\bar d\geq 3$, and (2) finite and infinite vertex-transitive graphs. In the first case, we show that for $1 \ll t \ll |V|$, $P_t$ decays in the order of $t^{-1}$, and $(tP_t)^{-1}$ is approximately the probability that two particles starting from the root of the corresponding unimodular Galton-Watson tree never collide after one of them leaves the root, which is also roughly $|V|/(2t_{\text{meet}})$, where $t_{\text{meet}}$ is the mean meeting time of two walkers. By taking the local weak limit, for the unimodular Galton-Watson tree we prove the convergence of $tP_t$ as $t\to\infty$. For the second family of graphs, if we take a sequence of finite graphs $G_n=(V_n, E_n)$, such that $t_{\text{meet}}=O(|V_n|)$ and the inverse of the spectral gap $t_{\text{rel}}$ is $o(|V_n|)$, then for $t_{\text{rel}}\ll t\ll t_{\text{coal}}$, $(tP_t)^{-1}$ is approximately the probability that two random walks never meet before time $t$, and also $|V|/(2t_{\text{meet}})$. In addition, we define a natural uniform transience condition, and show that it implies the above for all $1\ll t\ll t_{\text{coal}}$. Such estimates of $tP_t$ are also obtained for all infinite transient transitive unimodular graphs, in particular, all transient transitive amenable graphs.

math.PR↗

arXiv2022

CCL4Rec: Contrast over Contrastive Learning for Micro-video Recommendation

Micro-video recommender systems suffer from the ubiquitous noises in users' behaviors, which might render the learned user representation indiscriminating, and lead to trivial recommendations (e.g., popular items) or even weird ones that are far beyond users' interests. Contrastive learning is an emergent technique for learning discriminating representations with random data augmentations. However, due to neglecting the noises in user behaviors and treating all augmented samples equally, the existing contrastive learning framework is insufficient for learning discriminating user representations in recommendation. To bridge this research gap, we propose the Contrast over Contrastive Learning framework for training recommender models, named CCL4Rec, which models the nuances of different augmented views by further contrasting augmented positives/negatives with adaptive pulling/pushing strengths, i.e., the contrast over (vanilla) contrastive learning. To accommodate these contrasts, we devise the hardness-aware augmentations that track the importance of behaviors being replaced in the query user and the relatedness of substitutes, and thus determining the quality of augmented positives/negatives. The hardness-aware augmentation also permits controllable contrastive learning, leading to performance gains and robust training. In this way, CCL4Rec captures the nuances of historical behaviors for a given user, which explicitly shields off the learned user representation from the effects of noisy behaviors. We conduct extensive experiments on two micro-video recommendation benchmarks, which demonstrate that CCL4Rec with far less model parameters could achieve comparable performance to existing state-of-the-art method, and improve the training/inference speed by several orders of magnitude.

cs.IR↗

arXiv2022

Theoretical and experimental study on Noise Equivalent Power of X-ray semiconductor ultra-fast response material based on the rad-optic effect

Semiconductor material based on the rad-optic effect enables ultra-fast detection of X-rays and plays an important role in fusion diagnostics. Obtaining the accurate noise equivalent power (NEP) of the semiconductor ultrafast response material is the key to detecting X-rays. In this paper, the refractive index change mechanism of the semiconductor under X-ray irradiation was analyzed, and the quantitative relationship between the diffraction efficiency and the X-ray photon energy was established through the LT-AlGaAs diffraction imaging experiments. The impulse responses of LT-AlGaAs under 1 KeV-10 KeV X-ray radiation were calculated, revealing the variation of NEP density with radiated photon energy. In the case of bombarding the Al target to generate 1.5 KeV X-rays, the imaging experiments of LT-AlGaAs were performed. The diffraction image of LT-AlGaAs has a linear relationship with the radiation intensity, and the NEP density of LT-AlGaAs reaches 4.80*105W/cm2. This study has reference significance for the development of ultra-fast X-ray imaging systems based on the rad-optic effect.

physics.ins-det↗

arXiv2022

Contrastive Learning with Positive-Negative Frame Mask for Music Representation

Self-supervised learning, especially contrastive learning, has made an outstanding contribution to the development of many deep learning research fields. Recently, researchers in the acoustic signal processing field noticed its success and leveraged contrastive learning for better music representation. Typically, existing approaches maximize the similarity between two distorted audio segments sampled from the same music. In other words, they ensure a semantic agreement at the music level. However, those coarse-grained methods neglect some inessential or noisy elements at the frame level, which may be detrimental to the model to learn the effective representation of music. Towards this end, this paper proposes a novel Positive-nEgative frame mask for Music Representation based on the contrastive learning framework, abbreviated as PEMR. Concretely, PEMR incorporates a Positive-Negative Mask Generation module, which leverages transformer blocks to generate frame masks on the Log-Mel spectrogram. We can generate self-augmented negative and positive samples by masking important components or inessential components, respectively. We devise a novel contrastive learning objective to accommodate both self-augmented positives/negatives sampled from the same music. We conduct experiments on four public datasets. The experimental results of two music-related downstream tasks, music classification, and cover song identification, demonstrate the generalization ability and transferability of music representation learned by PEMR.

cs.SD↗

arXiv2021

CauseRec: Counterfactual User Sequence Synthesis for Sequential Recommendation

Learning user representations based on historical behaviors lies at the core of modern recommender systems. Recent advances in sequential recommenders have convincingly demonstrated high capability in extracting effective user representations from the given behavior sequences. Despite significant progress, we argue that solely modeling the observational behaviors sequences may end up with a brittle and unstable system due to the noisy and sparse nature of user interactions logged. In this paper, we propose to learn accurate and robust user representations, which are required to be less sensitive to (attack on) noisy behaviors and trust more on the indispensable ones, by modeling counterfactual data distribution. Specifically, given an observed behavior sequence, the proposed CauseRec framework identifies dispensable and indispensable concepts at both the fine-grained item level and the abstract interest level. CauseRec conditionally samples user concept sequences from the counterfactual data distributions by replacing dispensable and indispensable concepts within the original concept sequence. With user representations obtained from the synthesized user sequences, CauseRec performs contrastive user representation learning by contrasting the counterfactual with the observational. We conduct extensive experiments on real-world public recommendation benchmarks and justify the effectiveness of CauseRec with multi-aspects model analysis. The results demonstrate that the proposed CauseRec outperforms state-of-the-art sequential recommenders by learning accurate and robust user representations.

cs.IR↗

arXiv2021

Modeling High-order Interactions across Multi-interests for Micro-video Recommendation

Personalized recommendation system has become pervasive in various video platform. Many effective methods have been proposed, but most of them didn't capture the user's multi-level interest trait and dependencies between their viewed micro-videos well. To solve these problems, we propose a Self-over-Co Attention module to enhance user's interest representation. In particular, we first use co-attention to model correlation patterns across different levels and then use self-attention to model correlation patterns within a specific level. Experimental results on filtered public datasets verify that our presented module is useful.

cs.CV↗

arXiv2019

The Symbiotic Contact Process

We consider a contact process on $Z^d$ with two species that interact in a symbiotic manner. Each site can either be vacant or occupied by individuals of species $A$ and/or $B$. Multiple occupancy by the same species at a single site is prohibited. The name symbiotic comes from the fact that if only one species is present at a site then that particle dies with rate 1 but if both species are present then the death rate is reduced to $μ\le 1$ for each particle at that site. We show the critical birth rate $λ_c(μ)$ for weak survival is of order $\sqrtμ$ as $μ\to 0$. Mean-field calculations predict that when $μ< 1/2$ there is a discontinuous transition as $λ$ is varied. In contrast, we show that, in any dimension, the phase transition is continuous. To be fair to physicists the paper that introduced the model, the authors say that the symbiotic contact process is in the directed percolation universality class and hence has a continuous transition. However, a 2018 paper asserts that the transition is discontinuous above the upper critical dimension, which is 4 for oriented percolation.

math.PR↗

arXiv2019

Zeros of repeated derivatives of random polynomials

It has been shown that zeros of Kac polynomials $K_n(z)$ of degree $n$ cluster asymptotically near the unit circle as $n\to\infty$ under some assumptions. This property remains unchanged for the $l$-th derivative of the Kac polynomials $K^{(l)}_n(z)$ for any fixed order $l$. So it's natural to study the situation when the number of the derivatives we take depends on $n$, i.e., $l=N_n$. We will show that the limiting global behavior of zeros of $K_n^{(N_n)}(z)$ depends on the limit of the ratio $N_n/n$. In particular, we prove that when the limit of the ratio is strictly positive, the property of the uniform clustering around the unit circle fails; when the ratio is close to 1, the zeros have some rescaling phenomenon. Then we study such problem for random polynomials with more general coefficients. But things, especially the rescaling phenomenon, become very complicated for the general case when $N_n/n\to 1$, where we compute the case of the random elliptic polynomials to illustrate this.

math.PR↗

arXiv2017

Average nearest neighbor degrees in scale-free networks

The average nearest neighbor degree (ANND) of a node of degree $k$ is widely used to measure dependencies between degrees of neighbor nodes in a network. We formally analyze ANND in undirected random graphs when the graph size tends to infinity. The limiting behavior of ANND depends on the variance of the degree distribution. When the variance is finite, the ANND has a deterministic limit. When the variance is infinite, the ANND scales with the size of the graph, and we prove a corresponding central limit theorem in the configuration model (CM, a network with random connections). As ANND proved uninformative in the infinite variance scenario, we propose an alternative measure, the average nearest neighbor rank (ANNR). We prove that ANNR converges to a deterministic function whenever the degree distribution has finite mean. We then consider the erased configuration model (ECM), where self-loops and multiple edges are removed, and investigate the well-known `structural negative correlations', or `finite-size effects', that arise in simple graphs, such as ECM, because large nodes can only have a limited number of large neighbors. Interestingly, we prove that for any fixed $k$, ANNR in ECM converges to the same limit as in CM. However, numerical experiments show that finite-size effects occur when $k$ scales with $n$.

math.PR↗

arXiv2012

Tidal evolution of exo-planetary systems: WASP-50, GJ 1214 and CoRoT-7

We perform numerical simulations to investigate tidal evolution of two single-planet systems, that is, WASP-50 and GJ 1214 and a two-planet system CoRoT-7. The results of orbital evolution show that tidal decay and circularization may play a significant role in shaping their final orbits, which is related to the initial orbital data in the simulations. For GJ 1214 system, different cases of initial eccentricity are also considered as only an upper limit of its eccentricity (0.27) is shown, and the outcome suggests a possible maximum initial eccentricity (0.4) in the adopted dynamical model. Moreover, additional runs with alternative values of dissipation factor $Q^\prime_1$ are carried out to explore tidal evolution for GJ 1214b, and these results further indicate that the real $Q^\prime_1$ of GJ 1214b may be much larger than its typical value, which may reasonably suggest that GJ 1214b bears a present-day larger eccentricity, undergoing tidal circularization at a slow rate. For the CoRoT-7 system, tidal forces make two planets migrating towards their host star as well as producing tidal circularization, and in this process tidal effects and mutual gravitational interactions are coupled with each other. Various scenarios of the initial eccentricity of the outer planet have also been done to investigate final planetary configuration. Tidal decay arising from stellar tides may still work for each system as the eccentricity decreases to zero, and this is in association with the remaining lifetime of each planet used to predict its future.

astro-ph.EP↗