Searcharxiv⌕ Search

arXiv subjects

Golnaz Ghasemiesfeh

Publications and source records attributed to Golnaz Ghasemiesfeh.

7 recordsLinked to original sources
arXiv2026

Efficient Retrieval Scaling with Hierarchical Indexing for Large Scale Recommendation

The increase in data volume, computational resources, and model parameters during training has led to the development of numerous large-scale industrial retrieval models for recommendation tasks. However, effectively and efficiently deploying these large-scale foundational retrieval models remains a critical challenge that has not been fully addressed. Common quick-win solutions for deploying these massive models include relying on offline computations (such as cached user dictionaries) or distilling large models into smaller ones. Yet, both approaches fall short of fully leveraging the representational and inference capabilities of foundational models. In this paper, we explore whether it is possible to learn a hierarchical organization over the memory of foundational retrieval models. Such a hierarchical structure would enable more efficient search by reducing retrieval costs while preserving exactness. To achieve this, we propose jointly learning a hierarchical index using cross-attention and residual quantization for large-scale retrieval models. We also present its real-world deployment at Meta, supporting daily advertisement recommendations for billions of Facebook and Instagram users. Interestingly, we discovered that the intermediate nodes in the learned index correspond to a small set of high-quality data. Fine-tuning the model on this set further improves inference performance, and concretize the concept of "test-time training" within the recommendation system domain. We demonstrate these findings using both internal and public datasets with strong baseline comparisons and hope they contribute to the community's efforts in developing the next generation of foundational retrieval models.

cs.IR↗
arXiv2024

Hierarchical Structured Neural Network: Efficient Retrieval Scaling for Large Scale Recommendation

Retrieval, the initial stage of a recommendation system, is tasked with down-selecting items from a pool of tens of millions of candidates to a few thousands. Embedding Based Retrieval (EBR) has been a typical choice for this problem, addressing the computational demands of deep neural networks across vast item corpora. EBR utilizes Two Tower or Siamese Networks to learn representations for users and items, and employ Approximate Nearest Neighbor (ANN) search to efficiently retrieve relevant items. Despite its popularity in industry, EBR faces limitations. The Two Tower architecture, relying on a single dot product interaction, struggles to capture complex data distributions due to limited capability in learning expressive interactions between users and items. Additionally, ANN index building and representation learning for user and item are often separate, leading to inconsistencies exacerbated by representation (e.g. continuous online training) and item drift (e.g. items expired and new items added). In this paper, we introduce the Hierarchical Structured Neural Network (HSNN), an efficient deep neural network model to learn intricate user and item interactions beyond the commonly used dot product in retrieval tasks, achieving sublinear computational costs relative to corpus size. A Modular Neural Network (MoNN) is designed to maintain high expressiveness for interaction learning while ensuring efficiency. A mixture of MoNNs operate on a hierarchical item index to achieve extensive computation sharing, enabling it to scale up to large corpus size. MoNN and the hierarchical index are jointly learnt to continuously adapt to distribution shifts in both user interests and item distributions. HSNN achieves substantial improvement in offline evaluation compared to prevailing methods.

cs.IR↗
arXiv2014

How Complex Contagions Spread Quickly in the Preferential Attachment Model and Other Time-Evolving Networks

In this paper, we study the spreading speed of complex contagions in a social network. A $k$-complex contagion starts from a set of initially infected seeds such that any node with at least $k$ infected neighbors gets infected. Simple contagions, i.e., $k=1$, quickly spread to the entire network in small world graphs. However, fast spreading of complex contagions appears to be less likely and more delicate; the successful cases depend crucially on the network structure~\cite{G08,Ghasemiesfeh:2013:CCW}. Our main result shows that complex contagions can spread fast in a general family of time-evolving networks that includes the preferential attachment model~\cite{barabasi99emergence}. We prove that if the initial seeds are chosen as the oldest nodes in a network of this family, a $k$-complex contagion covers the entire network of $n$ nodes in $O(\log n)$ steps. We show that the choice of the initial seeds is crucial. If the initial seeds are uniformly randomly chosen in the PA model, even with a polynomial number of them, a complex contagion would stop prematurely. The oldest nodes in a preferential attachment model are likely to have high degrees. However, we remark that it is actually not the power law degree distribution per se that facilitates fast spreading of complex contagions, but rather the evolutionary graph structure of such models. Some members of the said family do not even have a power-law distribution. We also prove that complex contagions are fast in the copy model~\cite{KumarRaRa00}, a variant of the preferential attachment family. Finally, we prove that when a complex contagion starts from an arbitrary set of initial seeds on a general graph, determining if the number of infected vertices is above a given threshold is $\mathbf{P}$-complete. Thus, one cannot hope to categorize all the settings in which complex contagions percolate in a graph.

cs.SI↗
arXiv2014

Complex Contagions in Kleinberg's Small World Model

Complex contagions describe diffusion of behaviors in a social network in settings where spreading requires the influence by two or more neighbors. In a $k$-complex contagion, a cluster of nodes are initially infected, and additional nodes become infected in the next round if they have at least $k$ already infected neighbors. It has been argued that complex contagions better model behavioral changes such as adoption of new beliefs, fashion trends or expensive technology innovations. This has motivated rigorous understanding of spreading of complex contagions in social networks. Despite simple contagions ($k=1$) that spread fast in all small world graphs, how complex contagions spread is much less understood. Previous work~\cite{Ghasemiesfeh:2013:CCW} analyzes complex contagions in Kleinberg's small world model~\cite{kleinberg00small} where edges are randomly added according to a spatial distribution (with exponent $γ$) on top of a two dimensional grid structure. It has been shown in~\cite{Ghasemiesfeh:2013:CCW} that the speed of complex contagions differs exponentially when $γ=0$ compared to when $γ=2$. In this paper, we fully characterize the entire parameter space of $γ$ except at one point, and provide upper and lower bounds for the speed of $k$-complex contagions. We study two subtly different variants of Kleinberg's small world model and show that, with respect to complex contagions, they behave differently. For each model and each $k \geq 2$, we show that there is an intermediate range of values, such that when $γ$ takes any of these values, a $k$-complex contagion spreads quickly on the corresponding graph, in a polylogarithmic number of rounds. However, if $γ$ is outside this range, then a $k$-complex contagion requires a polynomial number of rounds to spread to the entire network.

cs.DS↗