arXiv subjects

Hongtao Lin

Publications and source records attributed to Hongtao Lin.

At least 19 recordsLinked to original sources

arXiv2026

UniPinRec: Unifying Generative Retrieval and Ranking at Pinterest Scale

Modern recommendation systems predominantly train retrieval and ranking as separate models despite both increasingly relying on large transformers encoding the same user behavior data, duplicating parameters, compute, and serving cost. Prior work unifies the model architecture but not the full pipeline: input formats, training procedures, and serving stacks remain fragmented across stages. We present UniPinRec, which achieves full-stack unification of retrieval and ranking at Pinterest: one input format, one model, one training stage, deployed within existing serving infrastructure. A shared transformer encodes the user action sequence into candidate-independent representations that branch into retrieval (ANN dot-product) and ranking (cross-attention) via task-specific heads. Three ideas make this work: (1) Masked Action Modeling (MAM) eliminates interleaving, enabling weight sharing without doubling context length; (2) Blended training examples pair action sequences with feedview impression slates to satisfy both objectives jointly; (3) Cross-stage KV cache sharing reuses user-history computation from retrieval for ranking, reducing total FLOPs versus serving two independent models. Deployed in the Pinterest core surfaces, UniPinRec delivers approximately +1% online engagement lift while cutting end-to-end serving latency by 11.1% and lifting QPS by 63.6%. To our knowledge, this is the first full-stack unification of retrieval and ranking, covering inputs, model, training and serving, deployed in a production recommendation system.

cs.IR↗

arXiv2025

Bootstrapping Conditional Retrieval for User-to-Item Recommendations

User-to-item retrieval has been an active research area in recommendation system, and two tower models are widely adopted due to model simplicity and serving efficiency. In this work, we focus on a variant called \textit{conditional retrieval}, where we expect retrieved items to be relevant to a condition (e.g. topic). We propose a method that uses the same training data as standard two tower models but incorporates item-side information as conditions in query. This allows us to bootstrap new conditional retrieval use cases and encourages feature interactions between user and condition. Experiments show that our method can retrieve highly relevant items and outperforms standard two tower models with filters on engagement metrics. The proposed model is deployed to power a topic-based notification feed at Pinterest and led to +0.26\% weekly active users.

cs.IR↗

arXiv2025

Synergizing Implicit and Explicit User Interests: A Multi-Embedding Retrieval Framework at Pinterest

Industrial recommendation systems are typically composed of multiple stages, including retrieval, ranking, and blending. The retrieval stage plays a critical role in generating a high-recall set of candidate items that covers a wide range of diverse user interests. Effectively covering the diverse and long-tail user interests within this stage poses a significant challenge: traditional two-tower models struggle in this regard due to limited user-item feature interaction and often bias towards top use cases. To address these issues, we propose a novel multi-embedding retrieval framework designed to enhance user interest representation by generating multiple user embeddings conditioned on both implicit and explicit user interests. Implicit interests are captured from user history through a Differentiable Clustering Module (DCM), whereas explicit interests, such as topics that the user has followed, are modeled via Conditional Retrieval (CR). These methodologies represent a form of conditioned user representation learning that involves condition representation construction and associating the target item with the relevant conditions. Synergizing implicit and explicit user interests serves as a complementary approach to achieve more effective and comprehensive candidate retrieval as they benefit on different user segments and extract conditions from different but supplementary sources. Extensive experiments and A/B testing reveal significant improvements in user engagements and feed diversity metrics. Our proposed framework has been successfully deployed on Pinterest home feed.

cs.IR↗

arXiv2024

A Field Theory Framework of Incompressible Fluid Dynamics

This study develops an effective theoretical framework that couples two vector fields: the velocity field $\mathbf{u}$ and an auxiliary vorticity field $\boldsymbol{\xi}$. Together, these fields form a larger conserved dynamical system. Within this framework, the incompressible Navier-Stokes (NS) equation and a complementary vorticity equation with negative viscosity are derived. By introducing the concept of light-cone vorticity $\boldsymbol{\eta}_\pm = \mathbf{w} \pm \boldsymbol{\xi}$, the paper constructs a unified framework for coupled dynamics. Furthermore, it explores the mechanism of spontaneous symmetry breaking from $SU(2)$ gauge theory to $U(1) \times U(1)$, which leads to the emergence of the coupled vector field theory in the non-relativistic limit. This approach uncovers a connection between fluid dynamics and fundamental gauge theories, suggesting that the NS equations describe a subsystem where dissipation results from energy transfer between the velocity and auxiliary fields. The study concludes by linking the complete dynamical framework to the Abrikosov-Nielsen-Olesen-Zumino (ANOZ) theory, a non-Abelian generalization of Bardeen-Cooper-Schrieffer (BCS) theory, offering new insights into fluid dynamics and quantum fluid theory.

physics.flu-dyn↗

arXiv2024

On-Chip Vectorial Structured Light Manipulation via Inverse Design

On-chip structured light, with potentially infinite complexity, has emerged as a linchpin in the realm of integrated photonics. However, the realization of arbitrarily tailoring a multitude of light field dimensions in complex media remains a challenge1, Through associating physical light fields and mathematical function spaces by introducing a mapping operator, we proposed a data-driven inverse design method to precisely manipulate between any two structured light fields in the on-chip high-dimensional Hilbert space. To illustrate, light field conversion in on-chip topological photonics was achieved. High-performance topological coupling devices with minimal insertion loss and customizable topological routing devices were designed and realized. Our method provides a new paradigm to enable precise manipulation over the on-chip vectorial structured light and paves the way for the realization of complex photonic functions.

physics.optics↗

arXiv2023

"Zero change" platform for monolithic back-end-of-line integration of phase change materials in silicon photonics

Monolithic integration of novel materials for unprecedented device functions without modifying the existing photonic component library is the key to advancing heterogeneous silicon photonic integrated circuits. To achieve this, the introduction of a silicon nitride etching stop layer at selective area, coupled with low-loss oxide trench to waveguide surface, enables the incorporation of various functional materials without disrupting the reliability of foundry-verified devices. As an illustration, two distinct chalcogenide phase change materials (PCM) with remarkable nonvolatile modulation capabilities, namely Sb2Se3 and Ge2Sb2Se4Te1, were monolithic back-end-of-line integrated into silicon photonics. The PCM enables compact phase and intensity tuning units with zero-static power consumption. Taking advantage of these building blocks, the phase error of a push-pull Mach-Zehnder interferometer optical switch could be trimmed by a nonvolatile phase shifter with a 48% peak power consumption reduction. Mirco-ring filters with a rejection ratio >25dB could be applied for >5-bit wavelength selective intensity modulation, and waveguide-based >7-bit intensity-modulation photonic attenuators could achieve >39dB broadband attenuation. The advanced "Zero change" back-end-of-line integration platform could not only facilitate the integration of PCMs for integrated reconfigurable photonics but also open up the possibilities for integrating other excellent optoelectronic materials in the future silicon photonic process design kits.

physics.optics↗

arXiv2023

Graphene/silicon heterojunction for reconfigurable phase-relevant activation function in coherent optical neural networks

Optical neural networks (ONNs) herald a new era in information and communication technologies and have implemented various intelligent applications. In an ONN, the activation function (AF) is a crucial component determining the network performances and on-chip AF devices are still in development. Here, we first demonstrate on-chip reconfigurable AF devices with phase activation fulfilled by dual-functional graphene/silicon (Gra/Si) heterojunctions. With optical modulation and detection in one device, time delays are shorter, energy consumption is lower, reconfigurability is higher and the device footprint is smaller than other on-chip AF strategies. The experimental modulation voltage (power) of our Gra/Si heterojunction achieves as low as 1 V (0.5 mW), superior to many pure silicon counterparts. In the photodetection aspect, a high responsivity of over 200 mA/W is realized. Special nonlinear functions generated are fed into a complex-valued ONN to challenge handwritten letters and image recognition tasks, showing improved accuracy and potential of high-efficient, all-component-integration on-chip ONN. Our results offer new insights for on-chip ONN devices and pave the way to high-performance integrated optoelectronic computing circuits.

physics.optics↗

arXiv2021

Waveguide-Integrated Mid-Infrared Photodetection using Graphene on a Scalable Chalcogenide Glass Platform

The development of compact and fieldable mid-infrared (mid-IR) spectroscopy devices represents a critical challenge for distributed sensing with applications from gas leak detection to environmental monitoring. Recent work has focused on mid-IR photonic integrated circuit (PIC) sensing platforms and waveguide-integrated mid-IR light sources and detectors based on semiconductors such as PbTe, black phosphorus and tellurene. However, material bandgaps and reliance on SiO$_2$ substrates limit operation to wavelengths $\lambda\lesssim4\,\mu\textrm{m}$. Here we overcome these challenges with a chalcogenide glass-on-CaF$_2$ PIC architecture incorporating split-gate photothermoelectric graphene photodetectors. Our design extends operation to $\lambda=5.2\,\mu\textrm{m}$ with a Johnson noise-limited noise-equivalent power of $1.1\,\mathrm{nW}/\mathrm{Hz}^{1/2}$, no fall-off in photoresponse up to $f = 1\,\mathrm{MHz}$, and a predicted 3-dB bandwidth of $f_{3\textrm{dB}}>1\,\mathrm{GHz}$. This mid-IR PIC platform readily extends to longer wavelengths and opens the door to applications from distributed gas sensing and portable dual comb spectroscopy to weather-resilient free space optical communications.

physics.optics↗

arXiv2021

A robust single-pixel particle image velocimetry based on fully convolutional networks with cross-correlation embedded

Particle image velocimetry (PIV) is essential in experimental fluid dynamics. In the current work, we propose a new velocity field estimation paradigm, which achieves a synergetic combination of the deep learning method and the traditional cross-correlation method. Specifically, the deep learning method is used to optimize and correct a coarse velocity guess to achieve a super-resolution calculation. And the cross-correlation method provides the initial velocity field based on a coarse correlation with a large interrogation window. As a reference, the coarse velocity guess helps with improving the robustness of the proposed algorithm. This fully convolutional network with embedded cross-correlation is named as CC-FCN. CC-FCN has two types of input layers, one is for the particle images, and the other is for the initial velocity field calculated using cross-correlation with a coarse resolution. Firstly, two pyramidal modules extract features of particle images and initial velocity field respectively. Then the fusion module appropriately fuses these features. Finally, CC-FCN achieves the super-resolution calculation through a series of deconvolution layers to obtain the single-pixel velocity field. As the supervised learning strategy is considered, synthetic data sets including ground-truth fluid motions are generated to train the network parameters. Synthetic and real experimental PIV data sets are used to test the trained neural network in terms of accuracy, precision, spatial resolution and robustness. The test results show that these attributes of CC-FCN are further improved compared with those of other tested PIV algorithms. The proposed model could therefore provide competitive and robust estimations for PIV experiments.

physics.flu-dyn↗

arXiv2019

NERO: A Neural Rule Grounding Framework for Label-Efficient Relation Extraction

Deep neural models for relation extraction tend to be less reliable when perfectly labeled data is limited, despite their success in label-sufficient scenarios. Instead of seeking more instance-level labels from human annotators, here we propose to annotate frequent surface patterns to form labeling rules. These rules can be automatically mined from large text corpora and generalized via a soft rule matching mechanism. Prior works use labeling rules in an exact matching fashion, which inherently limits the coverage of sentence matching and results in the low-recall issue. In this paper, we present a neural approach to ground rules for RE, named NERO, which jointly learns a relation extraction module and a soft matching module. One can employ any neural relation extraction models as the instantiation for the RE module. The soft matching module learns to match rules with semantically similar sentences such that raw corpora can be automatically labeled and leveraged by the RE module (in a much better coverage) as augmented supervision, in addition to the exactly matched sentences. Extensive experiments and analysis on two public and widely-used datasets demonstrate the effectiveness of the proposed NERO framework, comparing with both rule-based and semi-supervised methods. Through user studies, we find that the time efficiency for a human to annotate rules and sentences are similar (0.30 vs. 0.35 min per label). In particular, NERO's performance using 270 rules is comparable to the models trained using 3,000 labeled sentences, yielding a 9.5x speedup. Moreover, NERO can predict for unseen relations at test time and provide interpretable predictions. We release our code to the community for future research.

cs.CL↗

arXiv2019

Molecular monolayer stabilizer for multilayer 2D materials

2D van der Waals materials have rich and unique functional properties, but many are susceptible to corrosion under ambient conditions. Here we show that linear alkylamines are highly effective in protecting the optoelectronic properties of these materials such as black phosphorous (BP) and transitional metal dichalcogenides. As a representative example, n-hexylamine can be applied in the form of thin molecular monolayers on BP flakes with less-than-2nm thickness and can prolong BP's lifetime from a few hours to several weeks and even months in ambient environments. Characterizations combined with our theoretical analysis show that the thin monolayers selectively sift out water molecules, forming a drying layer to achieve the passivation of the protected 2D materials. The monolayer coating is also stable in air, hydrogen annealing, and organic solvents, but can be removed by certain organic acids.

physics.app-ph↗

arXiv2019

Learning Dual Retrieval Module for Semi-supervised Relation Extraction

Relation extraction is an important task in structuring content of text data, and becomes especially challenging when learning with weak supervision---where only a limited number of labeled sentences are given and a large number of unlabeled sentences are available. Most existing work exploits unlabeled data based on the ideas of self-training (i.e., bootstrapping a model) and multi-view learning (e.g., ensembling multiple model variants). However, these methods either suffer from the issue of semantic drift, or do not fully capture the problem characteristics of relation extraction. In this paper, we leverage a key insight that retrieving sentences expressing a relation is a dual task of predicting relation label for a given sentence---two tasks are complementary to each other and can be optimized jointly for mutual enhancement. To model this intuition, we propose DualRE, a principled framework that introduces a retrieval module which is jointly trained with the original relation prediction module. In this way, high-quality samples selected by retrieval module from unlabeled data can be used to improve prediction module, and vice versa. Experimental results\footnote{\small Code and data can be found at \url{https://github.com/INK-USC/DualRE}.} on two public datasets as well as case studies demonstrate the effectiveness of the DualRE approach.

cs.CL↗

arXiv2018

Extreme Broadband Transparent Optical Phase Change Materials for High-Performance Nonvolatile Photonics

Optical phase change materials (O-PCMs), a unique group of materials featuring drastic optical property contrast upon solid-state phase transition, have found widespread adoption in photonic switches and routers, reconfigurable meta-optics, reflective display, and optical neuromorphic computers. Current phase change materials, such as Ge-Sb-Te (GST), exhibit large contrast of both refractive index (delta n) and optical loss (delta k), simultaneously. The coupling of both optical properties fundamentally limits the function and performance of many potential applications. In this article, we introduce a new class of O-PCMs, Ge-Sb-Se-Te (GSST) which breaks this traditional coupling, as demonstrated with an optical figure of merit improvement of more than two orders of magnitude. The first-principle computationally optimized alloy, Ge2Sb2Se4Te1, combines broadband low optical loss (1-18.5 micron), large optical contrast (delta n = 2.0), and significantly improved glass forming ability, enabling an entirely new field of infrared and thermal photonic devices. We further leverage the material to demonstrate nonvolatile integrated optical switches with record low loss and large contrast ratio, as well as an electrically addressed, microsecond switched pixel level spatial light modulator, thereby validating its promise as a platform material for scalable nonvolatile photonics.

physics.app-ph↗

arXiv2018

Digital Fourier transform spectroscopy: a high-performance, scalable technology for on-chip spectrum analysis

Optical spectrum analysis is the cornerstone of spectroscopic sensing, optical network performance monitoring, and hyperspectral imaging. While conventional high-performance spectrometers used to perform such analysis are often large benchtop instruments, on-chip spectrometers have recently emerged as a promising alternative with apparent Size, Weight, and Power (SWaP) advantages. Existing on-chip spectrometer designs, however, are limited in spectral channel count and signal-to-noise ratio (SNR). Here we demonstrate a transformative on-chip digital Fourier transform (dFT) spectrometer that can acquire high-resolution spectra via time- domain modulation of a reconfigurable Mach-Zehnder interferometer. The device, fabricated and packaged using industry-standard silicon photonics technology, claims the multiplex advantage to dramatically boost SNR and unprecedented scalability capable of addressing exponentially increasing numbers of spectral channels. We further implemented machine learning regularization techniques to spectrum reconstruction and achieved significant noise suppression and spectral resolution enhancement beyond the classical Rayleigh criterion.

physics.app-ph↗

arXiv2017

Ultra-thin, High-efficiency Mid-Infrared Transmissive Huygens Meta-Optics

The mid-infrared (mid-IR) is a strategically important band for numerous applications ranging from night vision to biochemical sensing. Unlike visible or near-infrared optical parts which are commonplace and economically available off-the-shelf, mid-IR optics often requires exotic materials or complicated processing, which accounts for their high cost and inferior quality compared to their visible or near-infrared counterparts. Here we theoretically analyzed and experimentally realized a Huygens metasurface platform capable of fulfilling a diverse cross-section of optical functions in the mid-IR. The meta-optical elements were constructed using high-index chalcogenide films deposited on fluoride substrates:the choices of wide-band transparent materials allow the design to be scaled across a broad infrared spectrum. Capitalizing on a novel two-component Huygens' meta-atom design, the meta-optical devices feature an ultra-thin profile ($\lambda_0/8$ in thickness, where $\lambda_0$ is the free-space wavelength) and measured optical efficiencies up to 75% in transmissive mode, both of which represent major improvements over state-of-the-art. We have also demonstrated, for the first time, mid-IR transmissive meta-lenses with diffraction-limited focusing and imaging performance. The projected size, weight and power advantages, coupled with the manufacturing scalability leveraging standard microfabrication technologies, make the Huygens meta-optical devices promising for next-generation mid-IR system applications.

physics.optics↗

arXiv2017

Chalcogenide Glass-on-Graphene Photonics

Two-dimensional (2-D) materials are of tremendous interest to integrated photonics given their singular optical characteristics spanning light emission, modulation, saturable absorption, and nonlinear optics. To harness their optical properties, these atomically thin materials are usually attached onto prefabricated devices via a transfer process. In this paper, we present a new route for 2-D material integration with planar photonics. Central to this approach is the use of chalcogenide glass, a multifunctional material which can be directly deposited and patterned on a wide variety of 2-D materials and can simultaneously function as the light guiding medium, a gate dielectric, and a passivation layer for 2-D materials. Besides claiming improved fabrication yield and throughput compared to the traditional transfer process, our technique also enables unconventional multilayer device geometries optimally designed for enhancing light-matter interactions in the 2-D layers. Capitalizing on this facile integration method, we demonstrate a series of high-performance glass-on-graphene devices including ultra-broadband on-chip polarizers, energy-efficient thermo-optic switches, as well as graphene-based mid-infrared (mid-IR) waveguide-integrated photodetectors and modulators.

physics.optics↗

arXiv2016

High-Q mid-infrared chalcogenide glass resonators

In this letter, we fabricated and characterized chalcogenide glass resonators monolithically integrated on mid-infrared transparent CaF2 substrates. The devices feature an intrinsic Q-factor of 4E5 at 5.2 micron wavelength, which represents the highest Q-factor reported in planar mid-infrared resonators. It is found that moisture can significantly impact the device performance when the device was exposed to an ambient environment, although the high-Q characteristics can be restored after undergoing an annealing treatment. Using these devices, we further demonstrated on-chip cavity-enhanced spectroscopy to quantify mid-IR absorption of ethanol solutions around 5.2 micron wavelength.

cond-mat.mtrl-sci↗

arXiv2014

Heterogeneously integrated silicon photonics for the mid-infrared and spectroscopic sensing

Besides being the foundational material for microelectronics, in optics, crystalline silicon has long been used for making infrared lenses and mirrors. More recently, silicon has become the key material to achieve large-scale integration of photonic devices for optical interconnect and telecommunication. For optics, silicon has significant advantages: it offers a very high refractive index and is highly transparent in the spectral range from 1.2 to 8 micron. To fully exploit silicon's superior performance in a remarkably broad range and to enable new optoelectronic functionalities, here we describe a general method to integrate silicon photonic devices on arbitrary substrates. In particular, we apply the technique to successfully integrate silicon micro-ring resonators on mid-infrared compatible substrates for operation in the mid-infrared. These high-performance mid-infrared optical resonators are utilized to demonstrate, for the first time, on-chip cavity-enhanced mid-infrared spectroscopic analysis of organic chemicals with a limit of detection of less than 0.1 ng.

physics.optics↗