arXiv subjects

Pengjie Wang

Publications and source records attributed to Pengjie Wang.

At least 37 records · Page 2Linked to original sources

arXiv2026

LORE: A Large Generative Model for Search Relevance

Achievement. We introduce LORE, a systematic framework for Large Generative Model-based relevance in e-commerce search. Deployed and iterated over three years, LORE achieves a cumulative +27\% improvement in online GoodRate metrics. This report shares the valuable experience gained throughout its development lifecycle, spanning data, features, training, evaluation, and deployment. Insight. While existing works apply Chain-of-Thought (CoT) to enhance relevance, they often hit a performance ceiling. We argue this stems from treating relevance as a monolithic task, lacking principled deconstruction. Our key insight is that relevance comprises distinct capabilities: knowledge and reasoning, multi-modal matching, and rule adherence. We contend that a qualitative-driven decomposition is essential for breaking through current performance bottlenecks. Contributions. LORE provides a complete blueprint for the LLM relevance lifecycle. Key contributions include: (1) A two-stage training paradigm combining progressive CoT synthesis via SFT with human preference alignment via RL. (2) A comprehensive benchmark, RAIR, designed to evaluate these core capabilities. (3) A query frequency-stratified deployment strategy that efficiently transfers offline LLM capabilities to the online system. LORE serves as both a practical solution and a methodological reference for other vertical domains.

cs.IR↗

arXiv2025

VALUE: Value-Aware Large Language Model for Query Rewriting via Weighted Trie in Sponsored Search

Query-to-bidword(i.e., bidding keyword) rewriting is fundamental to sponsored search, transforming noisy user queries into semantically relevant and commercially valuable keywords. Recent advances in large language models (LLMs) improve semantic relevance through generative retrieval frameworks, but they rarely encode the commercial value of keywords. As a result, rewrites are often semantically correct yet economically suboptimal, and a reinforcement learning from human feedback (RLHF) stage is usually added after supervised fine-tuning(SFT) to mitigate this deficiency. However, conventional preference alignment frequently overemphasize the ordering of bidword values and is susceptible to overfitting, which degrades rewrite quality. In addition, bidword value changes rapidly, while existing generative methods do not respond to these fluctuations. To address this shortcoming, we introduce VALUE(Value-Aware Large language model for qUery rewriting via wEighted trie), a framework that integrates value awareness directly into generation and enhances value alignment during training. VALUE employs the Weighted Trie, a novel variant of the classical trie that stores real-time value signals for each token. During decoding, the framework adjusts the LLM's token probabilities with these signals, constraining the search space and steering generation toward high-value rewrites. The alignment stage uses a fine-grained preference learning strategy that emphasizes stable, high-value differences and down-weights noisy or transient fluctuations, thereby improving robustness and reducing overfitting. Offline experiments show that VALUE significantly outperforms baselines in both semantic matching and value-centric metrics. VALUE has been deployed on our advertising system since October 2024 and served the Double Eleven promotions, the biggest shopping carnival in China.

cs.IR↗

arXiv2025

MOON Embedding: Multimodal Representation Learning for E-commerce Search Advertising

We introduce MOON, our comprehensive set of sustainable iterative practices for multimodal representation learning for e-commerce applications. MOON has already been fully deployed across all stages of Taobao search advertising system, including retrieval, relevance, ranking, and so on. The performance gains are particularly significant on click-through rate (CTR) prediction task, which achieves an overall +20.00% online CTR improvement. Over the past three years, this project has delivered the largest improvement on CTR prediction task and undergone five full-scale iterations. Throughout the exploration and iteration of our MOON, we have accumulated valuable insights and practical experience that we believe will benefit the research community. MOON contains a three-stage training paradigm of "Pretraining, Post-training, and Application", allowing effective integration of multimodal representations with downstream tasks. Notably, to bridge the misalignment between the objectives of multimodal representation learning and downstream training, we define the exchange rate to quantify how effectively improvements in an intermediate metric can translate into downstream gains. Through this analysis, we identify the image-based search recall as a critical intermediate metric guiding the optimization of multimodal models. Over three years and five iterations, MOON has evolved along four critical dimensions: data processing, training strategy, model architecture, and downstream application. The lessons and insights gained through the iterative improvements will also be shared. As part of our exploration into scaling effects in the e-commerce field, we further conduct a systematic study of the scaling laws governing multimodal representation learning, examining multiple factors such as the number of training tokens, negative samples, and the length of user behavior sequences.

cs.IR↗

arXiv2025

RecIS: Sparse to Dense, A Unified Training Framework for Recommendation Models

In this paper, we propose RecIS, a unified Sparse-Dense training framework designed to achieve two primary goals: 1. Unified Framework To create a Unified sparse-dense training framework based on the PyTorch ecosystem that meets the training needs of industrial-grade recommendation models that integrated with large models. 2.System Optimization To optimize the sparse component, offering superior efficiency over the TensorFlow-based recommendation models. The dense component, meanwhile, leverages existing optimization technologies within the PyTorch ecosystem. Currently, RecIS is being used in Alibaba for numerous large-model enhanced recommendation training tasks, and some traditional sparse models have also begun training in it.

cs.IR↗

arXiv2025

Beyond Advertising: Mechanism Design for Platform-Wide Marketing Service "QuanZhanTui"

On e-commerce platforms, sellers typically bid for impressions from ad traffic to promote their products. However, for most sellers, the majority of their sales come from organic traffic. Consequently, the relationship between their ad spending and total sales remains uncertain, resulting in operational inefficiency. To address this issue, e-commerce platforms have recently introduced a novel platform-wide marketing service known as QuanZhanTui, which has reportedly enhanced marketing efficiency for sellers and driven substantial revenue growth for platforms. QuanZhanTui allows sellers to bid for impressions from the platform's entire traffic to boost their total sales without compromising the platform's user experience. In this paper, we investigate the mechanism design problem that arises from QuanZhanTui. The problem is formulated as a multi-objective optimization to balance sellers' welfare and platform's user experience. We first introduce the stock-constrained value maximizer model, which reflects sellers' dual requirements on marketing efficiency and platform-wide ROI. Then, we propose the Liquid Payment Auction (LPA), an auction designed to optimize the balanced objectives while accounting for sellers' requirements in the auto-bidding environment. It employs a simple payment rule based on sellers' liquid welfare, providing a clearer link between their investment and total sales. Under mild assumptions, we theoretically prove desirable properties of LPA, such as optimality and incentive compatibility. Extensive experiments demonstrate LPA's superior performance over conventional auctions in QuanZhanTui.

cs.GT↗

arXiv2025

UQABench: Evaluating User Embedding for Prompting LLMs in Personalized Question Answering

Large language models (LLMs) achieve remarkable success in natural language processing (NLP). In practical scenarios like recommendations, as users increasingly seek personalized experiences, it becomes crucial to incorporate user interaction history into the context of LLMs to enhance personalization. However, from a practical utility perspective, user interactions' extensive length and noise present challenges when used directly as text prompts. A promising solution is to compress and distill interactions into compact embeddings, serving as soft prompts to assist LLMs in generating personalized responses. Although this approach brings efficiency, a critical concern emerges: Can user embeddings adequately capture valuable information and prompt LLMs? To address this concern, we propose \name, a benchmark designed to evaluate the effectiveness of user embeddings in prompting LLMs for personalization. We establish a fair and standardized evaluation process, encompassing pre-training, fine-tuning, and evaluation stages. To thoroughly evaluate user embeddings, we design three dimensions of tasks: sequence understanding, action prediction, and interest perception. These evaluation tasks cover the industry's demands in traditional recommendation tasks, such as improving prediction accuracy, and its aspirations for LLM-based methods, such as accurately understanding user interests and enhancing the user experience. We conduct extensive experiments on various state-of-the-art methods for modeling user embeddings. Additionally, we reveal the scaling laws of leveraging user embeddings to prompt LLMs. The benchmark is available online.

cs.IR↗

arXiv2025

Gradient Deconfliction via Orthogonal Projections onto Subspaces For Multi-task Learning

Although multi-task learning (MTL) has been a preferred approach and successfully applied in many real-world scenarios, MTL models are not guaranteed to outperform single-task models on all tasks mainly due to the negative effects of conflicting gradients among the tasks. In this paper, we fully examine the influence of conflicting gradients and further emphasize the importance and advantages of achieving non-conflicting gradients which allows simple but effective trade-off strategies among the tasks with stable performance. Based on our findings, we propose the Gradient Deconfliction via Orthogonal Projections onto Subspaces (GradOPS) spanned by other task-specific gradients. Our method not only solves all conflicts among the tasks, but can also effectively search for diverse solutions towards different trade-off preferences among the tasks. Theoretical analysis on convergence is provided, and performance of our algorithm is fully testified on multiple benchmarks in various domains. Results demonstrate that our method can effectively find multiple state-of-the-art solutions with different trade-off strategies among the tasks on multiple datasets.

cs.LG↗

arXiv2025

MIM: Multi-modal Content Interest Modeling Paradigm for User Behavior Modeling

Click-Through Rate (CTR) prediction is a crucial task in recommendation systems, online searches, and advertising platforms, where accurately capturing users' real interests in content is essential for performance. However, existing methods heavily rely on ID embeddings, which fail to reflect users' true preferences for content such as images and titles. This limitation becomes particularly evident in cold-start and long-tail scenarios, where traditional approaches struggle to deliver effective results. To address these challenges, we propose a novel Multi-modal Content Interest Modeling paradigm (MIM), which consists of three key stages: Pre-training, Content-Interest-Aware Supervised Fine-Tuning (C-SFT), and Content-Interest-Aware UBM (CiUBM). The pre-training stage adapts foundational models to domain-specific data, enabling the extraction of high-quality multi-modal embeddings. The C-SFT stage bridges the semantic gap between content and user interests by leveraging user behavior signals to guide the alignment of embeddings with user preferences. Finally, the CiUBM stage integrates multi-modal embeddings and ID-based collaborative filtering signals into a unified framework. Comprehensive offline experiments and online A/B tests conducted on the Taobao, one of the world's largest e-commerce platforms, demonstrated the effectiveness and efficiency of MIM method. The method has been successfully deployed online, achieving a significant increase of +14.14% in CTR and +4.12% in RPM, showcasing its industrial applicability and substantial impact on platform performance. To promote further research, we have publicly released the code and dataset at https://pan.quark.cn/s/8fc8ec3e74f3.

cs.IR↗

arXiv2025

Explainable LLM-driven Multi-dimensional Distillation for E-Commerce Relevance Learning

Effective query-item relevance modeling is pivotal for enhancing user experience and safeguarding user satisfaction in e-commerce search systems. Recently, benefiting from the vast inherent knowledge, Large Language Model (LLM) approach demonstrates strong performance and long-tail generalization ability compared with previous neural-based specialized relevance learning methods. Though promising, current LLM-based methods encounter the following inadequacies in practice: First, the massive parameters and computational demands make it difficult to be deployed online. Second, distilling LLM models to online models is a feasible direction, but the LLM relevance modeling is a black box, and its rich intrinsic knowledge is difficult to extract and apply online. To improve the interpretability of LLM and boost the performance of online relevance models via LLM, we propose an Explainable LLM-driven Multi-dimensional Distillation framework for e-commerce relevance learning, which comprises two core components: (1) An Explainable LLM for relevance modeling (ELLM-rele), which decomposes the relevance learning into intermediate steps and models relevance learning as a Chain-of-Thought (CoT) reasoning, thereby enhancing both interpretability and performance of LLM. (2) A Multi-dimensional Knowledge Distillation (MKD) architecture that transfers the knowledge of ELLM-rele to current deployable interaction-based and representation-based student models from both the relevance score distribution and CoT reasoning aspects. Through distilling the probabilistic and CoT reasoning knowledge, MKD improves both the semantic interaction and long-tail generalization abilities of student models. Extensive offline evaluations and online experiments on Taobao search ad scene demonstrate that our proposed framework significantly enhances e-commerce relevance learning performance and user experience.

cs.IR↗

arXiv2024

An open dataset for oracle bone script recognition and decipherment

Oracle bone script, one of the earliest known forms of ancient Chinese writing, presents invaluable research materials for scholars studying the humanities and geography of the Shang Dynasty, dating back 3,000 years. The immense historical and cultural significance of these writings cannot be overstated. However, the passage of time has obscured much of their meaning, presenting a significant challenge in deciphering these ancient texts. With the advent of Artificial Intelligence (AI), employing AI to assist in deciphering Oracle Bone Characters (OBCs) has become a feasible option. Yet, progress in this area has been hindered by a lack of high-quality datasets. To address this issue, this paper details the creation of the HUST-OBC dataset. This dataset encompasses 77,064 images of 1,588 individual deciphered characters and 62,989 images of 9,411 undeciphered characters, with a total of 140,053 images, compiled from diverse sources. The hope is that this dataset could inspire and assist future research in deciphering those unknown OBCs. All the codes and datasets are available at https://github.com/Yuliang-Liu/Open-Oracle.

cs.CV↗

arXiv2024

Puzzle Pieces Picker: Deciphering Ancient Chinese Characters with Radical Reconstruction

Oracle Bone Inscriptions is one of the oldest existing forms of writing in the world. However, due to the great antiquity of the era, a large number of Oracle Bone Inscriptions (OBI) remain undeciphered, making it one of the global challenges in the field of paleography today. This paper introduces a novel approach, namely Puzzle Pieces Picker (P$^3$), to decipher these enigmatic characters through radical reconstruction. We deconstruct OBI into foundational strokes and radicals, then employ a Transformer model to reconstruct them into their modern (conterpart)\textcolor{blue}{counterparts}, offering a groundbreaking solution to ancient script analysis. To further this endeavor, a new Ancient Chinese Character Puzzles (ACCP) dataset was developed, comprising an extensive collection of character images from seven key historical stages, annotated with detailed radical sequences. The experiments have showcased considerable promising insights, underscoring the potential and effectiveness of our approach in deciphering the intricacies of ancient Chinese scripts. Through this novel dataset and methodology, we aim to bridge the gap between traditional philology and modern document analysis techniques, offering new insights into the rich history of Chinese linguistic heritage.

cs.CV↗

arXiv2024

Superconductivity from On-Chip Metallization on 2D Topological Chalcogenides

Two-dimensional (2D) transition metal dichalcogenides (TMDs) is a versatile class of quantum materials of interest to various fields including, e.g., nanoelectronics, optical devices, and topological and correlated quantum matter. Tailoring the electronic properties of TMDs is essential to their applications in many directions. Here, we report that a highly controllable and uniform on-chip 2D metallization process converts a class of atomically thin TMDs into robust superconductors, a property belonging to none of the starting materials. As examples, we demonstrate the introduction of superconductivity into a class of 2D air-sensitive topological TMDs, including monolayers of Td-WTe2, 1T'-MoTe2 and 2H-MoTe2, as well as their natural and twisted bilayers, metalized with an ultrathin layer of Palladium. This class of TMDs are known to exhibit intriguing topological phases ranging from topological insulator, Weyl semimetal to fractional Chern insulator. The unique, high-quality two-dimensional metallization process is based on our recent findings of the long-distance, non-Fickian in-plane mass transport and chemistry in 2D that occur at relatively low temperatures and in devices fully encapsulated with inert insulating layers. Highly compatible with existing nanofabrication techniques for van der Waals (vdW) stacks, our results offer a route to designing and engineering superconductivity and topological phases in a class of correlated 2D materials.

cond-mat.mes-hall↗

arXiv2024

Electron acceleration and X-ray generation from near-critical-density carbon nanotube foams driven by moderately relativistic lasers

Direct laser acceleration of electrons in near-critical-density (NCD) carbon nanotube foams (CNFs) has its advantages in the high-efficiency generation of relativistic electrons and broadband X-rays. Here, we report the first simultaneous measurement on the spectra of laser-driven electrons and X-rays from CNFs at moderately relativistic intensities of around 5\times{10}^{19}\ W/cm^2.\ The density and thickness of the CNFs were scanned in the experiments, indicating the optimized electrons temperature of 5.5 MeV and X-ray critical energy of 5 keV. Two-dimensional (2D) particle-in-cell (PIC) simulations confirm that the electrons, with a temperature significantly higher than the pondermotive scale, are directly accelerated by the laser along the NCD plasma channel, while the bright X-rays are emitted by these electrons through betatron radiation or Thomson backscattering inside the channel. The simultaneously generated electrons and X-rays, automatically synchronized with the femtosecond laser driver, are suitable for applications such as bi-modal radiography.

physics.plasm-ph↗

arXiv2024

An open dataset for the evolution of oracle bone characters: EVOBC

The earliest extant Chinese characters originate from oracle bone inscriptions, which are closely related to other East Asian languages. These inscriptions hold immense value for anthropology and archaeology. However, deciphering oracle bone script remains a formidable challenge, with only approximately 1,600 of the over 4,500 extant characters elucidated to date. Further scholarly investigation is required to comprehensively understand this ancient writing system. Artificial Intelligence technology is a promising avenue for deciphering oracle bone characters, particularly concerning their evolution. However, one of the challenges is the lack of datasets mapping the evolution of these characters over time. In this study, we systematically collected ancient characters from authoritative texts and websites spanning six historical stages: Oracle Bone Characters - OBC (15th century B.C.), Bronze Inscriptions - BI (13th to 221 B.C.), Seal Script - SS (11th to 8th centuries B.C.), Spring and Autumn period Characters - SAC (770 to 476 B.C.), Warring States period Characters - WSC (475 B.C. to 221 B.C.), and Clerical Script - CS (221 B.C. to 220 A.D.). Subsequently, we constructed an extensive dataset, namely EVolution Oracle Bone Characters (EVOBC), consisting of 229,170 images representing 13,714 distinct character categories. We conducted validation and simulated deciphering on the constructed dataset, and the results demonstrate its high efficacy in aiding the study of oracle bone script. This openly accessible dataset aims to digitalize ancient Chinese scripts across multiple eras, facilitating the decipherment of oracle bone script by examining the evolution of glyph forms.

cs.AI↗

arXiv2023

APG: Adaptive Parameter Generation Network for Click-Through Rate Prediction

In many web applications, deep learning-based CTR prediction models (deep CTR models for short) are widely adopted. Traditional deep CTR models learn patterns in a static manner, i.e., the network parameters are the same across all the instances. However, such a manner can hardly characterize each of the instances which may have different underlying distributions. It actually limits the representation power of deep CTR models, leading to sub-optimal results. In this paper, we propose an efficient, effective, and universal module, named as Adaptive Parameter Generation network (APG), which can dynamically generate parameters for deep CTR models on-the-fly based on different instances. Extensive experimental evaluation results show that APG can be applied to a variety of deep CTR models and significantly improve their performance. Meanwhile, APG can reduce the time cost by 38.7\% and memory usage by 96.6\% compared to a regular deep CTR model. We have deployed APG in the industrial sponsored search system and achieved 3\% CTR gain and 1\% RPM gain respectively.

cs.IR↗

arXiv2023

A Platform for Far-Infrared Spectroscopy of Quantum Materials at Millikelvin Temperatures

Optical spectroscopy of quantum materials at ultralow temperatures is rarely explored, yet it may provide critical characterizations of quantum phases not possible using other approaches. We describe the development of a novel experimental platform that enables optical spectroscopic studies, together with standard electronic transport, of materials at millikelvin temperatures inside a dilution refrigerator. The instrument is capable of measuring both bulk crystals and micron-sized two-dimensional van der Waals materials and devices. We demonstrate the performance by implementing photocurrent-based Fourier transform infrared spectroscopy on a monolayer WTe$_2$ device and a multilayer 1T-TaS$_2$ crystal, with a spectral range available from the near-infrared to the terahertz regime and in magnetic fields up to 5 T. In the far-infrared regime, we achieve spectroscopic measurements at a base temperature as low as ~ 43 mK and a sample electron temperature of ~ 450 mK. Possible experiments and potential future upgrades of this versatile instrumental platform are envisioned.

cond-mat.mes-hall↗

arXiv2023

Evidence for Two Dimensional Anisotropic Luttinger Liquids at Millikelvin Temperatures

While Landau's Fermi liquid theory provides the standard description for two- and three-dimensional (2D/3D) conductors, the physics of interacting one-dimensional (1D) conductors is governed by the distinct Luttinger liquid (LL) theory. Can a LL-like state, in which electronic excitations are fractionalized modes, emerge in a 2D system as a stable zero-temperature phase? This long-standing question, first brought up by Anderson decades ago, is crucial in the study of non-Fermi liquids but remains unsettled. A recent experiment identified a moiré superlattice of twisted bilayer tungsten ditelluride (tWTe_2) with a small interlayer twist angle as a 2D host of the LL physics at temperatures of a few kelvins. Here we report experimental evidence for a 2D anisotropic LL state in a substantially reduced temperature regime, down to at least 50 mK, spontaneously formed in a tWTe_2 system with a twist angle of ~ 3 degree. While the system is metallic-like and nearly isotropic above 2 K, a dramatically enhanced electronic anisotropy develops in the millikelvin regime, featuring distinct transport behaviors along two orthogonal in-plane directions. In the strongly anisotropic phase, we observe transport characteristics of a 2D LL phase, i.e., the universal power law scaling behaviors in across-wire conductance and a zero-bias dip in the differential resistance along the wire direction. Our results represent a step forward in the search for stable LL physics beyond 1D and related unconventional quantum matter.

cond-mat.mes-hall↗

arXiv2023

Surface-Confined Two-Dimensional Crystal Growth on a Monolayer

Conventional vapor deposition or epitaxial growth of two-dimensional (2D) materials and heterostructures is conducted in a large chamber in which masses transport from the source to the substrate. Here we report a chamber-free, on-chip approach for growing a 2D crystalline structures directly in a nanoscale surface-confined 2D space. The method is based on a surprising discovery of a rapid, long-distance, non-Fickian transport of a uniform layer of atomically thin palladium (Pd) on a monolayer crystal of tungsten ditelluride (WTe2), at temperatures well below the known melting points of all materials involved. The resulting nanoconfined growth realizes a controlled formation of a stable new 2D crystalline material, Pd7WTe2 , when the monolayer seed is either free-standing or fully encapsulated in a van der Waals stack. The approach is generalizable and highly compatible with nanodevice fabrication, promising to expand the library of 2D materials and their functionalities.

cond-mat.mes-hall↗