Searcharxiv⌕ Search

arXiv subjects

Yan Zhu

Publications and source records attributed to Yan Zhu.

At least 73 records · Page 4Linked to original sources
arXiv2024

User Welfare Optimization in Recommender Systems with Competing Content Creators

Driven by the new economic opportunities created by the creator economy, an increasing number of content creators rely on and compete for revenue generated from online content recommendation platforms. This burgeoning competition reshapes the dynamics of content distribution and profoundly impacts long-term user welfare on the platform. However, the absence of a comprehensive picture of global user preference distribution often traps the competition, especially the creators, in states that yield sub-optimal user welfare. To encourage creators to best serve a broad user population with relevant content, it becomes the platform's responsibility to leverage its information advantage regarding user preference distribution to accurately signal creators. In this study, we perform system-side user welfare optimization under a competitive game setting among content creators. We propose an algorithmic solution for the platform, which dynamically computes a sequence of weights for each user based on their satisfaction of the recommended content. These weights are then utilized to design mechanisms that adjust the recommendation policy or the post-recommendation rewards, thereby influencing creators' content production strategies. To validate the effectiveness of our proposed method, we report our findings from a series of experiments, including: 1. a proof-of-concept negative example illustrating how creators' strategies converge towards sub-optimal states without platform intervention; 2. offline experiments employing our proposed intervention mechanisms on diverse datasets; and 3. results from a three-week online experiment conducted on a leading short-video recommendation platform.

cs.IR↗
arXiv2024

MaskClustering: View Consensus based Mask Graph Clustering for Open-Vocabulary 3D Instance Segmentation

Open-vocabulary 3D instance segmentation is cutting-edge for its ability to segment 3D instances without predefined categories. However, progress in 3D lags behind its 2D counterpart due to limited annotated 3D data. To address this, recent works first generate 2D open-vocabulary masks through 2D models and then merge them into 3D instances based on metrics calculated between two neighboring frames. In contrast to these local metrics, we propose a novel metric, view consensus rate, to enhance the utilization of multi-view observations. The key insight is that two 2D masks should be deemed part of the same 3D instance if a significant number of other 2D masks from different views contain both these two masks. Using this metric as edge weight, we construct a global mask graph where each mask is a node. Through iterative clustering of masks showing high view consensus, we generate a series of clusters, each representing a distinct 3D instance. Notably, our model is training-free. Through extensive experiments on publicly available datasets, including ScanNet++, ScanNet200 and MatterPort3D, we demonstrate that our method achieves state-of-the-art performance in open-vocabulary 3D instance segmentation. Our project page is at https://pku-epic.github.io/MaskClustering.

cs.CV↗
arXiv2024

Learning quantum properties from short-range correlations using multi-task networks

Characterizing multipartite quantum systems is crucial for quantum computing and many-body physics. The problem, however, becomes challenging when the system size is large and the properties of interest involve correlations among a large number of particles. Here we introduce a neural network model that can predict various quantum properties of many-body quantum states with constant correlation length, using only measurement data from a small number of neighboring sites. The model is based on the technique of multi-task learning, which we show to offer several advantages over traditional single-task approaches. Through numerical experiments, we show that multi-task learning can be applied to sufficiently regular states to predict global properties, like string order parameters, from the observation of short-range correlations, and to distinguish between quantum phases that cannot be distinguished by single-task networks. Remarkably, our model appears to be able to transfer information learnt from lower dimensional quantum systems to higher dimensional ones, and to make accurate predictions for Hamiltonians that were not seen in the training.

quant-ph↗
arXiv2024

Bivariate $Q$-polynomial structures for the nonbinary Johnson scheme and the association scheme obtained from attenuated spaces

The study of $P$-polynomial association schemes (distance-regular graphs) and $Q$-polynomial association schemes, and in particular $P$- and $Q$-polynomial association schemes, has been a central theme not only in the theory of association schemes but also in the whole study of algebraic combinatorics in general. Leonard's theorem (1982) says that the spherical functions (or the character tables) of $P$- and $Q$-polynomial association schemes are described by Askey-Wilson orthogonal polynomials or their relatives. These polynomials are one-variable orthogonal polynomials. It seems that the new attempt to define and study higher rank $P$- and $Q$-polynomial association schemes had been hoped for, but had gotten only limited success. The first very successful attempt was initiated recently by Bernard-Crampé-d'Andecy-Vinet-Zaimi [arXiv:2212.10824], and then followed by Bannai-Kurihara-Zhao-Zhu [arXiv:2305.00707]. The general theory and some explicit examples of families of higher rank (multivariate) $P$- and/or $Q$-polynomial association schemes have been obtained there. The main purpose of the present paper is to prove that some important families of association schemes are shown to be bivariate $Q$-polynomial. Namely, we show that all the nonbinary Johnson association schemes and all the attenuated space association schemes are bivariate $Q$-polynomial. It should be noted that the parameter restrictions needed in the previous papers are completely lifted in this paper. Our proofs are done by explicitly calculating the Krein parameters of these association schemes. At the end, we mention some speculations and indications of what we can expect in the future study.

math.CO↗
arXiv2023

Quantum Process Learning Through Neural Emulation

Neural networks are a promising tool for characterizing intermediate-scale quantum devices from limited amounts of measurement data. A challenging problem in this area is to learn the action of an unknown quantum process on an ensemble of physically relevant input states. To tackle this problem, we introduce a neural network that emulates the unknown process by constructing an internal representation of the input ensemble and by mimicking the action of the process at the state representation level. After being trained with measurement data from a few pairs of input/output quantum states, the network becomes able to predict the measurement statistics for all inputs in the ensemble of interest. We show that our model exhibits high accuracy in applications to quantum computing, quantum photonics, and quantum many-body physics.

quant-ph↗
arXiv2023

Knowledge Extraction and Distillation from Large-Scale Image-Text Colonoscopy Records Leveraging Large Language and Vision Models

The development of artificial intelligence systems for colonoscopy analysis often necessitates expert-annotated image datasets. However, limitations in dataset size and diversity impede model performance and generalisation. Image-text colonoscopy records from routine clinical practice, comprising millions of images and text reports, serve as a valuable data source, though annotating them is labour-intensive. Here we leverage recent advancements in large language and vision models and propose EndoKED, a data mining paradigm for deep knowledge extraction and distillation. EndoKED automates the transformation of raw colonoscopy records into image datasets with pixel-level annotation. We validate EndoKED using multi-centre datasets of raw colonoscopy records (~1 million images), demonstrating its superior performance in training polyp detection and segmentation models. Furthermore, the EndoKED pre-trained vision backbone enables data-efficient and generalisable learning for optical biopsy, achieving expert-level performance in both retrospective and prospective validation.

cs.CV↗
arXiv2023

Evidence of Kitaev interaction in the monolayer 1T-CrTe$_2$

The two-dimensional 1T-CrTe$_2$ has been an attractive room-temperature van der Waals magnet which has a potential application in spintronic devices. Although it was recognized as a ferromagnetism in the past, the monolayer 1T-CrTe$_2$ was recently found to exhibit zigzag antiferromagnetism with the easy axis oriented at $70^\circ$ to the perpendicular direction of the plane. Therefore, the origin of the intricate anisotropic magnetic behavior therein is well worthy of thorough exploration. Here, by applying density functional theory with spin spiral method, we demonstrate that the Kitaev interaction, together with the single-ion anisotropy and other off-diagonal exchanges, is amenable to explain the magnetic orientation in the metallic 1T-CrTe$_2$. Moreover, the Ruderman-Kittle-Kasuya-Yosida interaction can also be extracted from the dispersion calculations, which explains the metallic behavior of 1T-CrTe$_2$. Our results demonstrate that 1T-CrTe$_2$ is potentially a rare metallic Kitaev material.

cond-mat.str-el↗
arXiv2023

Multivariate P- and/or Q-polynomial association schemes

The classification problem of $P$- and $Q$-polynomial association schemes has been one of the central problems in algebraic combinatorics. Generalizing the concept of $P$- and $Q$-polynomial association schemes to multivariate cases, namely to consider higher rank $P$- and $Q$-polynomial association schemes, has been tried by some authors, but it seems that so far there were neither very well-established definition nor results. Very recently, Bernard, Crampé, d'Andecy, Vinet, and Zaimi [arXiv:2212.10824], defined bivariate $P$-polynomial association schemes, as well as bivariate $Q$-polynomial association schemes. In this paper, we study these concepts and propose a new modified definition concerning a general monomial order, which is more general and more natural and also easy to handle. We prove that there are many interesting families of examples of multivariate $P$- and/or $Q$-polynomial association schemes.

math.CO↗
arXiv2023

Orthogonal Temporal Interpolation for Zero-Shot Video Recognition

Zero-shot video recognition (ZSVR) is a task that aims to recognize video categories that have not been seen during the model training process. Recently, vision-language models (VLMs) pre-trained on large-scale image-text pairs have demonstrated impressive transferability for ZSVR. To make VLMs applicable to the video domain, existing methods often use an additional temporal learning module after the image-level encoder to learn the temporal relationships among video frames. Unfortunately, for video from unseen categories, we observe an abnormal phenomenon where the model that uses spatial-temporal feature performs much worse than the model that removes temporal learning module and uses only spatial feature. We conjecture that improper temporal modeling on video disrupts the spatial feature of the video. To verify our hypothesis, we propose Feature Factorization to retain the orthogonal temporal feature of the video and use interpolation to construct refined spatial-temporal feature. The model using appropriately refined spatial-temporal feature performs better than the one using only spatial feature, which verifies the effectiveness of the orthogonal temporal feature for the ZSVR task. Therefore, an Orthogonal Temporal Interpolation module is designed to learn a better refined spatial-temporal video feature during training. Additionally, a Matching Loss is introduced to improve the quality of the orthogonal temporal feature. We propose a model called OTI for ZSVR by employing orthogonal temporal interpolation and the matching loss based on VLMs. The ZSVR accuracies on popular video datasets (i.e., Kinetics-600, UCF101 and HMDB51) show that OTI outperforms the previous state-of-the-art method by a clear margin.

cs.CV↗
arXiv2023

User-Controllable Recommendation via Counterfactual Retrospective and Prospective Explanations

Modern recommender systems utilize users' historical behaviors to generate personalized recommendations. However, these systems often lack user controllability, leading to diminished user satisfaction and trust in the systems. Acknowledging the recent advancements in explainable recommender systems that enhance users' understanding of recommendation mechanisms, we propose leveraging these advancements to improve user controllability. In this paper, we present a user-controllable recommender system that seamlessly integrates explainability and controllability within a unified framework. By providing both retrospective and prospective explanations through counterfactual reasoning, users can customize their control over the system by interacting with these explanations. Furthermore, we introduce and assess two attributes of controllability in recommendation systems: the complexity of controllability and the accuracy of controllability. Experimental evaluations on MovieLens and Yelp datasets substantiate the effectiveness of our proposed framework. Additionally, our experiments demonstrate that offering users control options can potentially enhance recommendation accuracy in the future. Source code and data are available at \url{https://github.com/chrisjtan/ucr}.

cs.IR↗
arXiv2023

Rethinking Incentives in Recommender Systems: Are Monotone Rewards Always Beneficial?

The past decade has witnessed the flourishing of a new profession as media content creators, who rely on revenue streams from online content recommendation platforms. The reward mechanism employed by these platforms creates a competitive environment among creators which affect their production choices and, consequently, content distribution and system welfare. It is thus crucial to design the platform's reward mechanism in order to steer the creators' competition towards a desirable welfare outcome in the long run. This work makes two major contributions in this regard: first, we uncover a fundamental limit about a class of widely adopted mechanisms, coined Merit-based Monotone Mechanisms, by showing that they inevitably lead to a constant fraction loss of the optimal welfare. To circumvent this limitation, we introduce Backward Rewarding Mechanisms (BRMs) and show that the competition game resultant from BRMs possesses a potential game structure. BRMs thus naturally induce strategic creators' collective behaviors towards optimizing the potential function, which can be designed to match any given welfare metric. In addition, the BRM class can be parameterized to allow the platform to directly optimize welfare within the feasible mechanism space even when the welfare metric is not explicitly defined.

cs.GT↗
arXiv2023

Searching for the nano-Hertz stochastic gravitational wave background with the Chinese Pulsar Timing Array Data Release I

Observing and timing a group of millisecond pulsars (MSPs) with high rotational stability enables the direct detection of gravitational waves (GWs). The GW signals can be identified from the spatial correlations encoded in the times-of-arrival of widely spaced pulsar-pairs. The Chinese Pulsar Timing Array (CPTA) is a collaboration aiming at the direct GW detection with observations carried out using Chinese radio telescopes. This short article serves as a `table of contents' for a forthcoming series of papers related to the CPTA Data Release 1 (CPTA DR1) which uses observations from the Five-hundred-meter Aperture Spherical radio Telescope (FAST). Here, after summarizing the time span and accuracy of CPTA DR1, we report the key results of our statistical inference finding a correlated signal with amplitude $\log A_{\rm c}= -14.4 \,^{+1.0}_{-2.8}$ for spectral index in the range of $α\in [-1.8, 1.5]$ assuming a GW background (GWB) induced quadrupolar correlation. The search for the Hellings-Downs (HD) correlation curve is also presented, where some evidence for the HD correlation has been found that a 4.6-$σ$ statistical significance is achieved using the discrete frequency method around the frequency of 14 nHz. We expect that the future International Pulsar Timing Array data analysis and the next CPTA data release will be more sensitive to the nHz GWB, which could verify the current results.

astro-ph.HE↗
arXiv2023

Performance of FAST with an Ultra-Wide Bandwidth Receiver at 500-3300 MHz

The Five-hundred-meter Aperture Spherical radio Telescope (FAST) has been running for several years. A new Ultra-Wide Bandwidth (UWB) receiver, simultaneously covering 500-3300 MHz, has been mounted in the FAST feed cabin and passed a series of observational tests. The whole UWB band is separated into four independent bands. Each band has 1048576 channels in total, resulted in a spectral resolution of 1 kHz. At 500-3300 MHz, the antenna gain is around 14.3-7.7 K/Jy, the aperture efficiency is around 0.56-0.30, the system temperature is around 88-130 K, and the HPBW is around 7.6-1.6 arcmin. The measured standard deviation of pointing accuracy is better than ~7.9 arcsec, when zenith angle (ZA) is within 26.4deg. The sensitivity and stability of the UWB receiver are confirmed to satisfy expectation by spectral observations, e.g., HI and OH. The FAST UWB receiver already has a good performance for taking sensitive observations in various scientific goals.

astro-ph.IM↗
arXiv2023

Quantum Similarity Testing with Convolutional Neural Networks

The task of testing whether two uncharacterized quantum devices behave in the same way is crucial for benchmarking near-term quantum computers and quantum simulators, but has so far remained open for continuous-variable quantum systems. In this Letter, we develop a machine learning algorithm for comparing unknown continuous variable states using limited and noisy data. The algorithm works on non-Gaussian quantum states for which similarity testing could not be achieved with previous techniques. Our approach is based on a convolutional neural network that assesses the similarity of quantum states based on a lower-dimensional state representation built from measurement data. The network can be trained offline with classically simulated data from a fiducial set of states sharing structural similarities with the states to be tested, or with experimental data generated by measurements on the fiducial states, or with a combination of simulated and experimental data. We test the performance of the model on noisy cat states and states generated by arbitrary selective number-dependent phase gates. Our network can also be applied to the problem of comparing continuous variable states across different experimental platforms, with different sets of achievable measurements, and to the problem of experimentally testing whether two states are equivalent up to Gaussian unitary transformations.

quant-ph↗
arXiv2023

GraspNeRF: Multiview-based 6-DoF Grasp Detection for Transparent and Specular Objects Using Generalizable NeRF

In this work, we tackle 6-DoF grasp detection for transparent and specular objects, which is an important yet challenging problem in vision-based robotic systems, due to the failure of depth cameras in sensing their geometry. We, for the first time, propose a multiview RGB-based 6-DoF grasp detection network, GraspNeRF, that leverages the generalizable neural radiance field (NeRF) to achieve material-agnostic object grasping in clutter. Compared to the existing NeRF-based 3-DoF grasp detection methods that rely on densely captured input images and time-consuming per-scene optimization, our system can perform zero-shot NeRF construction with sparse RGB inputs and reliably detect 6-DoF grasps, both in real-time. The proposed framework jointly learns generalizable NeRF and grasp detection in an end-to-end manner, optimizing the scene representation construction for the grasping. For training data, we generate a large-scale photorealistic domain-randomized synthetic dataset of grasping in cluttered tabletop scenes that enables direct transfer to the real world. Our extensive experiments in synthetic and real-world environments demonstrate that our method significantly outperforms all the baselines in all the experiments while remaining in real-time. Project page can be found at https://pku-epic.github.io/GraspNeRF

cs.RO↗
arXiv2023

Atlas of dynamic spectra of fast radio burst FRB 20201124A

Fast radio bursts (FRBs) are highly dispersed millisecond-duration radio bursts, of which the physical origin is still not fully understood. FRB 20201124A is one of the most actively repeating FRBs. In this paper, we present the collection of 1863 burst dynamic spectra of FRB 20201124A measured with the Five-hundred-meter Aperture Spherical radio Telescope (FAST). The current collection, taken from the observation during the FRB active phase from April to June 2021, is the largest burst sample detected in any FRB so far. The standard PSRFITs format is adopted, including dynamic spectra of the burst, and the time information of the dynamic spectra, in addition, mask files help readers to identify the pulse positions are also provided.

astro-ph.HE↗
arXiv2022

EndoBoost: a plug-and-play module for false positive suppression during computer-aided polyp detection in real-world colonoscopy (with dataset)

The advance of computer-aided detection systems using deep learning opened a new scope in endoscopic image analysis. However, the learning-based models developed on closed datasets are susceptible to unknown anomalies in complex clinical environments. In particular, the high false positive rate of polyp detection remains a major challenge in clinical practice. In this work, we release the FPPD-13 dataset, which provides a taxonomy and real-world cases of typical false positives during computer-aided polyp detection in real-world colonoscopy. We further propose a post-hoc module EndoBoost, which can be plugged into generic polyp detection models to filter out false positive predictions. This is realized by generative learning of the polyp manifold with normalizing flows and rejecting false positives through density estimation. Compared to supervised classification, this anomaly detection paradigm achieves better data efficiency and robustness in open-world settings. Extensive experiments demonstrate a promising false positive suppression in both retrospective and prospective validation. In addition, the released dataset can be used to perform 'stress' tests on established detection systems and encourages further research toward robust and reliable computer-aided endoscopic image analysis. The dataset and code will be publicly available at http://endoboost.miccai.cloud.

cs.CV↗
arXiv2022

Robust Time Series Chain Discovery with Incremental Nearest Neighbors

Time series motif discovery has been a fundamental task to identify meaningful repeated patterns in time series. Recently, time series chains were introduced as an expansion of time series motifs to identify the continuous evolving patterns in time series data. Informally, a time series chain (TSC) is a temporally ordered set of time series subsequences, in which every subsequence is similar to the one that precedes it, but the last and the first can be arbitrarily dissimilar. TSCs are shown to be able to reveal latent continuous evolving trends in the time series, and identify precursors of unusual events in complex systems. Despite its promising interpretability, unfortunately, we have observed that existing TSC definitions lack the ability to accurately cover the evolving part of a time series: the discovered chains can be easily cut by noise and can include non-evolving patterns, making them impractical in real-world applications. Inspired by a recent work that tracks how the nearest neighbor of a time series subsequence changes over time, we introduce a new TSC definition which is much more robust to noise in the data, in the sense that they can better locate the evolving patterns while excluding the non-evolving ones. We further propose two new quality metrics to rank the discovered chains. With extensive empirical evaluations, we demonstrate that the proposed TSC definition is significantly more robust to noise than the state of the art, and the top ranked chains discovered can reveal meaningful regularities in a variety of real world datasets.

cs.LG↗