arXiv subjects

Siyu Wang

Publications and source records attributed to Siyu Wang.

At least 73 records · Page 4Linked to original sources

arXiv2023

On the Opportunities and Challenges of Offline Reinforcement Learning for Recommender Systems

Reinforcement learning serves as a potent tool for modeling dynamic user interests within recommender systems, garnering increasing research attention of late. However, a significant drawback persists: its poor data efficiency, stemming from its interactive nature. The training of reinforcement learning-based recommender systems demands expensive online interactions to amass adequate trajectories, essential for agents to learn user preferences. This inefficiency renders reinforcement learning-based recommender systems a formidable undertaking, necessitating the exploration of potential solutions. Recent strides in offline reinforcement learning present a new perspective. Offline reinforcement learning empowers agents to glean insights from offline datasets and deploy learned policies in online settings. Given that recommender systems possess extensive offline datasets, the framework of offline reinforcement learning aligns seamlessly. Despite being a burgeoning field, works centered on recommender systems utilizing offline reinforcement learning remain limited. This survey aims to introduce and delve into offline reinforcement learning within recommender systems, offering an inclusive review of existing literature in this domain. Furthermore, we strive to underscore prevalent challenges, opportunities, and future pathways, poised to propel research in this evolving field.

cs.IR↗

arXiv2023

Nonstandard limit theorems and large deviation for beta -Jacobi ensembles with a different scaling

We consider $β$-Jacobi ensembles with parameters $p_1, p_2\geq n.$ We prove that the empirical measure of the rescaled Jacobi ensembles converges weakly to a modified Watcher law via the spectral measure method, which revisits the weak limits obtained in \cite{MaLDPJ} while replacing the condition $βn\!>\!> \log n$ by $βn\!>\!>1.$ We also provide the central limit theorem and the large deviation for the corresponding rescaled spectral measure.

math.PR↗

arXiv2023

Causal Disentangled Variational Auto-Encoder for Preference Understanding in Recommendation

Recommendation models are typically trained on observational user interaction data, but the interactions between latent factors in users' decision-making processes lead to complex and entangled data. Disentangling these latent factors to uncover their underlying representation can improve the robustness, interpretability, and controllability of recommendation models. This paper introduces the Causal Disentangled Variational Auto-Encoder (CaD-VAE), a novel approach for learning causal disentangled representations from interaction data in recommender systems. The CaD-VAE method considers the causal relationships between semantically related factors in real-world recommendation scenarios, rather than enforcing independence as in existing disentanglement methods. The approach utilizes structural causal models to generate causal representations that describe the causal relationship between latent factors. The results demonstrate that CaD-VAE outperforms existing methods, offering a promising solution for disentangling complex user behavior data in recommendation systems.

cs.IR↗

arXiv2023

Ada-Grouper: Accelerating Pipeline Parallelism in Preempted Network by Adaptive Group-Scheduling for Micro-Batches

Pipeline parallelism has been demonstrated to be a remarkable approach to improve throughput for training deep neural networks with billions of parameters over heterogeneous clusters. The 1F1B scheduling plan is a widely adopted strategy for memory and performance optimization, which interchanges the forward and backward stage computations of different micro-batches. On the other hand, a common issue in using the 1F1B scheduling is that stage computation is delayed due to the data transfer when network resources are preempted by other tasks, even with the minimum communication between stages. The exclusive access of these network resources cannot be guaranteed in cloud offerings. We present a general scheduling technique to accommodate pipeline parallelism to preempted network environments at the expense of a certain amount of memory pressure. The core concept is to extend 1F1B schedule scheme to kFkB, which groups k micro-batches, and alternately executes k forward and backward computations. We propose Ada-Grouper, an adaptive kFkB scheduler which regularly adjusts the number of group members k to maintain an optimal balance between communication and computation efficiency correspond to changes in a changing network environment under the memory limit. Experimental results demonstrate that our design maintain stable performance for pipeline parallelism, yielding a performance increase of up from 4% to 30%, compared with 1F1B in preempted network scenarios.

cs.DC↗

arXiv2023

Auto-Parallelizing Large Models with Rhino: A Systematic Approach on Production AI Platform

We present Rhino, a system for accelerating tensor programs with automatic parallelization on AI platform for real production environment. It transforms a tensor program written for a single device into an equivalent distributed program that is capable of scaling up to thousands of devices with no user configuration. Rhino firstly works on a semantically independent intermediate representation of tensor programs, which facilitates its generalization to unprecedented applications. Additionally, it implements a task-oriented controller and a distributed runtime for optimal performance. Rhino explores on a complete and systematic parallelization strategy space that comprises all the paradigms commonly employed in deep learning (DL), in addition to strided partitioning and pipeline parallelism on non-linear models. Aiming to efficiently search for a near-optimal parallel execution plan, our analysis of production clusters reveals general heuristics to speed up the strategy search. On top of it, two optimization levels are designed to offer users flexible trade-offs between the search time and strategy quality. Our experiments demonstrate that Rhino can not only re-discover the expert-crafted strategies of classic, research and production DL models, but also identify novel parallelization strategies which surpass existing systems for novel models.

cs.DC↗

arXiv2023

Expediting Distributed DNN Training with Device Topology-Aware Graph Deployment

This paper presents TAG, an automatic system to derive optimized DNN training graph and its deployment onto any device topology, for expedited training in device- and topology- heterogeneous ML clusters. We novelly combine both the DNN computation graph and the device topology graph as input to a graph neural network (GNN), and join the GNN with a search-based method to quickly identify optimized distributed training strategies. To reduce communication in a heterogeneous cluster, we further explore a lossless gradient compression technique and solve a combinatorial optimization problem to automatically apply the technique for training time minimization. We evaluate TAG with various representative DNN models and device topologies, showing that it can achieve up to 4.56x training speed-up as compared to existing schemes. TAG can produce efficient deployment strategies for both unseen DNN models and unseen device topologies, without heavy fine-tuning.

cs.LG↗

arXiv2022

Optical microcombs in whispering gallery mode crystalline resonators with dispersive intermode interactions

Soliton microcombs have shown great potential in a variety of applications ranging from chip scale frequency metrology to optical communications and photonic data center, in which light coupling among cavity transverse modes, termed as intermode interactions, are long-existing and usually give rise to localized impacts on the soliton state. Of particular interest are whispering gallery mode based crystalline resonators, which with dense mode families, potentially feature interactions of all kind. While effects of narrow-band interactions such as spectral power spikes have been well recognized in crystalline resonators, that of broadband interactions remains unexplored. Here, we demonstrate microcombs with broadband and dispersive intermode interactions, in home-developed magnesium fluoride microresonators with an intrinsic $\mathbf{Q}$-factor approaching 10 billion.In addition to conventional soliton comb generation in the single mode pumping scheme, comb states with broadband spectral tailoring effect have been observed, via an intermode pumping scheme.Remarkably, footprints of both constructive and destructive interference on the comb spectrum have been observed, which as confirmed by simulations, are connected to the dispersive effects of the coupled mode family.Our results not only contribute to the understanding of dissipative soliton dynamics in multi-mode or coupled resonator systems, but also extend the access to stable soliton combs in crystalline microresonators where mode control and dispersion engineering are usually challenging.

physics.optics↗

arXiv2022

Optimizing DNN Compilation for Distributed Training with Joint OP and Tensor Fusion

This paper proposes DisCo, an automatic deep learning compilation module for data-parallel distributed training. Unlike most deep learning compilers that focus on training or inference on a single device, DisCo optimizes a DNN model for distributed training over multiple GPU machines. Existing single-device compilation strategies do not work well in distributed training, due mainly to communication inefficiency that they incur. DisCo generates optimized, joint computation operator and communication tensor fusion strategies to enable highly efficient distributed training. A GNN-based simulator is built to effectively estimate per-iteration training time achieved by operator/tensor fusion candidates. A backtracking search algorithm is driven by the simulator, navigating efficiently in the large strategy space to identify good operator/tensor fusion strategies that minimize distributed training time. We compare DisCo with existing DL fusion schemes and show that it achieves good training speed-up close to the ideal, full computation-communication overlap case.

cs.DC↗

arXiv2022

Intrinsically Motivated Reinforcement Learning based Recommendation with Counterfactual Data Augmentation

Deep reinforcement learning (DRL) has been proven its efficiency in capturing users' dynamic interests in recent literature. However, training a DRL agent is challenging, because of the sparse environment in recommender systems (RS), DRL agents could spend times either exploring informative user-item interaction trajectories or using existing trajectories for policy learning. It is also known as the exploration and exploitation trade-off which affects the recommendation performance significantly when the environment is sparse. It is more challenging to balance the exploration and exploitation in DRL RS where RS agent need to deeply explore the informative trajectories and exploit them efficiently in the context of recommender systems. As a step to address this issue, We design a novel intrinsically ,otivated reinforcement learning method to increase the capability of exploring informative interaction trajectories in the sparse environment, which are further enriched via a counterfactual augmentation strategy for more efficient exploitation. The extensive experiments on six offline datasets and three online simulation platforms demonstrate the superiority of our model to a set of existing state-of-the-art methods.

cs.IR↗

arXiv2022

Two weight inequality for Hankel form on weighted Bergman spaces induced by doubling weights

The boundedness of the small Hankel operator $h_f^ν(g)=P_ν(f\bar{g})$, induced by an analytic symbol $f$ and the Bergman projection $P_ν$ associated to $ν$, acting from the weighted Bergman space $A^p_\om$ to $A^q_ν$ is characterized on the full range $0<p,q<\infty$ when $ω,ν$ belong to the class $\mathcal{D}$ of radial weights admitting certain two-sided doubling conditions. Certain results obtained are equivalent to the boundedness of bilinear Hankel forms, which are in turn used to establish the weak factorization $A_η^{q}=A_ω^{p_{1}}\odot A_ν^{p_{2}}$, where $1<q,p_{1},p_{2}<\infty$ such that $q^{-1}=p_{1}^{-1}+p_{2}^{-1}$ and $\widetildeη^{\frac{1}{q}}\asymp\widetildeω^{\frac{1}{p_{1}}}\widetildeν^{\frac{1}{p_{2}}}$. Here $\widetildeτ(r)=\int_r^1τ(t)\,dt/(1-t)$ for all $0\le r<1$.

math.FA↗

arXiv2022

Fast Lossless Neural Compression with Integer-Only Discrete Flows

By applying entropy codecs with learned data distributions, neural compressors have significantly outperformed traditional codecs in terms of compression ratio. However, the high inference latency of neural networks hinders the deployment of neural compressors in practical applications. In this work, we propose Integer-only Discrete Flows (IODF), an efficient neural compressor with integer-only arithmetic. Our work is built upon integer discrete flows, which consists of invertible transformations between discrete random variables. We propose efficient invertible transformations with integer-only arithmetic based on 8-bit quantization. Our invertible transformation is equipped with learnable binary gates to remove redundant filters during inference. We deploy IODF with TensorRT on GPUs, achieving 10x inference speedup compared to the fastest existing neural compressors, while retaining the high compression rates on ImageNet32 and ImageNet64.

cs.LG↗

arXiv2022

Model-agnostic Counterfactual Synthesis Policy for Interactive Recommendation

Interactive recommendation is able to learn from the interactive processes between users and systems to confront the dynamic interests of users. Recent advances have convinced that the ability of reinforcement learning to handle the dynamic process can be effectively applied in the interactive recommendation. However, the sparsity of interactive data may hamper the performance of the system. We propose to train a Model-agnostic Counterfactual Synthesis Policy to generate counterfactual data and address the data sparsity problem by modelling from observation and counterfactual distribution. The proposed policy can identify and replace the trivial components for any state in the training process with other agents, which can be deployed in any RL-based algorithm. The experimental results demonstrate the effectiveness and generality of our proposed policy.

cs.IR↗

arXiv2021

Refractive index and extinction coefficient of vapor-deposited water ice in the UV-Vis range

Laboratory results of the optical properties of vapor-deposited water ice, specifically the refractive index and extinction coefficient, are available mainly for a selective set of wavelengths and a limited number of deposition temperatures. Experimental limitations are the main reason for the lack of broadband data, which is unfortunate as these quantities are needed to interpret and predict astronomical and planetary observations. The goal of this work is to address these lacking data, using an experimental broadband method that is capable of rapidly providing reliable water ice data across the entire UV-visible range. This approach combines the simultaneous use of a monochromatic HeNe laser and a broadband Xe-arc lamp to record interference fringes of water ice during deposition at astronomically relevant ice temperatures. The ice thickness is typically more than 20 $μ$m. Analyzing the period and intensity patterns combining both the monochromatic and broadband interference patterns allows the determination of the wavelength-dependent refractive index and extinction coefficient. We present accurate refractive index and extinction coefficient graphs for wavelengths between 250 and 750 nm and ices deposited between 30 and 160 K. From our data, we find a possible structural change in the ice in the 110-130 K region that has not been reported before. We also discuss that the data presented in this paper can be used to interpret astronomical observations of icy surfaces.

astro-ph.IM↗

arXiv2021

Adversarial Robustness of Deep Reinforcement Learning based Dynamic Recommender Systems

Adversarial attacks, e.g., adversarial perturbations of the input and adversarial samples, pose significant challenges to machine learning and deep learning techniques, including interactive recommendation systems. The latent embedding space of those techniques makes adversarial attacks difficult to detect at an early stage. Recent advance in causality shows that counterfactual can also be considered one of ways to generate the adversarial samples drawn from different distribution as the training samples. We propose to explore adversarial examples and attack agnostic detection on reinforcement learning-based interactive recommendation systems. We first craft different types of adversarial examples by adding perturbations to the input and intervening on the casual factors. Then, we augment recommendation systems by detecting potential attacks with a deep learning-based classifier based on the crafted data. Finally, we study the attack strength and frequency of adversarial examples and evaluate our model on standard datasets with multiple crafting methods. Our extensive experiments show that most adversarial attacks are effective, and both attack strength and attack frequency impact the attack performance. The strategically-timed attack achieves comparative attack performance with only 1/3 to 1/2 attack frequency. Besides, our black-box detector trained with one crafting method has the generalization ability over several other crafting methods.

cs.LG↗

arXiv2021

Toeplitz operators on weighted Bergman spaces induced by a class of radial weights

Suppose that $ω$ is a radial weight on the unit disk that satisfies both forward and reverse doubling conditions. Using Carleson measures and $T1$-type conditions, we obtain necessary and sufficient conditions of the positive Borel measure $μ$ such that the Toeplitz operator $T_{μ,ω}:L^p_a(ω)\to L_a^1(ω)$ is bounded and compact for $0<p\leq 1$. In addition, we obtain a bump condition for the bounded Toeplitz operators with $L^1(ω)$ symbol on $L^1_a(ω)$. This generalizes a result of Zhu in \cite{zhu1989}.

math.FA↗

arXiv2021

Volterra type integration operators between weighted Bergman spaces and Hardy spaces

Let $\mathcal{D}$ be the class of radial weights on the unit disk which satisfy both forward and reverse doubling conditions. Let $g$ be an analytic function on the unit disk $\mathbb{D}$. We characterize bounded and compact Volterra type integration operators \[ J_{g}(f)(z)=\int_{0}^{z}f(λ)g'(λ)dλ\] between weighted Bergman spaces $L_{a}^{p}(ω)$ induced by $\mathcal{D}$ weights and Hardy spaces $H^{q}$ for $0<p,q<\infty$.

math.FA↗

arXiv2020

The seesaw impacts between reduced emissions and enhanced AOC on O$_3$ during the COVID-19

Due to the lockdown measures during the 2019 novel coronavirus (COVID-19) pandemic, the economic activities and the associated emissions have significantly declined. This reduction in emissions has created a natural experiment to assess the impact of the emitted precursor control policy on ozone (O$_3$) pollution, which has become a public concern in China during the last decade. In this study, we utilized comprehensive satellite, ground-level observations, and source-oriented chemical transport modeling to investigate the O$_3$ variations during the COVID-19 in China. Here we found that the O$_3$ formation regime shifted from a VOC-limited regime to a NOx-limited regime due to the lower NOx during the COVID-19 lockdown. However, instead of these changes of the O$_3$ formation region, the significant elevated O$_3$ in the North China Plain (40%) and Yangtze River Delta (35%) were mainly attributed to the enhanced atmospheric oxidant capacity (AOC) in these regions, which was different from previous studies. We suggest that future O$_3$ control policies should comprehensively consider the synergistic effects of O$_3$ formation regime and AOC on the O$_3$ elevation.

physics.ao-ph↗

arXiv2020

Auto-MAP: A DQN Framework for Exploring Distributed Execution Plans for DNN Workloads

The last decade has witnessed growth in the computational requirements for training deep neural networks. Current approaches (e.g., data/model parallelism, pipeline parallelism) parallelize training tasks onto multiple devices. However, these approaches always rely on specific deep learning frameworks and requires elaborate manual design, which make it difficult to maintain and share between different type of models. In this paper, we propose Auto-MAP, a framework for exploring distributed execution plans for DNN workloads, which can automatically discovering fast parallelization strategies through reinforcement learning on IR level of deep learning models. Efficient exploration remains a major challenge for reinforcement learning. We leverage DQN with task-specific pruning strategies to help efficiently explore the search space including optimized strategies. Our evaluation shows that Auto-MAP can find the optimal solution in two hours, while achieving better throughput on several NLP and convolution models.

cs.DC↗