Searcharxiv⌕ Search

arXiv subjects

Dong Wang

Publications and source records attributed to Dong Wang.

At least 397 records · Page 22Linked to original sources
arXiv2023

Ten issues of NetGPT

With the rapid development and application of foundation models (FMs), it is foreseeable that FMs will play an important role in future wireless communications. As current Artificial Intelligence (AI) algorithms applied in wireless networks are dedicated models that aim for different neural network architectures and objectives, drawbacks in aspects of generality, performance gain, management, collaboration, etc. need to be conquered. In this paper, we define NetGPT (Network Generative Pre-trained Transformer) -- the foundation models for wireless communications, and summarize ten issues regarding design and application of NetGPT.

cs.NI↗
arXiv2023

Page Curves in Holographic Superconductors

Considering a doubly holographic model, we study the black hole information paradox for the eternal AdSd-RN black hole coupled to and in equilibrium with a d-dimensional conformal bath whose state has been deformed by the charged scalar field coupled to a U(1) gauge field. Without a brane, the spontaneous symmetry breaking of the gauge field on boundary systems can induce a second-order phase transition of the charged scalar field at the critical temperature, known as holographic superconductors. The bath deformation can significantly change its entanglement dynamics with the black hole, resulting in variations in the Page curve and Page time. Our results indicate that characteristic parameters of the Page curve, such as entanglement velocity, initial area difference and Page time, can be used as suitable probes to detect superconducting phase transitions. In particular, the entanglement velocity can also probe both Kasner flows and Josephson oscillations. When keeping the endpoint of the radiation region fixed at twice the critical Page point, the entanglement velocity (the internal backreaction) has a more significant influence on the Page time compared to the initial area difference (the external backreaction).

hep-th↗
arXiv2023

Linear Recurrent Units for Sequential Recommendation

State-of-the-art sequential recommendation relies heavily on self-attention-based recommender models. Yet such models are computationally expensive and often too slow for real-time recommendation. Furthermore, the self-attention operation is performed at a sequence-level, thereby making low-cost incremental inference challenging. Inspired by recent advances in efficient language modeling, we propose linear recurrent units for sequential recommendation (LRURec). Similar to recurrent neural networks, LRURec offers rapid inference and can achieve incremental inference on sequential inputs. By decomposing the linear recurrence operation and designing recursive parallelization in our framework, LRURec provides the additional benefits of reduced model size and parallelizable training. Moreover, we optimize the architecture of LRURec by implementing a series of modifications to address the lack of non-linearity and improve training dynamics. To validate the effectiveness of our proposed LRURec, we conduct extensive experiments on multiple real-world datasets and compare its performance against state-of-the-art sequential recommenders. Experimental results demonstrate the effectiveness of LRURec, which consistently outperforms baselines by a significant margin. Results also highlight the efficiency of LRURec with our parallelized training paradigm and fast inference on long sequences, showing its potential to further enhance user experience in sequential recommendation.

cs.IR↗
arXiv2023

Analysis and Applications of Deep Learning with Finite Samples in Full Life-Cycle Intelligence of Nuclear Power Generation

The advent of Industry 4.0 has precipitated the incorporation of Artificial Intelligence (AI) methods within industrial contexts, aiming to realize intelligent manufacturing, operation as well as maintenance, also known as industrial intelligence. However, intricate industrial milieus, particularly those relating to energy exploration and production, frequently encompass data characterized by long-tailed class distribution, sample imbalance, and domain shift. These attributes pose noteworthy challenges to data-centric Deep Learning (DL) techniques, crucial for the realization of industrial intelligence. The present study centers on the intricate and distinctive industrial scenarios of Nuclear Power Generation (NPG), meticulously scrutinizing the application of DL techniques under the constraints of finite data samples. Initially, the paper expounds on potential employment scenarios for AI across the full life-cycle of NPG. Subsequently, we delve into an evaluative exposition of DL's advancement, grounded in the finite sample perspective. This encompasses aspects such as small-sample learning, few-shot learning, zero-shot learning, and open-set recognition, also referring to the unique data characteristics of NPG. The paper then proceeds to present two specific case studies. The first revolves around the automatic recognition of zirconium alloy metallography, while the second pertains to open-set recognition for signal diagnosis of machinery sensors. These cases, spanning the entirety of NPG's life-cycle, are accompanied by constructive outcomes and insightful deliberations. By exploring and applying DL methodologies within the constraints of finite sample availability, this paper not only furnishes a robust technical foundation but also introduces a fresh perspective toward the secure and efficient advancement and exploitation of this advanced energy source.

cs.LG↗
arXiv2023

Affordance-Driven Next-Best-View Planning for Robotic Grasping

Grasping occluded objects in cluttered environments is an essential component in complex robotic manipulation tasks. In this paper, we introduce an AffordanCE-driven Next-Best-View planning policy (ACE-NBV) that tries to find a feasible grasp for target object via continuously observing scenes from new viewpoints. This policy is motivated by the observation that the grasp affordances of an occluded object can be better-measured under the view when the view-direction are the same as the grasp view. Specifically, our method leverages the paradigm of novel view imagery to predict the grasps affordances under previously unobserved view, and select next observation view based on the highest imagined grasp quality of the target object. The experimental results in simulation and on a real robot demonstrate the effectiveness of the proposed affordance-driven next-best-view planning policy. Project page: https://sszxc.net/ace-nbv/.

cs.RO↗
arXiv2023

LlamaRec: Two-Stage Recommendation using Large Language Models for Ranking

Recently, large language models (LLMs) have exhibited significant progress in language understanding and generation. By leveraging textual features, customized LLMs are also applied for recommendation and demonstrate improvements across diverse recommendation scenarios. Yet the majority of existing methods perform training-free recommendation that heavily relies on pretrained knowledge (e.g., movie recommendation). In addition, inference on LLMs is slow due to autoregressive generation, rendering existing methods less effective for real-time recommendation. As such, we propose a two-stage framework using large language models for ranking-based recommendation (LlamaRec). In particular, we use small-scale sequential recommenders to retrieve candidates based on the user interaction history. Then, both history and retrieved items are fed to the LLM in text via a carefully designed prompt template. Instead of generating next-item titles, we adopt a verbalizer-based approach that transforms output logits into probability distributions over the candidate items. Therefore, the proposed LlamaRec can efficiently rank items without generating long text. To validate the effectiveness of the proposed framework, we compare against state-of-the-art baseline methods on benchmark datasets. Our experimental results demonstrate the performance of LlamaRec, which consistently achieves superior performance in both recommendation performance and efficiency.

cs.IR↗
arXiv2023

Cross-Domain Policy Adaptation via Value-Guided Data Filtering

Generalizing policies across different domains with dynamics mismatch poses a significant challenge in reinforcement learning. For example, a robot learns the policy in a simulator, but when it is deployed in the real world, the dynamics of the environment may be different. Given the source and target domain with dynamics mismatch, we consider the online dynamics adaptation problem, in which case the agent can access sufficient source domain data while online interactions with the target domain are limited. Existing research has attempted to solve the problem from the dynamics discrepancy perspective. In this work, we reveal the limitations of these methods and explore the problem from the value difference perspective via a novel insight on the value consistency across domains. Specifically, we present the Value-Guided Data Filtering (VGDF) algorithm, which selectively shares transitions from the source domain based on the proximity of paired value targets across the two domains. Empirical results on various environments with kinematic and morphology shifts demonstrate that our method achieves superior performance compared to prior approaches.

cs.LG↗
arXiv2023

Diffusion Model is an Effective Planner and Data Synthesizer for Multi-Task Reinforcement Learning

Diffusion models have demonstrated highly-expressive generative capabilities in vision and NLP. Recent studies in reinforcement learning (RL) have shown that diffusion models are also powerful in modeling complex policies or trajectories in offline datasets. However, these works have been limited to single-task settings where a generalist agent capable of addressing multi-task predicaments is absent. In this paper, we aim to investigate the effectiveness of a single diffusion model in modeling large-scale multi-task offline data, which can be challenging due to diverse and multimodal data distribution. Specifically, we propose Multi-Task Diffusion Model (\textsc{MTDiff}), a diffusion-based method that incorporates Transformer backbones and prompt learning for generative planning and data synthesis in multi-task offline settings. \textsc{MTDiff} leverages vast amounts of knowledge available in multi-task data and performs implicit knowledge sharing among tasks. For generative planning, we find \textsc{MTDiff} outperforms state-of-the-art algorithms across 50 tasks on Meta-World and 8 maps on Maze2D. For data synthesis, \textsc{MTDiff} generates high-quality data for testing tasks given a single demonstration as a prompt, which enhances the low-quality datasets for even unseen tasks.

cs.LG↗
arXiv2023

A Glance is Enough: Extract Target Sentence By Looking at A keyword

This paper investigates the possibility of extracting a target sentence from multi-talker speech using only a keyword as input. For example, in social security applications, the keyword might be "help", and the goal is to identify what the person who called for help is articulating while ignoring other speakers. To address this problem, we propose using the Transformer architecture to embed both the keyword and the speech utterance and then rely on the cross-attention mechanism to select the correct content from the concatenated or overlapping speech. Experimental results on Librispeech demonstrate that our proposed method can effectively extract target sentences from very noisy and mixed speech (SNR=-3dB), achieving a phone error rate (PER) of 26\%, compared to the baseline system's PER of 96%.

cs.CL↗
arXiv2023

Observing parity-time symmetry in diffusive systems

Phase modulation has scarcely been mentioned in diffusive systems since the diffusion process does not carry momentum like waves. Recently, the non-Hermitian physics provides a new perspective for understanding diffusion and shows prospects in the phase regulation of heat flow, for example, the discovery of anti-parity-time (APT) symmetry in diffusive systems. The precise control of thermal phase however remains elusive hitherto and can hardly be realized in APT-symmetric thermal systems due to the existence of phase oscillation. Here we construct the counterpart of APT-symmetric diffusive systems, i.e., PT-symmetric diffusive systems, which can achieve complete suppression of thermal phase oscillation. We find the real coupling of diffusive fields can be readily established through a strong convective background, where the decay-rate detuning is enabled by thermal metamaterial design. Moreover, we observe phase transition of PT symmetry breaking in diffusive systems with the symmetry-determined amplitude distribution and phase regulation of coupled temperature fields. Our work uncovers the existence of PT-symmetry in dissipative energy exchanges and provides a unique approach for harnessing the mass transfer of particles, the wave propagation in strongly scattering systems as well as thermal conduction.

physics.app-ph↗
arXiv2023

Multi-Domain Adaptation by Self-Supervised Learning for Speaker Verification

In real-world applications, speaker recognition models often face various domain-mismatch challenges, leading to a significant drop in performance. Although numerous domain adaptation techniques have been developed to address this issue, almost all present methods focus on a simple configuration where the model is trained in one domain and deployed in another. However, real-world environments are often complex and may contain multiple domains, making the methods designed for one-to-one adaptation suboptimal. In our paper, we propose a self-supervised learning method to tackle this multi-domain adaptation problem. Building upon the basic self-supervised adaptation algorithm, we designed three strategies to make it suitable for multi-domain adaptation: an in-domain negative sampling strategy, a MoCo-like memory bank scheme, and a CORAL-like distribution alignment. We conducted experiments using VoxCeleb2 as the source domain dataset and CN-Celeb1 as the target multi-domain dataset. Our results demonstrate that our method clearly outperforms the basic self-supervised adaptation method, which simply treats the data of CN-Celeb1 as a single domain. Importantly, the improvement is consistent in nearly all in-domain tests and cross-domain tests, demonstrating the effectiveness of our proposed method.

cs.SD↗
arXiv2023

An Investigation of Distribution Alignment in Multi-Genre Speaker Recognition

Multi-genre speaker recognition is becoming increasingly popular due to its ability to better represent the complexities of real-world applications. However, a major challenge is the significant shift in the distribution of speaker vectors across different genres. While distribution alignment is a common approach to address this challenge, previous studies have mainly focused on aligning a source domain with a target domain, and the performance of multi-genre data is unknown. This paper presents a comprehensive study of mainstream distribution alignment methods on multi-genre data, where multiple distributions need to be aligned. We analyze various methods both qualitatively and quantitatively. Our experiments on the CN-Celeb dataset show that within-between distribution alignment (WBDA) performs relatively better. However, we also found that none of the investigated methods consistently improved performance in all test cases. This suggests that solely aligning the distributions of speaker vectors may not fully address the challenges posed by multi-genre speaker recognition. Further investigation is necessary to develop a more comprehensive solution.

cs.SD↗
arXiv2023

Leveraging the Power of Data Augmentation for Transformer-based Tracking

Due to long-distance correlation and powerful pretrained models, transformer-based methods have initiated a breakthrough in visual object tracking performance. Previous works focus on designing effective architectures suited for tracking, but ignore that data augmentation is equally crucial for training a well-performing model. In this paper, we first explore the impact of general data augmentations on transformer-based trackers via systematic experiments, and reveal the limited effectiveness of these common strategies. Motivated by experimental observations, we then propose two data augmentation methods customized for tracking. First, we optimize existing random cropping via a dynamic search radius mechanism and simulation for boundary samples. Second, we propose a token-level feature mixing augmentation strategy, which enables the model against challenges like background interference. Extensive experiments on two transformer-based trackers and six benchmarks demonstrate the effectiveness and data efficiency of our methods, especially under challenging settings, like one-shot tracking and small image resolutions.

cs.CV↗
arXiv2023

Designing the pressure-dependent shear modulus using tessellated granular metamaterials

Jammed packings of granular materials display complex mechanical response. For example, the ensemble-averaged shear modulus $\left\langle G \right\rangle$ increases as a power-law in pressure $p$ for static packings of soft spherical particles that can rearrange during compression. We seek to design granular materials with shear moduli that can either increase {\it or} decrease with pressure without particle rearrangements even in the large-system limit. To do this, we construct {\it tessellated} granular metamaterials by joining multiple particle-filled cells together. We focus on cells that contain a small number of bidisperse disks in two dimensions. We first study the mechanical properties of individual disk-filled cells with three types of boundaries: periodic boundary conditions (PBC), fixed-length walls (FXW), and flexible walls (FLW). Hypostatic jammed packings are found for cells with FLW, but not in cells with PBC and FXW, and they are stabilized by quartic modes of the dynamical matrix. The shear modulus of a single cell depends linearly on $p$. We find that the slope of the shear modulus with pressure, $λ_c < 0$ for all packings in single cells with PBC where the number of particles per cell $N \ge 6$. In contrast, single cells with FXW and FLW can possess $λ_c > 0$, as well as $λ_c < 0$, for $N \le 16$. We show that we can force the mechanical properties of multi-cell granular metamaterials to possess those of single cells by constraining the endpoints of the outer walls and enforcing an affine shear response. These studies demonstrate that tessellated granular metamaterials provide a novel platform for the design of soft materials with specified mechanical properties.

cond-mat.soft↗
arXiv2023

Deep Variational Luenberger-type Observer for Stochastic Video Prediction

Considering the inherent stochasticity and uncertainty, predicting future video frames is exceptionally challenging. In this work, we study the problem of video prediction by combining interpretability of stochastic state space models and representation learning of deep neural networks. Our model builds upon an variational encoder which transforms the input video into a latent feature space and a Luenberger-type observer which captures the dynamic evolution of the latent features. This enables the decomposition of videos into static features and dynamics in an unsupervised manner. By deriving the stability theory of the nonlinear Luenberger-type observer, the hidden states in the feature space become insensitive with respect to the initial values, which improves the robustness of the overall model. Furthermore, the variational lower bound on the data log-likelihood can be derived to obtain the tractable posterior prediction distribution based on the variational principle. Finally, the experiments such as the Bouncing Balls dataset and the Pendulum dataset are provided to demonstrate the proposed model outperforms concurrent works.

cs.CV↗
arXiv2023

NMA: Neural Multi-slot Auctions with Externalities for Online Advertising

Online advertising driven by auctions brings billions of dollars in revenue for social networking services and e-commerce platforms. GSP auctions, which are simple and easy to understand for advertisers, have almost become the benchmark for ad auction mechanisms in the industry. However, most GSP-based industrial practices assume that the user click only relies on the ad itself, which overlook the effect of external items, referred to as externalities. Recently, DNA has attempted to upgrade GSP with deep neural networks and models local externalities to some extent. However, it only considers set-level contexts from auctions and ignores the order and displayed position of ads, which is still suboptimal. Although VCG-based multi-slot auctions (e.g., VCG, WVCG) make it theoretically possible to model global externalities (e.g., the order and positions of ads and so on), they lack an efficient balance of both revenue and social welfare. In this paper, we propose novel auction mechanisms named Neural Multi-slot Auctions (NMA) to tackle the above-mentioned challenges. Specifically, we model the global externalities effectively with a context-aware list-wise prediction module to achieve better performance. We design a list-wise deep rank module to guarantee incentive compatibility in end-to-end learning. Furthermore, we propose an auxiliary loss for social welfare to effectively reduce the decline of social welfare while maximizing revenue. Experiment results on both offline large-scale datasets and online A/B tests demonstrate that NMA obtains higher revenue with balanced social welfare than other existing auction mechanisms (i.e., GSP, DNA, WVCG) in industrial practice, and we have successfully deployed NMA on Meituan food delivery platform.

cs.AI↗
arXiv2023

Holographic entanglement entropy and subregion complexity for excited states of holographic superconductors

We investigate the holographic entanglement entropy (HEE) and the holographic subregion complexity (HSC) for holographic superconductors, both in the Einstein and in the Einstein-Gauss-Bonnet gravitational theories. For both ground and excited states, we show that, in the Einstein gravity, the HSC decreases as the temperature increases and the normal phase has a smaller HSC than the superconducting phase, which is opposite to the behavior of the HEE. Moreover, we find out that, for a given temperature $T$ in the superconducting phase, the higher excited state leads to a lager value of the HEE but a smaller value of the HSC. However, the Einstein-Gauss-Bonnet gravity has significantly different effect on the HSC, while the HEE always increases monotonously as the temperature increases and its value in the normal phase always larger than that in the superconducting phase. Our results indicate that the HEE and HSC provide richer physics in phase transitions and condensation of scalar hair for holographic superconductors with excited states.

hep-th↗
arXiv2023

Holographic superfluid with excited states

We construct a novel family of solutions of the holographic superfluid model with the excited states in the probe limit. We observe that the higher excited state or larger superfluid velocity will make the scalar hair more difficult to be developed, and the higher excited state or smaller mass of the scalar field makes it easier for the emergence of translating point from the second-order transition to the first-order one. We note that the difference of the critical chemical potential between the consecutive states increases as the superfluid velocity increases. Interestingly, the "Cave of Winds" phase structure will disappear but the first-order phase transition occurs for the excited states, which is completely different from the holographic superfluid model with the ground state. This means that the excited state will hinder the appearance of the Cave of Winds. Moreover, we find that there exist additional poles in Im[$σ(ω)$] and delta functions in Re[$σ(ω)$] arising at low temperature for the excited states, and the higher excited state or larger superfluid velocity results in the larger deviation from the expected relation in the gap frequency.

hep-th↗