Searcharxiv⌕ Search

arXiv subjects

Dong Li

Publications and source records attributed to Dong Li.

At least 343 records · Page 19Linked to original sources
arXiv2023

Phase Shift Compression for Control Signaling Reduction in IRS-Aided Wireless Systems: Global Attention and Lightweight Design

A potential 6G technology known as intelligent reflecting surface (IRS) has recently gained much attention from academia and industry. However, acquiring the optimized quantized phase shift (QPS) presents challenges for the IRS due to the phenomenon of signaling storms. In this paper, we attempt to solve the above problem by proposing two deep learning models, the global attention phase shift compression network (GAPSCN) and the simplified GAPSCN (S-GAPSCN). In GAPSCN, we propose a novel attention mechanism that emphasizes a greater number of meaningful features than previous attention-related works. Additionally, S-GAPSCN is built with an asymmetric architecture to meet the practical constraints on computation resources of the IRS controller. Moreover, in S-GAPSCN, to compensate for the performance degradation caused by simplifying the model, we design a low-computation complexity joint attention-assisted multi-scale network (JAAMSN) module in the decoder of S-GAPSCN. Simulation results demonstrate that the proposed global attention mechanism achieves prominent performance compared with the existing attention mechanisms and the proposed GAPSCN can achieve reliable reconstruction performance compared with existing state-of-the-art models. Furthermore, the proposed S-GAPSCN can approach the performance of the GAPSCN at a much lower computational cost.

eess.SP↗
arXiv2023

Computation-Efficient Backscatter-Blessed MEC with User Reciprocity

This letter proposes a new user cooperative offloading protocol called user reciprocity in backscatter communication (BackCom)-aided mobile edge computing systems with efficient computation, whose quintessence is that each user can switch alternately between the active or the BackCom mode in different slots, and one user works in the active mode and the other user works in the BackCom mode in each time slot. In particular, the user in the BackCom mode can always use the signal transmitted by the user in the active mode for more data transmission in a spectrum-sharing manner. To evaluate the proposed protocol, a computation efficiency (CE) maximization-based optimization problem is formulated by jointly power control, time scheduling, reflection coefficient adjustment, and computing frequency allocation, while satisfying various physical constraints on the maximum energy budget, the computing frequency threshold, the minimum computed bits, and harvested energy threshold. To solve this non-convex problem, Dinkelbach's method and quadratic transform are first employed to transform the complex fractional forms into linear ones. Then, an iterative algorithm is designed by decomposing the resulting problem to obtain the suboptimal solution. The closed-form solutions for the transmit power, the RC, and the local computing frequency are provided for more insights. Besides, the analytical performance gain with the reciprocal mode is also derived. Simulation results demonstrate that the proposed scheme outperforms benchmark schemes regarding the CE.

cs.IT↗
arXiv2023

Toeplitz Neural Network for Sequence Modeling

Sequence modeling has important applications in natural language processing and computer vision. Recently, the transformer-based models have shown strong performance on various sequence modeling tasks, which rely on attention to capture pairwise token relations, and position embedding to inject positional information. While showing good performance, the transformer models are inefficient to scale to long input sequences, mainly due to the quadratic space-time complexity of attention. To overcome this inefficiency, we propose to model sequences with a relative position encoded Toeplitz matrix and use a Toeplitz matrix-vector production trick to reduce the space-time complexity of the sequence modeling to log linear. A lightweight sub-network called relative position encoder is proposed to generate relative position coefficients with a fixed budget of parameters, enabling the proposed Toeplitz neural network to deal with varying sequence lengths. In addition, despite being trained on 512-token sequences, our model can extrapolate input sequence length up to 14K tokens in inference with consistent performance. Extensive experiments on autoregressive and bidirectional language modeling, image modeling, and the challenging Long-Range Arena benchmark show that our method achieves better performance than its competitors in most downstream tasks while being significantly faster. The code is available at https://github.com/OpenNLPLab/Tnn.

cs.CL↗
arXiv2023

Rethinking Memory Profiling and Migration for Multi-Tiered Large Memory Systems

Multi-tiered large memory systems call for rethinking of memory profiling and migration because of the unique problems unseen in the traditional memory systems with smaller capacity and fewer tiers. We develop MTM, an application-transparent page management system based on three principles: (1) connecting the control of profiling overhead with the profiling mechanism for high-quality profiling; (2) building a universal page migration policy on the complex multi-tiered memory for high performance; and (3) introducing huge page awareness. We evaluate MTM using common big-data applications with realistic working sets (hundreds of GB to 1 TB). MTM outperforms seven state-of-the-art solutions by up to 42% (17% on average)

cs.PF↗
arXiv2023

Global energetics of solar powerful events on 6 September 2017

Solar flares and coronal mass ejections (CMEs) are thought to be the most powerful events on the Sun. They can release energy as high as 10^32 erg in tens of minutes,and could produce solar energetic particles (SEPs) in the interplanetary space. We explore global energy budgets of solar major eruptions on 6 September 2017, including the energy partition of a powerful solar flare, the energy budget of the accompanied CME and SEPs. In the wavelength range shortward of 222 nm, a major contribution of the flare radiated energy is in the soft X-ray (SXR) 0.1-7 nm domain. The flare energy radiated at wavelengths of Ly-alpha and middle ultraviolet is larger than that radiated in the extreme ultraviolet wavelength, but it is much less than that radiated in the SXR waveband. The total flare radiated energy could be comparable to the thermal and nonthermal energies. The energies carried by the major flare and its accompanied CME are roughly equal, and they are both powered by the magnetic free energy in the AR NOAA 12673. Moreover, the CME is efficient in accelerating SEPs, and that the prompt component (whether it comes from the solar flare or the CME) contributes only a negligible fraction.

astro-ph.SR↗
arXiv2023

To Reflect or Not To Reflect: On-Off Control and Number Configuration for Reflecting Elements in RIS-Aided Wireless Systems

Reconfigurable intelligent surface (RIS) has been regarded as a promising technique due to its high array gain and low power. However, the traditional passive RIS suffers from the ``double fading'' effect, which has restricted the performance of passive RIS-aided communications. Fortunately, active RIS can alleviate this problem since it can adjust the phase shift and amplify the received signal simultaneously. Nevertheless, a high beamforming gain often requires a number of reflecting elements, which leads to non-negligible power consumption, especially for the active RIS. Thus, one challenge is how to improve the scalability of the RIS and the energy efficiency. Different from the existing works where all reflecting elements are activated, we propose a novel element on-off mechanism where reflecting elements can be flexibly activated and deactivated. Two different optimization problems for passive RIS and active RIS are formulated by maximizing the total energy efficiency. We develop two different alternating optimization-based iterative algorithms to obtain sub-optimal solutions. Furthermore, we consider special cases involving rate maximization problems for given the same total power budget, and respectively analyze the number configuration for passive RIS and active RIS. Simulation results verify that reflecting elements under the proposed algorithms can be flexibly activated and deactivated.

cs.IT↗
arXiv2023

Phase shift and magnetic anisotropy induced field splitting of impurity states in (Li1-xFex)OHFeSe superconductor

Revealing the energy and spatial characteristics of impurity induced states in superconductors is essential for understanding their mechanism and fabricating new quantum state by manipulating impurities. Here by using high-resolution scanning tunneling microscopy/spectroscopy, we investigated the spatial distribution and magnetic field response of the impurity states in (Li1-xFex)OHFeSe. We detected two pairs of strong in-gap states on the "dumbbell" shaped defects. They display clear damped oscillations with different phase shifts and a direct phase-energy correlation. These features have long been predicted for classical Yu-Shiba-Rusinov (YSR) state, which are demonstrated here with unprecedented resolution for the first time. Moreover, upon applying magnetic field, all the in-gap state peaks remarkably split into two rather than shift, and the splitting strength is field orientation dependent. Via detailed numerical model calculations, we found such anisotropic splitting behavior can be naturally induced by a high-spin impurity coupled to anisotropic environment, highlighting how magnetic anisotropy affects the behavior of YSR states.

cond-mat.supr-con↗
arXiv2023

AttMEMO : Accelerating Transformers with Memoization on Big Memory Systems

Transformer models gain popularity because of their superior inference accuracy and inference throughput. However, the transformer is computation-intensive, causing a long inference time. The existing works on transformer inference acceleration have limitations caused by either the modification of transformer architectures or the need of specialized hardware. In this paper, we identify the opportunities of using memoization to accelerate the self-attention mechanism in transformers without the above limitations. Built upon a unique observation that there is rich similarity in attention computation across inference sequences, we build a memoization database that leverages the emerging big memory system. We introduce a novel embedding technique to find semantically similar inputs to identify computation similarity. We also introduce a series of techniques such as memory mapping and selective memoization to avoid memory copy and unnecessary overhead. We enable 22% inference-latency reduction on average (up to 68%) with negligible loss in inference accuracy.

cs.PF↗
arXiv2023

Charge order driven by multiple-Q spin fluctuations in heavily electron-doped iron selenide superconductors

Intertwined spin and charge orders have been widely studied in high-temperature superconductors, since their fluctuations may facilitate electron pairing; however, they are rarely identified in heavily electron-doped iron selenides. Here, using scanning tunneling microscopy, we show that when the superconductivity of (Li0.84Fe0.16OH)Fe1-xSe is suppressed by introducing Fe-site defects, a short-ranged checkerboard charge order emerges, propagating along the Fe-Fe directions with an approximately 2aFe period. It persists throughout the whole phase space tuned by Fe-site defect density, from a defect-pinned local pattern in optimally doped samples to an extended order in samples with lower Tc or non-superconducting. Intriguingly, our simulations indicate that the charge order is likely driven by multiple-Q spin density waves originating from the spin fluctuations observed by inelastic neutron scattering. Our study proves the presence of a competing order in heavily electron-doped iron selenides, and demonstrates the potential of charge order as a tool to detect spin fluctuations.

cond-mat.supr-con↗
arXiv2023

Fine-grained Audible Video Description

We explore a new task for audio-visual-language modeling called fine-grained audible video description (FAVD). It aims to provide detailed textual descriptions for the given audible videos, including the appearance and spatial locations of each object, the actions of moving objects, and the sounds in videos. Existing visual-language modeling tasks often concentrate on visual cues in videos while undervaluing the language and audio modalities. On the other hand, FAVD requires not only audio-visual-language modeling skills but also paragraph-level language generation abilities. We construct the first fine-grained audible video description benchmark (FAVDBench) to facilitate this research. For each video clip, we first provide a one-sentence summary of the video, ie, the caption, followed by 4-6 sentences describing the visual details and 1-2 audio-related descriptions at the end. The descriptions are provided in both English and Chinese. We create two new metrics for this task: an EntityScore to gauge the completeness of entities in the visual descriptions, and an AudioScore to assess the audio descriptions. As a preliminary approach to this task, we propose an audio-visual-language transformer that extends existing video captioning model with an additional audio branch. We combine the masked language modeling and auto-regressive language modeling losses to optimize our model so that it can produce paragraph-level descriptions. We illustrate the efficiency of our model in audio-visual-language modeling by evaluating it against the proposed benchmark using both conventional captioning metrics and our proposed metrics. We further put our benchmark to the test in video generation models, demonstrating that employing fine-grained video descriptions can create more intricate videos than using captions.

cs.CV↗
arXiv2023

DR-Label: Improving GNN Models for Catalysis Systems by Label Deconstruction and Reconstruction

Attaining the equilibrium state of a catalyst-adsorbate system is key to fundamentally assessing its effective properties, such as adsorption energy. Machine learning methods with finer supervision strategies have been applied to boost and guide the relaxation process of an atomic system and better predict its properties at the equilibrium state. In this paper, we present a novel graph neural network (GNN) supervision and prediction strategy DR-Label. The method enhances the supervision signal, reduces the multiplicity of solutions in edge representation, and encourages the model to provide node predictions that are graph structural variation robust. DR-Label first Deconstructs finer-grained equilibrium state information to the model by projecting the node-level supervision signal to each edge. Reversely, the model Reconstructs a more robust equilibrium state prediction by transforming edge-level predictions to node-level with a sphere-fitting algorithm. The DR-Label strategy was applied to three radically distinct models, each of which displayed consistent performance enhancements. Based on the DR-Label strategy, we further proposed DRFormer, which achieved a new state-of-the-art performance on the Open Catalyst 2020 (OC20) dataset and the Cu-based single-atom-alloyed CO adsorption (SAA) dataset. We expect that our work will highlight crucial steps for the development of a more accurate model in equilibrium state property prediction of a catalysis system.

cs.LG↗
arXiv2023

Ultimate precision limit of SU(2) and SU(1,1) interferometers in noisy metrology

The quantum Fisher information (QFI) in SU(2) and SU(1,1) interferometers was considered, and the QFI-only calculation was overestimated. In general, the phase estimation as a two-parameter estimation problem, and the quantum Fisher information matrix (QFIM) is necessary. In this paper, we theoretically generalize the model developed by Escher et al [Nature Physics 7, 406 (2011)] to the QFIM case with noise and study the ultimate precision limits of SU(2) and SU(1,1) interferometers with photon losses because photon losses as a very usual noise may happen to the phase measurement process. Using coherent state and squeezed vacuum state as a specific example, we numerically analyze the variation of the overestimated QFI with the loss coefficient, and find its disappearance and recovery phenomenon.

quant-ph↗
arXiv2023

Enhanced phase sensitivity in a Mach-Zehnder interferometer via photon recycling

We propose an alternative scheme for phase estimation in a Mach-Zehnder interferometer (MZI) with photon recycling. It is demonstrated that with the same coherent-state input and homodyne detection, our proposal possesses a phase sensitivity beyond the traditional MZI. For instance, it can achieve an enhancement factor of 9.32 in the phase sensitivity compared with the conventional scheme even with a photon loss of 10% on the photon-recycled arm. From another point of view, the quantum Cramer-Rao bound (QCRB) is also investigated. It is found that our scheme is able to achieve a lower QCRB than the traditional one. Intriguingly, the QCRB of our scheme is dependent of the phase shift phi while the traditional scheme has a constant QCRB regardless of the phase shift. Finally, we present the underlying mechanisms behind the enhanced phase sensitivity. We believe that our results provide another angle from which to enhance the phase sensitivity in a MZI via photon recycling.

quant-ph↗
arXiv2023

Multi-Prototypes Convex Merging Based K-Means Clustering Algorithm

K-Means algorithm is a popular clustering method. However, it has two limitations: 1) it gets stuck easily in spurious local minima, and 2) the number of clusters k has to be given a priori. To solve these two issues, a multi-prototypes convex merging based K-Means clustering algorithm (MCKM) is presented. First, based on the structure of the spurious local minima of the K-Means problem, a multi-prototypes sampling (MPS) is designed to select the appropriate number of multi-prototypes for data with arbitrary shapes. A theoretical proof is given to guarantee that the multi-prototypes selected by MPS can achieve a constant factor approximation to the optimal cost of the K-Means problem. Then, a merging technique, called convex merging (CM), merges the multi-prototypes to get a better local minima without k being given a priori. Specifically, CM can obtain the optimal merging and estimate the correct k. By integrating these two techniques with K-Means algorithm, the proposed MCKM is an efficient and explainable clustering algorithm for escaping the undesirable local minima of K-Means problem without given k first. Experimental results performed on synthetic and real-world data sets have verified the effectiveness of the proposed algorithm.

cs.LG↗
arXiv2023

Accelerated Fuzzy C-Means Clustering Based on New Affinity Filtering and Membership Scaling

Fuzzy C-Means (FCM) is a widely used clustering method. However, FCM and its many accelerated variants have low efficiency in the mid-to-late stage of the clustering process. In this stage, all samples are involved in the update of their non-affinity centers, and the fuzzy membership grades of the most of samples, whose assignment is unchanged, are still updated by calculating the samples-centers distances. All those lead to the algorithms converging slowly. In this paper, a new affinity filtering technique is developed to recognize a complete set of the non-affinity centers for each sample with low computations. Then, a new membership scaling technique is suggested to set the membership grades between each sample and its non-affinity centers to 0 and maintain the fuzzy membership grades for others. By integrating those two techniques, FCM based on new affinity filtering and membership scaling (AMFCM) is proposed to accelerate the whole convergence process of FCM. Many experimental results performed on synthetic and real-world data sets have shown the feasibility and efficiency of the proposed algorithm. Compared with the state-of-the-art algorithms, AMFCM is significantly faster and more effective. For example, AMFCM reduces the number of the iteration of FCM by 80% on average.

cs.LG↗
arXiv2023

Scalable Deep Graph Clustering with Random-walk based Self-supervised Learning

Web-based interactions can be frequently represented by an attributed graph, and node clustering in such graphs has received much attention lately. Multiple efforts have successfully applied Graph Convolutional Networks (GCN), though with some limits on accuracy as GCNs have been shown to suffer from over-smoothing issues. Though other methods (particularly those based on Laplacian Smoothing) have reported better accuracy, a fundamental limitation of all the work is a lack of scalability. This paper addresses this open problem by relating the Laplacian smoothing to the Generalized PageRank and applying a random-walk based algorithm as a scalable graph filter. This forms the basis for our scalable deep clustering algorithm, RwSL, where through a self-supervised mini-batch training mechanism, we simultaneously optimize a deep neural network for sample-cluster assignment distribution and an autoencoder for a clustering-oriented embedding. Using 6 real-world datasets and 6 clustering metrics, we show that RwSL achieved improved results over several recent baselines. Most notably, we show that RwSL, unlike all other deep clustering frameworks, can continue to scale beyond graphs with more than one million nodes, i.e., handle web-scale. We also demonstrate how RwSL could perform node clustering on a graph with 1.8 billion edges using only a single GPU.

cs.LG↗
arXiv2023

Transformer in Transformer as Backbone for Deep Reinforcement Learning

Designing better deep networks and better reinforcement learning (RL) algorithms are both important for deep RL. This work focuses on the former. Previous methods build the network with several modules like CNN, LSTM and Attention. Recent methods combine the Transformer with these modules for better performance. However, it requires tedious optimization skills to train a network composed of mixed modules, making these methods inconvenient to be used in practice. In this paper, we propose to design \emph{pure Transformer-based networks} for deep RL, aiming at providing off-the-shelf backbones for both the online and offline settings. Specifically, the Transformer in Transformer (TIT) backbone is proposed, which cascades two Transformers in a very natural way: the inner one is used to process a single observation, while the outer one is responsible for processing the observation history; combining both is expected to extract spatial-temporal representations for good decision-making. Experiments show that TIT can achieve satisfactory performance in different settings consistently.

cs.LG↗
arXiv2022

Generation-Augmented Query Expansion For Code Retrieval

Pre-trained language models have achieved promising success in code retrieval tasks, where a natural language documentation query is given to find the most relevant existing code snippet. However, existing models focus only on optimizing the documentation code pairs by embedding them into latent space, without the association of external knowledge. In this paper, we propose a generation-augmented query expansion framework. Inspired by the human retrieval process - sketching an answer before searching, in this work, we utilize the powerful code generation model to benefit the code retrieval task. Specifically, we demonstrate that rather than merely retrieving the target code snippet according to the documentation query, it would be helpful to augment the documentation query with its generation counterpart - generated code snippets from the code generation model. To the best of our knowledge, this is the first attempt that leverages the code generation model to enhance the code retrieval task. We achieve new state-of-the-art results on the CodeSearchNet benchmark and surpass the baselines significantly.

cs.SE↗