arXiv subjects

Xingwei Wang

Publications and source records attributed to Xingwei Wang.

At least 37 records · Page 2Linked to original sources

arXiv2024

Physical Layer Security in AmBC-NOMA Networks with Random Eavesdroppers

In this work, we investigate the physical layer security (PLS) of ambient backscatter communication non-orthogonal multiple access (AmBC-NOMA) networks where non-colluding eavesdroppers (Eves) are randomly distributed. In the proposed system, a base station (BS) transmits a superimposed signal to a typical NOMA user pair, while a backscatter device~(BD) simultaneously transmits its unique signal by reflecting and modulating the BS's signal. Meanwhile, Eves passively attempt to wiretap the ongoing transmissions. Notably, the number and locations of Eves are unknown, posing a substantial security threat to the system. To address this challenge, the BS injects artificial noise (AN) to mislead the Eves, and a protected zone is employed to create an Eve-exclusion area around the BS. Theoretical expressions for outage probability (OP) and intercept probability (IP) are provided to evaluate the system's reliability-security trade-off. Asymptotic behavior at high signal-to-noise ratio (SNR) is further explored, including the derivation of diversity orders for the OP. Numerical results validate the analytical findings through extensive simulations, demonstrating that both the AN injection and protected zone can effectively enhance PLS. Furthermore, analysis and insights of different key parameters, including transmit SNR, reflection efficiency at the BD, power allocation coefficient, power fraction allocated to desired signal, Eve-exclusion area radius, Eve distribution density, and backscattered AN cancellation efficiency, on OP and IP are also provided.

eess.SY↗

arXiv2024

Self-supervised Hierarchical Representation for Medication Recommendation

Medication recommender is to suggest appropriate medication combinations based on a patient's health history, e.g., diagnoses and procedures. Existing works represent different diagnoses/procedures well separated by one-hot encodings. However, they ignore the latent hierarchical structures of these medical terms, undermining the generalization performance of the model. For example, "Respiratory Diseases", "Chronic Respiratory Diseases" and "Chronic Bronchiti" have a hierarchical relationship, progressing from general to specific. To address this issue, we propose a novel hierarchical encoder named HIER to hierarchically represent diagnoses and procedures, which is based on standard medical codes and compatible with any existing methods. Specifically, the proposed method learns relation embedding with a self-supervised objective for incorporating the neighbor hierarchical structure. Additionally, we develop the position encoding to explicitly introduce global hierarchical position. Extensive experiments demonstrate significant and consistent improvements in recommendation accuracy across four baselines and two real-world clinical datasets.

cs.IR↗

arXiv2024

CoRA: Collaborative Information Perception by Large Language Model's Weights for Recommendation

Involving collaborative information in Large Language Models (LLMs) is a promising technique for adapting LLMs for recommendation. Existing methods achieve this by concatenating collaborative features with text tokens into a unified sequence input and then fine-tuning to align these features with LLM's input space. Although effective, in this work, we identify two limitations when adapting LLMs to recommendation tasks, which hinder the integration of general knowledge and collaborative information, resulting in sub-optimal recommendation performance. (1) Fine-tuning LLM with recommendation data can undermine its inherent world knowledge and fundamental competencies, which are crucial for interpreting and inferring recommendation text. (2) Incorporating collaborative features into textual prompts disrupts the semantics of the original prompts, preventing LLM from generating appropriate outputs. In this paper, we propose a new paradigm, \textbf{Co}llaborative \textbf{Lo}RA (CoRA), with a collaborative query generator. Rather than input space alignment, this method aligns collaborative information with LLM's parameter space, representing them as incremental weights to update LLM's output. This way, LLM perceives collaborative information without altering its general knowledge and text inference capabilities. Specifically, we employ a collaborative filtering model to extract user and item embeddings and inject them into a set number of learnable queries. We then convert collaborative queries into collaborative weights with low-rank properties and merge the collaborative weights into LLM's weights, enabling LLM to perceive the collaborative signals and generate personalized recommendations without fine-tuning or extra collaborative tokens in prompts. Extensive experiments confirm that CoRA effectively integrates collaborative information into LLM, enhancing recommendation performance.

cs.IR↗

arXiv2024

Double-Side Delay Alignment Modulation for Multi-User Millimeter Wave and TeraHertz Communications

Delay alignment modulation (DAM) is an innovative broadband modulation technique well suited for millimeter wave (mmWave) and terahertz (THz) massive multiple-input multiple-output (MIMO) communication systems. Leveraging the high spatial resolution and sparsity of multi-path channels, DAM mitigates inter-symbol interference (ISI) effectively, by aligning all multi-path components through a combination of delay pre/post-compensation and path-based beamforming. As such, ISI is eliminated while preserving multi-path power gains. In this paper, we explore multi-user double-side DAM with both delay pre-compensation at the transmitter and post-compensation at the receiver, contrasting with prior one-side DAM that primarily focuses on delay pre-compensation only. Firstly, we reveal the constraint for the introduced delays and the delay pre/post-compensation vectors tailored for multi-user double-side DAM, given a specific number of delay pre/post-compensations. Furthermore, we show that as long as the number of base station (BS)/user equipment (UE) antennas is sufficiently large, single-side DAM, where delay compensation is only performed at the BS/UE, is preferred than double-side DAM since the former results in less ISI to be spatially eliminated. Next, we propose two low-complexity path-based beamforming strategies based on the eigen-beamforming transmission and ISI-zero forcing (ZF) principles, respectively, based on which the achievable sum rates are studied. Simulation results verify that with sufficiently large BS/UE antennas, single-side DAM is sufficient. Furthermore, compared to the benchmark scheme of orthogonal frequency division multiplexing (OFDM), multi-user BS-side DAM achieves higher spectral efficiency and/or lower peak-to-average power ratio (PAPR).

eess.SP↗

arXiv2024

SurgeryV2: Bridging the Gap Between Model Merging and Multi-Task Learning with Deep Representation Surgery

Model merging-based multitask learning (MTL) offers a promising approach for performing MTL by merging multiple expert models without requiring access to raw training data. However, in this paper, we examine the merged model's representation distribution and uncover a critical issue of "representation bias". This bias arises from a significant distribution gap between the representations of the merged and expert models, leading to the suboptimal performance of the merged MTL model. To address this challenge, we first propose a representation surgery solution called Surgery. Surgery is a lightweight, task-specific module that aligns the final layer representations of the merged model with those of the expert models, effectively alleviating bias and improving the merged model's performance. Despite these improvements, a performance gap remains compared to the traditional MTL method. Further analysis reveals that representation bias phenomena exist at each layer of the merged model, and aligning representations only in the last layer is insufficient for fully reducing systemic bias because biases introduced at each layer can accumulate and interact in complex ways. To tackle this, we then propose a more comprehensive solution, deep representation surgery (also called SurgeryV2), which mitigates representation bias across all layers, and thus bridges the performance gap between model merging-based MTL and traditional MTL. Finally, we design an unsupervised optimization objective to optimize both the Surgery and SurgeryV2 modules. Our experimental results show that incorporating these modules into state-of-the-art (SOTA) model merging schemes leads to significant performance gains. Notably, our SurgeryV2 scheme reaches almost the same level as individual expert models or the traditional MTL model. The code is available at \url{https://github.com/EnnengYang/SurgeryV2}.

cs.LG↗

arXiv2024

Data Augmentation for Sequential Recommendation: A Survey

As an essential branch of recommender systems, sequential recommendation (SR) has received much attention due to its well-consistency with real-world situations. However, the widespread data sparsity issue limits the SR model's performance. Therefore, researchers have proposed many data augmentation (DA) methods to mitigate this phenomenon and have achieved impressive progress. In this survey, we provide a comprehensive review of DA methods for SR. We start by introducing the research background and motivation. Then, we categorize existing methodologies regarding their augmentation principles, objects, and purposes. Next, we present a comparative discussion of their advantages and disadvantages, followed by the exhibition and analysis of representative experimental results. Finally, we outline directions for future research and summarize this survey. We also maintain a repository with a paper list at \url{https://github.com/KingGugu/DA-CL-4Rec}.

cs.IR↗

arXiv2024

Continual Learning From a Stream of APIs

Continual learning (CL) aims to learn new tasks without forgetting previous tasks. However, existing CL methods require a large amount of raw data, which is often unavailable due to copyright considerations and privacy risks. Instead, stakeholders usually release pre-trained machine learning models as a service (MLaaS), which users can access via APIs. This paper considers two practical-yet-novel CL settings: data-efficient CL (DECL-APIs) and data-free CL (DFCL-APIs), which achieve CL from a stream of APIs with partial or no raw data. Performing CL under these two new settings faces several challenges: unavailable full raw data, unknown model parameters, heterogeneous models of arbitrary architecture and scale, and catastrophic forgetting of previous APIs. To overcome these issues, we propose a novel data-free cooperative continual distillation learning framework that distills knowledge from a stream of APIs into a CL model by generating pseudo data, just by querying APIs. Specifically, our framework includes two cooperative generators and one CL model, forming their training as an adversarial game. We first use the CL model and the current API as fixed discriminators to train generators via a derivative-free method. Generators adversarially generate hard and diverse synthetic data to maximize the response gap between the CL model and the API. Next, we train the CL model by minimizing the gap between the responses of the CL model and the black-box API on synthetic data, to transfer the API's knowledge to the CL model. Furthermore, we propose a new regularization term based on network similarity to prevent catastrophic forgetting of previous APIs.Our method performs comparably to classic CL with full raw data on the MNIST and SVHN in the DFCL-APIs setting. In the DECL-APIs setting, our method achieves 0.97x, 0.75x and 0.69x performance of classic CL on CIFAR10, CIFAR100, and MiniImageNet.

cs.LG↗

arXiv2024

Symmetric Graph Contrastive Learning against Noisy Views for Recommendation

Graph Contrastive Learning (GCL) leverages data augmentation techniques to produce contrasting views, enhancing the accuracy of recommendation systems through learning the consistency between contrastive views. However, existing augmentation methods, such as directly perturbing interaction graph (e.g., node/edge dropout), may interfere with the original connections and generate poor contrasting views, resulting in sub-optimal performance. In this paper, we define the views that share only a small amount of information with the original graph due to poor data augmentation as noisy views (i.e., the last 20% of the views with a cosine similarity value less than 0.1 to the original view). We demonstrate through detailed experiments that noisy views will significantly degrade recommendation performance. Further, we propose a model-agnostic Symmetric Graph Contrastive Learning (SGCL) method with theoretical guarantees to address this issue. Specifically, we introduce symmetry theory into graph contrastive learning, based on which we propose a symmetric form and contrast loss resistant to noisy interference. We provide theoretical proof that our proposed SGCL method has a high tolerance to noisy views. Further demonstration is given by conducting extensive experiments on three real-world datasets. The experimental results demonstrate that our approach substantially increases recommendation accuracy, with relative improvements reaching as high as 12.25% over nine other competing models. These results highlight the efficacy of our method.

cs.LG↗

arXiv2024

AdaMerging: Adaptive Model Merging for Multi-Task Learning

Multi-task learning (MTL) aims to empower a model to tackle multiple tasks simultaneously. A recent development known as task arithmetic has revealed that several models, each fine-tuned for distinct tasks, can be directly merged into a single model to execute MTL without necessitating a retraining process using the initial training data. Nevertheless, this direct addition of models often leads to a significant deterioration in the overall performance of the merged model. This decline occurs due to potential conflicts and intricate correlations among the multiple tasks. Consequently, the challenge emerges of how to merge pre-trained models more effectively without using their original training data. This paper introduces an innovative technique called Adaptive Model Merging (AdaMerging). This approach aims to autonomously learn the coefficients for model merging, either in a task-wise or layer-wise manner, without relying on the original training data. Specifically, our AdaMerging method operates as an automatic, unsupervised task arithmetic scheme. It leverages entropy minimization on unlabeled test samples from the multi-task setup as a surrogate objective function to iteratively refine the merging coefficients of the multiple models. Our experimental findings across eight tasks demonstrate the efficacy of the AdaMerging scheme we put forth. Compared to the current state-of-the-art task arithmetic merging scheme, AdaMerging showcases a remarkable 11\% improvement in performance. Notably, AdaMerging also exhibits superior generalization capabilities when applied to unseen downstream tasks. Furthermore, it displays a significantly enhanced robustness to data distribution shifts that may occur during the testing phase.

cs.LG↗

arXiv2024

Representation Surgery for Multi-Task Model Merging

Multi-task learning (MTL) compresses the information from multiple tasks into a unified backbone to improve computational efficiency and generalization. Recent work directly merges multiple independently trained models to perform MTL instead of collecting their raw data for joint training, greatly expanding the application scenarios of MTL. However, by visualizing the representation distribution of existing model merging schemes, we find that the merged model often suffers from the dilemma of representation bias. That is, there is a significant discrepancy in the representation distribution between the merged and individual models, resulting in poor performance of merged MTL. In this paper, we propose a representation surgery solution called "Surgery" to reduce representation bias in the merged model. Specifically, Surgery is a lightweight task-specific module that takes the representation of the merged model as input and attempts to output the biases contained in the representation from the merged model. We then designed an unsupervised optimization objective that updates the Surgery module by minimizing the distance between the merged model's representation and the individual model's representation. Extensive experiments demonstrate significant MTL performance improvements when our Surgery module is applied to state-of-the-art (SOTA) model merging schemes.

cs.LG↗

arXiv2024

AmBC-NOMA-Aided Short-Packet Communication for High Mobility V2X Transmissions

In this paper, we investigate the performance of ambient backscatter communication non-orthogonal multiple access (AmBC-NOMA)-assisted short packet communication for high-mobility vehicle-to-everything transmissions. In the proposed system, a roadside unit (RSU) transmits a superimposed signal to a typical NOMA user pair. Simultaneously, the backscatter device (BD) transmits its own signal towards the user pair by reflecting and modulating the RSU's superimposed signals. Due to vehicles' mobility, we consider realistic assumptions of time-selective fading and channel estimation errors. Theoretical expressions for the average block error rates (BLERs) of both users are derived. Furthermore, analysis and insights on transmit signal-to-noise ratio, vehicles' mobility, imperfect channel estimation, the reflection efficiency at the BD, and blocklength are provided. Numerical results validate the theoretical findings and reveal that the AmBC-NOMA system outperforms its orthogonal multiple access counterpart in terms of BLER performance.

cs.PF↗

arXiv2024

ID Embedding as Subtle Features of Content and Structure for Multimodal Recommendation

Multimodal recommendation aims to model user and item representations comprehensively with the involvement of multimedia content for effective recommendations. Existing research has shown that it is beneficial for recommendation performance to combine (user- and item-) ID embeddings with multimodal salient features, indicating the value of IDs. However, there is a lack of a thorough analysis of the ID embeddings in terms of feature semantics in the literature. In this paper, we revisit the value of ID embeddings for multimodal recommendation and conduct a thorough study regarding its semantics, which we recognize as subtle features of \emph{content} and \emph{structure}. Based on our findings, we propose a novel recommendation model by incorporating ID embeddings to enhance the salient features of both content and structure. Specifically, we put forward a hierarchical attention mechanism to incorporate ID embeddings in modality fusing, coupled with contrastive learning, to enhance content representations. Meanwhile, we propose a lightweight graph convolution network for each modality to amalgamate neighborhood and ID embeddings for improving structural representations. Finally, the content and structure representations are combined to form the ultimate item embedding for recommendation. Extensive experiments on three real-world datasets (Baby, Sports, and Clothing) demonstrate the superiority of our method over state-of-the-art multimodal recommendation methods and the effectiveness of fine-grained ID embeddings. Our code is available at https://anonymous.4open.science/r/IDSF-code/.

cs.IR↗

arXiv2024

PreConfig: A Pretrained Model for Automating Network Configuration

Manual network configuration automation (NCA) tools face significant challenges in versatility and flexibility due to their reliance on extensive domain expertise and manual design, limiting their adaptability to diverse scenarios and complex application needs. This paper introduces PreConfig, an innovative NCA tool that leverages a pretrained language model for automating network configuration tasks. PreConfig is designed to address the complexity and variety of NCA tasks by framing them as text-to-text transformation problems, thus unifying the tasks of configuration generation, translation, and analysis under a single, versatile model. Our approach overcomes existing tools' limitations by utilizing advances in natural language processing to automatically comprehend and generate network configurations without extensive manual re-engineering. We confront the challenges of integrating domain-specific knowledge into pretrained models and the scarcity of supervision data in the network configuration field. Our solution involves constructing a specialized corpus and further pretraining on network configuration data, coupled with a novel data mining technique for generating task supervision data. The proposed model demonstrates robustness in configuration generation, translation, and analysis, outperforming conventional tools in handling complex networking environments. The experimental results validate the effectiveness of PreConfig, establishing a new direction for automating network configuration tasks with pretrained language models.

cs.NI↗

arXiv2024

Towards Unified Modeling for Positive and Negative Preferences in Sign-Aware Recommendation

Recently, sign-aware graph recommendation has drawn much attention as it will learn users' negative preferences besides positive ones from both positive and negative interactions (i.e., links in a graph) with items. To accommodate the different semantics of negative and positive links, existing works utilize two independent encoders to model users' positive and negative preferences, respectively. However, these approaches cannot learn the negative preferences from high-order heterogeneous interactions between users and items formed by multiple links with different signs, resulting in inaccurate and incomplete negative user preferences. To cope with these intractable issues, we propose a novel \textbf{L}ight \textbf{S}igned \textbf{G}raph Convolution Network specifically for \textbf{Rec}ommendation (\textbf{LSGRec}), which adopts a unified modeling approach to simultaneously model high-order users' positive and negative preferences on a signed user-item interaction graph. Specifically, for the negative preferences within high-order heterogeneous interactions, first-order negative preferences are captured by the negative links, while high-order negative preferences are propagated along positive edges. Then, recommendation results are generated based on positive preferences and optimized with negative ones. Finally, we train representations of users and items through different auxiliary tasks. Extensive experiments on three real-world datasets demonstrate that our method outperforms existing baselines regarding performance and computational efficiency. Our code is available at \url{https://anonymous.4open.science/r/LSGRec-BB95}.

cs.IR↗

arXiv2024

Pareto-wise Ranking Classifier for Multi-objective Evolutionary Neural Architecture Search

In the deployment of deep neural models, how to effectively and automatically find feasible deep models under diverse design objectives is fundamental. Most existing neural architecture search (NAS) methods utilize surrogates to predict the detailed performance (e.g., accuracy and model size) of a candidate architecture during the search, which however is complicated and inefficient. In contrast, we aim to learn an efficient Pareto classifier to simplify the search process of NAS by transforming the complex multi-objective NAS task into a simple Pareto-dominance classification task. To this end, we propose a classification-wise Pareto evolution approach for one-shot NAS, where an online classifier is trained to predict the dominance relationship between the candidate and constructed reference architectures, instead of using surrogates to fit the objective functions. The main contribution of this study is to change supernet adaption into a Pareto classifier. Besides, we design two adaptive schemes to select the reference set of architectures for constructing classification boundary and regulate the rate of positive samples over negative ones, respectively. We compare the proposed evolution approach with state-of-the-art approaches on widely-used benchmark datasets, and experimental results indicate that the proposed approach outperforms other approaches and have found a number of neural architectures with different model sizes ranging from 2M to 6M under diverse objectives and constraints.

cs.LG↗

arXiv2024

Federated Domain Generalization: A Survey

Machine learning typically relies on the assumption that training and testing distributions are identical and that data is centrally stored for training and testing. However, in real-world scenarios, distributions may differ significantly and data is often distributed across different devices, organizations, or edge nodes. Consequently, it is imperative to develop models that can effectively generalize to unseen distributions where data is distributed across different domains. In response to this challenge, there has been a surge of interest in federated domain generalization (FDG) in recent years. FDG combines the strengths of federated learning (FL) and domain generalization (DG) techniques to enable multiple source domains to collaboratively learn a model capable of directly generalizing to unseen domains while preserving data privacy. However, generalizing the federated model under domain shifts is a technically challenging problem that has received scant attention in the research area so far. This paper presents the first survey of recent advances in this area. Initially, we discuss the development process from traditional machine learning to domain adaptation and domain generalization, leading to FDG as well as provide the corresponding formal definition. Then, we categorize recent methodologies into four classes: federated domain alignment, data manipulation, learning strategies, and aggregation optimization, and present suitable algorithms in detail for each category. Next, we introduce commonly used datasets, applications, evaluations, and benchmarks. Finally, we conclude this survey by providing some potential research topics for the future.

cs.LG↗

arXiv2023

Uniform Sequence Better: Time Interval Aware Data Augmentation for Sequential Recommendation

Sequential recommendation is an important task to predict the next-item to access based on a sequence of interacted items. Most existing works learn user preference as the transition pattern from the previous item to the next one, ignoring the time interval between these two items. However, we observe that the time interval in a sequence may vary significantly different, and thus result in the ineffectiveness of user modeling due to the issue of \emph{preference drift}. In fact, we conducted an empirical study to validate this observation, and found that a sequence with uniformly distributed time interval (denoted as uniform sequence) is more beneficial for performance improvement than that with greatly varying time interval. Therefore, we propose to augment sequence data from the perspective of time interval, which is not studied in the literature. Specifically, we design five operators (Ti-Crop, Ti-Reorder, Ti-Mask, Ti-Substitute, Ti-Insert) to transform the original non-uniform sequence to uniform sequence with the consideration of variance of time intervals. Then, we devise a control strategy to execute data augmentation on item sequences in different lengths. Finally, we implement these improvements on a state-of-the-art model CoSeRec and validate our approach on four real datasets. The experimental results show that our approach reaches significantly better performance than the other 11 competing methods. Our implementation is available: https://github.com/KingGugu/TiCoSeRec.

cs.IR↗

arXiv2023

Achievable Rate Region and Path-Based Beamforming for Multi-User Single-Carrier Delay Alignment Modulation

Delay alignment modulation (DAM) is a novel wideband transmission technique for mmWave massive MIMO systems, which exploits the high spatial resolution and multi-path sparsity to mitigate ISI, without relying on channel equalization or multi-carrier transmission. In particular, DAM leverages the delay pre-compensation and path-based beamforming to effectively align the multi-path components, thus achieving the constructive multi-path combination for eliminating the ISI while preserving the multi-path power gain. Different from the existing works only considering single-user DAM, this paper investigates the DAM technique for multi-user mmWave massive MIMO communication. First, we consider the asymptotic regime when the number of antennas Mt at BS is sufficiently large. It is shown that by employing the simple delay pre-compensation and per-path-based MRT beamforming, the single-carrier DAM is able to perfectly eliminate both ISI and IUI. Next, we consider the general scenario with Mt being finite. In this scenario, we characterize the achievable rate region of the multi-user DAM system by finding its Pareto boundary. Specifically, we formulate a rate-profile-constrained sum rate maximization problem by optimizing the per-path-based beamforming. Furthermore, we present three low-complexity per-path-based beamforming strategies based on the MRT, zero-forcing, and regularized zero-forcing principles, respectively, based on which the achievable sum rates are studied. Finally, we provide simulation results to demonstrate the performance of our proposed strategies as compared to two benchmark schemes based on the strongest-path-based beamforming and the prevalent OFDM, respectively. It is shown that DAM achieves higher spectral efficiency and/or lower peak-to-average-ratio, for systems with high spatial resolution and multi-path diversity.

cs.IT↗