arXiv subjects

Jie Zhang

Publications and source records attributed to Jie Zhang.

At least 721 records · Page 40Linked to original sources

arXiv2023

Retraining A Graph-based Recommender with Interests Disentanglement

In a practical recommender system, new interactions are continuously observed. Some interactions are expected, because they largely follow users' long-term preferences. Some other interactions are indications of recent trends in user preference changes or marketing positions of new items. Accordingly, the recommender needs to be periodically retrained or updated to capture the new trends, and yet not to forget the long-term preferences. In this paper, we propose a novel and generic retraining framework called Disentangled Incremental Learning (DIL) for graph-based recommenders. We assume that long-term preferences are well captured in the existing model, in the form of model parameters learned from past interactions. New preferences can be learned from the user-item bipartite graph constructed using the newly observed interactions. In DIL, we design an Information Extraction Module to extract historical preferences from the existing model. Then we blend the historical and new preferences in the form of node embeddings in the new graph, through a Disentanglement Module. The essence of the disentanglement module is to decorrelate the historical and new preferences so that both can be well captured, via carefully designed losses. Through experiments on three benchmark datasets, we show the effectiveness of DIL in capturing dynamics of useritem interactions. We also demonstrate the robustness of DIL by attaching it to two base models - LightGCN and NGCF.

cs.IR↗

arXiv2023

Towards Unbiased Training in Federated Open-world Semi-supervised Learning

Federated Semi-supervised Learning (FedSSL) has emerged as a new paradigm for allowing distributed clients to collaboratively train a machine learning model over scarce labeled data and abundant unlabeled data. However, existing works for FedSSL rely on a closed-world assumption that all local training data and global testing data are from seen classes observed in the labeled dataset. It is crucial to go one step further: adapting FL models to an open-world setting, where unseen classes exist in the unlabeled data. In this paper, we propose a novel Federatedopen-world Semi-Supervised Learning (FedoSSL) framework, which can solve the key challenge in distributed and open-world settings, i.e., the biased training process for heterogeneously distributed unseen classes. Specifically, since the advent of a certain unseen class depends on a client basis, the locally unseen classes (exist in multiple clients) are likely to receive differentiated superior aggregation effects than the globally unseen classes (exist only in one client). We adopt an uncertainty-aware suppressed loss to alleviate the biased training between locally unseen and globally unseen classes. Besides, we enable a calibration module supplementary to the global aggregation to avoid potential conflicting knowledge transfer caused by inconsistent data distribution among different clients. The proposed FedoSSL can be easily adapted to state-of-the-art FL methods, which is also validated via extensive experiments on benchmarks and real-world datasets (CIFAR-10, CIFAR-100 and CINIC-10).

cs.LG↗

arXiv2023

SelfCF: A Simple Framework for Self-supervised Collaborative Filtering

Collaborative filtering (CF) is widely used to learn informative latent representations of users and items from observed interactions. Existing CF-based methods commonly adopt negative sampling to discriminate different items. Training with negative sampling on large datasets is computationally expensive. Further, negative items should be carefully sampled under the defined distribution, in order to avoid selecting an observed positive item in the training dataset. Unavoidably, some negative items sampled from the training dataset could be positive in the test set. In this paper, we propose a self-supervised collaborative filtering framework (SelfCF), that is specially designed for recommender scenario with implicit feedback. The proposed SelfCF framework simplifies the Siamese networks and can be easily applied to existing deep-learning based CF models, which we refer to as backbone networks. The main idea of SelfCF is to augment the output embeddings generated by backbone networks, because it is infeasible to augment raw input of user/item ids. We propose and study three output perturbation techniques that can be applied to different types of backbone networks including both traditional CF models and graph-based models. The framework enables learning informative representations of users and items without negative samples, and is agnostic to the encapsulated backbones. We conduct comprehensive experiments on four datasets to show that our framework may achieve even better recommendation accuracy than the encapsulated supervised counterpart with a 2$\times$--4$\times$ faster training speed. We also show that SelfCF can boost up the accuracy by up to 17.79% on average, compared with a self-supervised framework BUIR.

cs.IR↗

arXiv2023

Towards Fairer and More Efficient Federated Learning via Multidimensional Personalized Edge Models

Federated learning (FL) is an emerging technique that trains massive and geographically distributed edge data while maintaining privacy. However, FL has inherent challenges in terms of fairness and computational efficiency due to the rising heterogeneity of edges, and thus usually results in sub-optimal performance in recent state-of-the-art (SOTA) solutions. In this paper, we propose a Customized Federated Learning (CFL) system to eliminate FL heterogeneity from multiple dimensions. Specifically, CFL tailors personalized models from the specially designed global model for each client jointly guided by an online trained model-search helper and a novel aggregation algorithm. Extensive experiments demonstrate that CFL has full-stack advantages for both FL training and edge reasoning and significantly improves the SOTA performance w.r.t. model accuracy (up to 7.2% in the non-heterogeneous environment and up to 21.8% in the heterogeneous environment), efficiency, and FL fairness.

cs.LG↗

arXiv2023

Fed-FSNet: Mitigating Non-I.I.D. Federated Learning via Fuzzy Synthesizing Network

Federated learning (FL) has emerged as a promising privacy-preserving distributed machine learning framework recently. It aims at collaboratively learning a shared global model by performing distributed training locally on edge devices and aggregating local models into a global one without centralized raw data sharing in the cloud server. However, due to the large local data heterogeneities (Non-I.I.D. data) across edge devices, the FL may easily obtain a global model that can produce more shifted gradients on local datasets, thereby degrading the model performance or even suffering from the non-convergence during training. In this paper, we propose a novel FL training framework, dubbed Fed-FSNet, using a properly designed Fuzzy Synthesizing Network (FSNet) to mitigate the Non-I.I.D. FL at-the-source. Concretely, we maintain an edge-agnostic hidden model in the cloud server to estimate a less-accurate while direction-aware inversion of the global model. The hidden model can then fuzzily synthesize several mimic I.I.D. data samples (sample features) conditioned on only the global model, which can be shared by edge devices to facilitate the FL training towards faster and better convergence. Moreover, since the synthesizing process involves neither access to the parameters/updates of local models nor analyzing individual local model outputs, our framework can still ensure the privacy of FL. Experimental results on several FL benchmarks demonstrate that our method can significantly mitigate the Non-I.I.D. issue and obtain better performance against other representative methods.

cs.CR↗

arXiv2023

CCLAP: Controllable Chinese Landscape Painting Generation via Latent Diffusion Model

With the development of deep generative models, recent years have seen great success of Chinese landscape painting generation. However, few works focus on controllable Chinese landscape painting generation due to the lack of data and limited modeling capabilities. In this work, we propose a controllable Chinese landscape painting generation method named CCLAP, which can generate painting with specific content and style based on Latent Diffusion Model. Specifically, it consists of two cascaded modules, i.e., content generator and style aggregator. The content generator module guarantees the content of generated paintings specific to the input text. While the style aggregator module is to generate paintings of a style corresponding to a reference image. Moreover, a new dataset of Chinese landscape paintings named CLAP is collected for comprehensive evaluation. Both the qualitative and quantitative results demonstrate that our method achieves state-of-the-art performance, especially in artfully-composed and artistic conception. Codes are available at https://github.com/Robin-WZQ/CCLAP.

cs.CV↗

arXiv2023

Analytic Network Traffic Prediction Based on User Behavior Modeling

This paper proposes an interpretable user-behavior-based (UBB) network traffic prediction (NTP) method. Based on user behavior, a weekly traffic demand profile can be naturally sorted into three categories, i.e., weekday, Saturday, and Sunday. For each category, the traffic pattern is divided into three components which are mainly generated in three time periods, i.e., morning, afternoon, and evening. Each component is modeled as a normal-distributed signal. Numerical results indicate the UBB NTP method matches the practical wireless traffic demand very well. Compared with existing methods, the proposed UBB NTP method improves the computational efficiency and increases the predictive accuracy.

cs.NI↗

arXiv2023

Prophet: Conflict-Free Sharding Blockchain via Byzantine-Tolerant Deterministic Ordering

Sharding scales throughput by splitting blockchain nodes into parallel groups. However, different shards' independent and random scheduling for cross-shard transactions results in numerous conflicts and aborts, since cross-shard transactions from different shards may access the same account. A deterministic ordering can eliminate conflicts by determining a global order for transactions before processing, as proved in the database field. Unfortunately, due to the intertwining of the Byzantine environment and information isolation among shards, there is no trusted party able to predetermine such an order for cross-shard transactions. To tackle this challenge, this paper proposes Prophet, a conflict-free sharding blockchain based on Byzantine-tolerant deterministic ordering. It first depends on untrusted self-organizing coalitions of nodes from different shards to pre-execute cross-shard transactions for prerequisite information about ordering. It then determines a trusted global order based on stateless ordering and post-verification for pre-executed results, through shard cooperation. Following the order, the shards thus orderly execute and commit transactions without conflicts. Prophet orchestrates the pre-execution, ordering, and execution processes in the sharding consensus for minimal overhead. We rigorously prove the determinism and serializability of transactions under the Byzantine and sharded environment. An evaluation of our prototype shows that Prophet improves the throughput by $3.11\times$ and achieves nearly no aborts on 1 million Ethereum transactions compared with state-of-the-art sharding.

cs.CR↗

arXiv2023

Accelerating Dataset Distillation via Model Augmentation

Dataset Distillation (DD), a newly emerging field, aims at generating much smaller but efficient synthetic training datasets from large ones. Existing DD methods based on gradient matching achieve leading performance; however, they are extremely computationally intensive as they require continuously optimizing a dataset among thousands of randomly initialized models. In this paper, we assume that training the synthetic data with diverse models leads to better generalization performance. Thus we propose two model augmentation techniques, i.e. using early-stage models and parameter perturbation to learn an informative synthetic set with significantly reduced training cost. Extensive experiments demonstrate that our method achieves up to 20x speedup and comparable performance on par with state-of-the-art methods.

cs.LG↗

arXiv2023

An Efficient Deep Learning Image Condition for Locating Earthquakes

Migration-based earthquake location methods may encounter the polarity reversal issue due to the non-explosive components of seismic source, leading to an unfocused migration image. Various methods have been proposed, yet producing an ideally focused migration source image is still a challenge. In this study, by taking advantage of the general pattern recognition ability of convolutional neural network, we propose a novel Deep Learning Image Condition (DLIC) to address this issue. The proposed DLIC measures the goodness of waveform alignments for both P- and S-waves after correcting their traveltime moveouts and it follows the geophysical principle of seismic imaging that the best-aligned waveforms utterly represent a best-imaged source location. Both a synthetic test and real data application are used to show the effectiveness and merits of the proposed DLIC. A test on synthetic data shows that the DLIC can effectively overcome the polarity reversal issues in the data. Real data application to southern California shows that the DLIC can greatly enhance the focusing of migrated source image over the widely used source scanning algorithm. Further tests show that the DLIC is applicable to continuous seismic data, to regions with few historical earthquakes, and has the potential to locate small earthquakes. The proposed DLIC shall benefit the migration-based source location methods.

physics.geo-ph↗

arXiv2023

Power-law Distribution of Solar-Cycle Modulated Coronal Jets

Power-law distributions have been studied as a significant characteristic of non-linear dissipative systems. Since discovering the power-law distribution of solar flares that was later extended to nano-flares and stellar flares, it has been widely accepted that different scales of flares share the same physical process. Here, we present the newly developed Semi-Automated Jet Identification Algorithm (SAJIA) and its application for detecting more than 1200 off-limb solar jets during Solar Cycle 24. Power-law distributions have been revealed between the intensity/energy and frequency of these events, with indices found to be analogous to those for flares and coronal mass ejections (CMEs). These jets are also found to be spatially and temporally modulated by the solar cycle forming a butterfly diagram in their latitudinal-temporal evolution, experiencing quasi-annual oscillations in their analysed properties, and very likely gathering in certain active longitudinal belts. Our results show that coronal jets display the same nonlinear behaviour as that observed in flares and CMEs, in solar and stellar atmospheres, strongly suggesting that they result from the same nonlinear statistics of scale-free processes as their counterparts in different scales of eruptive events. Although these jets, like flares and other large-scale dynamic phenomena, are found to be significantly modulated by the solar cycle, their corresponding power-law indices still remain similar.

astro-ph.SR↗

arXiv2023

Spatial Scattering Modulation with Multipath Component Aggregation Based on Antenna Arrays

In this paper, a multipath component aggregation (MCA) mechanism is introduced for spatial scattering modulation (SSM) to overcome the limitation in conventional SSM that the transmit antenna array steers the beam to a single multipath (MP) component at each instance. In the proposed MCA-SSM system, information bits are divided into two streams. One is mapped to an amplitude-phase-modulation (APM) constellation symbol, and the other is mapped to a beam vector symbol which steers multiple beams to selected strongest MP components via an MCA matrix. In comparison with the conventional SSM system, the proposed MCA-SSM enhances the bit error performance by avoiding both low receiving power due to steering the beam to a single weak MP component and inter-MP interference due to MP components with close values of angle of arrival (AoA) or angle of departure (AoD). For the proposed MCA-SSM, a union upper bound (UUB) on the average bit error probability (ABEP) with any MCA matrix is analytically derived and validated via Monte Carlo simulations. Based on the UUB, the MCA matrix is analytically optimized to minimize the ABEP of the MCA-SSM. Finally, numerical experiments are carried out, which show that the proposed MCA-SSM system remarkably outperforms the state-of-the-art SSM system in terms of ABEP under a typical indoor environment.

cs.IT↗

arXiv2023

Route to stabilize cubic gauche polynitrogen to ambient conditions via surface-saturation by hydrogen

Cubic gauche polynitrogen (cg-N) is an attractive high-energy density material. However, high-pressure synthesized cg-N will decompose at low-pressure and cannot exist at ambient conditions. Here, the stabilities of cg-N surfaces with and without saturations at different pressures and temperatures are investigated systematically. Pristine surfaces at 0 GPa are very brittle and will decompose at 300 K, especially (1 1 0) surface will collapse completely just after structural relaxation, whereas the decompositions of surfaces can be suppressed by applying pressure, indicating that surface instability causes the cg-N decomposition at low-pressure. Due to the saturation of dangling bonds and transferring electrons to the surfaces, saturation with H can stabilize surfaces at ambient conditions, while OH saturation cannot because of getting electrons from the surfaces. An acidic environment or surface saturation with less electronegative adsorbates is more favorable for the stability of polymerized nitrogen.

cond-mat.mtrl-sci↗

arXiv2023

DeAR: A Deep-learning-based Audio Re-recording Resilient Watermarking

Audio watermarking is widely used for leaking source tracing. The robustness of the watermark determines the traceability of the algorithm. With the development of digital technology, audio re-recording (AR) has become an efficient and covert means to steal secrets. AR process could drastically destroy the watermark signal while preserving the original information. This puts forward a new requirement for audio watermarking at this stage, that is, to be robust to AR distortions. Unfortunately, none of the existing algorithms can effectively resist AR attacks due to the complexity of the AR process. To address this limitation, this paper proposes DeAR, a deep-learning-based audio re-recording resistant watermarking. Inspired by DNN-based image watermarking, we pioneer a deep learning framework for audio carriers, based on which the watermark signal can be effectively embedded and extracted. Meanwhile, in order to resist the AR attack, we delicately analyze the distortions that occurred in the AR process and design the corresponding distortion layer to cooperate with the proposed watermarking framework. Extensive experiments show that the proposed algorithm can resist not only common electronic channel distortions but also AR distortions. Under the premise of high-quality embedding (SNR=25.86dB), in the case of a common re-recording distance (20cm), the algorithm can effectively achieve an average bit recovery accuracy of 98.55%.

cs.SD↗

arXiv2023

From RDMA to RDCA: Toward High-Speed Last Mile of Data Center Networks Using Remote Direct Cache Access

In this paper, we conduct systematic measurement studies to show that the high memory bandwidth consumption of modern distributed applications can lead to a significant drop of network throughput and a large increase of tail latency in high-speed RDMA networks.We identify its root cause as the high contention of memory bandwidth between application processes and network processes. This contention leads to frequent packet drops at the NIC of receiving hosts, which triggers the congestion control mechanism of the network and eventually results in network performance degradation. To tackle this problem, we make a key observation that given the distributed storage service, the vast majority of data it receives from the network will be eventually written to high-speed storage media (e.g., SSD) by CPU. As such, we propose to bypass host memory when processing received data to completely circumvent this performance bottleneck. In particular, we design Lamda, a novel receiver cache processing system that consumes a small amount of CPU cache to process received data from the network at line rate. We implement a prototype of Lamda and evaluate its performance extensively in a Clos-based testbed. Results show that for distributed storage applications, Lamda improves network throughput by 4.7% with zero memory bandwidth consumption on storage nodes, and improves network throughput by up 17% and 45% for large block size and small size under the memory bandwidth pressure, respectively. Lamda can also be applied to latency-sensitive HPC applications, which reduces their communication latency by 35.1%.

cs.NI↗

arXiv2023

DiffusionCT: Latent Diffusion Model for CT Image Standardization

Computed tomography (CT) is one of the modalities for effective lung cancer screening, diagnosis, treatment, and prognosis. The features extracted from CT images are now used to quantify spatial and temporal variations in tumors. However, CT images obtained from various scanners with customized acquisition protocols may introduce considerable variations in texture features, even for the same patient. This presents a fundamental challenge to downstream studies that require consistent and reliable feature analysis. Existing CT image harmonization models rely on GAN-based supervised or semi-supervised learning, with limited performance. This work addresses the issue of CT image harmonization using a new diffusion-based model, named DiffusionCT, to standardize CT images acquired from different vendors and protocols. DiffusionCT operates in the latent space by mapping a latent non-standard distribution into a standard one. DiffusionCT incorporates an Unet-based encoder-decoder, augmented by a diffusion model integrated into the bottleneck part. The model is designed in two training phases. The encoder-decoder is first trained, without embedding the diffusion model, to learn the latent representation of the input data. The latent diffusion model is then trained in the next training phase while fixing the encoder-decoder. Finally, the decoder synthesizes a standardized image with the transformed latent representation. The experimental results demonstrate a significant improvement in the performance of the standardization task using DiffusionCT.

eess.IV↗

arXiv2023

A Distributed Machine Learning-Based Approach for IRS-Enhanced Cell-Free MIMO Networks

In cell-free multiple input multiple output (MIMO) networks, multiple base stations (BSs) collaborate to achieve high spectral efficiency. Nevertheless, high penetration loss due to large blockages in harsh propagation environments is often an issue that severely degrades communication performance. Considering that intelligent reflecting surface (IRS) is capable of constructing digitally controllable reflection links in a low-cost manner, we investigate an IRS-enhanced downlink cell-free MIMO network in this paper. We aim to maximize the sum rate of all the users by jointly optimizing the transmit beamforming at the BSs and the reflection coefficients at the IRS. To address the optimization problem, we propose a fully distributed machine learning algorithm. Different from the conventional iterative optimization algorithms that require a central processing at the central processing unit (CPU) and large amount of channel state information and signaling exchange between the BSs and the CPU, in the proposed algorithm, each BS can locally design its beamforming vectors. Meanwhile, the IRS reflection coefficients are determined by one of the BSs. Simulation results show that the deployment of IRS can significantly boost the sum user rate and that the proposed algorithm can achieve a high sum user rate with a low computational complexity.

cs.IT↗

arXiv2023

Real Face Foundation Representation Learning for Generalized Deepfake Detection

The emergence of deepfake technologies has become a matter of social concern as they pose threats to individual privacy and public security. It is now of great significance to develop reliable deepfake detectors. However, with numerous face manipulation algorithms present, it is almost impossible to collect sufficient representative fake faces, and it is hard for existing detectors to generalize to all types of manipulation. Therefore, we turn to learn the distribution of real faces, and indirectly identify fake images that deviate from the real face distribution. In this study, we propose Real Face Foundation Representation Learning (RFFR), which aims to learn a general representation from large-scale real face datasets and detect potential artifacts outside the distribution of RFFR. Specifically, we train a model on real face datasets by masked image modeling (MIM), which results in a discrepancy between input faces and the reconstructed ones when applying the model on fake samples. This discrepancy reveals the low-level artifacts not contained in RFFR, making it easier to build a deepfake detector sensitive to all kinds of potential artifacts outside the distribution of RFFR. Extensive experiments demonstrate that our method brings about better generalization performance, as it significantly outperforms the state-of-the-art methods in cross-manipulation evaluations, and has the potential to further improve by introducing extra real faces for training RFFR.

cs.CV↗