arXiv subjects

Wei Jiang

Publications and source records attributed to Wei Jiang.

At least 235 records · Page 13Linked to original sources

arXiv2024

QSMDiff: Unsupervised 3D Diffusion Models for Quantitative Susceptibility Mapping

Quantitative Susceptibility Mapping (QSM) dipole inversion is an ill-posed inverse problem for quantifying magnetic susceptibility distributions from MRI tissue phases. While supervised deep learning methods have shown success in specific QSM tasks, their generalizability across different acquisition scenarios remains constrained. Recent developments in diffusion models have demonstrated potential for solving 2D medical imaging inverse problems. However, their application to 3D modalities, such as QSM, remains challenging due to high computational demands. In this work, we developed a 3D image patch-based diffusion model, namely QSMDiff, for robust QSM reconstruction across different scan parameters, alongside simultaneous super-resolution and image-denoising tasks. QSMDiff adopts unsupervised 3D image patch training and full-size measurement guidance during inference for controlled image generation. Evaluation on simulated and in-vivo human brains, using gradient-echo and echo-planar imaging sequences across different acquisition parameters, demonstrates superior performance. The method proposed in QSMDiff also holds promise for impacting other 3D medical imaging applications beyond QSM.

eess.IV↗

arXiv2024

LVC-LGMC: Joint Local and Global Motion Compensation for Learned Video Compression

Existing learned video compression models employ flow net or deformable convolutional networks (DCN) to estimate motion information. However, the limited receptive fields of flow net and DCN inherently direct their attentiveness towards the local contexts. Global contexts, such as large-scale motions and global correlations among frames are ignored, presenting a significant bottleneck for capturing accurate motions. To address this issue, we propose a joint local and global motion compensation module (LGMC) for leaned video coding. More specifically, we adopt flow net for local motion compensation. To capture global context, we employ the cross attention in feature domain for motion compensation. In addition, to avoid the quadratic complexity of vanilla cross attention, we divide the softmax operations in attention into two independent softmax operations, leading to linear complexity. To validate the effectiveness of our proposed LGMC, we integrate it with DCVC-TCM and obtain learned video compression with joint local and global motion compensation (LVC-LGMC). Extensive experiments demonstrate that our LVC-LGMC has significant rate-distortion performance improvements over baseline DCVC-TCM.

eess.IV↗

arXiv2024

Orbital torque switching in perpendicularly magnetized materials

The orbital Hall effect in light materials has attracted considerable attention for developing novel orbitronic devices. Here we investigate the orbital torque efficiency and demonstrate the switching of the perpendicularly magnetized materials through the orbital Hall material (OHM), i.e., Zirconium (Zr). The orbital torque efficiency of approximately 0.78 is achieved in the Zr OHM with the perpendicularly magnetized [Co/Pt]3 sample, which significantly surpasses that of the perpendicularly magnetized CoFeB/Gd/CoFeB sample (approximately 0.04). Such notable difference is attributed to the different spin-orbit correlation strength between the [Co/Pt]3 sample and the CoFeB/Gd/CoFeB sample, which has been confirmed through the theoretical calculations. Furthermore, the full magnetization switching of the [Co/Pt]3 sample with a switching current density of approximately 2.6x106 A/cm2 has been realized through Zr, which even outperforms that of the W spin Hall material. Our finding provides a guideline to understand orbital torque efficiency and paves the way to develop energy-efficient orbitronic devices.

cond-mat.mtrl-sci↗

arXiv2024

CoCA: Fusing Position Embedding with Collinear Constrained Attention in Transformers for Long Context Window Extending

Self-attention and position embedding are two key modules in transformer-based Large Language Models (LLMs). However, the potential relationship between them is far from well studied, especially for long context window extending. In fact, anomalous behaviors harming long context extrapolation exist between Rotary Position Embedding (RoPE) and vanilla self-attention unveiled by our work. To address this issue, we propose a novel attention mechanism, CoCA (Collinear Constrained Attention). Specifically, we enforce a collinear constraint between $Q$ and $K$ to seamlessly integrate RoPE and self-attention. While only adding minimal computational and spatial complexity, this integration significantly enhances long context window extrapolation ability. We provide an optimized implementation, making it a drop-in replacement for any existing transformer-based models. Extensive experiments show that CoCA performs extraordinarily well in extending context windows. A CoCA-based GPT model, trained with a context length of 512, can seamlessly extend the context window up to 32K (60$\times$), without any fine-tuning. Additionally, by dropping CoCA in LLaMA-7B, we achieve extrapolation up to 32K within only 2K training length. Our code is publicly available at: https://github.com/codefuse-ai/Collinear-Constrained-Attention

cs.LG↗

arXiv2024

Challenging Low Homophily in Social Recommendation

Social relations are leveraged to tackle the sparsity issue of user-item interaction data in recommendation under the assumption of social homophily. However, social recommendation paradigms predominantly focus on homophily based on user preferences. While social information can enhance recommendations, its alignment with user preferences is not guaranteed, thereby posing the risk of introducing informational redundancy. We empirically discover that social graphs in real recommendation data exhibit low preference-aware homophily, which limits the effect of social recommendation models. To comprehensively extract preference-aware homophily information latent in the social graph, we propose Social Heterophily-alleviating Rewiring (SHaRe), a data-centric framework for enhancing existing graph-based social recommendation models. We adopt Graph Rewiring technique to capture and add highly homophilic social relations, and cut low homophilic (or heterophilic) relations. To better refine the user representations from reliable social relations, we integrate a contrastive learning method into the training of SHaRe, aiming to calibrate the user representations for enhancing the result of Graph Rewiring. Experiments on real-world datasets show that the proposed framework not only exhibits enhanced performances across varying homophily ratios but also improves the performance of existing state-of-the-art (SOTA) social recommendation models.

cs.IR↗

arXiv2024

Giant piezoelectricity in group IV monochalcogenides with ferroelectric AA layer stacking

The piezoelectricity of group IV monochalcogenides (MXs, with M = Ge, Sn and X = S, Se) has attracted much attention due to their substantially higher piezoelectric coefficients compared to other 2D materials. However, with increasing layer number, their piezoelectricity rapidly disappears due to the antiferroelectric stacking order, severely limiting their practical applications. Using first-principles calculations, we investigated the piezoelectricity of MXs with the ferroelectric AA stacking configuration, which has recently been stabilized in experiments. We found that AA-stacked MXs have a ferroelectric ground state with the smallest lattice constant among other stacking configurations, resulting in a giant piezoelectric coefficient, which is the first demonstration of a strategy where the piezoelectric coefficients can increase with the number of layers. This can be attributed to a strong negative correlation between the lattice constant along the armchair direction and the piezoelectric coefficient, and spontaneous compressive strain stabilized in ferroelectric AA stacking configuration.

cond-mat.mtrl-sci↗

arXiv2024

UCVC: A Unified Contextual Video Compression Framework with Joint P-frame and B-frame Coding

This paper presents a learned video compression method in response to video compression track of the 6th Challenge on Learned Image Compression (CLIC), at DCC 2024.Specifically, we propose a unified contextual video compression framework (UCVC) for joint P-frame and B-frame coding. Each non-intra frame refers to two neighboring decoded frames, which can be either both from the past for P-frame compression, or one from the past and one from the future for B-frame compression. In training stage, the model parameters are jointly optimized with both P-frames and B-frames. Benefiting from the designs, the framework can support both P-frame and B-frame coding and achieve comparable compression efficiency with that specifically designed for P-frame or B-frame.As for challenge submission, we report the optimal compression efficiency by selecting appropriate frame types for each test sequence. Our team name is PKUSZ-LVC.

cs.CV↗

arXiv2024

Tensor-based process control and monitoring for semiconductor manufacturing with unstable disturbances

With the development and popularity of sensors installed in manufacturing systems, complex data are collected during manufacturing processes, which brings challenges for traditional process control methods. This paper proposes a novel process control and monitoring method for the complex structure of high-dimensional image-based overlay errors (modeled in tensor form), which are collected in semiconductor manufacturing processes. The proposed method aims to reduce overlay errors using limited control recipes. We first build a high-dimensional process model and propose different tensor-on-vector regression algorithms to estimate parameters in the model to alleviate the curse of dimensionality. Then, based on the estimate of tensor parameters, the exponentially weighted moving average (EWMA) controller for tensor data is designed whose stability is theoretically guaranteed. Considering the fact that low-dimensional control recipes cannot compensate for all high-dimensional disturbances on the image, control residuals are monitored to prevent significant drifts of uncontrollable high-dimensional disturbances. Through extensive simulations and real case studies, the performances of parameter estimation algorithms and the EWMA controller in tensor space are evaluated. Compared with existing image-based feedback controllers, the superiority of our method is verified especially when disturbances are not stable.

stat.ML↗

arXiv2024

Hierarchical Cell-Free Massive MIMO for High Capacity with Simple Implementation

Cell-free massive multi-input multi-output (MIMO) has recently gained much attention for its potential in shaping the landscape of sixth-generation (6G) wireless systems. This paper proposes a hierarchical network architecture tailored for cell-free massive MIMO, seamlessly integrating co-located and distributed antennas. A central base station (CBS), equipped with an antenna array, positions itself near the center of the coverage area, complemented by distributed access points spanning the periphery. The proposed architecture remarkably outperforms conventional cell-free networks, demonstrating superior sum throughput while maintaining a comparable worst-case per-user spectral efficiency. Meanwhile, the implementation cost associated with the fronthaul network is substantially diminished.

cs.IT↗

arXiv2024

CodeFuse-13B: A Pretrained Multi-lingual Code Large Language Model

Code Large Language Models (Code LLMs) have gained significant attention in the industry due to their wide applications in the full lifecycle of software engineering. However, the effectiveness of existing models in understanding non-English inputs for multi-lingual code-related tasks is still far from well studied. This paper introduces CodeFuse-13B, an open-sourced pre-trained code LLM. It is specifically designed for code-related tasks with both English and Chinese prompts and supports over 40 programming languages. CodeFuse achieves its effectiveness by utilizing a high quality pre-training dataset that is carefully filtered by program analyzers and optimized during the training process. Extensive experiments are conducted using real-world usage scenarios, the industry-standard benchmark HumanEval-x, and the specially designed CodeFuseEval for Chinese prompts. To assess the effectiveness of CodeFuse, we actively collected valuable human feedback from the AntGroup's software development process where CodeFuse has been successfully deployed. The results demonstrate that CodeFuse-13B achieves a HumanEval pass@1 score of 37.10%, positioning it as one of the top multi-lingual code LLMs with similar parameter sizes. In practical scenarios, such as code generation, code translation, code comments, and testcase generation, CodeFuse performs better than other models when confronted with Chinese prompts.

cs.SE↗

arXiv2024

Summarizing Stream Data for Memory-Constrained Online Continual Learning

Replay-based methods have proved their effectiveness on online continual learning by rehearsing past samples from an auxiliary memory. With many efforts made on improving training schemes based on the memory, however, the information carried by each sample in the memory remains under-investigated. Under circumstances with restricted storage space, the informativeness of the memory becomes critical for effective replay. Although some works design specific strategies to select representative samples, by only employing a small number of original images, the storage space is still not well utilized. To this end, we propose to Summarize the knowledge from the Stream Data (SSD) into more informative samples by distilling the training characteristics of real images. Through maintaining the consistency of training gradients and relationship to the past tasks, the summarized samples are more representative for the stream data compared to the original images. Extensive experiments are conducted on multiple online continual learning benchmarks to support that the proposed SSD method significantly enhances the replay effects. We demonstrate that with limited extra computational overhead, SSD provides more than 3% accuracy boost for sequential CIFAR-100 under extremely restricted memory buffer. Code in https://github.com/vimar-gu/SSD.

cs.CV↗

arXiv2024

Beam-Based Multiple Access for IRS-Aided Millimeter-Wave and Terahertz Communications

Recently, intelligent reflecting surface (IRS)-aided millimeter-wave (mmWave) and terahertz (THz) communications are considered in the wireless community. This paper aims to design a beam-based multiple-access strategy for this new paradigm. Its key idea is to make use of multiple sub-arrays over a hybrid digital-analog array to form independent beams, each of which is steered towards the desired direction to mitigate inter-user interference and suppress unwanted signal reflection. The proposed scheme combines the advantages of both orthogonal multiple access (i.e., no inter-user interference) and non-orthogonal multiple access (i.e., full time-frequency resource use). Consequently, it can substantially boost the system capacity, as verified by Monte-Carlo simulations.

cs.IT↗

arXiv2023

RecAGT: Shard Testable Codes with Adaptive Group Testing for Malicious Nodes Identification in Sharding Permissioned Blockchain

Recently, permissioned blockchain has been extensively explored in various fields, such as asset management, supply chain, healthcare, and many others. Many scholars are dedicated to improving its verifiability, scalability, and performance based on sharding techniques, including grouping nodes and handling cross-shard transactions. However, they ignore the node vulnerability problem, i.e., there is no guarantee that nodes will not be maliciously controlled throughout their life cycle. Facing this challenge, we propose RecAGT, a novel identification scheme aimed at reducing communication overhead and identifying potential malicious nodes. First, shard testable codes are designed to encode the original data in case of a leak of confidential data. Second, a new identity proof protocol is presented as evidence against malicious behavior. Finally, adaptive group testing is chosen to identify malicious nodes. Notably, our work focuses on the internal operation within the committee and can thus be applied to any sharding permissioned blockchains. Simulation results show that our proposed scheme can effectively identify malicious nodes with low communication and computational costs.

cs.CR↗

arXiv2023

Breakdown modes of capacitively coupled plasma II: unsustainable discharges

In this work, the one-dimensional implicit particle-in-cell/Monte-Carlo collision code (PIC/MCC) is used to study the discharge of a capacitively coupled plasma (CCP) under extremely low pressure driven by high-frequency rf power in pure argon. With the introduction of high-coefficient electron-induced secondary electron emission (ESEE) and a blocking capacitor, the discharge that cannot be sustained shows a variety of different characteristics: including the normal failure discharge (NFD) of the electron avalanche, bias failure discharge caused by the charging effect of the blocking capacitor, and runaway failure discharge caused by the decrease in the ESEE rate during the forming of the sheath. The discharges in low-pressure regions exhibit a range of discharge characteristics, the sustainable discharges of which have been analyzed in more detail. The study of unsustainable discharge helps to find the reasons for failure discharge and then determine the parameters of sustainable discharge, which is of great value in preventing plasma crack, equipment product yield, and equipment safety to help prevent industrial losses.

physics.plasm-ph↗

arXiv2023

Distributed Optimization with Quantized Gradient Descent

In this paper, we consider the unconstrained distributed optimization problem, in which the exchange of information in the network is captured by a directed graph topology, thus, nodes can only communicate with their neighbors. Additionally, in our problem, the communication channels among the nodes have limited bandwidth. In order to alleviate this limitation, quantized messages should be exchanged among the nodes. For solving this distributed optimization problem, we combine a gradient descent method with a distributed quantized consensus algorithm (which requires the nodes to exchange quantized messages and converges in a finite number of steps). Specifically, at every optimization step, each node (i) performs a gradient descent step (i.e., subtracts the scaled gradient from its current estimate), and (ii) performs a finite-time calculation of the quantized average of every node's estimate in the network. As a consequence, this algorithm approximately mimics the centralized gradient descent algorithm. We show that our algorithm asymptotically converges to a neighborhood of the optimal solution with linear convergence rate. The performance of the proposed algorithm is demonstrated via simple illustrative examples.

eess.SY↗

arXiv2023

Breakdown modes of capacitively coupled plasma I: transitions from glow discharge to multipactor

In this work, a one-dimensional direct implicit particle-in-cell/Monte Carlo collision (PIC/MCC) code is used to study the capacitive discharge driven under 60 MHz rf power in the background gas of pure argon. The electron-induced secondary electron emission (ESEE) model suitable for SiO$_2$ electrode is taken into account. Several discharge modes are found in the transition region between the higher-pressure glow discharge and the abnormal multipactor discharge in this simulation. In a nerrow voltage range of hundreds of volts, the abnormal multipactor will transform to normal multipactor with the contining decrease of pressure. The characteristic of the three sustainale discharges are given and anaysed, and their fomation porcess are discussed. These new discharges have higher electron energy and electron flux at the boundary and are mainly sustained by higher electrode-induced SEE coefficient and high frequency. The appearence of the two multipactor modes in 60 MHz range might broaden the theory of gas discharge and expand the application of capacitive discharges.

physics.plasm-ph↗

arXiv2023

VGSG: Vision-Guided Semantic-Group Network for Text-based Person Search

Text-based Person Search (TBPS) aims to retrieve images of target pedestrian indicated by textual descriptions. It is essential for TBPS to extract fine-grained local features and align them crossing modality. Existing methods utilize external tools or heavy cross-modal interaction to achieve explicit alignment of cross-modal fine-grained features, which is inefficient and time-consuming. In this work, we propose a Vision-Guided Semantic-Group Network (VGSG) for text-based person search to extract well-aligned fine-grained visual and textual features. In the proposed VGSG, we develop a Semantic-Group Textual Learning (SGTL) module and a Vision-guided Knowledge Transfer (VGKT) module to extract textual local features under the guidance of visual local clues. In SGTL, in order to obtain the local textual representation, we group textual features from the channel dimension based on the semantic cues of language expression, which encourages similar semantic patterns to be grouped implicitly without external tools. In VGKT, a vision-guided attention is employed to extract visual-related textual features, which are inherently aligned with visual cues and termed vision-guided textual features. Furthermore, we design a relational knowledge transfer, including a vision-language similarity transfer and a class probability transfer, to adaptively propagate information of the vision-guided textual features to semantic-group textual features. With the help of relational knowledge transfer, VGKT is capable of aligning semantic-group textual features with corresponding visual features without external tools and complex pairwise interaction. Experimental results on two challenging benchmarks demonstrate its superiority over state-of-the-art methods.

cs.CV↗

arXiv2023

Readout Electronics for CBM-TOF Super Module Quality Evaluation

A super module assembled with MRPC detectors is a component of TOF (Time of Flight) system for the Compressed Baryonic Matter (CBM) experiment. Before the super modules are applied to CBM-TOF, their quality needs to be evaluated. The readout electronics is confronted with a tremendous challenge of transmitting data at the maximal speed of 6 Gbps. In this paper, the readout method for CBM-TOF super module quality evaluation is presented. A parallel architecture based on the Gigabit Ethernet is designed to meet the requirement for data transmission rate. First, the data is sent from the front-end electronics to four readout module groups via optical fibers at the maximal rate of 1.5 Gbps per fiber. Next, the data are further distributed to sixteen parallel daughter readout models so that the maximal data throughput of each daughter readout module is 375 Mbps, within its capacity of 550 Mbps. Finally, the readout daughter modules send data to the data acquisition (DAQ) software through standard Gigabit Ethernet. The proposed readout method has the advantage of good scalability, so it can meet different requirements for variable data rate. The preliminary test result shows that each of four parallel readout groups can transmit data at the speed of 1.6 Gbps, which indicates that the overall readout system can meet the requirement for the maximal data-transmission speed of 6 Gbps.

physics.ins-det↗