arXiv subjects

Man Liu

Publications and source records attributed to Man Liu.

13 recordsLinked to original sources

arXiv2026

FP4Quant: Efficient 4-Bit Communication Compression for Model-Parallel LLM Training

The rapid scaling of large language models (LLMs) has made communication overhead a critical bottleneck in distributed model-parallel training. Although existing low-precision compression methods can effectively reduce communication costs, further reducing the communication footprint to ultra-low-bit formats such as FP4 introduces new challenges, including limited numerical range, sub-byte storage overhead, and increased computational cost from fine-grained quantization. In this paper, we present \textbf{FP4Quant}, a micro-grouped FP4 compression framework for efficient model-parallel LLM training. FP4Quant integrates a micro-grouped dual-scale FP4 quantization mechanism to preserve local tensor distribution characteristics, a warp-cooperative FP4 packing scheme to enable efficient sub-byte data representation, and a hybrid two-level FWHT pipeline to mitigate quantization errors with GPU-aware execution optimization. Experiments on NVIDIA H100 GPUs with Qwen-7B demonstrate up to 1.56$\times$ end-to-end throughput improvement over BF16, revealing a trade-off between throughput and model quality.

cs.AI↗

arXiv2026

Tools-CC-Bench: a Benchmark Suite for Collective Communication with Compression in HPC and AI Workloads

Distributed HPC and LLM workloads increasingly require efficient communication for scalability, yet growing data movement has become a major performance bottleneck. Communication compression can reduce this overhead and complement execution-level optimizations, but its benefits remain difficult to assess because existing benchmarks lack support for diverse backends, realistic datasets, application-specific accuracy metrics, and overlap-induced resource contention. We present CC-Bench, a lightweight, extensible, and application-oriented benchmark suite for evaluating communication compression under realistic execution conditions. CC-Bench uses declarative application-environment modeling to decouple profiling logic from communication libraries, datasets, and fidelity metrics, enabling portable cross-library evaluation. It further combines function-level interception and hardware counter monitoring to characterize per-phase latency, hardware utilization, numerical fidelity, and computation interference. With representative datasets from HPC and LLM workloads, CC-Bench evaluates three compression-enabled communication libraries on CPU and GPU clusters, revealing accuracy-performance trade-offs and bottlenecks to guide practical deployment and optimization.

cs.DC↗

arXiv2026

TACO: Efficient Communication Compression of Intermediate Tensors for Scalable Tensor-Parallel LLM Training

Handling communication overhead in large-scale tensor-parallel training remains a critical challenge due to the dense, near-zero distributions of intermediate tensors, which exacerbate errors under frequent communication and introduce significant computational overhead during compression. To this end, we propose TACO (Tensor-parallel Adaptive COmmunication compression), a robust FP8-based framework for compressing TP intermediate tensors. First, we employ a data-driven reshaping strategy combined with an Adaptive Scale-Hadamard Transform to enable high-fidelity FP8 quantization, while its Dual-Scale Quantization mechanism ensures numerical stability throughout training. Second, we design a highly fused compression operator to reduce memory traffic and kernel launch overhead, allowing efficient overlap with communication. Finally, we integrate TACO with existing state-of-the-art methods for Data and Pipeline Parallelism to develop a compression-enabled 3D-parallel training framework. Detailed experiments on GPT models and Qwen model demonstrate up to 1.87X end-to-end throughput improvement while maintaining near-lossless accuracy, validating the effectiveness and efficiency of TACO in large-scale training.

cs.DC↗

arXiv2025

Harnessing Group-Oriented Consistency Constraints for Semi-Supervised Semantic Segmentation in CdZnTe Semiconductors

Labeling Cadmium Zinc Telluride (CdZnTe) semiconductor images is challenging due to the low-contrast defect boundaries, necessitating annotators to cross-reference multiple views. These views share a single ground truth (GT), forming a unique ``many-to-one'' relationship. This characteristic renders advanced semi-supervised semantic segmentation (SSS) methods suboptimal, as they are generally limited by a ``one-to-one'' relationship, where each image is independently associated with its GT. Such limitation may lead to error accumulation in low-contrast regions, further exacerbating confirmation bias. To address this issue, we revisit the SSS pipeline from a group-oriented perspective and propose a human-inspired solution: the Intra-group Consistency Augmentation Framework (ICAF). First, we experimentally validate the inherent consistency constraints within CdZnTe groups, establishing a group-oriented baseline using the Intra-group View Sampling (IVS). Building on this insight, we introduce the Pseudo-label Correction Network (PCN) to enhance consistency representation, which consists of two key modules. The View Augmentation Module (VAM) improves boundary details by dynamically synthesizing a boundary-aware view through the aggregation of multiple views. In the View Correction Module (VCM), this synthesized view is paired with other views for information interaction, effectively emphasizing salient regions while minimizing noise. Extensive experiments demonstrate the effectiveness of our solution for CdZnTe materials. Leveraging DeepLabV3+ with a ResNet-101 backbone as our segmentation model, we achieve a 70.6\% mIoU on the CdZnTe dataset using only 2 group-annotated data (5\textperthousand). The code is available at \href{https://github.com/pipixiapipi/ICAF}{https://github.com/pipixiapipi/ICAF}.

cs.CV↗

arXiv2025

Attend and Enrich: Enhanced Visual Prompt for Zero-Shot Learning

Zero-shot learning (ZSL) endeavors to transfer knowledge from seen categories to recognize unseen categories, which mostly relies on the semantic-visual interactions between image and attribute tokens. Recently, prompt learning has emerged in ZSL and demonstrated significant potential as it allows the zero-shot transfer of diverse visual concepts to downstream tasks. However, current methods explore the fixed adaption of learnable prompt on seen domains, which makes them over-emphasize the primary visual features observed during training, limiting their generalization capabilities to unseen domains. In this work, we propose AENet, which endows semantic information into the visual prompt to distill semantic-enhanced prompt for visual representation enrichment, enabling effective knowledge transfer for ZSL. AENet comprises two key steps: 1) exploring the concept-harmonized tokens for the visual and attribute modalities, grounded on the modal-sharing token that represents consistent visual-semantic concepts; and 2) yielding semantic-enhanced prompt via the visual residual refinement unit with attribute consistency supervision. These are further integrated with primary visual features to attend to semantic-related information for visual enhancement, thus strengthening transferable ability. Experimental results on three benchmarks show that our AENet outperforms existing state-of-the-art ZSL methods. The code is provided in the zip file of supplementary materials.

cs.CV↗

arXiv2024

Metasurface-Integrated Polarization-Insensitive LCoS for Projection Displays

Liquid crystal on silicon (LCoS) panels, renowned for their high resolution and fill-factor, are integral to modern projection displays. However, their inherent polarization sensitivity constrains the upper limit of light utilization, increases system complexity and restricts broader applicability. Here, we demonstrate a dual-layer metasurface-integrated LCoS prototype that achieves polarization-insensitive, addressable amplitude modulation in the visible. Polarization sensitivity is eliminated in the reflective architecture through polarization conversion in the underlying metasurface and polarization-sensitive phase modulation of the liquid crystals (LC). This is further enhanced by the electrically tunable subwavelength grating formed by the upper metasurface and LC, resulting in a high-contrast, polarization-insensitive optical switch. We showcase a 64-pixel 2D addressable prototype capable of generating diverse projection patterns with high contrast. Compatible with existing LCoS processes, our metasurface device reduces system size and enhances energy efficiency, offering applications in projectors and AR/VR displays, with the potential to redefine projection chip technology.

physics.optics↗

arXiv2024

PSVMA+: Exploring Multi-granularity Semantic-visual Adaption for Generalized Zero-shot Learning

Generalized zero-shot learning (GZSL) endeavors to identify the unseen categories using knowledge from the seen domain, necessitating the intrinsic interactions between the visual features and attribute semantic features. However, GZSL suffers from insufficient visual-semantic correspondences due to the attribute diversity and instance diversity. Attribute diversity refers to varying semantic granularity in attribute descriptions, ranging from low-level (specific, directly observable) to high-level (abstract, highly generic) characteristics. This diversity challenges the collection of adequate visual cues for attributes under a uni-granularity. Additionally, diverse visual instances corresponding to the same sharing attributes introduce semantic ambiguity, leading to vague visual patterns. To tackle these problems, we propose a multi-granularity progressive semantic-visual mutual adaption (PSVMA+) network, where sufficient visual elements across granularity levels can be gathered to remedy the granularity inconsistency. PSVMA+ explores semantic-visual interactions at different granularity levels, enabling awareness of multi-granularity in both visual and semantic elements. At each granularity level, the dual semantic-visual transformer module (DSVTM) recasts the sharing attributes into instance-centric attributes and aggregates the semantic-related visual regions, thereby learning unambiguous visual features to accommodate various instances. Given the diverse contributions of different granularities, PSVMA+ employs selective cross-granularity learning to leverage knowledge from reliable granularities and adaptively fuses multi-granularity features for comprehensive representations. Experimental results demonstrate that PSVMA+ consistently outperforms state-of-the-art methods.

cs.CV↗

arXiv2023

Exploring Resolution Fields for Scalable Image Compression with Uncertainty Guidance

Recently, there are significant advancements in learning-based image compression methods surpassing traditional coding standards. Most of them prioritize achieving the best rate-distortion performance for a particular compression rate, which limits their flexibility and adaptability in various applications with complex and varying constraints. In this work, we explore the potential of resolution fields in scalable image compression and propose the reciprocal pyramid network (RPN) that fulfills the need for more adaptable and versatile compression. Specifically, RPN first builds a compression pyramid and generates the resolution fields at different levels in a top-down manner. The key design lies in the cross-resolution context mining module between adjacent levels, which performs feature enriching and distillation to mine meaningful contextualized information and remove unnecessary redundancy, producing informative resolution fields as residual priors. The scalability is achieved by progressive bitstream reusing and resolution field incorporation varying at different levels. Furthermore, between adjacent compression levels, we explicitly quantify the aleatoric uncertainty from the bottom decoded representations and develop an uncertainty-guided loss to update the upper-level compression parameters, forming a reverse pyramid process that enforces the network to focus on the textured pixels with high variance for more reliable and accurate reconstruction. Combining resolution field exploration and uncertainty guidance in a pyramid manner, RPN can effectively achieve spatial and quality scalable image compression. Experiments show the superiority of RPN against existing classical and deep learning-based scalable codecs. Code will be available at https://github.com/JGIroro/RPNSIC.

eess.IV↗

arXiv2023

Progressive Semantic-Visual Mutual Adaption for Generalized Zero-Shot Learning

Generalized Zero-Shot Learning (GZSL) identifies unseen categories by knowledge transferred from the seen domain, relying on the intrinsic interactions between visual and semantic information. Prior works mainly localize regions corresponding to the sharing attributes. When various visual appearances correspond to the same attribute, the sharing attributes inevitably introduce semantic ambiguity, hampering the exploration of accurate semantic-visual interactions. In this paper, we deploy the dual semantic-visual transformer module (DSVTM) to progressively model the correspondences between attribute prototypes and visual features, constituting a progressive semantic-visual mutual adaption (PSVMA) network for semantic disambiguation and knowledge transferability improvement. Specifically, DSVTM devises an instance-motivated semantic encoder that learns instance-centric prototypes to adapt to different images, enabling the recast of the unmatched semantic-visual pair into the matched one. Then, a semantic-motivated instance decoder strengthens accurate cross-domain interactions between the matched pair for semantic-related instance adaption, encouraging the generation of unambiguous visual representations. Moreover, to mitigate the bias towards seen classes in GZSL, a debiasing loss is proposed to pursue response consistency between seen and unseen predictions. The PSVMA consistently yields superior performances against other state-of-the-art methods. Code will be available at: https://github.com/ManLiuCoder/PSVMA.

cs.CV↗

arXiv2017

Direct Vehicle-to-Vehicle Communication with Infrastructure Assistance in 5G Network

Compared with today's 4G wireless communication network, the next generation of wireless system should be able to provide a wider range of services with different QoS requirements. One emerging new service is to exploit cooperative driving to actively avoid accidents and improve traffic efficiency. A key challenge for cooperative driving is on vehicle-to-vehicle (V2V) communication which requires a high reliability and a low end-to-end (E2E) latency. In order to meet these requirements, 5G should be evaluated by new key performance indicators (KPIs) rather than the conventional metric, as throughput in the legacy cellular networks. In this work, we exploit network controlled direct V2V communication for information exchange among vehicles. This communication process refers to packet transmission directly among vehicles without the involvement of network infrastructure in U-plane. In order to have a network architecture to enable direct V2V communication, the architecture of the 4G network is enhanced by deploying a new central entity with specific functionality for V2V communication. Moreover, a resource allocation scheme is also specifically designed to adapt to traffic model and service requirements of V2V communication. Last but not least, different technologies are considered and simulated in this work to improve the performance of direct V2V communication.

cs.NI↗

arXiv2017

Applying Device-to-Device Communication to Enhance IoT Services

Massive Machine Type Communication (mMTC) to serve billions of IoT devices is considered to open a potential new market for the next generation cellular network. Legacy cellular networks cannot meet the requirements of emerging mMTC applications, since they were designed for human-driven services. In order to provide supports for mMTC services, current research and standardization work focus on the improvement and adaptation of legacy networks. However, these solutions face challenges to enhance the service availability and improve the battery life of mMTC devices simultaneously. In this article, we propose to exploit a network controlled sidelink communication scheme to enable cellular network with better support for mMTC services. Moreover, a context-aware algorithm is applied to ensure the efficiency of the proposed scheme and multiple context information of devices are taken into account. Correspondingly, signaling schemes are also designed and illustrated in this work to facilitate the proposed technology. The signaling schemes enable the network to collect required context information with light signaling effort and thus network can derive a smart configuration for both the sidelink and cellular link. In order to demonstrate the improvements brought by our scheme, a system-level simulator is implemented and numerical results show that our scheme can simultaneously enhance both the service availability and battery life of sensors.

cs.NI↗

arXiv2013

Proposal for implementing the three-qubit refined Deutsch-Jozsa quantum algorithm

We propose a way to implement a three-qubit refined Deutsh-Jozsa (DJ) algorithm. The present proposal is based on the construction of the 35 $f$-controlled phase gates, which uses single-qubit $σ_z$ gates and two-qubit {\it standard} controlled-phase (CP) gates only. This proposal is implementable because a single-qubit $σ_z$ gate can be easily realized by applying a single classical pulse and a two-qubit CP gate has been experimentally demonstrated in various physical systems. Finally, it is noted that this proposal is quite general, and can be applied to implement a three-qubit refined DJ algorithm in a cavity-based or noncavity-based physical system.

quant-ph↗

arXiv2012

Realizing an $n$-target-qubit controlled phase gate in cavity QED: An approach without classical pulses

We propose a way to realize a multiqubit controlled phase gate with one qubit simultaneously controlling $n$ target qubits using atoms in cavity QED. In this proposal, there is no need of using classical pulses during the entire gate operation. The gate operation time scales as $\sqrt{n}$ only and thus the gate can be performed faster when compared with sending atoms through the cavity one at a time. In addition, only three steps of operations are required for realizing this $n$-target-qubit controlled phase gate. This proposal is quite general, which can be applied to other physical systems such as various superconducting qubits coupled to a resonator, NV centers coupled to a microsphere cavity or quantum dots in cavity QED.

quant-ph↗