arXiv subjects

Sha Zhang

Publications and source records attributed to Sha Zhang.

24 records · Page 2Linked to original sources

arXiv2024

HVDistill: Transferring Knowledge from Images to Point Clouds via Unsupervised Hybrid-View Distillation

We present a hybrid-view-based knowledge distillation framework, termed HVDistill, to guide the feature learning of a point cloud neural network with a pre-trained image network in an unsupervised manner. By exploiting the geometric relationship between RGB cameras and LiDAR sensors, the correspondence between the two modalities based on both image-plane view and bird-eye view can be established, which facilitates representation learning. Specifically, the image-plane correspondences can be simply obtained by projecting the point clouds, while the bird-eye-view correspondences can be achieved by lifting pixels to the 3D space with the predicted depths under the supervision of projected point clouds. The image teacher networks provide rich semantics from the image-plane view and meanwhile acquire geometric information from the bird-eye view. Indeed, image features from the two views naturally complement each other and together can ameliorate the learned feature representation of the point cloud student networks. Moreover, with a self-supervised pre-trained 2D network, HVDistill requires neither 2D nor 3D annotations. We pre-train our model on nuScenes dataset and transfer it to several downstream tasks on nuScenes, SemanticKITTI, and KITTI datasets for evaluation. Extensive experimental results show that our method achieves consistent improvements over the baseline trained from scratch and significantly outperforms the existing schemes. Codes are available at git@github.com:zhangsha1024/HVDistill.git.

cs.CV↗

arXiv2024

Agent3D-Zero: An Agent for Zero-shot 3D Understanding

The ability to understand and reason the 3D real world is a crucial milestone towards artificial general intelligence. The current common practice is to finetune Large Language Models (LLMs) with 3D data and texts to enable 3D understanding. Despite their effectiveness, these approaches are inherently limited by the scale and diversity of the available 3D data. Alternatively, in this work, we introduce Agent3D-Zero, an innovative 3D-aware agent framework addressing the 3D scene understanding in a zero-shot manner. The essence of our approach centers on reconceptualizing the challenge of 3D scene perception as a process of understanding and synthesizing insights from multiple images, inspired by how our human beings attempt to understand 3D scenes. By consolidating this idea, we propose a novel way to make use of a Large Visual Language Model (VLM) via actively selecting and analyzing a series of viewpoints for 3D understanding. Specifically, given an input 3D scene, Agent3D-Zero first processes a bird's-eye view image with custom-designed visual prompts, then iteratively chooses the next viewpoints to observe and summarize the underlying knowledge. A distinctive advantage of Agent3D-Zero is the introduction of novel visual prompts, which significantly unleash the VLMs' ability to identify the most informative viewpoints and thus facilitate observing 3D scenes. Extensive experiments demonstrate the effectiveness of the proposed framework in understanding diverse and previously unseen 3D environments.

cs.CV↗

arXiv2023

TrajMatch: Towards Automatic Spatio-temporal Calibration for Roadside LiDARs through Trajectory Matching

Recently, it has become popular to deploy sensors such as LiDARs on the roadside to monitor the passing traffic and assist autonomous vehicle perception. Unlike autonomous vehicle systems, roadside sensors are usually affiliated with different subsystems and lack synchronization both in time and space. Calibration is a key technology which allows the central server to fuse the data generated by different location infrastructures, which can deliver improve the sensing range and detection robustness. Unfortunately, existing calibration algorithms often assume that the LiDARs are significantly overlapped or that the temporal calibration is already achieved. Since these assumptions do not always hold in the real world, the calibration results from the existing algorithms are often unsatisfactory and always need human involvement, which brings high labor costs. In this paper, we propose TrajMatch -- the first system that can automatically calibrate for roadside LiDARs in both time and space. The main idea is to automatically calibrate the sensors based on the result of the detection/tracking task instead of extracting special features. More deeply, we propose a mechanism for evaluating calibration parameters that is consistent with our algorithm, and we demonstrate the effectiveness of this scheme experimentally, which can also be used to guide parameter iterations for multiple calibration. Finally, to evaluate the performance of TrajMatch , we collect two dataset, one simulated dataset LiDARnet-sim 1.0 and a real-world dataset. Experiment results show that TrajMatch can achieve a spatial calibration error of less than 10cm and a temporal calibration error of less than 1.5ms.

cs.RO↗

arXiv2021

Robust Beamforming Design for Covert Communications

In this paper, we consider a common unicast beamforming network where Alice utilizes the communication to Carol as a cover and covertly transmits a message to Bob without being recognized by Willie. We investigate the beamformer design of Alice to maximize the covert rate to Bob when Alice has either perfect or imperfect knowledge about Willie's channel state information (WCSI). For the perfect WCSI case, the problem is formulated under the perfect covert constraint, and we develop a covert beamformer by applying semidefinite relaxation and the bisection method. Then, to reduce the computational complexity, we further propose a zero-forcing beamformer design with a single iteration processing. For the case of the imperfect WCSI, the robust beamformer is developed based on a relaxation and restriction approach by utilizing the property of Kullback-Leibler divergence. Furthermore, we derive the optimal decision threshold of Willie, and analyze the false alarm and the missed detection probabilities in this case. Finally, the performance of the proposed beamformer designs is evaluated through numerical experiments.

cs.IT↗

arXiv2017

Hybrid Beamforming via the Kronecker Decomposition for the Millimeter-Wave Massive MIMO Systems

Despite its promising performance gain, the realization of mmWave massive MIMO still faces several practical challenges. In particular, implementing massive MIMO in the digital domain requires hundreds of RF chains matching the number of antennas. Furthermore, designing these components to operate at the mmWave frequencies is challenging and costly. These motivated the recent development of hybrid-beamforming where MIMO processing is divided for separate implementation in the analog and digital domains, called the analog and digital beamforming, respectively. Analog beamforming using a phase array introduces uni-modulus constraints on the beamforming coefficients, rendering the conventional MIMO techniques unsuitable and call for new designs. In this paper, we present a systematic design framework for hybrid beamforming for multi-cell multiuser massive MIMO systems over mmWave channels characterized by sparse propagation paths. The framework relies on the decomposition of analog beamforming vectors and path observation vectors into Kronecker products of factors being uni-modulus vectors. Exploiting properties of Kronecker mixed products, different factors of the analog beamformer are designed for either nulling interference paths or coherently combining data paths. Furthermore, a channel estimation scheme is designed for enabling the proposed hybrid beamforming. The scheme estimates the AoA of data and interference paths by analog beam scanning and data-path gains by analog beam steering. The performance of the channel estimation scheme is analyzed. In particular, the AoA spectrum resulting from beam scanning, which displays the magnitude distribution of paths over the AoA range, is derived in closed-form. It is shown that the inter-cell interference level diminishes inversely with the array size, the square root of pilot sequence length and the spatial separation between paths.

cs.IT↗

arXiv2010

Detection of spin bias in four-terminal quantum-dot ring

In this work, we show that in a four-quantum-dot ring, via introducing a local Rashba spin-orbit interaction the spin bias in the transverse terminals can be detected by observing the charge currents in the longitudinal probes. It is found that due to the Rashba interaction, the quantum interference in this system becomes spin-dependent and the opposite-spin currents induced by the spin bias can present different magnitudes, so charge currents emerge. Besides, the charge currents rely on both the magnitude and spin polarization direction of the spin bias. It is believed that this method provides an electrical but practical scheme to detect the spin bias (or the spin current).

cond-mat.mes-hall↗