arXiv subjects

Weidong Yang

Publications and source records attributed to Weidong Yang.

52 records · Page 3Linked to original sources

arXiv2024

3D Gaussian as a New Era: A Survey

3D Gaussian Splatting (3D-GS) has emerged as a significant advancement in the field of Computer Graphics, offering explicit scene representation and novel view synthesis without the reliance on neural networks, such as Neural Radiance Fields (NeRF). This technique has found diverse applications in areas such as robotics, urban mapping, autonomous navigation, and virtual reality/augmented reality, just name a few. Given the growing popularity and expanding research in 3D Gaussian Splatting, this paper presents a comprehensive survey of relevant papers from the past year. We organize the survey into taxonomies based on characteristics and applications, providing an introduction to the theoretical underpinnings of 3D Gaussian Splatting. Our goal through this survey is to acquaint new researchers with 3D Gaussian Splatting, serve as a valuable reference for seminal works in the field, and inspire future research directions, as discussed in our concluding section.

cs.CV↗

arXiv2024

RefGaussian: Disentangling Reflections from 3D Gaussian Splatting for Realistic Rendering

3D Gaussian Splatting (3D-GS) has made a notable advancement in the field of neural rendering, 3D scene reconstruction, and novel view synthesis. Nevertheless, 3D-GS encounters the main challenge when it comes to accurately representing physical reflections, especially in the case of total reflection and semi-reflection that are commonly found in real-world scenes. This limitation causes reflections to be mistakenly treated as independent elements with physical presence, leading to imprecise reconstructions. Herein, to tackle this challenge, we propose RefGaussian to disentangle reflections from 3D-GS for realistically modeling reflections. Specifically, we propose to split a scene into transmitted and reflected components and represent these components using two Spherical Harmonics (SH). Given that this decomposition is not fully determined, we employ local regularization techniques to ensure local smoothness for both the transmitted and reflected components, thereby achieving more plausible decomposition outcomes than 3D-GS. Experimental results demonstrate that our approach achieves superior novel view synthesis and accurate depth estimation outcomes. Furthermore, it enables the utilization of scene editing applications, ensuring both high-quality results and physical coherence.

cs.CV↗

arXiv2024

Lightening Anything in Medical Images

The development of medical imaging techniques has made a significant contribution to clinical decision-making. However, the existence of suboptimal imaging quality, as indicated by irregular illumination or imbalanced intensity, presents significant obstacles in automating disease screening, analysis, and diagnosis. Existing approaches for natural image enhancement are mostly trained with numerous paired images, presenting challenges in data collection and training costs, all while lacking the ability to generalize effectively. Here, we introduce a pioneering training-free Diffusion Model for Universal Medical Image Enhancement, named UniMIE. UniMIE demonstrates its unsupervised enhancement capabilities across various medical image modalities without the need for any fine-tuning. It accomplishes this by relying solely on a single pre-trained model from ImageNet. We conduct a comprehensive evaluation on 13 imaging modalities and over 15 medical types, demonstrating better qualities, robustness, and accuracy than other modality-specific and data-inefficient models. By delivering high-quality enhancement and corresponding accuracy downstream tasks across a wide range of tasks, UniMIE exhibits considerable potential to accelerate the advancement of diagnostic tools and customized treatment plans.

eess.IV↗

arXiv2024

Towards Unified Representation of Multi-Modal Pre-training for 3D Understanding via Differentiable Rendering

State-of-the-art 3D models, which excel in recognition tasks, typically depend on large-scale datasets and well-defined category sets. Recent advances in multi-modal pre-training have demonstrated potential in learning 3D representations by aligning features from 3D shapes with their 2D RGB or depth counterparts. However, these existing frameworks often rely solely on either RGB or depth images, limiting their effectiveness in harnessing a comprehensive range of multi-modal data for 3D applications. To tackle this challenge, we present DR-Point, a tri-modal pre-training framework that learns a unified representation of RGB images, depth images, and 3D point clouds by pre-training with object triplets garnered from each modality. To address the scarcity of such triplets, DR-Point employs differentiable rendering to obtain various depth images. This approach not only augments the supply of depth images but also enhances the accuracy of reconstructed point clouds, thereby promoting the representative learning of the Transformer backbone. Subsequently, using a limited number of synthetically generated triplets, DR-Point effectively learns a 3D representation space that aligns seamlessly with the RGB-Depth image space. Our extensive experiments demonstrate that DR-Point outperforms existing self-supervised learning methods in a wide range of downstream tasks, including 3D object classification, part segmentation, point cloud completion, semantic segmentation, and detection. Additionally, our ablation studies validate the effectiveness of DR-Point in enhancing point cloud understanding.

cs.MM↗

arXiv2024

GetMesh: A Controllable Model for High-quality Mesh Generation and Manipulation

Mesh is a fundamental representation of 3D assets in various industrial applications, and is widely supported by professional softwares. However, due to its irregular structure, mesh creation and manipulation is often time-consuming and labor-intensive. In this paper, we propose a highly controllable generative model, GetMesh, for mesh generation and manipulation across different categories. By taking a varying number of points as the latent representation, and re-organizing them as triplane representation, GetMesh generates meshes with rich and sharp details, outperforming both single-category and multi-category counterparts. Moreover, it also enables fine-grained control over the generation process that previous mesh generative models cannot achieve, where changing global/local mesh topologies, adding/removing mesh parts, and combining mesh parts across categories can be intuitively, efficiently, and robustly accomplished by adjusting the number, positions or features of latent points. Project page is https://getmesh.github.io.

cs.CV↗

arXiv2024

Visual Foundation Models Boost Cross-Modal Unsupervised Domain Adaptation for 3D Semantic Segmentation

Unsupervised domain adaptation (UDA) is vital for alleviating the workload of labeling 3D point cloud data and mitigating the absence of labels when facing a newly defined domain. Various methods of utilizing images to enhance the performance of cross-domain 3D segmentation have recently emerged. However, the pseudo labels, which are generated from models trained on the source domain and provide additional supervised signals for the unseen domain, are inadequate when utilized for 3D segmentation due to their inherent noisiness and consequently restrict the accuracy of neural networks. With the advent of 2D visual foundation models (VFMs) and their abundant knowledge prior, we propose a novel pipeline VFMSeg to further enhance the cross-modal unsupervised domain adaptation framework by leveraging these models. In this work, we study how to harness the knowledge priors learned by VFMs to produce more accurate labels for unlabeled target domains and improve overall performance. We first utilize a multi-modal VFM, which is pre-trained on large scale image-text pairs, to provide supervised labels (VFM-PL) for images and point clouds from the target domain. Then, another VFM trained on fine-grained 2D masks is adopted to guide the generation of semantically augmented images and point clouds to enhance the performance of neural networks, which mix the data from source and target domains like view frustums (FrustumMixing). Finally, we merge class-wise prediction across modalities to produce more accurate annotations for unlabeled target domains. Our method is evaluated on various autonomous driving datasets and the results demonstrate a significant improvement for 3D segmentation task.

cs.CV↗

arXiv2024

Topology-Aware Latent Diffusion for 3D Shape Generation

We introduce a new generative model that combines latent diffusion with persistent homology to create 3D shapes with high diversity, with a special emphasis on their topological characteristics. Our method involves representing 3D shapes as implicit fields, then employing persistent homology to extract topological features, including Betti numbers and persistence diagrams. The shape generation process consists of two steps. Initially, we employ a transformer-based autoencoding module to embed the implicit representation of each 3D shape into a set of latent vectors. Subsequently, we navigate through the learned latent space via a diffusion model. By strategically incorporating topological features into the diffusion process, our generative module is able to produce a richer variety of 3D shapes with different topological structures. Furthermore, our framework is flexible, supporting generation tasks constrained by a variety of inputs, including sparse and partial point clouds, as well as sketches. By modifying the persistence diagrams, we can alter the topology of the shapes generated from these input modalities.

cs.CV↗

arXiv2023

Blockchain-Based Identity Authentication Oriented to Multi-Cluster UAV Networking

Unmanned Aerial Vehicle (UAV) networking is increasingly used in field environments such as power inspection, agricultural plant protection, and emergency rescue. To guarantee UAV networking security, UAV identity authentication attracts wide attention, especially in the field environment without perfect infrastructure. Some blockchain-based UAV identity authentication solutions are proposed to establish decentralized and trusted authentication systems without relying on infrastructure. However, these solutions do not support disconnected UAV reconnection or even disband a cluster directly after its head UAV disconnection, which compromises cluster robustness and task result integrity. In this paper, we propose a blockchain-based identity authentication solution oriented to multi-cluster UAV networking with a UAV disconnection mechanism and a task result backup mechanism. Specifically, we build a blockchain maintained by head UAVs of all clusters, managing identity information to guarantee the security of decentralized identity management. The UAV disconnection mechanism permits a verified distributed UAV reconnection to ensure the robustness of the UAV cluster, and on this basis, the task result backup mechanism ensures the integrity of the task results stored in a cluster even any UAV disconnection. Finally, extensive experimental results prove the superiority of our solutions in terms of robustness, integrity, delay, and energy consumption.

cs.CR↗

arXiv2023

Self-supervised Learning for Pre-Training 3D Point Clouds: A Survey

Point cloud data has been extensively studied due to its compact form and flexibility in representing complex 3D structures. The ability of point cloud data to accurately capture and represent intricate 3D geometry makes it an ideal choice for a wide range of applications, including computer vision, robotics, and autonomous driving, all of which require an understanding of the underlying spatial structures. Given the challenges associated with annotating large-scale point clouds, self-supervised point cloud representation learning has attracted increasing attention in recent years. This approach aims to learn generic and useful point cloud representations from unlabeled data, circumventing the need for extensive manual annotations. In this paper, we present a comprehensive survey of self-supervised point cloud representation learning using DNNs. We begin by presenting the motivation and general trends in recent research. We then briefly introduce the commonly used datasets and evaluation metrics. Following that, we delve into an extensive exploration of self-supervised point cloud representation learning methods based on these techniques. Finally, we share our thoughts on some of the challenges and potential issues that future research in self-supervised learning for pre-training 3D point clouds may encounter.

cs.CV↗

arXiv2023

Generative Diffusion Prior for Unified Image Restoration and Enhancement

Existing image restoration methods mostly leverage the posterior distribution of natural images. However, they often assume known degradation and also require supervised training, which restricts their adaptation to complex real applications. In this work, we propose the Generative Diffusion Prior (GDP) to effectively model the posterior distributions in an unsupervised sampling manner. GDP utilizes a pre-train denoising diffusion generative model (DDPM) for solving linear inverse, non-linear, or blind problems. Specifically, GDP systematically explores a protocol of conditional guidance, which is verified more practical than the commonly used guidance way. Furthermore, GDP is strength at optimizing the parameters of degradation model during the denoising process, achieving blind image restoration. Besides, we devise hierarchical guidance and patch-based methods, enabling the GDP to generate images of arbitrary resolutions. Experimentally, we demonstrate GDP's versatility on several image datasets for linear problems, such as super-resolution, deblurring, inpainting, and colorization, as well as non-linear and blind issues, such as low-light enhancement and HDR image recovery. GDP outperforms the current leading unsupervised methods on the diverse benchmarks in reconstruction quality and perceptual quality. Moreover, GDP also generalizes well for natural images or synthesized images with arbitrary sizes from various tasks out of the distribution of the ImageNet training set.

cs.CV↗

arXiv2023

UniDA3D: Unified Domain Adaptive 3D Semantic Segmentation Pipeline

State-of-the-art 3D semantic segmentation models are trained on off-the-shelf public benchmarks, but they will inevitably face the challenge of recognition accuracy drop when these well-trained models are deployed to a new domain. In this paper, we introduce a Unified Domain Adaptive 3D semantic segmentation pipeline (UniDA3D) to enhance the weak generalization ability, and bridge the point distribution gap between domains. Different from previous studies that only focus on a single adaptation task, UniDA3D can tackle several adaptation tasks in 3D segmentation field, by designing a unified source-and-target active sampling strategy, which selects a maximally-informative subset from both source and target domains for effective model adaptation. Besides, benefiting from the rise of multi-modal 2D-3D datasets, UniDA3D investigates the possibility of achieving a multi-modal sampling strategy, by developing a cross-modality feature interaction module that can extract a representative pair of image and point features to achieve a bi-directional image-point feature interaction for safe model adaptation. Experimentally, UniDA3D is verified to be effective in many adaptation tasks including: 1) unsupervised domain adaptation, 2) unsupervised few-shot domain adaptation; 3) active domain adaptation. Their results demonstrate that, by easily coupling UniDA3D with off-the-shelf 3D segmentation baselines, domain generalization ability of these baselines can be enhanced.

cs.CV↗

arXiv2022

Comprehensive Review of Deep Learning-Based 3D Point Cloud Completion Processing and Analysis

Point cloud completion is a generation and estimation issue derived from the partial point clouds, which plays a vital role in the applications in 3D computer vision. The progress of deep learning (DL) has impressively improved the capability and robustness of point cloud completion. However, the quality of completed point clouds is still needed to be further enhanced to meet the practical utilization. Therefore, this work aims to conduct a comprehensive survey on various methods, including point-based, convolution-based, graph-based, and generative model-based approaches, etc. And this survey summarizes the comparisons among these methods to provoke further research insights. Besides, this review sums up the commonly used datasets and illustrates the applications of point cloud completion. Eventually, we also discussed possible research trends in this promptly expanding field.

cs.CV↗

arXiv2020

Identifying the elastic isotropy of architectured materials based on deep learning method

With the achievement on the additive manufacturing, the mechanical properties of architectured materials can be precisely designed by tailoring microstructures. As one of the primary design objectives, the elastic isotropy is of great significance for many engineering applications. However, the prevailing experimental and numerical methods are normally too costly and time-consuming to determine the elastic isotropy of architectured materials with tens of thousands of possible microstructures in design space. The quick mechanical characterization is thus desired for the advanced design of architectured materials. Here, a deep learning-based approach is developed as a portable and efficient tool to identify the elastic isotropy of architectured materials directly from the images of their representative microstructures with arbitrary component distributions. The measure of elastic isotropy for architectured materials is derived firstly in this paper to construct a database with associated images of microstructures. Then a convolutional neural network is trained with the database. It is found that the convolutional neural network shows good performance on the isotropy identification. Meanwhile, it exhibits enough robustness to maintain the performance under fluctuated material properties in practical fabrications. Moreover, the well-trained convolutional neural network can be successfully transferred among different types of architectured materials, including two-phase composites and porous materials, which greatly enhance the efficiency of the deep learning-based approach. This study can give new inspirations on the fast mechanical characterization for the big-data driven design of architectured materials.

physics.app-ph↗

arXiv2014

Radar In-Band and Out-of-Band Interference into LTE Macro and Small Cell Uplinks in the 3.5 GHz Band

National Telecommunications and Information Administration (NTIA) has proposed vast exclusions zones between radar and Worldwide Interoperability for Microwave Access (WiMAX) (WiMAX) systems which are also being considered as geographic separations between radars and 3.5 GHz Long Term Evolution (LTE) systems without investigating any changes induced by the distinct nature of LTE as opposed to WiMAX. This paper performs a detailed system-level analysis of the interference effects from shipborne radar systems into LTE systems. Even though the results reveal impacts of radar interference on LTE systems performance, they provide clear indications of conspicuously narrower exclusion zones for LTE vis-à-vis those for WiMAX and pave the way toward deploying LTE at 3.5 GHz within the coastline populous areas.

cs.NI↗

arXiv2014

Radar in-Band Interference Effects on Macrocell LTE Uplink Deployments in the U.S. 3.5 GHz Band

National Telecommunications and Information Administration (NTIA) has proposed vast exclusions zones between radar and Worldwide Interoperability for Microwave Access (WiMAX) systems which are also being considered as geographic separations between radars and 3.5 GHz Long Term Evolution (LTE) systems without investigating any changes induced by the distinct nature of LTE as opposed to WiMAX. This paper performs a detailed system-level analysis of the interference effects from shipborne radar systems into LTE systems. Even though the results reveal impacts of radar interference on LTE systems performance, they provide clear indications of conspicuously narrower exclusion zones for LTE vis-à-vis those of WiMAX and pave the way toward deploying LTE at 3.5 GHz within the coastline populous areas.

cs.NI↗

arXiv1998

Metal-insulator transition in VO$_{2}$: a Peierls-Mott-Hubbard mechanism

The electronic structure of VO$_2$ is studied in the frameworks of local density approximation (LDA) and LDA+$U$ to give a quantitative description of the metal-insulator (MI) transition in this system. It is found that, both structural distortion and the local Coulomb interaction, play important roles in the transition. An optical gap, comparable to the experimental value has been obtained in the monoclinic structure by using the LDA+$U$ method. Based on our results, we believe that both, the Peierls and the Mott-Hubbard mechanism, are essential for a description of the MI transition in this system.

cond-mat.mtrl-sci↗