arXiv subjects

Hu Wang

Publications and source records attributed to Hu Wang.

At least 55 records · Page 3Linked to original sources

arXiv2024

Segment Beyond View: Handling Partially Missing Modality for Audio-Visual Semantic Segmentation

Augmented Reality (AR) devices, emerging as prominent mobile interaction platforms, face challenges in user safety, particularly concerning oncoming vehicles. While some solutions leverage onboard camera arrays, these cameras often have limited field-of-view (FoV) with front or downward perspectives. Addressing this, we propose a new out-of-view semantic segmentation task and Segment Beyond View (SBV), a novel audio-visual semantic segmentation method. SBV supplements the visual modality, which miss the information beyond FoV, with the auditory information using a teacher-student distillation model (Omni2Ego). The model consists of a vision teacher utilising panoramic information, an auditory teacher with 8-channel audio, and an audio-visual student that takes views with limited FoV and binaural audio as input and produce semantic segmentation for objects outside FoV. SBV outperforms existing models in comparative evaluations and shows a consistent performance across varying FoV ranges and in monaural audio settings.

cs.CV↗

arXiv2024

Unraveling Instance Associations: A Closer Look for Audio-Visual Segmentation

Audio-visual segmentation (AVS) is a challenging task that involves accurately segmenting sounding objects based on audio-visual cues. The effectiveness of audio-visual learning critically depends on achieving accurate cross-modal alignment between sound and visual objects. Successful audio-visual learning requires two essential components: 1) a challenging dataset with high-quality pixel-level multi-class annotated images associated with audio files, and 2) a model that can establish strong links between audio information and its corresponding visual object. However, these requirements are only partially addressed by current methods, with training sets containing biased audio-visual data, and models that generalise poorly beyond this biased training set. In this work, we propose a new cost-effective strategy to build challenging and relatively unbiased high-quality audio-visual segmentation benchmarks. We also propose a new informative sample mining method for audio-visual supervised contrastive learning to leverage discriminative contrastive samples to enforce cross-modal understanding. We show empirical results that demonstrate the effectiveness of our benchmark. Furthermore, experiments conducted on existing AVS datasets and on our new benchmark show that our method achieves state-of-the-art (SOTA) segmentation accuracy.

cs.CV↗

arXiv2024

Deformable Feature Alignment and Refinement for Moving Infrared Dim-small Target Detection

The detection of moving infrared dim-small targets has been a challenging and prevalent research topic. The current state-of-the-art methods are mainly based on ConvLSTM to aggregate information from adjacent frames to facilitate the detection of the current frame. However, these methods implicitly utilize motion information only in the training stage and fail to explicitly explore motion compensation, resulting in poor performance in the case of a video sequence including large motion. In this paper, we propose a Deformable Feature Alignment and Refinement (DFAR) method based on deformable convolution to explicitly use motion context in both the training and inference stages. Specifically, a Temporal Deformable Alignment (TDA) module based on the designed Dilated Convolution Attention Fusion (DCAF) block is developed to explicitly align the adjacent frames with the current frame at the feature level. Then, the feature refinement module adaptively fuses the aligned features and further aggregates useful spatio-temporal information by means of the proposed Attention-guided Deformable Fusion (AGDF) block. In addition, to improve the alignment of adjacent frames with the current frame, we extend the traditional loss function by introducing a new motion compensation loss. Extensive experimental results demonstrate that the proposed DFAR method achieves the state-of-the-art performance on two benchmark datasets including DAUB and IRDST.

cs.CV↗

arXiv2024

Multi-modal Learning with Missing Modality via Shared-Specific Feature Modelling

The missing modality issue is critical but non-trivial to be solved by multi-modal models. Current methods aiming to handle the missing modality problem in multi-modal tasks, either deal with missing modalities only during evaluation or train separate models to handle specific missing modality settings. In addition, these models are designed for specific tasks, so for example, classification models are not easily adapted to segmentation tasks and vice versa. In this paper, we propose the Shared-Specific Feature Modelling (ShaSpec) method that is considerably simpler and more effective than competing approaches that address the issues above. ShaSpec is designed to take advantage of all available input modalities during training and evaluation by learning shared and specific features to better represent the input data. This is achieved from a strategy that relies on auxiliary tasks based on distribution alignment and domain classification, in addition to a residual feature fusion procedure. Also, the design simplicity of ShaSpec enables its easy adaptation to multiple tasks, such as classification and segmentation. Experiments are conducted on both medical image segmentation and computer vision classification, with results indicating that ShaSpec outperforms competing methods by a large margin. For instance, on BraTS2018, ShaSpec improves the SOTA by more than 3% for enhancing tumour, 5% for tumour core and 3% for whole tumour. The code repository address is https://github.com/billhhh/ShaSpec/.

cs.CV↗

arXiv2024

Disentangling Specificity for Abstractive Multi-document Summarization

Multi-document summarization (MDS) generates a summary from a document set. Each document in a set describes topic-relevant concepts, while per document also has its unique contents. However, the document specificity receives little attention from existing MDS approaches. Neglecting specific information for each document limits the comprehensiveness of the generated summaries. To solve this problem, in this paper, we propose to disentangle the specific content from documents in one document set. The document-specific representations, which are encouraged to be distant from each other via a proposed orthogonal constraint, are learned by the specific representation learner. We provide extensive analysis and have interesting findings that specific information and document set representations contribute distinctive strengths and their combination yields a more comprehensive solution for the MDS. Also, we find that the common (i.e. shared) information could not contribute much to the overall performance under the MDS settings. Implemetation codes are available at https://github.com/congboma/DisentangleSum.

cs.IR↗

arXiv2023

Dynamically configured physics-informed neural network in topology optimization applications

Integration of machine learning (ML) into the topology optimization (TO) framework is attracting increasing attention, but data acquisition in data-driven models is prohibitive. Compared with popular ML methods, the physics-informed neural network (PINN) can avoid generating enormous amounts of data when solving forward problems and additionally provide better inference. To this end, a dynamically configured PINN-based topology optimization (DCPINN-TO) method is proposed. The DCPINN is composed of two subnetworks, namely the backbone neural network (NN) and the coefficient NN, where the coefficient NN has fewer trainable parameters. The designed architecture aims to dynamically configure trainable parameters; that is, an inexpensive NN is used to replace an expensive one at certain optimization cycles. Furthermore, an active sampling strategy is proposed to selectively sample collocations depending on the pseudo-densities at each optimization cycle. In this manner, the number of collocations will decrease with the optimization process but will hardly affect it. The Gaussian integral is used to calculate the strain energy of elements, which yields a byproduct of decoupling the mapping of the material at the collocations. Several examples with different resolutions validate the feasibility of the DCPINN-TO method, and multiload and multiconstraint problems are employed to illustrate its generalization. In addition, compared to finite element analysis-based TO (FEA-TO), the accuracy of the displacement prediction and optimization results indicate that the DCPINN-TO method is effective and efficient.

cs.LG↗

arXiv2023

Resolution-independent generative models based on operator learning for physics-constrained Bayesian inverse problems

The Bayesian inference approach is widely used to tackle inverse problems due to its versatile and natural ability to handle ill-posedness. However, it often faces challenges when dealing with situations involving continuous fields or large-resolution discrete representations (high-dimensional). Moreover, the prior distribution of unknown parameters is commonly difficult to be determined. In this study, an Operator Learning-based Generative Adversarial Network (OL-GAN) is proposed and integrated into the Bayesian inference framework to handle these issues. Unlike most Bayesian approaches, the distinctive characteristic of the proposed method is to learn the joint distribution of parameters and responses. By leveraging the trained generative model, the posteriors of the unknown parameters can theoretically be approximated by any sampling algorithm (e.g., Markov Chain Monte Carlo, MCMC) in a low-dimensional latent space shared by the components of the joint distribution. The latent space is typically a simple and easy-to-sample distribution (e.g., Gaussian, uniform), which significantly reduces the computational cost associated with the Bayesian inference while avoiding prior selection concerns. Furthermore, incorporating operator learning enables resolution-independent in the generator. Predictions can be obtained at desired coordinates, and inversions can be performed even if the observation data are misaligned with the training data. Finally, the effectiveness of the proposed method is validated through several numerical experiments.

math.NA↗

arXiv2023

A novel reduced basis method for adjoint sensitivity analysis of dynamic topology optimization

In gradient-based time domain topology optimization, design sensitivity analysis (DSA) of the dynamic response is essential, and requires high computational cost to directly differentiate, especially for high-order dynamic system. To address this issue, this study develops an efficient reduced basis method (RBM)-based discrete adjoint sensitivity analysis method, which on the one hand significantly improves the efficiency of sensitivity analysis and on the other hand avoids the consistency errors caused by the continuum method. In this algorithm, the basis functions of the adjoint problem are constructed in the offline phase based on the greedy-POD method, and a novel model-based estimation is developed to facilitate the acceleration of this process. Based on these basis functions, a fast and reasonably accurate model is then built by Galerkin projection for sensitivity analysis in each dynamic topology optimization iteration. Finally, the effectiveness of the error measures, the efficiency and the accuracy of the presented reduced-order method are verified by 2D and 3D dynamic structure studies.

math.NA↗

arXiv2023

BoMD: Bag of Multi-label Descriptors for Noisy Chest X-ray Classification

Deep learning methods have shown outstanding classification accuracy in medical imaging problems, which is largely attributed to the availability of large-scale datasets manually annotated with clean labels. However, given the high cost of such manual annotation, new medical imaging classification problems may need to rely on machine-generated noisy labels extracted from radiology reports. Indeed, many Chest X-ray (CXR) classifiers have already been modelled from datasets with noisy labels, but their training procedure is in general not robust to noisy-label samples, leading to sub-optimal models. Furthermore, CXR datasets are mostly multi-label, so current noisy-label learning methods designed for multi-class problems cannot be easily adapted. In this paper, we propose a new method designed for the noisy multi-label CXR learning, which detects and smoothly re-labels samples from the dataset, which is then used to train common multi-label classifiers. The proposed method optimises a bag of multi-label descriptors (BoMD) to promote their similarity with the semantic descriptors produced by BERT models from the multi-label image annotation. Our experiments on diverse noisy multi-label training sets and clean testing sets show that our model has state-of-the-art accuracy and robustness in many CXR multi-label classification benchmarks.

eess.IV↗

arXiv2023

Distilling Missing Modality Knowledge from Ultrasound for Endometriosis Diagnosis with Magnetic Resonance Images

Endometriosis is a common chronic gynecological disorder that has many characteristics, including the pouch of Douglas (POD) obliteration, which can be diagnosed using Transvaginal gynecological ultrasound (TVUS) scans and magnetic resonance imaging (MRI). TVUS and MRI are complementary non-invasive endometriosis diagnosis imaging techniques, but patients are usually not scanned using both modalities and, it is generally more challenging to detect POD obliteration from MRI than TVUS. To mitigate this classification imbalance, we propose in this paper a knowledge distillation training algorithm to improve the POD obliteration detection from MRI by leveraging the detection results from unpaired TVUS data. More specifically, our algorithm pre-trains a teacher model to detect POD obliteration from TVUS data, and it also pre-trains a student model with 3D masked auto-encoder using a large amount of unlabelled pelvic 3D MRI volumes. Next, we distill the knowledge from the teacher TVUS POD obliteration detector to train the student MRI model by minimizing a regression loss that approximates the output of the student to the teacher using unpaired TVUS and MRI data. Experimental results on our endometriosis dataset containing TVUS and MRI data demonstrate the effectiveness of our method to improve the POD detection accuracy from MRI.

eess.IV↗

arXiv2023

A novel multi-stage concurrent topology optimization for variable-stiffness structures

The concurrent optimization of topology and fibre orientation is a promising approach to pursue higher strength and lighter weight of variable-stiffness structure. This study proposes a novel discrete-continuous scheme for the concurrent optimization. Considering the global convergence, Discrete Material Optimization (DMO) is firstly utilized to select a dominant angle for each element from several predefined candidate angles. However, it is still difficult to obtain excellent fibre angle convergence due to difficulty in selection of candidate materials for some elements. Therefore, the Sequential Binary-Phase Topology Optimization is proposed to guarantee the uniqueness of the element to candidate angle mapping. Moreover, to obtain better mechanical properties, Continuous Fibre Angle Optimization (CFAO) with spatial filter is introduced to optimize fibre continuity. Several classic numerical examples are used to verify the excellent performance of the proposed method in terms of fibre angle convergence and stable optimization ability.

math.GT↗

arXiv2023

An efficient online successive reanalysis method for dynamic topology optimization

In this study, an efficient reanalysis strategy for dynamic topology optimization is proposed. Compared with other related studies, an online successive dynamic reanalysis method and POD-based approximate dynamic displacement strategy are integrated. In dynamic reanalysis, the storage of the stiffness matrix decomposition can be avoided and the reduced basis vectors should be updated successively according to the structural status in each iteration. Therefore, the bottleneck of combined approximation method for large-scale dynamic topology optimization can be handled. Sequentially, the Proper Orthogonal Decomposition (POD) is employed to obtain the approximate dynamic displacement, in which the Proper Orthogonal Mode (POM) of the displacement field is employed to establish the approximated equivalent static loads of Equivalent Static Load (ESL) method. Compared with the exact equivalent static loads at all the time intervals, the number of equivalent static loads is significantly reduced. Finally, the 2D and 3D test results indicate that the proposed method has remarkable speed-up effect on the premise of small relative error, support the strength of the proposed strategy.

math.NA↗

arXiv2023

Data Hiding with Deep Learning: A Survey Unifying Digital Watermarking and Steganography

The advancement of secure communication and identity verification fields has significantly increased through the use of deep learning techniques for data hiding. By embedding information into a noise-tolerant signal such as audio, video, or images, digital watermarking and steganography techniques can be used to protect sensitive intellectual property and enable confidential communication, ensuring that the information embedded is only accessible to authorized parties. This survey provides an overview of recent developments in deep learning techniques deployed for data hiding, categorized systematically according to model architectures and noise injection methods. The objective functions, evaluation metrics, and datasets used for training these data hiding models are comprehensively summarised. Additionally, potential future research directions that unite digital watermarking and steganography on software engineering to enhance security and mitigate risks are suggested and deliberated. This contribution furthers the creation of a more trustworthy digital world and advances Responsible AI.

cs.CV↗

arXiv2023

An efficient topology optimization method based on adaptive reanalysis with projection reduction

Efficient topology optimization based on the adaptive auxiliary reduced model reanalysis (AARMR) is proposed to improve computational efficiency and scale. In this method, a projection auxiliary reduced model (PARM) is integrated into the combined approximation reduced model (CARM) to reduce the dimension of the model in different aspects. First, the CARM restricts the solution space to avoid large matrix factorization. Second, the PARM is proposed to construct the CARM dynamically to save computational cost. Furthermore, the multi-grid conjugate gradient method is suggested to update PARM adaptively. Finally, several classic numerical examples are tested to show that the proposed method not only significantly improves computational efficiency, but also can solve large-scale problems that are difficult to solve by direct solvers due to the memory limitations.

cs.CE↗

arXiv2022

Robust Data Hiding Using Inverse Gradient Attention

Data hiding is the procedure of encoding desired information into a certain types of cover media (e.g. images) to resist potential noises for data recovery, while ensuring the embedded image has few perceptual perturbations. Recently, with the tremendous successes gained by deep neural networks in various fields, the research on data hiding with deep learning models has attracted an increasing amount of attentions. In deep data hiding models, to maximize the encoding capacity, each pixel of the cover image ought to be treated differently since they have different sensitivities w.r.t. visual quality. The neglecting to consider the sensitivity of each pixel inevitably affects the model's robustness for information hiding. In this paper, we propose a novel deep data hiding scheme with Inverse Gradient Attention (IGA), combining the idea of attention mechanism to endow different attention weights for different pixels. Equipped with the proposed modules, the model can spotlight pixels with more robustness for data hiding. Extensive experiments demonstrate that the proposed model outperforms the mainstream deep learning based data hiding methods on two prevalent datasets under multiple evaluation metrics. Besides, we further identify and discuss the connections between the proposed inverse gradient attention and high-frequency regions within images, which can serve as an informative reference to the deep data hiding research community. The codes are available at: https://github.com/hongleizhang/IGA.

cs.CV↗

arXiv2022

An efficient topology optimization based on multigrid assisted reanalysis for heat transfer problem

To improve the computational efficiency of heat transfer topology optimization, a Multigrid Assisted Reanalysis (MGAR) method is proposed in this study. The MGAR not only significantly improves the computational efficiency, but also relieves the hardware burden, and thus can efficiently solve large-scale heat transfer topology optimization problems. In addition, a projection-based post-processing strategy is also proposed and integrated with a continuous density filtering strategy to successfully obtain smooth boundary while eliminating some small-sized features. Several 2D and 3D numerical examples demonstrate that the computational efficiency of the MGAR is close to or even higher than that of the MGCG with almost identical optimization results, moreover, the efficiency improvement in the 3D scenario is superior than that of the 2D scenario, which reveals the excellent potential of the MGAR to save computational cost for large-scale problems.

cs.CE↗

arXiv2022

An E-PINN assisted practical uncertainty quantification for inverse problems

How to solve inverse problems is the challenge of many engineering and industrial applications. Recently, physics-informed neural networks (PINNs) have emerged as a powerful approach to solve inverse problems efficiently. However, it is difficult for PINNs to quantify the uncertainty of results. Therefore, this study proposed ensemble PINNs (E-PINNs) to handle this issue. The E-PINN uses ensemble statistics of several basic models to provide uncertainty quantifications for the inverse solution based on the PINN framework, and it is employed to solve the inverse problems in which the unknown quantity is propagated through partial differential equations (PDEs), especially the identification of the unknown field (e.g., space function) of a given physical system. Compared with other data-driven approaches, the suggested method is more than straightforward to implement, and also obtains high-quality uncertainty estimates of the quantity of interest (QoI) without significantly increasing the complexity of the algorithm. This work discusses the good properties of ensemble learning in field inversion and uncertainty quantification. The effectiveness of the proposed method is demonstrated through several numerical experiments. To enhance the robustness of models, adversarial training (AT) is applied. Furthermore, an adaptive active sampling (AS) strategy based on the uncertainty estimates from E-PINNs is also proposed to improve the accuracy of material field inversion problems.

math.NA↗

arXiv2022

Multi-view Local Co-occurrence and Global Consistency Learning Improve Mammogram Classification Generalisation

When analysing screening mammograms, radiologists can naturally process information across two ipsilateral views of each breast, namely the cranio-caudal (CC) and mediolateral-oblique (MLO) views. These multiple related images provide complementary diagnostic information and can improve the radiologist's classification accuracy. Unfortunately, most existing deep learning systems, trained with globally-labelled images, lack the ability to jointly analyse and integrate global and local information from these multiple views. By ignoring the potentially valuable information present in multiple images of a screening episode, one limits the potential accuracy of these systems. Here, we propose a new multi-view global-local analysis method that mimics the radiologist's reading procedure, based on a global consistency learning and local co-occurrence learning of ipsilateral views in mammograms. Extensive experiments show that our model outperforms competing methods, in terms of classification accuracy and generalisation, on a large-scale private dataset and two publicly available datasets, where models are exclusively trained and tested with global labels.

cs.CV↗