arXiv subjects

Zhihua Wang

Publications and source records attributed to Zhihua Wang.

At least 55 records · Page 3Linked to original sources

arXiv2024

EndoFinder: Online Image Retrieval for Explainable Colorectal Polyp Diagnosis

Determining the necessity of resecting malignant polyps during colonoscopy screen is crucial for patient outcomes, yet challenging due to the time-consuming and costly nature of histopathology examination. While deep learning-based classification models have shown promise in achieving optical biopsy with endoscopic images, they often suffer from a lack of explainability. To overcome this limitation, we introduce EndoFinder, a content-based image retrieval framework to find the 'digital twin' polyp in the reference database given a newly detected polyp. The clinical semantics of the new polyp can be inferred referring to the matched ones. EndoFinder pioneers a polyp-aware image encoder that is pre-trained on a large polyp dataset in a self-supervised way, merging masked image modeling with contrastive learning. This results in a generic embedding space ready for different downstream clinical tasks based on image retrieval. We validate the framework on polyp re-identification and optical biopsy tasks, with extensive experiments demonstrating that EndoFinder not only achieves explainable diagnostics but also matches the performance of supervised classification models. EndoFinder's reliance on image retrieval has the potential to support diverse downstream decision-making tasks during real-time colonoscopy procedures.

cs.CV↗

arXiv2024

Multiscale Sliced Wasserstein Distances as Perceptual Color Difference Measures

Contemporary color difference (CD) measures for photographic images typically operate by comparing co-located pixels, patches in a ``perceptually uniform'' color space, or features in a learned latent space. Consequently, these measures inadequately capture the human color perception of misaligned image pairs, which are prevalent in digital photography (e.g., the same scene captured by different smartphones). In this paper, we describe a perceptual CD measure based on the multiscale sliced Wasserstein distance, which facilitates efficient comparisons between non-local patches of similar color and structure. This aligns with the modern understanding of color perception, where color and structure are inextricably interdependent as a unitary process of perceptual organization. Meanwhile, our method is easy to implement and training-free. Experimental results indicate that our CD measure performs favorably in assessing CDs in photographic images, and consistently surpasses competing models in the presence of image misalignment. Additionally, we empirically verify that our measure functions as a metric in the mathematical sense, and show its promise as a loss function for image and video color transfer tasks. The code is available at https://github.com/real-hjq/MS-SWD.

cs.CV↗

arXiv2024

THQA: A Perceptual Quality Assessment Database for Talking Heads

In the realm of media technology, digital humans have gained prominence due to rapid advancements in computer technology. However, the manual modeling and control required for the majority of digital humans pose significant obstacles to efficient development. The speech-driven methods offer a novel avenue for manipulating the mouth shape and expressions of digital humans. Despite the proliferation of driving methods, the quality of many generated talking head (TH) videos remains a concern, impacting user visual experiences. To tackle this issue, this paper introduces the Talking Head Quality Assessment (THQA) database, featuring 800 TH videos generated through 8 diverse speech-driven methods. Extensive experiments affirm the THQA database's richness in character and speech features. Subsequent subjective quality assessment experiments analyze correlations between scoring results and speech-driven methods, ages, and genders. In addition, experimental results show that mainstream image and video quality assessment methods have limitations for the THQA database, underscoring the imperative for further research to enhance TH video quality assessment. The THQA database is publicly accessible at https://github.com/zyj-2000/THQA.

cs.CV↗

arXiv2024

LightSleepNet: Design of a Personalized Portable Sleep Staging System Based on Single-Channel EEG

This paper proposed LightSleepNet - a light-weight, 1-d Convolutional Neural Network (CNN) based personalized architecture for real-time sleep staging, which can be implemented on various mobile platforms with limited hardware resources. The proposed architecture only requires an input of 30s single-channel EEG signal for the classification. Two residual blocks consisting of group 1-d convolution are used instead of the traditional convolution layers to remove the redundancy in the CNN. Channel shuffles are inserted into each convolution layer to improve the accuracy. In order to avoid over-fitting to the training set, a Global Average Pooling (GAP) layer is used to replace the fully connected layer, which further reduces the total number of the model parameters significantly. A personalized algorithm combining Adaptive Batch Normalization (AdaBN) and gradient re-weighting is proposed for unsupervised domain adaptation. A higher priority is given to examples that are easy to transfer to the new subject, and the algorithm could be personalized for new subjects without re-training. Experimental results show a state-of-the-art overall accuracy of 83.8% with only 45.76 Million Floating-point Operations per Second (MFLOPs) computation and 43.08 K parameters.

eess.SP↗

arXiv2024

Multi-Channel Multi-Domain based Knowledge Distillation Algorithm for Sleep Staging with Single-Channel EEG

This paper proposed a Multi-Channel Multi-Domain (MCMD) based knowledge distillation algorithm for sleep staging using single-channel EEG. Both knowledge from different domains and different channels are learnt in the proposed algorithm, simultaneously. A multi-channel pre-training and single-channel fine-tuning scheme is used in the proposed work. The knowledge from different channels in the source domain is transferred to the single-channel model in the target domain. A pre-trained teacher-student model scheme is used to distill knowledge from the multi-channel teacher model to the single-channel student model combining with output transfer and intermediate feature transfer in the target domain. The proposed algorithm achieves a state-of-the-art single-channel sleep staging accuracy of 86.5%, with only 0.6% deterioration from the state-of-the-art multi-channel model. There is an improvement of 2% compared to the baseline model. The experimental results show that knowledge from multiple domains (different datasets) and multiple channels (e.g. EMG, EOG) could be transferred to single-channel sleep staging.

eess.SP↗

arXiv2023

Variability of Radar Backscattering from Bare Soil Disturbed by Surface Parameters Uncertainties

Surface roughness and dielectric properties are crucial in characterizing radar backscattering from bare soil surfaces. However, their estimation depends on the surface size of the sampling profile, and the complex relative permittivity is disturbed by different dielectric spectrum models. Hence, it is desirable to know how the uncertainty associated with roughness and complex relative permittivity propagates to radar backscattering coefficients. To identify the extent to which the uncertainty propagates, we examined the roughness sample variance and bias of complex relative permittivity. Then, we evaluated the error of radar backscattering coefficients as a function of incident angle, frequency, and polarization induced by each of the two uncertainty sources and their coupling. The results help interpret the discrepancy among model predictions and in-situ measurements and suggest a minimum surface size to estimate the RMS height and correlation length to confine the radar backscattering coefficients' error.

physics.app-ph↗

arXiv2023

RetroOOD: Understanding Out-of-Distribution Generalization in Retrosynthesis Prediction

Machine learning-assisted retrosynthesis prediction models have been gaining widespread adoption, though their performances oftentimes degrade significantly when deployed in real-world applications embracing out-of-distribution (OOD) molecules or reactions. Despite steady progress on standard benchmarks, our understanding of existing retrosynthesis prediction models under the premise of distribution shifts remains stagnant. To this end, we first formally sort out two types of distribution shifts in retrosynthesis prediction and construct two groups of benchmark datasets. Next, through comprehensive experiments, we systematically compare state-of-the-art retrosynthesis prediction models on the two groups of benchmarks, revealing the limitations of previous in-distribution evaluation and re-examining the advantages of each model. More remarkably, we are motivated by the above empirical insights to propose two model-agnostic techniques that can improve the OOD generalization of arbitrary off-the-shelf retrosynthesis prediction algorithms. Our preliminary experiments show their high potential with an average performance improvement of 4.6%, and the established benchmarks serve as a foothold for further retrosynthesis prediction research towards OOD generalization.

cs.LG↗

arXiv2023

Measuring Perceptual Color Differences of Smartphone Photographs

Measuring perceptual color differences (CDs) is of great importance in modern smartphone photography. Despite the long history, most CD measures have been constrained by psychophysical data of homogeneous color patches or a limited number of simplistic natural photographic images. It is thus questionable whether existing CD measures generalize in the age of smartphone photography characterized by greater content complexities and learning-based image signal processors. In this paper, we put together so far the largest image dataset for perceptual CD assessment, in which the photographic images are 1) captured by six flagship smartphones, 2) altered by Photoshop, 3) post-processed by built-in filters of the smartphones, and 4) reproduced with incorrect color profiles. We then conduct a large-scale psychophysical experiment to gather perceptual CDs of 30,000 image pairs in a carefully controlled laboratory environment. Based on the newly established dataset, we make one of the first attempts to construct an end-to-end learnable CD formula based on a lightweight neural network, as a generalization of several previous metrics. Extensive experiments demonstrate that the optimized formula outperforms 33 existing CD measures by a large margin, offers reasonable local CD maps without the use of dense supervision, generalizes well to homogeneous color patch data, and empirically behaves as a proper metric in the mathematical sense. Our dataset and code are publicly available at https://github.com/hellooks/CDNet.

cs.CV↗

arXiv2023

Learning a Deep Color Difference Metric for Photographic Images

Most well-established and widely used color difference (CD) metrics are handcrafted and subject-calibrated against uniformly colored patches, which do not generalize well to photographic images characterized by natural scene complexities. Constructing CD formulae for photographic images is still an active research topic in imaging/illumination, vision science, and color science communities. In this paper, we aim to learn a deep CD metric for photographic images with four desirable properties. First, it well aligns with the observations in vision science that color and form are linked inextricably in visual cortical processing. Second, it is a proper metric in the mathematical sense. Third, it computes accurate CDs between photographic images, differing mainly in color appearances. Fourth, it is robust to mild geometric distortions (e.g., translation or due to parallax), which are often present in photographic images of the same scene captured by different digital cameras. We show that all these properties can be satisfied at once by learning a multi-scale autoregressive normalizing flow for feature transform, followed by the Euclidean distance which is linearly proportional to the human perceptual CD. Quantitative and qualitative experiments on the large-scale SPCD dataset demonstrate the promise of the learned CD metric.

cs.CV↗

arXiv2022

Learning Instrumental Variable from Data Fusion for Treatment Effect Estimation

The advent of the big data era brought new opportunities and challenges to draw treatment effect in data fusion, that is, a mixed dataset collected from multiple sources (each source with an independent treatment assignment mechanism). Due to possibly omitted source labels and unmeasured confounders, traditional methods cannot estimate individual treatment assignment probability and infer treatment effect effectively. Therefore, we propose to reconstruct the source label and model it as a Group Instrumental Variable (GIV) to implement IV-based Regression for treatment effect estimation. In this paper, we conceptualize this line of thought and develop a unified framework (Meta-EM) to (1) map the raw data into a representation space to construct Linear Mixed Models for the assigned treatment variable; (2) estimate the distribution differences and model the GIV for the different treatment assignment mechanisms; and (3) adopt an alternating training strategy to iteratively optimize the representations and the joint distribution to model GIV for IV regression. Empirical results demonstrate the advantages of our Meta-EM compared with state-of-the-art methods.

cs.AI↗

arXiv2022

ConfounderGAN: Protecting Image Data Privacy with Causal Confounder

The success of deep learning is partly attributed to the availability of massive data downloaded freely from the Internet. However, it also means that users' private data may be collected by commercial organizations without consent and used to train their models. Therefore, it's important and necessary to develop a method or tool to prevent unauthorized data exploitation. In this paper, we propose ConfounderGAN, a generative adversarial network (GAN) that can make personal image data unlearnable to protect the data privacy of its owners. Specifically, the noise produced by the generator for each image has the confounder property. It can build spurious correlations between images and labels, so that the model cannot learn the correct mapping from images to labels in this noise-added dataset. Meanwhile, the discriminator is used to ensure that the generated noise is small and imperceptible, thereby remaining the normal utility of the encrypted image for humans. The experiments are conducted in six image classification datasets, consisting of three natural object datasets and three medical datasets. The results demonstrate that our method not only outperforms state-of-the-art methods in standard settings, but can also be applied to fast encryption scenarios. Moreover, we show a series of transferability and stability experiments to further illustrate the effectiveness and superiority of our method.

cs.CV↗

arXiv2022

The Grothendieck algebras of certain smash product semisimple Hopf algebras

Let $H$ be a semisimple Hopf algebra over an algebraically closed field $\mathbbm{k}$ of characteristic $p>\dim_{\mathbbm{k}}(H)^{1/2}$ and $p\nmid 2\dim_{\mathbbm{k}}(H)$. In this paper, we consider the smash product semisimple Hopf algebra $H\#\mathbbm{k}G$, where $G$ is a cyclic group of order $n:=2\dim_{\mathbbm{k}}(H)$. Using irreducible representations of $H$ and those of $\mathbbm{k}G$, we determine all non-isomorphic irreducible representations of $H\#\mathbbm{k}G$. There is a close relationship between the Grothendieck algebra $(G_0(H\#\mathbbm{k}G)\otimes_{\mathbb{Z}}\mathbbm{k},*)$ of $H\#\mathbbm{k}G$ and the Grothendieck algebra $(G_0(H)\otimes_{\mathbb{Z}}\mathbbm{k},*)$ of $H$. To establish this connection, we endow with a new multiplication operator $\star$ on $G_0(H)\otimes_{\mathbb{Z}}\mathbbm{k}$ and show that the Grothendieck algebra $(G_0(H\#\mathbbm{k}G)\otimes_{\mathbb{Z}}\mathbbm{k},\ast)$ is isomorphic to the direct sum of $(G_0(H)\otimes_{\mathbb{Z}}\mathbbm{k},*)^{\oplus\frac{n}{2}}$ and $(G_0(H)\otimes_{\mathbb{Z}}\mathbbm{k},\star)^{\oplus\frac{n}{2}}$.

math.RT↗

arXiv2022

Higher Frobenius-Schur indicators for semisimple Hopf algebras in positive characteristic

Let $H$ be a semisimple Hopf algebra over an algebraically closed field $\mathbbm{k}$ of characteristic $p>\dim_{\mathbbm{k}}(H)^{1/2}$. We show that the antipode $S$ of $H$ satisfies the equality $S^2(h)=\mathbf{u}h\mathbf{u}^{-1}$, where $h\in H$, $\mathbf{u}=S(Λ_{(2)})Λ_{(1)}$ and $Λ$ is a nonzero integral of $H$. The formula of $S^2$ enables us to define higher Frobenius-Schur indicators for the Hopf algebra $H$. This generalizes the notions of higher Frobenius-Schur indicators from the case of characteristic 0 to the case of characteristic $p>\dim_{\mathbbm{k}}(H)^{1/2}$. These indicators defined here share some properties with the ones defined over a field of characteristic 0. Especially, all these indicators are gauge invariants for the tensor category Rep$(H)$ of finite dimensional representations of $H$.

math.RT↗

arXiv2022

Invariants from the Sweedler power maps on integrals

For a finite-dimensional Hopf algebra $A$ with a nonzero left integral $Λ$, we investigate a relationship between $P_n(Λ)$ and $P_n^J(Λ)$, where $P_n$ and $P_n^J$ are respectively the $n$-th Sweedler power maps of $A$ and the twisted Hopf algebra $A^J$. We use this relation to give several invariants of the representation category Rep$(A)$ considered as a tensor category. As applications, we distinguish the representation categories of 12-dimensional pointed nonsemisimple Hopf algebras. Also, these invariants are sufficient to distinguish the representation categories Rep$(K_8)$, Rep$(\kk Q_8)$ and Rep$(\kk D_4)$, although they have been completely distinguished by their Frobenius-Schur indicators. We further reveal a relationship between the right integrals $λ$ in $A^*$ and $λ^J$ in $(A^J)^*$. This can be used to give a uniform proof of the remarkable result which says that the $n$-th indicator $ν_n(A)$ is a gauge invariant of $A$ for any $n\in \mathbb{Z}$. We also use the expression for $λ^J$ to give an alternative proof of the known result that the Killing form of the Hopf algebra $A$ is invariant under twisting. As a result, the dimension of the Killing radical of $A$ is a gauge invariant of $A$.

math.QA↗

arXiv2021

Fusion categories containing a fusion subcategory with maximal rank

In this paper, we study fusion categories which contain a proper fusion subcategory with maximal rank. They can be viewed as generalizations of near-group fusion categories. We first prove that they admit spherical structure. We then classify those which are non-degenerate or symmetric. Finally, we classify such fusion categories of rank 4.

math.QA↗

arXiv2021

Learning from Synthetic Data for Opinion-free Blind Image Quality Assessment in the Wild

Nowadays, most existing blind image quality assessment (BIQA) models 1) are developed for synthetically-distorted images and often generalize poorly to authentic ones; 2) heavily rely on human ratings, which are prohibitively labor-expensive to collect. Here, we propose an $opinion$-$free$ BIQA method that learns from synthetically-distorted images and multiple agents to assess the perceptual quality of authentically-distorted ones captured in the wild without relying on human labels. Specifically, we first assemble a large number of image pairs from synthetically-distorted images and use a set of full-reference image quality assessment (FR-IQA) models to assign pseudo-binary labels of each pair indicating which image has higher quality as the supervisory signal. We then train a convolutional neural network (CNN)-based BIQA model to rank the perceptual quality, optimized for consistency with the binary labels. Since there exists domain shift between the synthetically- and authentically-distorted images, an unsupervised domain adaptation (UDA) module is introduced to alleviate this issue. Extensive experiments demonstrate the effectiveness of our proposed $opinion$-$free$ BIQA model, yielding state-of-the-art performance in terms of correlation with human opinion scores, as well as gMAD competition. Codes will be made publicly available upon acceptance.

cs.MM↗

arXiv2021

Learning Semantic Segmentation of Large-Scale Point Clouds with Random Sampling

We study the problem of efficient semantic segmentation of large-scale 3D point clouds. By relying on expensive sampling techniques or computationally heavy pre/post-processing steps, most existing approaches are only able to be trained and operate over small-scale point clouds. In this paper, we introduce RandLA-Net, an efficient and lightweight neural architecture to directly infer per-point semantics for large-scale point clouds. The key to our approach is to use random point sampling instead of more complex point selection approaches. Although remarkably computation and memory efficient, random sampling can discard key features by chance. To overcome this, we introduce a novel local feature aggregation module to progressively increase the receptive field for each 3D point, thereby effectively preserving geometric details. Comparative experiments show that our RandLA-Net can process 1 million points in a single pass up to 200x faster than existing approaches. Moreover, extensive experiments on five large-scale point cloud datasets, including Semantic3D, SemanticKITTI, Toronto3D, NPM3D and S3DIS, demonstrate the state-of-the-art semantic segmentation performance of our RandLA-Net.

cs.CV↗

arXiv2021

Semi-Supervised Deep Ensembles for Blind Image Quality Assessment

Ensemble methods are generally regarded to be better than a single model if the base learners are deemed to be "accurate" and "diverse." Here we investigate a semi-supervised ensemble learning strategy to produce generalizable blind image quality assessment models. We train a multi-head convolutional network for quality prediction by maximizing the accuracy of the ensemble (as well as the base learners) on labeled data, and the disagreement (i.e., diversity) among them on unlabeled data, both implemented by the fidelity loss. We conduct extensive experiments to demonstrate the advantages of employing unlabeled data for BIQA, especially in model generalization and failure identification.

cs.CV↗