Searcharxiv⌕ Search

arXiv subjects

Xin Zhao

Publications and source records attributed to Xin Zhao.

At least 127 records · Page 7Linked to original sources
arXiv2022

3D Matting: A Soft Segmentation Method Applied in Computed Tomography

Three-dimensional (3D) images, such as CT, MRI, and PET, are common in medical imaging applications and important in clinical diagnosis. Semantic ambiguity is a typical feature of many medical image labels. It can be caused by many factors, such as the imaging properties, pathological anatomy, and the weak representation of the binary masks, which brings challenges to accurate 3D segmentation. In 2D medical images, using soft masks instead of binary masks generated by image matting to characterize lesions can provide rich semantic information, describe the structural characteristics of lesions more comprehensively, and thus benefit the subsequent diagnoses and analyses. In this work, we introduce image matting into the 3D scenes to describe the lesions in 3D medical images. The study of image matting in 3D modality is limited, and there is no high-quality annotated dataset related to 3D matting, therefore slowing down the development of data-driven deep-learning-based methods. To address this issue, we constructed the first 3D medical matting dataset and convincingly verified the validity of the dataset through quality control and downstream experiments in lung nodules classification. We then adapt the four selected state-of-the-art 2D image matting algorithms to 3D scenes and further customize the methods for CT images. Also, we propose the first end-to-end deep 3D matting network and implement a solid 3D medical image matting benchmark, which will be released to encourage further research.

eess.IV↗
arXiv2022

Multimodal foundation models are better simulators of the human brain

Multimodal learning, especially large-scale multimodal pre-training, has developed rapidly over the past few years and led to the greatest advances in artificial intelligence (AI). Despite its effectiveness, understanding the underlying mechanism of multimodal pre-training models still remains a grand challenge. Revealing the explainability of such models is likely to enable breakthroughs of novel learning paradigms in the AI field. To this end, given the multimodal nature of the human brain, we propose to explore the explainability of multimodal learning models with the aid of non-invasive brain imaging technologies such as functional magnetic resonance imaging (fMRI). Concretely, we first present a newly-designed multimodal foundation model pre-trained on 15 million image-text pairs, which has shown strong multimodal understanding and generalization abilities in a variety of cognitive downstream tasks. Further, from the perspective of neural encoding (based on our foundation model), we find that both visual and lingual encoders trained multimodally are more brain-like compared with unimodal ones. Particularly, we identify a number of brain regions where multimodally-trained encoders demonstrate better neural encoding performance. This is consistent with the findings in existing studies on exploring brain multi-sensory integration. Therefore, we believe that multimodal foundation models are more suitable tools for neuroscientists to study the multimodal signal processing mechanisms in the human brain. Our findings also demonstrate the potential of multimodal foundation models as ideal computational simulators to promote both AI-for-brain and brain-for-AI research.

cs.NE↗
arXiv2022

Attribute Controllable Beautiful Caucasian Face Generation by Aesthetics Driven Reinforcement Learning

In recent years, image generation has made great strides in improving the quality of images, producing high-fidelity ones. Also, quite recently, there are architecture designs, which enable GAN to unsupervisedly learn the semantic attributes represented in different layers. However, there is still a lack of research on generating face images more consistent with human aesthetics. Based on EigenGAN [He et al., ICCV 2021], we build the techniques of reinforcement learning into the generator of EigenGAN. The agent tries to figure out how to alter the semantic attributes of the generated human faces towards more preferable ones. To accomplish this, we trained an aesthetics scoring model that can conduct facial beauty prediction. We also can utilize this scoring model to analyze the correlation between face attributes and aesthetics scores. Empirically, using off-the-shelf techniques from reinforcement learning would not work well. So instead, we present a new variant incorporating the ingredients emerging in the reinforcement learning communities in recent years. Compared to the original generated images, the adjusted ones show clear distinctions concerning various attributes. Experimental results using the MindSpore, show the effectiveness of the proposed method. Altered facial images are commonly more attractive, with significantly improved aesthetic levels.

cs.CV↗
arXiv2022

Multi-target Tracking of Zebrafish based on Particle Filter

Zebrafish is an excellent model organism, which has been widely used in the fields of biological experiments, drug screening, and swarm intelligence. In recent years, there are a large number of techniques for tracking of zebrafish involved in the study of behaviors, which makes it attack much attention of scientists from many fields. Multi-target tracking of zebrafish is still facing many challenges. The high mobility and uncertainty make it difficult to predict its motion; the similar appearances and texture features make it difficult to establish an appearance model; it is even hard to link the trajectories because of the frequent occlusion. In this paper, we use particle filter to approximate the uncertainty of the motion. Firstly, by analyzing the motion characteristics of zebrafish, we establish an efficient hybrid motion model to predict its positions; then we establish an appearance model based on the predicted positions to predict the postures of every targets, meanwhile weigh the particles by comparing the difference of predicted pose and observation pose ; finally, we get the optimal position of single zebrafish through the weighted position, and use the joint particle filter to process trajectory linking of multiple zebrafish.

cs.CV↗
arXiv2022

JDRec: Practical Actor-Critic Framework for Online Combinatorial Recommender System

A combinatorial recommender (CR) system feeds a list of items to a user at a time in the result page, in which the user behavior is affected by both contextual information and items. The CR is formulated as a combinatorial optimization problem with the objective of maximizing the recommendation reward of the whole list. Despite its importance, it is still a challenge to build a practical CR system, due to the efficiency, dynamics, personalization requirement in online environment. In particular, we tear the problem into two sub-problems, list generation and list evaluation. Novel and practical model architectures are designed for these sub-problems aiming at jointly optimizing effectiveness and efficiency. In order to adapt to online case, a bootstrap algorithm forming an actor-critic reinforcement framework is given to explore better recommendation mode in long-term user interaction. Offline and online experiment results demonstrate the efficacy of proposed JDRec framework. JDRec has been applied in online JD recommendation, improving click through rate by 2.6% and synthetical value for the platform by 5.03%. We will publish the large-scale dataset used in this study to contribute to the research community.

cs.IR↗
arXiv2022

An application of Pixel Interval Down-sampling (PID) for dense tiny microorganism counting on environmental microorganism images

This paper proposes a novel pixel interval down-sampling network (PID-Net) for dense tiny object (yeast cells) counting tasks with higher accuracy. The PID-Net is an end-to-end convolutional neural network (CNN) model with an encoder--decoder architecture. The pixel interval down-sampling operations are concatenated with max-pooling operations to combine the sparse and dense features. This addresses the limitation of contour conglutination of dense objects while counting. The evaluation was conducted using classical segmentation metrics (the Dice, Jaccard and Hausdorff distance) as well as counting metrics. The experimental results show that the proposed PID-Net had the best performance and potential for dense tiny object counting tasks, which achieved 96.97\% counting accuracy on the dataset with 2448 yeast cell images. By comparing with the state-of-the-art approaches, such as Attention U-Net, Swin U-Net and Trans U-Net, the proposed PID-Net can segment dense tiny objects with clearer boundaries and fewer incorrect debris, which shows the great potential of PID-Net in the task of accurate counting.

cs.CV↗
arXiv2022

QueryProp: Object Query Propagation for High-Performance Video Object Detection

Video object detection has been an important yet challenging topic in computer vision. Traditional methods mainly focus on designing the image-level or box-level feature propagation strategies to exploit temporal information. This paper argues that with a more effective and efficient feature propagation framework, video object detectors can gain improvement in terms of both accuracy and speed. For this purpose, this paper studies object-level feature propagation, and proposes an object query propagation (QueryProp) framework for high-performance video object detection. The proposed QueryProp contains two propagation strategies: 1) query propagation is performed from sparse key frames to dense non-key frames to reduce the redundant computation on non-key frames; 2) query propagation is performed from previous key frames to the current key frame to improve feature representation by temporal context modeling. To further facilitate query propagation, an adaptive propagation gate is designed to achieve flexible key frame selection. We conduct extensive experiments on the ImageNet VID dataset. QueryProp achieves comparable accuracy with state-of-the-art methods and strikes a decent accuracy/speed trade-off. Code is available at https://github.com/hf1995/QueryProp.

cs.CV↗
arXiv2022

Unsupervised Domain Adaptive Fundus Image Segmentation with Category-level Regularization

Existing unsupervised domain adaptation methods based on adversarial learning have achieved good performance in several medical imaging tasks. However, these methods focus only on global distribution adaptation and ignore distribution constraints at the category level, which would lead to sub-optimal adaptation performance. This paper presents an unsupervised domain adaptation framework based on category-level regularization that regularizes the category distribution from three perspectives. Specifically, for inter-domain category regularization, an adaptive prototype alignment module is proposed to align feature prototypes of the same category in the source and target domains. In addition, for intra-domain category regularization, we tailored a regularization technique for the source and target domains, respectively. In the source domain, a prototype-guided discriminative loss is proposed to learn more discriminative feature representations by enforcing intra-class compactness and inter-class separability, and as a complement to traditional supervised loss. In the target domain, an augmented consistency category regularization loss is proposed to force the model to produce consistent predictions for augmented/unaugmented target images, which encourages semantically similar regions to be given the same label. Extensive experiments on two publicly fundus datasets show that the proposed approach significantly outperforms other state-of-the-art comparison algorithms.

cs.CV↗
arXiv2022

CUGE: A Chinese Language Understanding and Generation Evaluation Benchmark

Realizing general-purpose language intelligence has been a longstanding goal for natural language processing, where standard evaluation benchmarks play a fundamental and guiding role. We argue that for general-purpose language intelligence evaluation, the benchmark itself needs to be comprehensive and systematic. To this end, we propose CUGE, a Chinese Language Understanding and Generation Evaluation benchmark with the following features: (1) Hierarchical benchmark framework, where datasets are principally selected and organized with a language capability-task-dataset hierarchy. (2) Multi-level scoring strategy, where different levels of model performance are provided based on the hierarchical framework. To facilitate CUGE, we provide a public leaderboard that can be customized to support flexible model judging criteria. Evaluation results on representative pre-trained language models indicate ample room for improvement towards general-purpose language intelligence. CUGE is publicly available at cuge.baai.ac.cn.

cs.CL↗
arXiv2022

PanopticDepth: A Unified Framework for Depth-aware Panoptic Segmentation

This paper presents a unified framework for depth-aware panoptic segmentation (DPS), which aims to reconstruct 3D scene with instance-level semantics from one single image. Prior works address this problem by simply adding a dense depth regression head to panoptic segmentation (PS) networks, resulting in two independent task branches. This neglects the mutually-beneficial relations between these two tasks, thus failing to exploit handy instance-level semantic cues to boost depth accuracy while also producing sub-optimal depth maps. To overcome these limitations, we propose a unified framework for the DPS task by applying a dynamic convolution technique to both the PS and depth prediction tasks. Specifically, instead of predicting depth for all pixels at a time, we generate instance-specific kernels to predict depth and segmentation masks for each instance. Moreover, leveraging the instance-wise depth estimation scheme, we add additional instance-level depth cues to assist with supervising the depth learning via a new depth loss. Extensive experiments on Cityscapes-DPS and SemKITTI-DPS show the effectiveness and promise of our method. We hope our unified solution to DPS can lead a new paradigm in this area. Code is available at https://github.com/NaiyuGao/PanopticDepth.

cs.CV↗
arXiv2022

Physical Mechanism of Superconductivity Part II Superconductivity and Superfluidity

The transition mechanism of metal-insulator in metal oxides is discussed in detail, which is a part of the mechanism of superconductivity. Through the study of magic angle twisted bilayer graphene superconductor and other new findings on superconductivity, we further demonstrate that the physical mechanism of superconductivity proposed in Part I is the only correct way to handle the properties of superconductivity in various materials. We propose that superfluid helium consists of normal liquid helium mixed with high-energy helium atoms. Based on this new model, all peculiar features discovered in superfluid helium can be truly understood, such as the climb of superfluid helium on the container's wall, the fountain effect, the discontinuity of specific heat capacity at phase transition point, and the maintaining mass current in a ring-shaped container. We demonstrate that the high-energy particles play a driving force role in both superconductors and superfluid helium, and therefore dominate their properties.

cond-mat.supr-con↗
arXiv2022

A Comprehensive Survey with Quantitative Comparison of Image Analysis Methods for Microorganism Biovolume Measurements

With the acceleration of urbanization and living standards, microorganisms play increasingly important roles in industrial production, bio-technique, and food safety testing. Microorganism biovolume measurements are one of the essential parts of microbial analysis. However, traditional manual measurement methods are time-consuming and challenging to measure the characteristics precisely. With the development of digital image processing techniques, the characteristics of the microbial population can be detected and quantified. The changing trend can be adjusted in time and provided a basis for the improvement. The applications of the microorganism biovolume measurement method have developed since the 1980s. More than 62 articles are reviewed in this study, and the articles are grouped by digital image segmentation methods with periods. This study has high research significance and application value, which can be referred to microbial researchers to have a comprehensive understanding of microorganism biovolume measurements using digital image analysis methods and potential applications.

cs.CV↗
arXiv2022

The Nature of Schrodinger Equation -- On Quantum Physics Part I

We propose that the Schrodinger equation results from applying the classical wave equation to describe the physical system in which subatomic particles play random motion, thereby leading to quantum mechanics. The physical reality described by the wave function is subatomic particle moving randomly. Therefore, the characteristics of quantum mechanics have a dual nature, one of them is the deterministic nature carried on from classical physics, and the other is the probabilistic nature coined by particle's random motion. Based on this model, almost all of open questions in quantum mechanics can be explained consistently, which include the particle-wave duality, the principle of quantum superposition, the interference pattern of double-slit experiments, and the boundary between the classical world and the quantum world. The current quantum mechanics is a mixture of matrix mechanics and wave mechanics, which are sharply conflicting in principle. Matrix mechanics treats quantum particles as classical particles with fixed relation between the particle's position and its momentum. The matrix mechanics, in fact, belongs to the old quantum theory. Both Born's non-commutative relation and Heisenberg uncertainty relation originate from matrix mechanics. However, in wave mechanics, there is no any fixed relation between the particle's position and its momentum, and the particle's position and its momentum belong to immeasurable physical quantities. Therefore, there is no need for non-commutative relation and uncertainty relation in wave mechanics.

physics.gen-ph↗
arXiv2022

A State-of-the-art Survey of Object Detection Techniques in Microorganism Image Analysis: From Classical Methods to Deep Learning Approaches

Microorganisms play a vital role in human life. Therefore, microorganism detection is of great significance to human beings. However, the traditional manual microscopic detection methods have the disadvantages of long detection cycle, low detection accuracy in large orders, and great difficulty in detecting uncommon microorganisms. Therefore, it is meaningful to apply computer image analysis technology to the field of microorganism detection. Computer image analysis can realize high-precision and high-efficiency detection of microorganisms. In this review, first,we analyse the existing microorganism detection methods in chronological order, from traditional image processing and traditional machine learning to deep learning methods. Then, we analyze and summarize these existing methods and introduce some potential methods, including visual transformers. In the end, the future development direction and challenges of microorganism detection are discussed. In general, we have summarized 142 related technical papers from 1985 to the present. This review will help researchers have a more comprehensive understanding of the development process, research status, and future trends in the field of microorganism detection and provide a reference for researchers in other fields.

cs.CV↗
arXiv2022

Solenoid-free current drive via ECRH in EXL-50 spherical torus plasmas

As a new spherical tokamak (ST) designed to simplify engineering requirements of a possible future fusion power source, the EXL-50 experiment features a low aspect ratio (A) vacuum vessel (VV), encircling a central post assembly containing the toroidal field coil conductors without a central solenoid. Multiple electron cyclotron resonance heating (ECRH) resonances are located within the VV to improve current drive effectiveness. Copious energetic electrons are produced and measured with hard X-ray detectors, carry the bulk of the plasma current ranging from 50kA to 150kA, which is maintained for more than 1s duration. It is observed that over one Ampere current can be maintained per Watt of ECRH power issued from the 28-GHz gyrotrons. The plasma current reaches Ip>80kA for high density (>5e18me-2) discharge with 150kW ECHR heating. An analysis was carried out combining reconstructed multi-fluid equilibrium, guiding-center orbits of energetic electrons, and resonant heating mechanisms. It is verified that in EXL-50 a broadly distributed current of energetic electrons creates smaller closed magnetic-flux surfaces of low aspect ratio that in turn confine the thermal plasma electrons and ions and participate in maintaining the equilibrium force-balance.

physics.plasm-ph↗
arXiv2022

Global Instance Tracking: Locating Target More Like Humans

Target tracking, the essential ability of the human visual system, has been simulated by computer vision tasks. However, existing trackers perform well in austere experimental environments but fail in challenges like occlusion and fast motion. The massive gap indicates that researches only measure tracking performance rather than intelligence. How to scientifically judge the intelligence level of trackers? Distinct from decision-making problems, lacking three requirements (a challenging task, a fair environment, and a scientific evaluation procedure) makes it strenuous to answer the question. In this article, we first propose the global instance tracking (GIT) task, which is supposed to search an arbitrary user-specified instance in a video without any assumptions about camera or motion consistency, to model the human visual tracking ability. Whereafter, we construct a high-quality and large-scale benchmark VideoCube to create a challenging environment. Finally, we design a scientific evaluation procedure using human capabilities as the baseline to judge tracking intelligence. Additionally, we provide an online platform with toolkit and an updated leaderboard. Although the experimental results indicate a definite gap between trackers and humans, we expect to take a step forward to generate authentic human-like trackers. The database, toolkit, evaluation server, and baseline results are available at http://videocube.aitestunion.com.

cs.CV↗
arXiv2022

Neural Architecture Search for Inversion

Over the year, people have been using deep learning to tackle inversion problems, and we see the framework has been applied to build relationship between recording wavefield and velocity (Yang et al., 2016). Here we will extend the work from 2 perspectives, one is deriving a more appropriate loss function, as we now, pixel-2-pixel comparison might not be the best choice to characterize image structure, and we will elaborate on how to construct cost function to capture high level feature to enhance the model performance. Another dimension is searching for the more appropriate neural architecture, which is a subset of an even bigger picture, the automatic machine learning, or AutoML. There are several famous networks, U-net, ResNet (He et al., 2016) and DenseNet (Huang et al., 2017), and they achieve phenomenal results for certain problems, yet it's hard to argue they are the best for inversion problems without thoroughly searching within certain space. Here we will be showing our architecture search results for inversion.

cs.LG↗