arXiv subjects

Zhen Li

Publications and source records attributed to Zhen Li.

At least 325 records · Page 18Linked to original sources

arXiv2023

Comprehensive Visual Question Answering on Point Clouds through Compositional Scene Manipulation

Visual Question Answering on 3D Point Cloud (VQA-3D) is an emerging yet challenging field that aims at answering various types of textual questions given an entire point cloud scene. To tackle this problem, we propose the CLEVR3D, a large-scale VQA-3D dataset consisting of 171K questions from 8,771 3D scenes. Specifically, we develop a question engine leveraging 3D scene graph structures to generate diverse reasoning questions, covering the questions of objects' attributes (i.e., size, color, and material) and their spatial relationships. Through such a manner, we initially generated 44K questions from 1,333 real-world scenes. Moreover, a more challenging setup is proposed to remove the confounding bias and adjust the context from a common-sense layout. Such a setup requires the network to achieve comprehensive visual understanding when the 3D scene is different from the general co-occurrence context (e.g., chairs always exist with tables). To this end, we further introduce the compositional scene manipulation strategy and generate 127K questions from 7,438 augmented 3D scenes, which can improve VQA-3D models for real-world comprehension. Built upon the proposed dataset, we baseline several VQA-3D models, where experimental results verify that the CLEVR3D can significantly boost other 3D scene understanding tasks. Our code and dataset will be made publicly available at https://github.com/yanx27/CLEVR3D.

cs.CV↗

arXiv2023

Autonomous Navigation for Robot-assisted Intraluminal and Endovascular Procedures: A Systematic Review

Increased demand for less invasive procedures has accelerated the adoption of Intraluminal Procedures (IP) and Endovascular Interventions (EI) performed through body lumens and vessels. As navigation through lumens and vessels is quite complex, interest grows to establish autonomous navigation techniques for IP and EI for reaching the target area. Current research efforts are directed toward increasing the Level of Autonomy (LoA) during the navigation phase. One key ingredient for autonomous navigation is Motion Planning (MP) techniques. This paper provides an overview of MP techniques categorizing them based on LoA. Our analysis investigates advances for the different clinical scenarios. Through a systematic literature analysis using the PRISMA method, the study summarizes relevant works and investigates the clinical aim, LoA, adopted MP techniques, and validation types. We identify the limitations of the corresponding MP methods and provide directions to improve the robustness of the algorithms in dynamic intraluminal environments. MP for IP and EI can be classified into four subgroups: node, sampling, optimization, and learning-based techniques, with a notable rise in learning-based approaches in recent years. One of the review's contributions is the identification of the limiting factors in IP and EI robotic systems hindering higher levels of autonomous navigation. In the future, navigation is bound to become more autonomous, placing the clinician in a supervisory position to improve control precision and reduce workload.

cs.RO↗

arXiv2023

Energy extraction via Comisso-Asenjo mechanism from rotating hairy black hole

It was demonstrated by Comisso and Asenjo that the magnetic reconnection in the ergosphere is a promising mechanism to extract energy from the rotating Kerr black hole. In this work, we investigate the role of Comisso-Asenjo mechanism in energy extraction from the newly suggested rotating hairy black holes which have an extra hair due to the additional surrounding sources, such as dark matter or dark energy. We examine how the hairy parameters characterized the hair affect the magnetic reconnection process in addition to other important variables of the Comisso-Asenjo process, including the parameter spaces that permit energy extraction, the power, efficiency and power ratios with respect to the Blandford-Znajek mechanism.

gr-qc↗

arXiv2023

Energy extraction from rotating regular black hole via Comisso-Asenjo mechanism

Recently, it has been demonstrated by Comisso and Asenjo that magnetic reconnection processes in the ergosphere of a Kerr black hole can provide us with a promising mechanism for extracting the rotational energy from it. In this paper, we study the energy extraction from the the newly proposed rotating regular black holes via this Comisso-Asenjo mechanism. This novel rotating regular black hole has an exponential convergence factor $e^{-k/r}$ on the mass term characterized by the regular parameter $k$ in the exponent. We explore the effects of this regular parameter on the magnetic reconnection as well as other critical parameters determining the Comisso-Asenjo process. The parameter spaces allowing energy extraction to occur are investigated. The power, efficiency and the power ratio to the Blandford-Znajek mechanism are studied. The results show that the regularity of the rotating black hole has significant effects on the energy extraction via the Comisso-Asenjo mechanism.

gr-qc↗

arXiv2023

Toward Unpaired Multi-modal Medical Image Segmentation via Learning Structured Semantic Consistency

Integrating multi-modal data to promote medical image analysis has recently gained great attention. This paper presents a novel scheme to learn the mutual benefits of different modalities to achieve better segmentation results for unpaired multi-modal medical images. Our approach tackles two critical issues of this task from a practical perspective: (1) how to effectively learn the semantic consistencies of various modalities (e.g., CT and MRI), and (2) how to leverage the above consistencies to regularize the network learning while preserving its simplicity. To address (1), we leverage a carefully designed External Attention Module (EAM) to align semantic class representations and their correlations of different modalities. To solve (2), the proposed EAM is designed as an external plug-and-play one, which can be discarded once the model is optimized. We have demonstrated the effectiveness of the proposed method on two medical image segmentation scenarios: (1) cardiac structure segmentation, and (2) abdominal multi-organ segmentation. Extensive results show that the proposed method outperforms its counterparts by a wide margin.

cs.CV↗

arXiv2023

Hierarchical Weight Averaging for Deep Neural Networks

Despite the simplicity, stochastic gradient descent (SGD)-like algorithms are successful in training deep neural networks (DNNs). Among various attempts to improve SGD, weight averaging (WA), which averages the weights of multiple models, has recently received much attention in the literature. Broadly, WA falls into two categories: 1) online WA, which averages the weights of multiple models trained in parallel, is designed for reducing the gradient communication overhead of parallel mini-batch SGD, and 2) offline WA, which averages the weights of one model at different checkpoints, is typically used to improve the generalization ability of DNNs. Though online and offline WA are similar in form, they are seldom associated with each other. Besides, these methods typically perform either offline parameter averaging or online parameter averaging, but not both. In this work, we firstly attempt to incorporate online and offline WA into a general training framework termed Hierarchical Weight Averaging (HWA). By leveraging both the online and offline averaging manners, HWA is able to achieve both faster convergence speed and superior generalization performance without any fancy learning rate adjustment. Besides, we also analyze the issues faced by existing WA methods, and how our HWA address them, empirically. Finally, extensive experiments verify that HWA outperforms the state-of-the-art methods significantly.

cs.LG↗

arXiv2023

Decouple Non-parametric Knowledge Distillation For End-to-end Speech Translation

Existing techniques often attempt to make knowledge transfer from a powerful machine translation (MT) to speech translation (ST) model with some elaborate techniques, which often requires transcription as extra input during training. However, transcriptions are not always available, and how to improve the ST model performance without transcription, i.e., data efficiency, has rarely been studied in the literature. In this paper, we propose Decoupled Non-parametric Knowledge Distillation (DNKD) from data perspective to improve the data efficiency. Our method follows the knowledge distillation paradigm. However, instead of obtaining the teacher distribution from a sophisticated MT model, we construct it from a non-parametric datastore via k-Nearest-Neighbor (kNN) retrieval, which removes the dependence on transcription and MT model. Then we decouple the classic knowledge distillation loss into target and non-target distillation to enhance the effect of the knowledge among non-target logits, which is the prominent "dark knowledge". Experiments on MuST-C corpus show that, the proposed method can achieve consistent improvement over the strong baseline without requiring any transcription.

cs.CL↗

arXiv2023

AMT: All-Pairs Multi-Field Transforms for Efficient Frame Interpolation

We present All-Pairs Multi-Field Transforms (AMT), a new network architecture for video frame interpolation. It is based on two essential designs. First, we build bidirectional correlation volumes for all pairs of pixels, and use the predicted bilateral flows to retrieve correlations for updating both flows and the interpolated content feature. Second, we derive multiple groups of fine-grained flow fields from one pair of updated coarse flows for performing backward warping on the input frames separately. Combining these two designs enables us to generate promising task-oriented flows and reduce the difficulties in modeling large motions and handling occluded areas during frame interpolation. These qualities promote our model to achieve state-of-the-art performance on various benchmarks with high efficiency. Moreover, our convolution-based model competes favorably compared to Transformer-based models in terms of accuracy and efficiency. Our code is available at https://github.com/MCG-NKU/AMT.

cs.CV↗

arXiv2023

Superradiance and quasinormal modes of the gravitational perturbation around rotating hairy black hole

The No Hair theorem in classical general relativity predicts that rotating black holes are specified by the Kerr metric, which is uniquely identified by the mass and spin. However, as a pioneering study beyond general relativity, the rotating hairy black hole has been proposed, which encompasses the Kerr black hole as a special case. In these black holes, there are extra hair which could appear due to the additional surrounding sources such as dark matter or dark energy. In this work, we study the phenomenology of the rotating hairy black hole in terms of gravitational perturbations. In particular, the supperradiance and the quasinormal modes. Using the matching-asymptotic method, we derive the amplification factor and the superradiance conditions. We also calculate the quasinormal modes using the continued fraction method. The results are in very good agreement with previous studies in the Kerr limit. We also show how the amplification and quasinormal modes will shift in response to variations in the hairy parameters, black hole spin, and quantum numbers.

gr-qc↗

arXiv2023

Semantic Human Parsing via Scalable Semantic Transfer over Multiple Label Domains

This paper presents Scalable Semantic Transfer (SST), a novel training paradigm, to explore how to leverage the mutual benefits of the data from different label domains (i.e. various levels of label granularity) to train a powerful human parsing network. In practice, two common application scenarios are addressed, termed universal parsing and dedicated parsing, where the former aims to learn homogeneous human representations from multiple label domains and switch predictions by only using different segmentation heads, and the latter aims to learn a specific domain prediction while distilling the semantic knowledge from other domains. The proposed SST has the following appealing benefits: (1) it can capably serve as an effective training scheme to embed semantic associations of human body parts from multiple label domains into the human representation learning process; (2) it is an extensible semantic transfer framework without predetermining the overall relations of multiple label domains, which allows continuously adding human parsing datasets to promote the training. (3) the relevant modules are only used for auxiliary training and can be removed during inference, eliminating the extra reasoning cost. Experimental results demonstrate SST can effectively achieve promising universal human parsing performance as well as impressive improvements compared to its counterparts on three human parsing benchmarks (i.e., PASCAL-Person-Part, ATR, and CIHP). Code is available at https://github.com/yangjie-cv/SST.

cs.CV↗

arXiv2023

Fair-CDA: Continuous and Directional Augmentation for Group Fairness

In this work, we propose {\it Fair-CDA}, a fine-grained data augmentation strategy for imposing fairness constraints. We use a feature disentanglement method to extract the features highly related to the sensitive attributes. Then we show that group fairness can be achieved by regularizing the models on transition paths of sensitive features between groups. By adjusting the perturbation strength in the direction of the paths, our proposed augmentation is controllable and auditable. To alleviate the accuracy degradation caused by fairness constraints, we further introduce a calibrated model to impute labels for the augmented data. Our proposed method does not assume any data generative model and ensures good generalization for both accuracy and fairness. Experimental results show that Fair-CDA consistently outperforms state-of-the-art methods on widely-used benchmarks, e.g., Adult, CelebA and MovieLens. Especially, Fair-CDA obtains an 86.3\% relative improvement for fairness while maintaining the accuracy on the Adult dataset. Moreover, we evaluate Fair-CDA in an online recommendation system to demonstrate the effectiveness of our method in terms of accuracy and fairness.

cs.LG↗

arXiv2023

Deep neural operator for learning transient response of interpenetrating phase composites subject to dynamic loading

Additive manufacturing has been recognized as an industrial technological revolution for manufacturing, which allows fabrication of materials with complex three-dimensional (3D) structures directly from computer-aided design models. The mechanical properties of interpenetrating phase composites (IPCs), especially response to dynamic loading, highly depend on their 3D structures. In general, for each specified structural design, it could take hours or days to perform either finite element analysis (FEA) or experiments to test the mechanical response of IPCs to a given dynamic load. To accelerate the physics-based prediction of mechanical properties of IPCs for various structural designs, we employ a deep neural operator (DNO) to learn the transient response of IPCs under dynamic loading as surrogate of physics-based FEA models. We consider a 3D IPC beam formed by two metals with a ratio of Young's modulus of 2.7, wherein random blocks of constituent materials are used to demonstrate the generality and robustness of the DNO model. To obtain FEA results of IPC properties, 5,000 random time-dependent strain loads generated by a Gaussian process kennel are applied to the 3D IPC beam, and the reaction forces and stress fields inside the IPC beam under various loading are collected. Subsequently, the DNO model is trained using an incremental learning method with sequence-to-sequence training implemented in JAX, leading to a 100X speedup compared to widely used vanilla deep operator network models. After an offline training, the DNO model can act as surrogate of physics-based FEA to predict the transient mechanical response in terms of reaction force and stress distribution of the IPCs to various strain loads in one second at an accuracy of 98%. Also, the learned operator is able to provide extended prediction of the IPC beam subject to longer random strain loads at a reasonably well accuracy.

cs.LG↗

arXiv2023

Multi-view Inverse Rendering for Large-scale Real-world Indoor Scenes

We present a efficient multi-view inverse rendering method for large-scale real-world indoor scenes that reconstructs global illumination and physically-reasonable SVBRDFs. Unlike previous representations, where the global illumination of large scenes is simplified as multiple environment maps, we propose a compact representation called Texture-based Lighting (TBL). It consists of 3D mesh and HDR textures, and efficiently models direct and infinite-bounce indirect lighting of the entire large scene. Based on TBL, we further propose a hybrid lighting representation with precomputed irradiance, which significantly improves the efficiency and alleviates the rendering noise in the material optimization. To physically disentangle the ambiguity between materials, we propose a three-stage material optimization strategy based on the priors of semantic segmentation and room segmentation. Extensive experiments show that the proposed method outperforms the state-of-the-art quantitatively and qualitatively, and enables physically-reasonable mixed-reality applications such as material editing, editable novel view synthesis and relighting. The project page is at https://lzleejean.github.io/TexIR.

cs.CV↗

arXiv2023

Active Learning for Event Extraction with Memory-based Loss Prediction Model

Event extraction (EE) plays an important role in many industrial application scenarios, and high-quality EE methods require a large amount of manual annotation data to train supervised learning models. However, the cost of obtaining annotation data is very high, especially for annotation of domain events, which requires the participation of experts from corresponding domain. So we introduce active learning (AL) technology to reduce the cost of event annotation. But the existing AL methods have two main problems, which make them not well used for event extraction. Firstly, the existing pool-based selection strategies have limitations in terms of computational cost and sample validity. Secondly, the existing evaluation of sample importance lacks the use of local sample information. In this paper, we present a novel deep AL method for EE. We propose a batch-based selection strategy and a Memory-Based Loss Prediction model (MBLP) to select unlabeled samples efficiently. During the selection process, we use an internal-external sample loss ranking method to evaluate the sample importance by using local information. Finally, we propose a delayed training strategy to train the MBLP model. Extensive experiments are performed on three domain datasets, and our method outperforms other state-of-the-art methods.

cs.CL↗

arXiv2023

Arbitrary slip length for fluid-solid interface of arbitrary geometry in smoothed particle dynamics

We model a slip boundary condition at fluid-solid interface of an arbitrary geometry in smoothed particle hydrodynamics and smoothed dissipative particle dynamics simulations. Under an assumption of linear profile of the tangential velocity at quasi-steady state near the interface, an arbitrary slip length $b$ can be specified and correspondingly, an artificial velocity for every boundary particle can be calculated. Therefore, $b$ as an input parameter affects the calculation of dissipative and random forces near the interface. For $b \to 0$, the no-slip is recovered while for $b \to \infty$, the free-slip is achieved. Technically, we devise two different approaches to calculate the artificial velocity of any boundary particle. The first has a succinct principle and is competent for simple geometries, while the second is subtle and affordable for complex geometries. Slip lengths in simulations for both steady and transient flows coincide with the expected ones. As demonstration, we apply the two approaches extensively to simulate curvy channel flows, dynamics of an ellipsoid in pipe flow and flows within complex microvessels, where desired slip lengths at fluid-solid interfaces are prescribed. The proposed methodology may apply equally well to other particle methods such as dissipative particle dynamics and moving particle semi-implicit methods.

physics.flu-dyn↗

arXiv2023

Dynamic spreading and infiltration of a molten sand droplet on a porous surface

Compared to smooth surfaces, droplet spreading on porous surfaces is more complex and has relevance in many engineering applications. In this work, we investigate the infiltration dynamics of molten sand droplets on structured porous surfaces using the multiphase many-body dissipative particle dynamics (mDPD) method. We carry out three-dimensional simulations with different equilibrium contact angles and surface porosities. The temporal evolution of the radius of the wetted area follows a power law, as in the case of a smooth surface. The infiltration rate on the other hand is dictated by the competition between spreading and capillary inhibition of the pores. Additionally, the temporal evolution of the droplet height and the contact angle on the porous surface is also presented.

physics.flu-dyn↗

arXiv2023

Scalar Perturbation Around Rotating Regular Black Hole: Superradiance Instability and Quasinormal Modes

Black holes provide a natural laboratory to study particle physics and astrophysics. When black holes are surrounded by matter fields, there will be plenty of phenomena which can have observational consequences, from which we can learn about the matter fields as well as black hole spacetime. In this work, we investigate the massive scalar field in the vicinity of a newly proposed rotating regular black hole inspired by quantum gravity. We will especially investigate how this non-singular spactime will affect the superradiance instability and quasinormal modes of the scalar filed. We derive the superradiant conditions and the amplification factor by using the Matching-asymptotic Method, and the quasinormal modes are computed through Continued Fraction Method. In the Kerr limit, the results are in excellent agreements with previous research. We also demonstrate how the quasinormal modes will change as a function of black hole spin, regularity described by a parameter $k$ and scalar field mass respectively, with other parameters taking specific values.

gr-qc↗

arXiv2023

Adaptive Context Selection for Polyp Segmentation

Accurate polyp segmentation is of great significance for the diagnosis and treatment of colorectal cancer. However, it has always been very challenging due to the diverse shape and size of polyp. In recent years, state-of-the-art methods have achieved significant breakthroughs in this task with the help of deep convolutional neural networks. However, few algorithms explicitly consider the impact of the size and shape of the polyp and the complex spatial context on the segmentation performance, which results in the algorithms still being powerless for complex samples. In fact, segmentation of polyps of different sizes relies on different local and global contextual information for regional contrast reasoning. To tackle these issues, we propose an adaptive context selection based encoder-decoder framework which is composed of Local Context Attention (LCA) module, Global Context Module (GCM) and Adaptive Selection Module (ASM). Specifically, LCA modules deliver local context features from encoder layers to decoder layers, enhancing the attention to the hard region which is determined by the prediction map of previous layer. GCM aims to further explore the global context features and send to the decoder layers. ASM is used for adaptive selection and aggregation of context features through channel-wise attention. Our proposed approach is evaluated on the EndoScene and Kvasir-SEG Datasets, and shows outstanding performance compared with other state-of-the-art methods. The code is available at https://github.com/ReaFly/ACSNet.

cs.CV↗