arXiv subjects

Jia Wei

Publications and source records attributed to Jia Wei.

51 records · Page 3Linked to original sources

arXiv2023

SFusion: Self-attention based N-to-One Multimodal Fusion Block

People perceive the world with different senses, such as sight, hearing, smell, and touch. Processing and fusing information from multiple modalities enables Artificial Intelligence to understand the world around us more easily. However, when there are missing modalities, the number of available modalities is different in diverse situations, which leads to an N-to-One fusion problem. To solve this problem, we propose a self-attention based fusion block called SFusion. Different from preset formulations or convolution based methods, the proposed block automatically learns to fuse available modalities without synthesizing or zero-padding missing ones. Specifically, the feature representations extracted from upstream processing model are projected as tokens and fed into self-attention module to generate latent multimodal correlations. Then, a modal attention mechanism is introduced to build a shared representation, which can be applied by the downstream decision model. The proposed SFusion can be easily integrated into existing multimodal analysis networks. In this work, we apply SFusion to different backbone networks for human activity recognition and brain tumor segmentation tasks. Extensive experimental results show that the SFusion block achieves better performance than the competing fusion strategies. Our code is available at https://github.com/scut-cszcl/SFusion.

cs.CV↗

arXiv2022

AMORETTO: A Method for Deriving IoT-enriched Event Logs

Process analytics aims to gain insights into the behaviour and performance of business processes through the analysis of event logs, which record the execution of processes. With the widespread use of the Internet of Things (IoT), IoT data has become readily available and can provide valuable context information about business processes. As such, process analytics can benefit from incorporating IoT data into event logs to support more comprehensive, context-aware analyses. However, most existing studies focus on enhancing business process models with IoT data, whereas little attention has been paid to incorporating IoT data into event logs for process analytics. Hence, this paper aims to systematically integrate IoT data into event logs to support context-aware process analytics. To this end, we propose AMORETTO - a method for deriving IoT-enriched event logs. Firstly, we provide a classification of context data, referred to as the IoT-Pro context classification, which encompasses two context dimensions: IoT context and process context. Next, we present a method for integrating IoT data with event logs, guided by IoT-Pro, to yield IoT-enriched event logs. To demonstrate the applicability of AMORETTO, we applied it to a real-life use case and examined whether the derived IoT-enriched event log sufficed to address certain specific analytical questions.

cs.DB↗

arXiv2022

Color flux-tube nature of the states $T_{cs}(2900)$ and $T^a_{c\bar{s}}(2900)$

Inspired by the states $T_{cs0}(2900)^0$, $T_{cs1}(2900)^0$, $T^a_{c\bar{s}0}(2900)^{0}$ and $T^a_{c\bar{s}0}(2900)^{++}$ reported by the LHCb Collaboration, we carry out a systematical investigation on the properties of the ground and $P$-wave states $[cs][\bar{u}\bar{d}]$ and $[cu][\bar{s}\bar{d}]$ with various spin, isospin or $U$-spin, and color combinations in a multiquark color flux-tube model. Matching our results with the spin-parity and mass of the states $T_{cs0}(2900)^0$ and $T_{cs1}(2900)^0$, we can describe them as the compact states $[cs][\bar{u}\bar{d}]$ with $I(J^{P})=1(0^+)$ and $0(1^-)$ in the model, respectively. The ground state $T_{cs0}(2900)^0$ is mainly made of strongly overlapped an axial-vector $[cs]_{\bar{\mathbf{3}}_c}$ and an axial-vector $[\bar{u}\bar{d}]_{\mathbf{3}_c}$. The $P$-wave state $T_{cs1}(2900)^0$ is dominantly consisted of a gradually separated scalar or axial vector $[cs]_{\bar{\mathbf{3}}_c}$ and a scalar $[\bar{u}\bar{d}]_{\mathbf{3}_c}$ in the shape of a dumbbell. Supposing the states $T^a_{c\bar{s}0}(2900)^{0}$ and $T^a_{c\bar{s}0}(2900)^{++}$ belong to the same isospin triplet, the mass of the state $\left [[cu]_{\bar{\mathbf{3}}_c}[\bar{s}\bar{d}]_ {\mathbf{3}_c}\right ]_{\mathbf{1}_c}$ with symmetrical $U$-spin and $J^P=0^+$ is highly consistent with that of the states $T^a_{c\bar{s}0}(2900)^{0}$ and $T^a_{c\bar{s}0}(2900)^{++}$ in the model. After coupling two color configurations, the state $[cu][\bar{s}\bar{d}]$ is slightly lighter than the states $T^a_{c\bar{s}0}(2900)^{0}$ and $T^a_{c\bar{s}0}(2900)^{++}$. In addition, we also discuss the properties of other states in the model.

hep-ph↗

arXiv2022

Learning Multi-Modal Brain Tumor Segmentation from Privileged Semi-Paired MRI Images with Curriculum Disentanglement Learning

Due to the difficulties of obtaining multimodal paired images in clinical practice, recent studies propose to train brain tumor segmentation models with unpaired images and capture complementary information through modality translation. However, these models cannot fully exploit the complementary information from different modalities. In this work, we thus present a novel two-step (intra-modality and inter-modality) curriculum disentanglement learning framework to effectively utilize privileged semi-paired images, i.e. limited paired images that are only available in training, for brain tumor segmentation. Specifically, in the first step, we propose to conduct reconstruction and segmentation with augmented intra-modality style-consistent images. In the second step, the model jointly performs reconstruction, unsupervised/supervised translation, and segmentation for both unpaired and paired inter-modality images. A content consistency loss and a supervised translation loss are proposed to leverage complementary information from different modalities in this step. Through these two steps, our method effectively extracts modality-specific style codes describing the attenuation of tissue features and image contrast, and modality-invariant content codes containing anatomical and functional information from the input images. Experiments on three brain tumor segmentation tasks show that our model outperforms competing segmentation models based on unpaired images.

cs.CV↗

arXiv2022

Unsupervised Multi-Modal Medical Image Registration via Discriminator-Free Image-to-Image Translation

In clinical practice, well-aligned multi-modal images, such as Magnetic Resonance (MR) and Computed Tomography (CT), together can provide complementary information for image-guided therapies. Multi-modal image registration is essential for the accurate alignment of these multi-modal images. However, it remains a very challenging task due to complicated and unknown spatial correspondence between different modalities. In this paper, we propose a novel translation-based unsupervised deformable image registration approach to convert the multi-modal registration problem to a mono-modal one. Specifically, our approach incorporates a discriminator-free translation network to facilitate the training of the registration network and a patchwise contrastive loss to encourage the translation network to preserve object shapes. Furthermore, we propose to replace an adversarial loss, that is widely used in previous multi-modal image registration methods, with a pixel loss in order to integrate the output of translation into the target modality. This leads to an unsupervised method requiring no ground-truth deformation or pairs of aligned images for training. We evaluate four variants of our approach on the public Learn2Reg 2021 datasets \cite{hering2021learn2reg}. The experimental results demonstrate that the proposed architecture achieves state-of-the-art performance. Our code is available at https://github.com/heyblackC/DFMIR.

cs.CV↗

arXiv2022

Slice Imputation: Intermediate Slice Interpolation for Anisotropic 3D Medical Image Segmentation

We introduce a novel frame-interpolation-based method for slice imputation to improve segmentation accuracy for anisotropic 3D medical images, in which the number of slices and their corresponding segmentation labels can be increased between two consecutive slices in anisotropic 3D medical volumes. Unlike previous inter-slice imputation methods, which only focus on the smoothness in the axial direction, this study aims to improve the smoothness of the interpolated 3D medical volumes in all three directions: axial, sagittal, and coronal. The proposed multitask inter-slice imputation method, in particular, incorporates a smoothness loss function to evaluate the smoothness of the interpolated 3D medical volumes in the through-plane direction (sagittal and coronal). It not only improves the resolution of the interpolated 3D medical volumes in the through-plane direction but also transforms them into isotropic representations, which leads to better segmentation performances. Experiments on whole tumor segmentation in the brain, liver tumor segmentation, and prostate segmentation indicate that our method outperforms the competing slice imputation methods on both computed tomography and magnetic resonance images volumes in most cases.

eess.IV↗

arXiv2021

A Compact Neural Network-based Algorithm for Robust Image Watermarking

Digital image watermarking seeks to protect the digital media information from unauthorized access, where the message is embedded into the digital image and extracted from it, even some noises or distortions are applied under various data processing including lossy image compression and interactive content editing. Traditional image watermarking solutions easily suffer from robustness when specified with some prior constraints, while recent deep learning-based watermarking methods could not tackle the information loss problem well under various separate pipelines of feature encoder and decoder. In this paper, we propose a novel digital image watermarking solution with a compact neural network, named Invertible Watermarking Network (IWN). Our IWN architecture is based on a single Invertible Neural Network (INN), this bijective propagation framework enables us to effectively solve the challenge of message embedding and extraction simultaneously, by taking them as a pair of inverse problems for each other and learning a stable invertible mapping. In order to enhance the robustness of our watermarking solution, we specifically introduce a simple but effective bit message normalization module to condense the bit message to be embedded, and a noise layer is designed to simulate various practical attacks under our IWN framework. Extensive experiments demonstrate the superiority of our solution under various distortions.

cs.CV↗

arXiv2021

TarGAN: Target-Aware Generative Adversarial Networks for Multi-modality Medical Image Translation

Paired multi-modality medical images, can provide complementary information to help physicians make more reasonable decisions than single modality medical images. But they are difficult to generate due to multiple factors in practice (e.g., time, cost, radiation dose). To address these problems, multi-modality medical image translation has aroused increasing research interest recently. However, the existing works mainly focus on translation effect of a whole image instead of a critical target area or Region of Interest (ROI), e.g., organ and so on. This leads to poor-quality translation of the localized target area which becomes blurry, deformed or even with extra unreasonable textures. In this paper, we propose a novel target-aware generative adversarial network called TarGAN, which is a generic multi-modality medical image translation model capable of (1) learning multi-modality medical image translation without relying on paired data, (2) enhancing quality of target area generation with the help of target area labels. The generator of TarGAN jointly learns mapping at two levels simultaneously - whole image translation mapping and target area translation mapping. These two mappings are interrelated through a proposed crossing loss. The experiments on both quantitative measures and qualitative evaluations demonstrate that TarGAN outperforms the state-of-the-art methods in all cases. Subsequent segmentation task is conducted to demonstrate effectiveness of synthetic images generated by TarGAN in a real-world application. Our code is available at https://github.com/2165998/TarGAN.

eess.IV↗

arXiv2020

Tips and Tricks for Webly-Supervised Fine-Grained Recognition: Learning from the WebFG 2020 Challenge

WebFG 2020 is an international challenge hosted by Nanjing University of Science and Technology, University of Edinburgh, Nanjing University, The University of Adelaide, Waseda University, etc. This challenge mainly pays attention to the webly-supervised fine-grained recognition problem. In the literature, existing deep learning methods highly rely on large-scale and high-quality labeled training data, which poses a limitation to their practicability and scalability in real world applications. In particular, for fine-grained recognition, a visual task that requires professional knowledge for labeling, the cost of acquiring labeled training data is quite high. It causes extreme difficulties to obtain a large amount of high-quality training data. Therefore, utilizing free web data to train fine-grained recognition models has attracted increasing attentions from researchers in the fine-grained community. This challenge expects participants to develop webly-supervised fine-grained recognition methods, which leverages web images in training fine-grained recognition models to ease the extreme dependence of deep learning methods on large-scale manually labeled datasets and to enhance their practicability and scalability. In this technical report, we have pulled together the top WebFG 2020 solutions of total 54 competing teams, and discuss what methods worked best across the set of winning teams, and what surprisingly did not help.

cs.CV↗

arXiv2020

Inter-slice image augmentation based on frame interpolation for boosting medical image segmentation accuracy

We introduce the idea of inter-slice image augmentation whereby the numbers of the medical images and the corresponding segmentation labels are increased between two consecutive images in order to boost medical image segmentation accuracy. Unlike conventional data augmentation methods in medical imaging, which only increase the number of training samples directly by adding new virtual samples using simple parameterized transformations such as rotation, flipping, scaling, etc., we aim to augment data based on the relationship between two consecutive images, which increases not only the number but also the information of training samples. For this purpose, we propose a frame-interpolation-based data augmentation method to generate intermediate medical images and the corresponding segmentation labels between two consecutive images. We train and test a supervised U-Net liver segmentation network on SLIVER07 and CHAOS2019, respectively, with the augmented training samples, and obtain segmentation scores exhibiting significant improvement compared to the conventional augmentation methods.

eess.IV↗

arXiv2019

Unified Attentional Generative Adversarial Network for Brain Tumor Segmentation From Multimodal Unpaired Images

In medical applications, the same anatomical structures may be observed in multiple modalities despite the different image characteristics. Currently, most deep models for multimodal segmentation rely on paired registered images. However, multimodal paired registered images are difficult to obtain in many cases. Therefore, developing a model that can segment the target objects from different modalities with unpaired images is significant for many clinical applications. In this work, we propose a novel two-stream translation and segmentation unified attentional generative adversarial network (UAGAN), which can perform any-to-any image modality translation and segment the target objects simultaneously in the case where two or more modalities are available. The translation stream is used to capture modality-invariant features of the target anatomical structures. In addition, to focus on segmentation-related features, we add attentional blocks to extract valuable features from the translation stream. Experiments on three-modality brain tumor segmentation indicate that UAGAN outperforms the existing methods in most cases.

cs.CV↗

arXiv2018

The degrees, number of edges, spectral radius and weakly Hamilton-connectedness of bipartite graphs

A path of a graph $G$ is called a Hamilton path if it passes through all the vertices of $G$. A graph is Hamilton-connected if any two vertices are connected by a Hamilton path. Note that any bipartite graph is not Hamilton-connected. We consider the weak version of Hamilton-connected property among bipartite graphs. A weakly Hamilton-connected graph is a balanced bipartite graph $G=(X,Y,E)$ that there is a Hamilton path between any vertex $x\in X$ and $y\in Y$. In this paper, we present some degrees, number of edges, and spectral radius conditions for a simple balanced bipartite graph to be weakly Hamilton-connected.

math.CO↗

arXiv2013

A hybrid HDMR for mixed multiscale finite element method with application for flows in random porous media

Stochastic modeling has become a popular approach to quantify uncertainty in flows through heterogeneous porous media. The uncertainty in heterogeneous structure properties is often parameterized by a high-dimensional random variable. This leads to a deterministic problem in a high-dimensional parameter space and the numerical computation becomes very challengeable as the dimension of the parameter space increases. To efficiently tackle the high-dimensionality, we propose a hybrid high dimensional model representation (HDMR) technique, through which the high-dimensional stochastic model is decomposed into a moderate-dimensional stochastic model in a most active random space and a few one-dimensional stochastic models. The derived low-dimensional stochastic models are solved by incorporating sparse grid stochastic collocation method into the proposed hybrid HDMR. The porous media properties such as permeability are often heterogeneous. To treat the heterogeneity, we use a mixed multiscale finite element method (MMsFEM) to simulate each of derived stochastic models. To capture the non-local spatial features of the porous media and the important effects of random variables, we can hierarchically incorporate the global information individually from each of random parameters. This significantly enhances the accuracy of the multiscale simulation. The synergy of the hybrid HDMR and the MMsFEM reduces the stochastic model of flows in both stochastic space and physical space, and significantly decreases the computation complexity. We carefully analyze the proposed HDMR technique and the derived stochastic MMsFEM. A few numerical experiments are carried out for two-phase flows in random porous media and support the efficiency and accuracy of the MMsFEM based on the hybrid HDMR.

math.NA↗

arXiv2010

High-resolution angle-resolved photoemission spectroscopy study of the electronic structure of EuFe2As2

We report the high-resolution angle-resolved photoemission spectroscopy studies of electronic structure of EuFe2As2. The paramagnetic state data are found to be consistent with density-functional calculations. In the antiferromagnetic ordering state of Fe, our results show that the band splitting, folding, and hybridization evolve with temperature, which cannot be explained by a simple folding picture. Detailed measurements reveal that a tiny electron Fermi pocket and a tiny hole pocket are formed near (pi,pi) in the (0,0)-(pi,pi) direction, which qualitatively agree with the results of quantum oscillations, considering kz variation of Fermi surface. Furthermore, no noticeable change within the energy resolution is observed across the antiferromagnetic transition of Eu2+ ordering, suggesting weak coupling between Eu sublattice and FeAs sublattice.

cond-mat.str-el↗

arXiv2010

Electronic structure of Fe1.04(Te0.66Se0.34)

We report the electronic structure of the iron-chalcogenide superconductor, Fe1.04(Te0.66Se0.34), obtained with high resolution angle-resolved photoemission spectroscopy and density functional calculations. In photoemission measurements, various photon energies and polarizations are exploited to study the Fermi surface topology and symmetry properties of the bands. The measured band structure and their symmetry characters qualitatively agree with our density function theory calculations of Fe(Te0.66Se0.34), although the band structure is renormalized by about a factor of three. We find that the electronic structures of this iron-chalcogenides and the iron-pnictides have many aspects in common, however, significant differences exist near the Gamma-point. For Fe1.04(Te0.66Se0.34), there are clearly separated three bands with distinct even or odd symmetry that cross the Fermi energy (EF) near the zone center, which contribute to three hole-like Fermi surfaces. Especially, both experiments and calculations show a hole-like elliptical Fermi surface at the zone center. Moreover, no sign of spin density wave was observed in the electronic structure and susceptibility measurements of this compound.

cond-mat.str-el↗