arXiv subjects

Jing He

Publications and source records attributed to Jing He.

At least 55 records · Page 3Linked to original sources

arXiv2024

OmniBooth: Learning Latent Control for Image Synthesis with Multi-modal Instruction

We present OmniBooth, an image generation framework that enables spatial control with instance-level multi-modal customization. For all instances, the multimodal instruction can be described through text prompts or image references. Given a set of user-defined masks and associated text or image guidance, our objective is to generate an image, where multiple objects are positioned at specified coordinates and their attributes are precisely aligned with the corresponding guidance. This approach significantly expands the scope of text-to-image generation, and elevates it to a more versatile and practical dimension in controllability. In this paper, our core contribution lies in the proposed latent control signals, a high-dimensional spatial feature that provides a unified representation to integrate the spatial, textual, and image conditions seamlessly. The text condition extends ControlNet to provide instance-level open-vocabulary generation. The image condition further enables fine-grained control with personalized identity. In practice, our method empowers users with more flexibility in controllable generation, as users can choose multi-modal conditions from text or images as needed. Furthermore, thorough experiments demonstrate our enhanced performance in image synthesis fidelity and alignment across different tasks and datasets. Project page: https://len-li.github.io/omnibooth-web/

cs.CV↗

arXiv2024

The axioms for right (n+2)-angulated categories

Drawing inspiration from the works of Beligiannis-Marmaridis and Lin, we refine the axioms for a right $(n+2)$-angulated category and give some examples of such categories. Interestingly, we show that the morphism axiom for a right $(n+2)$-angulated category is actually redundant. Moreover, we prove that the higher octahedral axiom is equivalent to the mapping cone axiom for a right $(n+2)$-angulated category.

math.RT↗

arXiv2024

$n$-cotorsion pairs and recollements of extriangulated categories

In this article, we prove that if $(\mathcal A ,\mathcal B,\mathcal C)$ is a recollement of extriangulated categories, then $n$-cotorsion pairs in $\mathcal A$ and $\mathcal C$ can induce $n$-cotorsion pairs in $\mathcal B$. Conversely, this holds true under natural assumptions. Besides, we give mild conditions on a pseudo cluster tilting subcategory on the middle category of a recollement of extriangulated categories, for the corresponding additive quotients to form a recollement of semi-abelian categories.

math.RT↗

arXiv2024

Towards Efficient Diffusion-Based Image Editing with Instant Attention Masks

Diffusion-based Image Editing (DIE) is an emerging research hot-spot, which often applies a semantic mask to control the target area for diffusion-based editing. However, most existing solutions obtain these masks via manual operations or off-line processing, greatly reducing their efficiency. In this paper, we propose a novel and efficient image editing method for Text-to-Image (T2I) diffusion models, termed Instant Diffusion Editing(InstDiffEdit). In particular, InstDiffEdit aims to employ the cross-modal attention ability of existing diffusion models to achieve instant mask guidance during the diffusion steps. To reduce the noise of attention maps and realize the full automatics, we equip InstDiffEdit with a training-free refinement scheme to adaptively aggregate the attention distributions for the automatic yet accurate mask generation. Meanwhile, to supplement the existing evaluations of DIE, we propose a new benchmark called Editing-Mask to examine the mask accuracy and local editing ability of existing methods. To validate InstDiffEdit, we also conduct extensive experiments on ImageNet and Imagen, and compare it with a bunch of the SOTA methods. The experimental results show that InstDiffEdit not only outperforms the SOTA methods in both image quality and editing results, but also has a much faster inference speed, i.e., +5 to +6 times.

cs.CV↗

arXiv2024

Hereditary $n$-exangulated categories

Herschend-Liu-Nakaoka introduced the concept of $n$-exangulated categories as higher-dimensional analogues of extriangulated categories defined by Nakaoka-Palu. The class of $n$-exangulated categories contains $n$-exact categories and $(n+2)$-angulated categories as specific examples. In this article, we introduce the notion of hereditary $n$-exangulated categories, which generalize hereditary extriangulated categories. We provide two classes of hereditary $n$-exangulated categories through closed subfunctors. Additionally, we define the concept of $0$-Auslander $n$-exangulated categories and discuss the circumstances under which these two classes of hereditary $n$-exangulated categories become $0$-Auslander.

math.RT↗

arXiv2023

Traffic Video Object Detection using Motion Prior

Traffic videos inherently differ from generic videos in their stationary camera setup, thus providing a strong motion prior where objects often move in a specific direction over a short time interval. Existing works predominantly employ generic video object detection framework for traffic video object detection, which yield certain advantages such as broad applicability and robustness to diverse scenarios. However, they fail to harness the strength of motion prior to enhance detection accuracy. In this work, we propose two innovative methods to exploit the motion prior and boost the performance of both fully-supervised and semi-supervised traffic video object detection. Firstly, we introduce a new self-attention module that leverages the motion prior to guide temporal information integration in the fully-supervised setting. Secondly, we utilise the motion prior to develop a pseudo-labelling mechanism to eliminate noisy pseudo labels for the semi-supervised setting. Both of our motion-prior-centred methods consistently demonstrates superior performance, outperforming existing state-of-the-art approaches by a margin of 2% in terms of mAP.

cs.CV↗

arXiv2023

Can LSH (Locality-Sensitive Hashing) Be Replaced by Neural Network?

With the rapid development of GPU (Graphics Processing Unit) technologies and neural networks, we can explore more appropriate data structures and algorithms. Recent progress shows that neural networks can partly replace traditional data structures. In this paper, we proposed a novel DNN (Deep Neural Network)-based learned locality-sensitive hashing, called LLSH, to efficiently and flexibly map high-dimensional data to low-dimensional space. LLSH replaces the traditional LSH (Locality-sensitive Hashing) function families with parallel multi-layer neural networks, which reduces the time and memory consumption and guarantees query accuracy simultaneously. The proposed LLSH demonstrate the feasibility of replacing the hash index with learning-based neural networks and open a new door for developers to design and configure data organization more accurately to improve information-searching performance. Extensive experiments on different types of datasets show the superiority of the proposed method in query accuracy, time consumption, and memory usage.

cs.IR↗

arXiv2023

Semi-abelian categories arising from pseudo cluster tilting subcategories

The notion of a pseudo cluster tilting subcategory $\mathcal X$ in an extriangulated category $\mathcal C$ is defined in this article. We prove that the quotient category $\mathcal C/\mathcal X$, obtained by factoring an extriangulated category by a pseudo cluster tilting subcategory, is a semi-abelian category. Furthermore, we also show that the quotient category $\mathcal C/\mathcal X$ is an abelian category if and only if certain self-orthogonal conditions are satisfied. As an application, these results generalize the work of Xu and Zheng in the exact category.

math.RT↗

arXiv2023

Statistical inferences for complex dependence of multimodal imaging data

Statistical analysis of multimodal imaging data is a challenging task, since the data involves high-dimensionality, strong spatial correlations and complex data structures. In this paper, we propose rigorous statistical testing procedures for making inferences on the complex dependence of multimodal imaging data. Motivated by the analysis of multi-task fMRI data in the Human Connectome Project (HCP) study, we particularly address three hypothesis testing problems: (a) testing independence among imaging modalities over brain regions, (b) testing independence between brain regions within imaging modalities, and (c) testing independence between brain regions across different modalities. Considering a general form for all the three tests, we develop a global testing procedure and a multiple testing procedure controlling the false discovery rate. We study theoretical properties of the proposed tests and develop a computationally efficient distributed algorithm. The proposed methods and theory are general and relevant for many statistical problems of testing independence structure among the components of high-dimensional random vectors with arbitrary dependence structures. We also illustrate our proposed methods via extensive simulations and analysis of five task fMRI contrast maps in the HCP study.

stat.ME↗

arXiv2023

Deep Learning and Medical Imaging for COVID-19 Diagnosis: A Comprehensive Survey

COVID-19 (Coronavirus disease 2019) has been quickly spreading since its outbreak, impacting financial markets and healthcare systems globally. Countries all around the world have adopted a number of extraordinary steps to restrict the spreading virus, where early COVID-19 diagnosis is essential. Medical images such as X-ray images and Computed Tomography scans are becoming one of the main diagnostic tools to combat COVID-19 with the aid of deep learning-based systems. In this survey, we investigate the main contributions of deep learning applications using medical images in fighting against COVID-19 from the aspects of image classification, lesion localization, and severity quantification, and review different deep learning architectures and some image preprocessing techniques for achieving a preciser diagnosis. We also provide a summary of the X-ray and CT image datasets used in various studies for COVID-19 detection. The key difficulties and potential applications of deep learning in fighting against COVID-19 are finally discussed. This work summarizes the latest methods of deep learning using medical images to diagnose COVID-19, highlighting the challenges and inspiring more studies to keep utilizing the advantages of deep learning to combat COVID-19.

eess.IV↗

arXiv2022

Exploiting Expert Knowledge for Assigning Firms to Industries: A Novel Deep Learning Method

Industry assignment, which assigns firms to industries according to a predefined Industry Classification System (ICS), is fundamental to a large number of critical business practices, ranging from operations and strategic decision making by firms to economic analyses by government agencies. Three types of expert knowledge are essential to effective industry assignment: definition-based knowledge (i.e., expert definitions of each industry), structure-based knowledge (i.e., structural relationships among industries as specified in an ICS), and assignment-based knowledge (i.e., prior firm-industry assignments performed by domain experts). Existing industry assignment methods utilize only assignment-based knowledge to learn a model that classifies unassigned firms to industries, and overlook definition-based and structure-based knowledge. Moreover, these methods only consider which industry a firm has been assigned to, but ignore the time-specificity of assignment-based knowledge, i.e., when the assignment occurs. To address the limitations of existing methods, we propose a novel deep learning-based method that not only seamlessly integrates the three types of knowledge for industry assignment but also takes the time-specificity of assignment-based knowledge into account. Methodologically, our method features two innovations: dynamic industry representation and hierarchical assignment. The former represents an industry as a sequence of time-specific vectors by integrating the three types of knowledge through our proposed temporal and spatial aggregation mechanisms. The latter takes industry and firm representations as inputs, computes the probability of assigning a firm to different industries, and assigns the firm to the industry with the highest probability.

cs.LG↗

arXiv2022

PixelFolder: An Efficient Progressive Pixel Synthesis Network for Image Generation

Pixel synthesis is a promising research paradigm for image generation, which can well exploit pixel-wise prior knowledge for generation. However, existing methods still suffer from excessive memory footprint and computation overhead. In this paper, we propose a progressive pixel synthesis network towards efficient image generation, coined as PixelFolder. Specifically, PixelFolder formulates image generation as a progressive pixel regression problem and synthesizes images via a multi-stage structure, which can greatly reduce the overhead caused by large tensor transformations. In addition, we introduce novel pixel folding operations to further improve model efficiency while maintaining pixel-wise prior knowledge for end-to-end regression. With these innovative designs, we greatly reduce the expenditure of pixel synthesis, e.g., reducing 89% computation and 53% parameters compared with the latest pixel synthesis method CIPS. To validate our approach, we conduct extensive experiments on two benchmark datasets, namely FFHQ and LSUN Church. The experimental results show that with much less expenditure, PixelFolder obtains new state-of-the-art (SOTA) performance on two benchmark datasets, i.e., 3.77 FID and 2.45 FID on FFHQ and LSUN Church, respectively.Meanwhile, PixelFolder is also more efficient than the SOTA methods like StyleGAN2, reducing about 72% computation and 31% parameters, respectively. These results greatly validate the effectiveness of the proposed PixelFolder.

cs.CV↗

arXiv2022

Modelling matrix time series via a tensor CP-decomposition

We consider to model matrix time series based on a tensor CP-decomposition. Instead of using an iterative algorithm which is the standard practice for estimating CP-decompositions, we propose a new and one-pass estimation procedure based on a generalized eigenanalysis constructed from the serial dependence structure of the underlying process. To overcome the intricacy of solving a rank-reduced generalized eigenequation, we propose a further refined approach which projects it into a lower-dimensional full-ranked eigenequation. This refined method improves significantly the finite-sample performance of the estimation. The asymptotic theory has been established under a general setting without the stationarity. It shows, for example, that all the component coefficient vectors in the CP-decomposition are estimated consistently with certain convergence rates. The proposed model and the estimation method are also illustrated with both simulated and real data; showing effective dimension-reduction in modelling and forecasting matrix time series.

stat.ME↗

arXiv2022

Efficient Embedding VNFs in 5G Network Slicing: A Deep Reinforcement Learning Approach

5G radio access network (RAN) slicing aims to logically split an infrastructure into a set of self-contained programmable RAN slices, with each slice built on top of the underlying physical RAN (substrate) is a separate logical mobile network, which delivers a set of services with similar characteristics. Each RAN slice is constituted by various virtual network functions (VNFs) distributed geographically in numerous substrate nodes. A key challenge in building a robust RAN slicing is, therefore, designing a RAN slicing (RS)-configuration scheme that can utilize information such as resource availability in substrate networks as well as the interdependent relationships among slices to map (embed) VNFs onto live substrate nodes. With such motivation, we propose a machine-learning-powered RAN slicing scheme that aims to accommodate maximum numbers of slices (a set of connected Virtual Network Functions - VNFs) within a given request set. More specifically, we present a deep reinforcement scheme that is called Deep Allocation Agent (DAA). In short, DAA utilizes an empirically designed deep neural network that observes the current states of the substrate network and the requested slices to schedule the slices of which VNFs are then mapped to substrate nodes using an optimization algorithm. DAA is trained towards the goal of maximizing the number of accommodated slices in the given set by using an explicitly designed reward function. Our experiment study shows that, on average, DAA is able to maintain a rate of successfully routed slices above 80% in a resource-limited substrate network, and about 60% in extreme conditions, i.e., the available resources are much less than the demands.

cs.NI↗

arXiv2022

Pre-$(n+2)$-angulated categories

In this article, we introduce the notion of pre-$(n+2)$-angulated categories as higher dimensional analogues of pre-triangulated categories defined by Beligiannis-Reiten. We first show that the idempotent completion of a pre-$(n+2)$-angulated category admits a unique structure of pre-$(n+2)$-angulated category. Let $(\mathscr{C},\mathbb{E},\mathfrak{s})$ be an $n$-exangulated category and $\mathscr{X}$ be a strongly functorially finite subcategory of $\mathscr{C}$. We then show that the quotient category $\mathscr{C}/\mathscr{X}$ is a pre-$(n+2)$-angulated category.These results allow to construct several examples of pre-$(n+2)$-angulated categories. Moreover, we also give a necessary and sufficient condition for the quotient $\mathscr{C}/\mathscr{X}$ to be an $(n+2)$-angulated category.

math.RT↗

arXiv2022

Idempotent completion of certain $n$-exangulated categories

It was shown recently that an $n$-extension closed subcategory $\mathscr A$ of a Krull-Schmidt $(n+2)$-angulated category has a natural structure of an $n$-exangulated category. In this article, we prove that its idempotent completion $\widetilde{\mathscr A}$ admits an $n$-exangulated structure. It is not only a generalization of the main result of Lin, but also gives an $n$-exangulated category which is neither $n$-exact nor $(n+2)$-angulated in general.

math.RT↗

arXiv2022

Homotopy cartesian squares in extriangulated categories

Let $(\mathcal{C},\mathbb{E},\mathfrak{s})$ be an extriangulated category. Given a composition of two commutative squares in $\mathcal{C}$, if two commutative squares are homotopy cartesian, then their composition is also a homotopy cartesian. This covers the result by Mac Lane (1998) for abelian categories and the result by Christensen and Frankland (2022) for triangulated categories.

math.RT↗

arXiv2022

Two results of $n$-exangulated categories

$n$-exangulated categories were introduced by Herschend-Liu-Nakaoka which are a simultaneous generalization of $n$-exact categories and $(n+2)$-angulated categories. This paper consists of two results on $n$-exangulated categories: (1) we give an equivalent characterization of the axiom (EA2); (2) we provide a new way to construct a closed subfunctor of an $n$-exangulated category.

math.RT↗