Searcharxiv⌕ Search

arXiv subjects

Yu Zeng

Publications and source records attributed to Yu Zeng.

At least 73 records · Page 4Linked to original sources
arXiv2024

Holo-Relighting: Controllable Volumetric Portrait Relighting from a Single Image

At the core of portrait photography is the search for ideal lighting and viewpoint. The process often requires advanced knowledge in photography and an elaborate studio setup. In this work, we propose Holo-Relighting, a volumetric relighting method that is capable of synthesizing novel viewpoints, and novel lighting from a single image. Holo-Relighting leverages the pretrained 3D GAN (EG3D) to reconstruct geometry and appearance from an input portrait as a set of 3D-aware features. We design a relighting module conditioned on a given lighting to process these features, and predict a relit 3D representation in the form of a tri-plane, which can render to an arbitrary viewpoint through volume rendering. Besides viewpoint and lighting control, Holo-Relighting also takes the head pose as a condition to enable head-pose-dependent lighting effects. With these novel designs, Holo-Relighting can generate complex non-Lambertian lighting effects (e.g., specular highlights and cast shadows) without using any explicit physical lighting priors. We train Holo-Relighting with data captured with a light stage, and propose two data-rendering techniques to improve the data quality for training the volumetric relighting system. Through quantitative and qualitative experiments, we demonstrate Holo-Relighting can achieve state-of-the-arts relighting quality with better photorealism, 3D consistency and controllability.

cs.CV↗
arXiv2023

Finite groups with a small proportion of vanishing elements

The function $\mathrm{P}_{\mathbf{v}}(G)$, measuring the proportion of the elements of a finite group $G$ that are zeros of irreducible characters of $G$, takes (as proved in [12]) only values $\frac{m-1}{m}$, for $1 \leq m \leq 6$, in the interval $[0, \mathrm{P}_{\mathbf{v}}(A_7))$.In this paper, we give a complete classification of the finite groups $G$ such that $\mathrm{P}_{\mathbf{v}}(G)=\frac{m-1}{m}$ for $m=1,2,\cdots ,6$.

math.GR↗
arXiv2023

Single-Image-Based Deep Learning for Segmentation of Early Esophageal Cancer Lesions

Accurate segmentation of lesions is crucial for diagnosis and treatment of early esophageal cancer (EEC). However, neither traditional nor deep learning-based methods up to today can meet the clinical requirements, with the mean Dice score - the most important metric in medical image analysis - hardly exceeding 0.75. In this paper, we present a novel deep learning approach for segmenting EEC lesions. Our approach stands out for its uniqueness, as it relies solely on a single image coming from one patient, forming the so-called "You-Only-Have-One" (YOHO) framework. On one hand, this "one-image-one-network" learning ensures complete patient privacy as it does not use any images from other patients as the training data. On the other hand, it avoids nearly all generalization-related problems since each trained network is applied only to the input image itself. In particular, we can push the training to "over-fitting" as much as possible to increase the segmentation accuracy. Our technical details include an interaction with clinical physicians to utilize their expertise, a geometry-based rendering of a single lesion image to generate the training set (the \emph{biggest} novelty), and an edge-enhanced UNet. We have evaluated YOHO over an EEC data-set created by ourselves and achieved a mean Dice score of 0.888, which represents a significant advance toward clinical applications.

eess.IV↗
arXiv2023

Securing Deep Generative Models with Universal Adversarial Signature

Recent advances in deep generative models have led to the development of methods capable of synthesizing high-quality, realistic images. These models pose threats to society due to their potential misuse. Prior research attempted to mitigate these threats by detecting generated images, but the varying traces left by different generative models make it challenging to create a universal detector capable of generalizing to new, unseen generative models. In this paper, we propose to inject a universal adversarial signature into an arbitrary pre-trained generative model, in order to make its generated contents more detectable and traceable. First, the imperceptible optimal signature for each image can be found by a signature injector through adversarial training. Subsequently, the signature can be incorporated into an arbitrary generator by fine-tuning it with the images processed by the signature injector. In this way, the detector corresponding to the signature can be reused for any fine-tuned generator for tracking the generator identity. The proposed method is validated on the FFHQ and ImageNet datasets with various state-of-the-art generative models, consistently showing a promising detection rate. Code will be made publicly available at \url{https://github.com/zengxianyu/genwm}.

cs.CV↗
arXiv2022

SceneComposer: Any-Level Semantic Image Synthesis

We propose a new framework for conditional image synthesis from semantic layouts of any precision levels, ranging from pure text to a 2D semantic canvas with precise shapes. More specifically, the input layout consists of one or more semantic regions with free-form text descriptions and adjustable precision levels, which can be set based on the desired controllability. The framework naturally reduces to text-to-image (T2I) at the lowest level with no shape information, and it becomes segmentation-to-image (S2I) at the highest level. By supporting the levels in-between, our framework is flexible in assisting users of different drawing expertise and at different stages of their creative workflow. We introduce several novel techniques to address the challenges coming with this new setup, including a pipeline for collecting training data; a precision-encoded mask pyramid and a text feature map representation to jointly encode precision level, semantics, and composition information; and a multi-scale guided diffusion model to synthesize images. To evaluate the proposed method, we collect a test dataset containing user-drawn layouts with diverse scenes and styles. Experimental results show that the proposed method can generate high-quality images following the layout at given precision, and compares favorably against existing methods. Project page \url{https://zengxianyu.github.io/scenec/}

cs.CV↗
arXiv2022

Nonsolvable groups with three nonlinear irreducible character codegrees

For an irreducible character $χ$ of a finite group $G$, the codegree of $χ$ is defined as $|G:\ker(χ)|/χ(1)$. In this paper, we determine finite nonsolvable groups with exactly three nonlinear irreducible character codegrees, and they are $\mathrm{L}_2(2^f)$ for $f\ge 2$, $\mathrm{PGL}_2(q)$ for odd $q\ge 5$ or $\mathrm{M}_{10}$.

math.GR↗
arXiv2022

Finite groups with some bounded codegrees

For a character $χ$ of a finite group $G$, the number cod$(χ):=|G:\mathrm{ker}(χ)|/χ(1)$ is called the codegree of $χ$.In this paper, we give a solvability criterion for a finite group $G$ depending on the minimum of the ratio $χ(1)^2 /\mathrm{cod}(χ)$, when $χ$ varies among the irreducible characters of $G$.

math.GR↗
arXiv2022

On the proportion of vanishing elements in finite groups

We prove that the function $\mathrm{P}_{\mathrm{v}}(G)$, measuring the proportion of the elements of a finite group $G$ that are zeros of irreducible characters of $G$, takes very sparse values in a large segment of the $[0,1]$ interval.

math.GR↗
arXiv2022

Shape-guided Object Inpainting

Previous works on image inpainting mainly focus on inpainting background or partially missing objects, while the problem of inpainting an entire missing object remains unexplored. This work studies a new image inpainting task, i.e. shape-guided object inpainting. Given an incomplete input image, the goal is to fill in the hole by generating an object based on the context and implicit guidance given by the hole shape. Since previous methods for image inpainting are mainly designed for background inpainting, they are not suitable for this task. Therefore, we propose a new data preparation method and a novel Contextual Object Generator (CogNet) for the object inpainting task. On the data side, we incorporate object priors into training data by using object instances as holes. The CogNet has a two-stream architecture that combines the standard bottom-up image completion process with a top-down object generation process. A predictive class embedding module bridges the two streams by predicting the class of the missing object from the bottom-up features, from which a semantic object map is derived as the input of the top-down stream. Experiments demonstrate that the proposed method can generate realistic objects that fit the context in terms of both visual appearance and semantic meanings. Code can be found at the project page \url{https://zengxianyu.github.io/objpaint}

cs.CV↗
arXiv2022

On finite groups with certain complemented $p$-subgroups

Given a prime power $p^d$ with $p$ a prime and $d$ a positive integer, we classify the finite groups $G$ with $p^{2d}$ dividing $|G|$ in which all subgroups of order $p^d$ are complemented and the finite groups $G$ having a normal elementary abelian Sylow $p$-subgroup $P$ such that $p^d<|P|$ in which all subgroups of order $p^d$ are complemented.

math.GR↗
arXiv2021

SketchEdit: Mask-Free Local Image Manipulation with Partial Sketches

Sketch-based image manipulation is an interactive image editing task to modify an image based on input sketches from users. Existing methods typically formulate this task as a conditional inpainting problem, which requires users to draw an extra mask indicating the region to modify in addition to sketches. The masked regions are regarded as holes and filled by an inpainting model conditioned on the sketch. With this formulation, paired training data can be easily obtained by randomly creating masks and extracting edges or contours. Although this setup simplifies data preparation and model design, it complicates user interaction and discards useful information in masked regions. To this end, we investigate a new paradigm of sketch-based image manipulation: mask-free local image manipulation, which only requires sketch inputs from users and utilizes the entire original image. Given an image and sketch, our model automatically predicts the target modification region and encodes it into a structure agnostic style vector. A generator then synthesizes the new image content based on the style vector and sketch. The manipulated image is finally produced by blending the generator output into the modification region of the original image. Our model can be trained in a self-supervised fashion by learning the reconstruction of an image region from the style vector and sketch. The proposed method offers simpler and more intuitive user workflows for sketch-based image manipulation and provides better results than previous approaches. More results, code and interactive demo will be available at \url{https://zengxianyu.github.io/sketchedit}.

cs.CV↗
arXiv2021

On $\mathcal{M}$-supplemented subgroups

Let $G$ be a finite group and $p^k$ be a prime power dividing $|G|$. A subgroup $H$ of $G$ is called to be $\mathcal{M}$-supplemented in $G$ if there exists a subgroup $K$ of $G$ such that $G=HK$ and $H_iK<G$ for every maximal subgroup $H_i$ of $H$. In this paper, we complete the classification of the finite groups $G$ in which all subgroups of order $p^k$ are $\mathcal{M}$-supplemented. In particular, we show that if $k\geq 2$, then $G/\mathrm{O}_{p'}(G)$ is supersolvable with a normal Sylow $p$-subgroup and a cyclic $p$-complement.

math.GR↗
arXiv2021

Advancing COVID-19 Diagnosis with Privacy-Preserving Collaboration in Artificial Intelligence

Artificial intelligence (AI) provides a promising substitution for streamlining COVID-19 diagnoses. However, concerns surrounding security and trustworthiness impede the collection of large-scale representative medical data, posing a considerable challenge for training a well-generalised model in clinical practices. To address this, we launch the Unified CT-COVID AI Diagnostic Initiative (UCADI), where the AI model can be distributedly trained and independently executed at each host institution under a federated learning framework (FL) without data sharing. Here we show that our FL model outperformed all the local models by a large yield (test sensitivity /specificity in China: 0.973/0.951, in the UK: 0.730/0.942), achieving comparable performance with a panel of professional radiologists. We further evaluated the model on the hold-out (collected from another two hospitals leaving out the FL) and heterogeneous (acquired with contrast materials) data, provided visual explanations for decisions made by the model, and analysed the trade-offs between the model performance and the communication costs in the federated training process. Our study is based on 9,573 chest computed tomography scans (CTs) from 3,336 patients collected from 23 hospitals located in China and the UK. Collectively, our work advanced the prospects of utilising federated learning for privacy-preserving AI in digital health.

cs.AI↗
arXiv2021

Multipartite entanglement of the topologically ordered state in a perturbed toric code

We demonstrate that multipartite entanglement, witnessed by the quantum Fisher information (QFI), can characterize topological quantum phase transitions in the spin-$\frac{1}{2}$ toric code model on a square lattice with external fields. We show that the QFI density of the ground state can be written in terms of the expectation values of gauge-invariant Wilson loops for different sizes of square regions and identify $\mathbb{Z}_2$ topological order by its scaling behavior. Furthermore, we use this multipartite entanglement witness to investigate thermalization and disorder-assisted stabilization of topological order after a quantum quench. Moreover, with an upper bound of the QFI, we demonstrate the absence of finite-temperature topological order in the 2D toric code model in the thermodynamic limit. Our results provide insights to topological phases, which are robust against external disturbances, and are candidates for topologically protected quantum computation.

quant-ph↗
arXiv2021

CR-Fill: Generative Image Inpainting with Auxiliary Contexutal Reconstruction

Recent deep generative inpainting methods use attention layers to allow the generator to explicitly borrow feature patches from the known region to complete a missing region. Due to the lack of supervision signals for the correspondence between missing regions and known regions, it may fail to find proper reference features, which often leads to artifacts in the results. Also, it computes pair-wise similarity across the entire feature map during inference bringing a significant computational overhead. To address this issue, we propose to teach such patch-borrowing behavior to an attention-free generator by joint training of an auxiliary contextual reconstruction task, which encourages the generated output to be plausible even when reconstructed by surrounding regions. The auxiliary branch can be seen as a learnable loss function, i.e. named as contextual reconstruction (CR) loss, where query-reference feature similarity and reference-based reconstructor are jointly optimized with the inpainting generator. The auxiliary branch (i.e. CR loss) is required only during training, and only the inpainting generator is required during the inference. Experimental results demonstrate that the proposed inpainting model compares favourably against the state-of-the-art in terms of quantitative and visual performance.

cs.CV↗
arXiv2021

Protecting topological order by dynamical localization

As a prototype model of topological quantum memory, two-dimensional toric code is genuinely immune to generic local static perturbations, but fragile at finite temperature and also after non-equilibrium time evolution at zero temperature. We show that dynamical localization induced by disorder makes the time evolution a local unitary transformation at all times, which keeps topological order robust after a quantum quench. We verify this conclusion by investigating the Wilson loop expectation value and topological entanglement entropy. Our results suggest that the two dimensional topological quantum memory can be dynamically robust at zero temperature.

quant-ph↗
arXiv2020

High-Resolution Image Inpainting with Iterative Confidence Feedback and Guided Upsampling

Existing image inpainting methods often produce artifacts when dealing with large holes in real applications. To address this challenge, we propose an iterative inpainting method with a feedback mechanism. Specifically, we introduce a deep generative model which not only outputs an inpainting result but also a corresponding confidence map. Using this map as feedback, it progressively fills the hole by trusting only high-confidence pixels inside the hole at each iteration and focuses on the remaining pixels in the next iteration. As it reuses partial predictions from the previous iterations as known pixels, this process gradually improves the result. In addition, we propose a guided upsampling network to enable generation of high-resolution inpainting results. We achieve this by extending the Contextual Attention module to borrow high-resolution feature patches in the input image. Furthermore, to mimic real object removal scenarios, we collect a large object mask dataset and synthesize more realistic training data that better simulates user inputs. Experiments show that our method significantly outperforms existing methods in both quantitative and qualitative evaluations. More results and Web APP are available at https://zengxianyu.github.io/iic.

cs.CV↗