Searcharxiv⌕ Search

arXiv subjects

Andrey Kuznetsov

Publications and source records attributed to Andrey Kuznetsov.

63 records · Page 4Linked to original sources
arXiv2023

Kandinsky: an Improved Text-to-Image Synthesis with Image Prior and Latent Diffusion

Text-to-image generation is a significant domain in modern computer vision and has achieved substantial improvements through the evolution of generative architectures. Among these, there are diffusion-based models that have demonstrated essential quality enhancements. These models are generally split into two categories: pixel-level and latent-level approaches. We present Kandinsky1, a novel exploration of latent diffusion architecture, combining the principles of the image prior models with latent diffusion techniques. The image prior model is trained separately to map text embeddings to image embeddings of CLIP. Another distinct feature of the proposed model is the modified MoVQ implementation, which serves as the image autoencoder component. Overall, the designed model contains 3.3B parameters. We also deployed a user-friendly demo system that supports diverse generative modes such as text-to-image generation, image fusion, text and image fusion, image variations generation, and text-guided inpainting/outpainting. Additionally, we released the source code and checkpoints for the Kandinsky models. Experimental evaluations demonstrate a FID score of 8.03 on the COCO-30K dataset, marking our model as the top open-source performer in terms of measurable image generation quality.

cs.CV↗
arXiv2023

RusTitW: Russian Language Text Dataset for Visual Text in-the-Wild Recognition

Information surrounds people in modern life. Text is a very efficient type of information that people use for communication for centuries. However, automated text-in-the-wild recognition remains a challenging problem. The major limitation for a DL system is the lack of training data. For the competitive performance, training set must contain many samples that replicate the real-world cases. While there are many high-quality datasets for English text recognition; there are no available datasets for Russian language. In this paper, we present a large-scale human-labeled dataset for Russian text recognition in-the-wild. We also publish a synthetic dataset and code to reproduce the generation process

cs.CV↗
arXiv2022

Many Heads but One Brain: Fusion Brain -- a Competition and a Single Multimodal Multitask Architecture

Supporting the current trend in the AI community, we present the AI Journey 2021 Challenge called Fusion Brain, the first competition which is targeted to make the universal architecture which could process different modalities (in this case, images, texts, and code) and solve multiple tasks for vision and language. The Fusion Brain Challenge combines the following specific tasks: Code2code Translation, Handwritten Text recognition, Zero-shot Object Detection, and Visual Question Answering. We have created datasets for each task to test the participants' submissions on it. Moreover, we have collected and made publicly available a new handwritten dataset in both English and Russian, which consists of 94,128 pairs of images and texts. We also propose a multimodal and multitask architecture - a baseline solution, in the center of which is a frozen foundation model and which has been trained in Fusion mode along with Single-task mode. The proposed Fusion approach proves to be competitive and more energy-efficient compared to the task-specific one.

cs.CV↗
arXiv2022

Design and Performance of the Prototype Schwarzschild-Couder Telescope Camera

The prototype Schwarzschild-Couder Telescope (pSCT) is a candidate for a medium-sized telescope in the Cherenkov Telescope Array. The pSCT is based on a novel dual mirror optics design which reduces the plate scale and allows for the use of silicon photomultipliers as photodetectors. The prototype pSCT camera currently has only the central sector instrumented with 25 camera modules (1600 pixels), providing a 2.68$^{\circ}$ field of view (FoV). The camera electronics are based on custom TARGET (TeV array readout with GSa/s sampling and event trigger) application specific integrated circuits. Field programmable gate arrays sample incoming signals at a gigasample per second. A single backplane provides camera-wide triggers. An upgrade of the pSCT camera is in progress, which will fully populate the focal plane. This will increase the number of pixels to 11,328, the number of backplanes to 9, and the FoV to 8.04$^{\circ}$. Here we give a detailed description of the pSCT camera, including the basic concept, mechanical design, detectors, electronics, current status and first light.

astro-ph.IM↗
arXiv2022

RuCLIP -- new models and experiments: a technical report

In the report we propose six new implementations of ruCLIP model trained on our 240M pairs. The accuracy results are compared with original CLIP model with Ru-En translation (OPUS-MT) on 16 datasets from different domains. Our best implementations outperform CLIP + OPUS-MT solution on most of the datasets in few-show and zero-shot tasks. In the report we briefly describe the implementations and concentrate on the conducted experiments. Inference execution time comparison is also presented in the report.

cs.CV↗
arXiv2022

A new face swap method for image and video domains: a technical report

Deep fake technology became a hot field of research in the last few years. Researchers investigate sophisticated Generative Adversarial Networks (GAN), autoencoders, and other approaches to establish precise and robust algorithms for face swapping. Achieved results show that the deep fake unsupervised synthesis task has problems in terms of the visual quality of generated data. These problems usually lead to high fake detection accuracy when an expert analyzes them. The first problem is that existing image-to-image approaches do not consider video domain specificity and frame-by-frame processing leads to face jittering and other clearly visible distortions. Another problem is the generated data resolution, which is low for many existing methods due to high computational complexity. The third problem appears when the source face has larger proportions (like bigger cheeks), and after replacement it becomes visible on the face border. Our main goal was to develop such an approach that could solve these problems and outperform existing solutions on a number of clue metrics. We introduce a new face swap pipeline that is based on FaceShifter architecture and fixes the problems stated above. With a new eye loss function, super-resolution block, and Gaussian-based face mask generation leads to improvements in quality which is confirmed during evaluation.

cs.CV↗
arXiv2020

Feathers dataset for Fine-Grained Visual Categorization

This paper introduces a novel dataset FeatherV1, containing 28,272 images of feathers categorized by 595 bird species. It was created to perform taxonomic identification of bird species by a single feather, which can be applied in amateur and professional ornithology. FeatherV1 is the first publicly available bird's plumage dataset for machine learning, and it can raise interest for a new task in fine-grained visual recognition domain. The latest version of the dataset can be downloaded at https://github.com/feathers-dataset/feathersv1-dataset. We also present feathers classification task results. We selected several deep learning architectures (DenseNet based) for categorical crossentropy values comparison on the provided dataset.

cs.CV↗
arXiv2013

Photosensor Characterization for the Cherenkov Telescope Array: Silicon Photomultiplier versus Multi-Anode Photomultiplier Tube

Photomultiplier tube technology has been the photodetector of choice for the technique of imaging atmospheric Cherenkov telescopes since its birth more than 50 years ago. Recently, new types of photosensors are being contemplated for the next generation Cherenkov Telescope Array. It is envisioned that the array will be partly composed of telescopes using a Schwarzschild-Couder two mirror design never built before which has significantly improved optics. The camera of this novel optical design has a small plate scale which enables the use of compact photosensors. We present an extensive and detailed study of the two most promising devices being considered for this telescope design: the silicon photomultiplier and the multi-anode photomultiplier tube. We evaluated their most critical performance characteristics for imaging gamma-ray showers, and we present our results in a cohesive manner to clearly evaluate the advantages and disadvantages that both types of device have to offer in the context of GeV-TeV gamma-ray astronomy.

astro-ph.IM↗