arXiv subjects

Lei Yu

Publications and source records attributed to Lei Yu.

At least 199 records · Page 11Linked to original sources

arXiv2022

BSRT: Improving Burst Super-Resolution with Swin Transformer and Flow-Guided Deformable Alignment

This work addresses the Burst Super-Resolution (BurstSR) task using a new architecture, which requires restoring a high-quality image from a sequence of noisy, misaligned, and low-resolution RAW bursts. To overcome the challenges in BurstSR, we propose a Burst Super-Resolution Transformer (BSRT), which can significantly improve the capability of extracting inter-frame information and reconstruction. To achieve this goal, we propose a Pyramid Flow-Guided Deformable Convolution Network (Pyramid FG-DCN) and incorporate Swin Transformer Blocks and Groups as our main backbone. More specifically, we combine optical flows and deformable convolutions, hence our BSRT can handle misalignment and aggregate the potential texture information in multi-frames more efficiently. In addition, our Transformer-based structure can capture long-range dependency to further improve the performance. The evaluation on both synthetic and real-world tracks demonstrates that our approach achieves a new state-of-the-art in BurstSR task. Further, our BSRT wins the championship in the NTIRE2022 Burst Super-Resolution Challenge.

cs.CV↗

arXiv2022

Unifying Motion Deblurring and Frame Interpolation with Events

Slow shutter speed and long exposure time of frame-based cameras often cause visual blur and loss of inter-frame information, degenerating the overall quality of captured videos. To this end, we present a unified framework of event-based motion deblurring and frame interpolation for blurry video enhancement, where the extremely low latency of events is leveraged to alleviate motion blur and facilitate intermediate frame prediction. Specifically, the mapping relation between blurry frames and sharp latent images is first predicted by a learnable double integral network, and a fusion network is then proposed to refine the coarse results via utilizing the information from consecutive blurry inputs and the concurrent events. By exploring the mutual constraints among blurry frames, latent images, and event streams, we further propose a self-supervised learning framework to enable network training with real-world blurry videos and events. Extensive experiments demonstrate that our method compares favorably against the state-of-the-art approaches and achieves remarkable performance on both synthetic and real-world datasets.

cs.CV↗

arXiv2022

On the Rate-Distortion-Perception Function

Rate-distortion-perception theory generalizes Shannon's rate-distortion theory by introducing a constraint on the perceptual quality of the output. The perception constraint complements the conventional distortion constraint and aims to enforce distribution-level consistencies. In this new theory, the information-theoretic limit is characterized by the rate-distortion-perception function. Although a coding theorem for the rate-distortion-perception function has recently been established, the fundamental nature of the optimal coding schemes remains unclear, especially regarding the role of randomness in encoding and decoding. It is shown in the present work that except for certain extreme cases, the rate-distortion-perception function is achievable by deterministic codes. This paper also clarifies the subtle differences between two notions of perfect perceptual quality and explores some alternative formulations of the perception constraint.

cs.IT↗

arXiv2022

Backpropagation through Time and Space: Learning Numerical Methods with Multi-Agent Reinforcement Learning

We introduce Backpropagation Through Time and Space (BPTTS), a method for training a recurrent spatio-temporal neural network, that is used in a homogeneous multi-agent reinforcement learning (MARL) setting to learn numerical methods for hyperbolic conservation laws. We treat the numerical schemes underlying partial differential equations (PDEs) as a Partially Observable Markov Game (POMG) in Reinforcement Learning (RL). Similar to numerical solvers, our agent acts at each discrete location of a computational space for efficient and generalizable learning. To learn higher-order spatial methods by acting on local states, the agent must discern how its actions at a given spatiotemporal location affect the future evolution of the state. The manifestation of this non-stationarity is addressed by BPTTS, which allows for the flow of gradients across both space and time. The learned numerical policies are comparable to the SOTA numerics in two settings, the Burgers' Equation and the Euler Equations, and generalize well to other simulation set-ups.

cs.LG↗

arXiv2022

Deep Constrained Least Squares for Blind Image Super-Resolution

In this paper, we tackle the problem of blind image super-resolution(SR) with a reformulated degradation model and two novel modules. Following the common practices of blind SR, our method proposes to improve both the kernel estimation as well as the kernel-based high-resolution image restoration. To be more specific, we first reformulate the degradation model such that the deblurring kernel estimation can be transferred into the low-resolution space. On top of this, we introduce a dynamic deep linear filter module. Instead of learning a fixed kernel for all images, it can adaptively generate deblurring kernel weights conditional on the input and yield a more robust kernel estimation. Subsequently, a deep constrained least square filtering module is applied to generate clean features based on the reformulation and estimated kernel. The deblurred feature and the low input image feature are then fed into a dual-path structured SR network and restore the final high-resolution result. To evaluate our method, we further conduct evaluations on several benchmarks, including Gaussian8 and DIV2KRK. Our experiments demonstrate that the proposed method achieves better accuracy and visual improvements against state-of-the-art methods.

eess.IV↗

arXiv2022

Autofocus for Event Cameras

Focus control (FC) is crucial for cameras to capture sharp images in challenging real-world scenarios. The autofocus (AF) facilitates the FC by automatically adjusting the focus settings. However, due to the lack of effective AF methods for the recently introduced event cameras, their FC still relies on naive AF like manual focus adjustments, leading to poor adaptation in challenging real-world conditions. In particular, the inherent differences between event and frame data in terms of sensing modality, noise, temporal resolutions, etc., bring many challenges in designing an effective AF method for event cameras. To address these challenges, we develop a novel event-based autofocus framework consisting of an event-specific focus measure called event rate (ER) and a robust search strategy called event-based golden search (EGS). To verify the performance of our method, we have collected an event-based autofocus dataset (EAD) containing well-synchronized frames, events, and focal positions in a wide variety of challenging scenes with severe lighting and motion conditions. The experiments on this dataset and additional real-world scenarios demonstrated the superiority of our method over state-of-the-art approaches in terms of efficiency and accuracy.

cs.CV↗

arXiv2022

First implementation of full-workflow automation in radiotherapy: the All-in-One solution on rectal cancer

The aim of this work is to describe the technical characteristics of an AI-powered radiotherapy workflow that enables full-process automation (All-in-One), evaluate its performance implemented for on-couch initial treatment of rectal cancer, and provide insight into the behavior of full-workflow automation in the specialty of radiotherapy. The All-in-One workflow was developed based on a CT-integrated linear accelerator. It incorporates routine radiotherapy procedures from simulation, autosegmentation, autoplanning, image guidance, beam delivery, and in vivo quality assurance (QA) into one scheme, with critical decision points involved, while the patient is on the treatment couch during the whole process. For the enrolled ten patients with rectal cancer, minor modifications of the autosegmented target volumes were required, and the Dice similarity coefficient and 95% Hausdorff distance before and after modifications were 0.892{\pm}0.061 and 18.2{\pm}13.0 mm, respectively. The autosegmented normal tissues and automatic plans were clinically acceptable without any modifications or reoptimization. The pretreatment IGRT corrections were within 2 mm in all directions, and the EPID-based in vivo QA showed a γ passing rate better than 97{\%} (3{\%}/3 mm/10{\%} threshold). The duration of the whole process was 23.2{\pm}3.5 minutes, depending mostly on the time required for manual modification and plan evaluation. The All-in-One workflow enables full automation of the entire radiotherapy process by seamlessly integrating multiple routine procedures. The one-stop solution shortens the time scale it takes to ready the first treatment from days to minutes, significantly improving the patient experience and the efficiency of the workflow, and shows potential to facilitate the clinical application of online adaptive replanning.

physics.med-ph↗

arXiv2022

Enabling arbitrary translation objectives with Adaptive Tree Search

We introduce an adaptive tree search algorithm, that can find high-scoring outputs under translation models that make no assumptions about the form or structure of the search objective. This algorithm -- a deterministic variant of Monte Carlo tree search -- enables the exploration of new kinds of models that are unencumbered by constraints imposed to make decoding tractable, such as autoregressivity or conditional independence assumptions. When applied to autoregressive models, our algorithm has different biases than beam search has, which enables a new analysis of the role of decoding bias in autoregressive models. Empirically, we show that our adaptive tree search algorithm finds outputs with substantially better model scores compared to beam search in autoregressive models, and compared to reranking techniques in models whose scores do not decompose additively with respect to the words in the output. We also characterise the correlation of several translation model objectives with respect to BLEU. We find that while some standard models are poorly calibrated and benefit from the beam search bias, other often more robust models (autoregressive models tuned to maximize expected automatic metric scores, the noisy channel model and a newly proposed objective) benefit from increasing amounts of search using our proposed decoder, whereas the beam search bias limits the improvements obtained from such objectives. Thus, we argue that as models improve, the improvements may be masked by over-reliance on beam search or reranking based methods.

cs.CL↗

arXiv2022

Asymptotics for Strassen's Optimal Transport Problem

In this paper, we consider Strassen's version of optimal transport (OT) problem, which concerns minimizing the excess-cost probability (i.e., the probability that the cost is larger than a given value) over all couplings of two given distributions. We derive large deviation, moderate deviation, and central limit theorems for this problem. Our proof is based on Strassen's dual formulation of the OT problem, Sanov's theorem on the large deviation principle (LDP) of empirical measures, as well as the moderate deviation principle (MDP) and central limit theorems (CLT) of empirical measures. In order to apply the LDP, MDP, and CLT to Strassen's OT problem, nested formulas for Strassen's OT problem are derived. Based on these nested formulas and using a splitting technique, we construct asymptotically optimal solutions to Strassen's OT problem and its dual formulation.

math.PR↗

arXiv2022

Documentation based Semantic-Aware Log Parsing

With the recent advances of deep learning techniques, there are rapidly growing interests in applying machine learning to log data. As a fundamental part of log analytics, accurate log parsing that transforms raw logs to structured events is critical for subsequent machine learning and data mining tasks. Previous approaches either analyze the source code for parsing or are data-driven such as text clustering. They largely neglect to exploit another widely available and valuable resource, software documentation that provides detailed explanations for the messages, to improve accuracy. In this paper, we propose an approach and system framework to use documentation knowledge for log parsing. With parameter value identification, it not only can improve the parsing accuracy for documented messages but also for undocumented messages. In addition, it can discover the linkages between event templates that are established by sharing parameters and indicate the correlation of the event context.

cs.SE↗

arXiv2022

Sequential Channel Synthesis

The channel synthesis problem has been widely investigated over the last decade. In this paper, we consider the sequential version in which the encoder and the decoder work in a sequential way. Under a mild assumption on the target joint distribution we provide a complete (single-letter) characterization of the solution for the point-to-point case, which shows that the canonical symbol-by-symbol mapping is not optimal in general, but is indeed optimal if we make some additional assumptions on the encoder and decoder. We also extend this result to the broadcast scenario and the interactive communication scenario. We provide bounds in the broadcast setting and a complete characterization of the solution under a mild condition on the target joint distribution in the interactive communication case. Our proofs are based on a Rényi entropy method.

cs.IT↗

arXiv2021

Edge-Isoperimetric Inequalities and Ball-Noise Stability: Linear Programming and Probabilistic Approaches

Let $Q_{n}^{r}$ be the graph with vertex set $\{-1,1\}^{n}$ in which two vertices are joined if their Hamming distance is at most $r$. The edge-isoperimetric problem for $Q_{n}^{r}$ is that: For every $(n,r,M)$ such that $1\le r\le n$ and $1\le M\le2^{n}$, determine the minimum edge-boundary size of a subset of vertices of $Q_{n}^{r}$ with a given size $M$. In this paper, we apply two different approaches to prove bounds for this problem. The first approach is a linear programming approach and the second is a probabilistic approach. Our bound derived by the first approach generalizes the tight bound for $M=2^{n-1}$ derived by Kahn, Kalai, and Linial in 1989. Moreover, our bound is also tight for $M=2^{n-2}$ and $r\le\frac{n}{2}-1$. Our bounds derived by the second approach are expressed in terms of the \emph{noise stability}, and they are shown to be asymptotically tight as $n\to\infty$ when $r=2\lfloor\frac{βn}{2}\rfloor+1$ and $M=\lfloor\alpha2^{n}\rfloor$ for fixed $α,β\in(0,1)$, and is tight up to a factor $2$ when $r=2\lfloor\frac{βn}{2}\rfloor$ and $M=\lfloor\alpha2^{n}\rfloor$. In fact, the edge-isoperimetric problem is equivalent to a ball-noise stability problem which is a variant of the traditional (i.i.d.-) noise stability problem. Our results can be interpreted as bounds for the ball-noise stability problem.

math.CO↗

arXiv2021

Strong Brascamp-Lieb Inequalities

In this paper, we derive sharp nonlinear dimension-free Brascamp--Lieb inequalities (including hypercontractivity inequalities) for distributions on Polish spaces, which strengthen the classic Brascamp--Lieb inequalities. Applications include the extension of Mrs. Gerber's lemma to the cases of Rényi divergences and distributions on Polish spaces, the strengthening of small-set expansion theorems, and the characterization of the exponent of the $q$-stability. Our proofs in this paper are based on information-theoretic and coupling techniques.

math.FA↗

arXiv2021

MAAC: Novel Alert Correlation Method To Detect Multi-step Attack

With the continuous improvement of attack methods, there are more and more distributed, complex, targeted attacks in which the attackers use combined attack methods to achieve the purpose. Advanced cyber attacks include multiple stages to achieve the ultimate goal. Traditional intrusion detection systems such as endpoint security management tools, firewalls, and other monitoring tools generate a large number of alerts during the attack. These alerts include attack clues, as well as many false positives unrelated to attacks. Security analysts need to analyze a large number of alerts and find useful clues from them and reconstruct attack scenarios. However, most traditional security monitoring tools cannot correlate alerts from different sources, so many multi-step attacks are still completely unnoticed, requiring manual analysis by security analysts like finding a needle in a haystack. We propose MAAC, a multi-step attack alert correlation system, which reduces repeated alerts and combines multi-step attack paths based on alert semantics and attack stages. The evaluation results of the real-world datasets show that MAAC can effectively reduce the alerts by 90\% and find attack paths from a large number of alerts.

cs.CR↗

arXiv2021

Motion Deblurring with Real Events

In this paper, we propose an end-to-end learning framework for event-based motion deblurring in a self-supervised manner, where real-world events are exploited to alleviate the performance degradation caused by data inconsistency. To achieve this end, optical flows are predicted from events, with which the blurry consistency and photometric consistency are exploited to enable self-supervision on the deblurring network with real-world data. Furthermore, a piece-wise linear motion model is proposed to take into account motion non-linearities and thus leads to an accurate model for the physical formation of motion blurs in the real-world scenario. Extensive evaluation on both synthetic and real motion blur datasets demonstrates that the proposed algorithm bridges the gap between simulated and real-world motion blurs and shows remarkable performance for event-based motion deblurring in real-world scenarios.

cs.CV↗

arXiv2021

Predicting emergent linguistic compositions through time: Syntactic frame extension via multimodal chaining

Natural language relies on a finite lexicon to express an unbounded set of emerging ideas. One result of this tension is the formation of new compositions, such that existing linguistic units can be combined with emerging items into novel expressions. We develop a framework that exploits the cognitive mechanisms of chaining and multimodal knowledge to predict emergent compositional expressions through time. We present the syntactic frame extension model (SFEM) that draws on the theory of chaining and knowledge from "percept", "concept", and "language" to infer how verbs extend their frames to form new compositions with existing and novel nouns. We evaluate SFEM rigorously on the 1) modalities of knowledge and 2) categorization models of chaining, in a syntactically parsed English corpus over the past 150 years. We show that multimodal SFEM predicts newly emerged verb syntax and arguments substantially better than competing models using purely linguistic or unimodal knowledge. We find support for an exemplar view of chaining as opposed to a prototype view and reveal how the joint approach of multimodal chaining may be fundamental to the creation of literal and figurative language uses including metaphor and metonymy.

cs.CL↗

arXiv2021

Asymptotic Coupling and Its Applications in Information Theory

A coupling of two distributions $P_{X}$ and $P_{Y}$ is a joint distribution $P_{XY}$ with marginal distributions equal to $P_{X}$ and $P_{Y}$. Given marginals $P_{X}$ and $P_{Y}$ and a real-valued function $f$ of the joint distribution $P_{XY}$, what is its minimum over all couplings $P_{XY}$ of $P_{X}$ and $P_{Y}$? We study the asymptotics of such coupling problems with different $f$'s and with $X$ and $Y$ replaced by $X^{n}=(X_{1},\ldots,X_{n})$ and $Y^{n}=(Y_{1},\ldots,Y_{n})$ where $X_{i}$ and $Y_{i}$ are i.i.d.\ copies of random variables $X$ and $Y$ with distributions $P_{X}$ and $P_{Y}$ respectively. These include the maximal coupling, minimum distance coupling, maximal guessing coupling, and minimum entropy coupling problems. We characterize the limiting values of these coupling problems as $n$ tends to infinity. We show that they typically converge at least exponentially fast to their limits. Moreover, for the problems of maximal coupling and minimum excess-distance probability coupling, we also characterize (or bound) the optimal convergence rates (exponents). Furthermore, for the maximal guessing coupling problem we show that it is equivalent to the distribution approximation problem. Therefore, some existing results for the latter problem can be used to derive the asymptotics of the maximal guessing coupling problem. We also study the asymptotics of the maximal guessing coupling problem for two \emph{general} sources and a generalization of this problem, named the \emph{maximal guessing coupling through a channel problem}. We apply the preceding results to several new information-theoretic problems, including exact intrinsic randomness, exact resolvability, channel capacity with input distribution constraint, and perfect stealth and secrecy communication.

cs.IT↗

arXiv2021

Kernel Learning for High-Resolution Time-Frequency Distribution

The design of high-resolution and cross-term (CT) free time-frequency distributions (TFDs) has been an open problem. Classical kernel based methods are limited by the trade-off between TFD resolution and CT suppression, even under optimally derived parameters. To break the current limitation, we propose a data-driven kernel learning model directly based on Wigner-Ville distribution (WVD). The proposed kernel learning based TFD (KL-TFD) model includes several stacked multi-channel learning convolutional kernels. Specifically, a skipping operator is utilized to maintain correct information transmission, and a weighted block is employed to exploit spatial and channel dependencies. These two designs simultaneously achieve high TFD resolution and CT elimination. Numerical experiments on both synthetic and real-world data confirm the superiority of the proposed KL-TFD over traditional kernel function methods.

eess.SP↗