arXiv subjects

Zhongxuan Luo

Publications and source records attributed to Zhongxuan Luo.

At least 37 records · Page 2Linked to original sources

arXiv2022

Target-aware Dual Adversarial Learning and a Multi-scenario Multi-Modality Benchmark to Fuse Infrared and Visible for Object Detection

This study addresses the issue of fusing infrared and visible images that appear differently for object detection. Aiming at generating an image of high visual quality, previous approaches discover commons underlying the two modalities and fuse upon the common space either by iterative optimization or deep networks. These approaches neglect that modality differences implying the complementary information are extremely important for both fusion and subsequent detection task. This paper proposes a bilevel optimization formulation for the joint problem of fusion and detection, and then unrolls to a target-aware Dual Adversarial Learning (TarDAL) network for fusion and a commonly used detection network. The fusion network with one generator and dual discriminators seeks commons while learning from differences, which preserves structural information of targets from the infrared and textural details from the visible. Furthermore, we build a synchronized imaging system with calibrated infrared and optical sensors, and collect currently the most comprehensive benchmark covering a wide range of scenarios. Extensive experiments on several public datasets and our benchmark demonstrate that our method outputs not only visually appealing fusion but also higher detection mAP than the state-of-the-art approaches.

cs.CV↗

arXiv2021

Learning with Nested Scene Modeling and Cooperative Architecture Search for Low-Light Vision

Images captured from low-light scenes often suffer from severe degradations, including low visibility, color cast and intensive noises, etc. These factors not only affect image qualities, but also degrade the performance of downstream Low-Light Vision (LLV) applications. A variety of deep learning methods have been proposed to enhance the visual quality of low-light images. However, these approaches mostly rely on significant architecture engineering to obtain proper low-light models and often suffer from high computational burden. Furthermore, it is still challenging to extend these enhancement techniques to handle other LLVs. To partially address above issues, we establish Retinex-inspired Unrolling with Architecture Search (RUAS), a general learning framework, which not only can address low-light enhancement task, but also has the flexibility to handle other more challenging downstream vision applications. Specifically, we first establish a nested optimization formulation, together with an unrolling strategy, to explore underlying principles of a series of LLV tasks. Furthermore, we construct a differentiable strategy to cooperatively search specific scene and task architectures for RUAS. Last but not least, we demonstrate how to apply RUAS for both low- and high-level LLV applications (e.g., enhancement, detection and segmentation). Extensive experiments verify the flexibility, effectiveness, and efficiency of RUAS.

cs.CV↗

arXiv2021

Learning Deep Context-Sensitive Decomposition for Low-Light Image Enhancement

Enhancing the quality of low-light images plays a very important role in many image processing and multimedia applications. In recent years, a variety of deep learning techniques have been developed to address this challenging task. A typical framework is to simultaneously estimate the illumination and reflectance, but they disregard the scene-level contextual information encapsulated in feature spaces, causing many unfavorable outcomes, e.g., details loss, color unsaturation, artifacts, and so on. To address these issues, we develop a new context-sensitive decomposition network architecture to exploit the scene-level contextual dependencies on spatial scales. More concretely, we build a two-stream estimation mechanism including reflectance and illumination estimation network. We design a novel context-sensitive decomposition connection to bridge the two-stream mechanism by incorporating the physical principle. The spatially-varying illumination guidance is further constructed for achieving the edge-aware smoothness property of the illumination component. According to different training patterns, we construct CSDNet (paired supervision) and CSDGAN (unpaired supervision) to fully evaluate our designed architecture. We test our method on seven testing benchmarks to conduct plenty of analytical and evaluated experiments. Thanks to our designed context-sensitive decomposition connection, we successfully realized excellent enhanced results, which fully indicates our superiority against existing state-of-the-art approaches. Finally, considering the practical needs for high-efficiency, we develop a lightweight CSDNet (named LiteCSDNet) by reducing the number of channels. Further, by sharing an encoder for these two components, we obtain a more lightweight version (SLiteCSDNet for short). SLiteCSDNet just contains 0.0301M parameters but achieves the almost same performance as CSDNet.

eess.IV↗

arXiv2021

Learning Deformable Image Registration from Optimization: Perspective, Modules, Bilevel Training and Beyond

Conventional deformable registration methods aim at solving an optimization model carefully designed on image pairs and their computational costs are exceptionally high. In contrast, recent deep learning based approaches can provide fast deformation estimation. These heuristic network architectures are fully data-driven and thus lack explicit geometric constraints, e.g., topology-preserving, which are indispensable to generate plausible deformations. We design a new deep learning based framework to optimize a diffeomorphic model via multi-scale propagation in order to integrate advantages and avoid limitations of these two categories of approaches. Specifically, we introduce a generic optimization model to formulate diffeomorphic registration and develop a series of learnable architectures to obtain propagative updating in the coarse-to-fine feature space. Moreover, we propose a novel bilevel self-tuned training strategy, allowing efficient search of task-specific hyper-parameters. This training strategy increases the flexibility to various types of data while reduces computational and human burdens. We conduct two groups of image registration experiments on 3D volume datasets including image-to-atlas registration on brain MRI data and image-to-image registration on liver CT data. Extensive results demonstrate the state-of-the-art performance of the proposed method with diffeomorphic guarantee and extreme efficiency. We also apply our framework to challenging multi-modal image registration, and investigate how our registration to support the down-streaming tasks for medical image analysis including multi-modal fusion and image segmentation.

cs.CV↗

arXiv2021

An Underwater Image Semantic Segmentation Method Focusing on Boundaries and a Real Underwater Scene Semantic Segmentation Dataset

With the development of underwater object grabbing technology, underwater object recognition and segmentation of high accuracy has become a challenge. The existing underwater object detection technology can only give the general position of an object, unable to give more detailed information such as the outline of the object, which seriously affects the grabbing efficiency. To address this problem, we label and establish the first underwater semantic segmentation dataset of real scene(DUT-USEG:DUT Underwater Segmentation Dataset). The DUT- USEG dataset includes 6617 images, 1487 of which have semantic segmentation and instance segmentation annotations, and the remaining 5130 images have object detection box annotations. Based on this dataset, we propose a semi-supervised underwater semantic segmentation network focusing on the boundaries(US-Net: Underwater Segmentation Network). By designing a pseudo label generator and a boundary detection subnetwork, this network realizes the fine learning of boundaries between underwater objects and background, and improves the segmentation effect of boundary areas. Experiments show that the proposed method improves by 6.7% in three categories of holothurian, echinus, starfish in DUT-USEG dataset, and achieves state-of-the-art results. The DUT- USEG dataset will be released at https://github.com/baxiyi/DUT-USEG.

cs.CV↗

arXiv2020

Retinex-inspired Unrolling with Cooperative Prior Architecture Search for Low-light Image Enhancement

Low-light image enhancement plays very important roles in low-level vision field. Recent works have built a large variety of deep learning models to address this task. However, these approaches mostly rely on significant architecture engineering and suffer from high computational burden. In this paper, we propose a new method, named Retinex-inspired Unrolling with Architecture Search (RUAS), to construct lightweight yet effective enhancement network for low-light images in real-world scenario. Specifically, building upon Retinex rule, RUAS first establishes models to characterize the intrinsic underexposed structure of low-light images and unroll their optimization processes to construct our holistic propagation structure. Then by designing a cooperative reference-free learning strategy to discover low-light prior architectures from a compact search space, RUAS is able to obtain a top-performing image enhancement network, which is with fast speed and requires few computational resources. Extensive experiments verify the superiority of our RUAS framework against recently proposed state-of-the-art methods.

cs.CV↗

arXiv2020

Quadrilateral Mesh Generation III: Optimizing Singularity Configuration Based on Abel-Jacobi Theory

This work proposes a rigorous and practical algorithm for generating meromorphic quartic differentials for the purpose of quad-mesh generation. The work is based on the Abel-Jacobi theory of algebraic curve. The algorithm pipeline can be summarized as follows: calculate the homology group; compute the holomorphic differential group; construct the period matrix of the surface and Jacobi variety; calculate the Abel-Jacobi map for a given divisor; optimize the divisor to satisfy the Abel-Jacobi condition by an integer programming; compute the flat Riemannian metric with cone singularities at the divisor by Ricci flow; isometric immerse the surface punctured at the divisor onto the complex plane and pull back the canonical holomorphic differential to the surface to obtain the meromorphic quartic differential; construct the motor-graph to generate the resulting T-Mesh. The proposed method is rigorous and practical. The T-mesh results can be applied for constructing T-Spline directly. The efficiency and efficacy of the proposed algorithm are demonstrated by experimental results.

cs.CG↗

arXiv2020

Investigating Task-driven Latent Feasibility for Nonconvex Image Modeling

Properly modeling latent image distributions plays an important role in a variety of image-related vision problems. Most exiting approaches aim to formulate this problem as optimization models (e.g., Maximum A Posterior, MAP) with handcrafted priors. In recent years, different CNN modules are also considered as deep priors to regularize the image modeling process. However, these explicit regularization techniques require deep understandings on the problem and elaborately mathematical skills. In this work, we provide a new perspective, named Task-driven Latent Feasibility (TLF), to incorporate specific task information to narrow down the solution space for the optimization-based image modeling problem. Thanks to the flexibility of TLF, both designed and trained constraints can be embedded into the optimization process. By introducing control mechanisms based on the monotonicity and boundedness conditions, we can also strictly prove the convergence of our proposed inference process. We demonstrate that different types of image modeling problems, such as image deblurring and rain streaks removals, can all be appropriately addressed within our TLF framework. Extensive experiments also verify the theoretical results and show the advantages of our method against existing state-of-the-art approaches.

cs.CV↗

arXiv2019

Converged Deep Framework Assembling Principled Modules for CS-MRI

Compressed Sensing Magnetic Resonance Imaging (CS-MRI) significantly accelerates MR data acquisition at a sampling rate much lower than the Nyquist criterion. A major challenge for CS-MRI lies in solving the severely ill-posed inverse problem to reconstruct aliasing-free MR images from the sparse k-space data. Conventional methods typically optimize an energy function, producing reconstruction of high quality, but their iterative numerical solvers unavoidably bring extremely slow processing. Recent data-driven techniques are able to provide fast restoration by either learning direct prediction to final reconstruction or plugging learned modules into the energy optimizer. Nevertheless, these data-driven predictors cannot guarantee the reconstruction following constraints underlying the regularizers of conventional methods so that the reliability of their reconstruction results are questionable. In this paper, we propose a converged deep framework assembling principled modules for CS-MRI that fuses learning strategy with the iterative solver of a conventional reconstruction energy. This framework embeds an optimal condition checking mechanism, fostering \emph{efficient} and \emph{reliable} reconstruction. We also apply the framework to two practical tasks, \emph{i.e.}, parallel imaging and reconstruction with Rician noise. Extensive experiments on both benchmark and manufacturer-testing images demonstrate that the proposed method reliably converges to the optimal solution more efficiently and accurately than the state-of-the-art in various scenarios.

eess.IV↗

arXiv2019

Underexposed Image Correction via Hybrid Priors Navigated Deep Propagation

Enhancing visual qualities for underexposed images is an extensively concerned task that plays important roles in various areas of multimedia and computer vision. Most existing methods often fail to generate high-quality results with appropriate luminance and abundant details. To address these issues, we in this work develop a novel framework, integrating both knowledge from physical principles and implicit distributions from data to solve the underexposed image correction task. More concretely, we propose a new perspective to formulate this task as an energy-inspired model with advanced hybrid priors. A propagation procedure navigated by the hybrid priors is well designed for simultaneously propagating the reflectance and illumination toward desired results. We conduct extensive experiments to verify the necessity of integrating both underlying principles (i.e., with knowledge) and distributions (i.e., from data) as navigated deep propagation. Plenty of experimental results of underexposed image correction demonstrate that our proposed method performs favorably against the state-of-the-art methods on both subjective and objective assessments. Additionally, we execute the task of face detection to further verify the naturalness and practical value of underexposed image correction. What's more, we employ our method to single image haze removal whose experimental results further demonstrate its superiorities.

cs.CV↗

arXiv2019

Quadrilateral Mesh Generation II : Meromorphic Quartic Differentials and Abel-Jacobi Condition

This work discovers the equivalence relation between quadrilateral meshes and meromorphic quartic. Each quad-mesh induces a conformal structure of the surface, and a meromorphic differential, where the configuration of singular vertices correspond to the configurations the poles and zeros (divisor) of the meroromorphic differential. Due to Riemann surface theory, the configuration of singularities of a quad-mesh satisfies the Abel-Jacobi condition. Inversely, if a satisfies the Abel-Jacobi condition, then there exists a meromorphic quartic differential whose equals to the given one. Furthermore, if the meromorphic quadric differential is with finite, then it also induces a a quad-mesh, the poles and zeros of the meromorphic differential to the singular vertices of the quad-mesh. Besides the theoretic proofs, the computational algorithm for verification of Abel-Jacobi condition is explained in details. Furthermore, constructive algorithm of meromorphic quartic differential on zero surfaces is proposed, which is based on the global algebraic representation of meromorphic. Our experimental results demonstrate the efficiency and efficacy of the algorithm. This opens up a direction for quad-mesh generation using algebraic geometric approach.

cs.CG↗

arXiv2019

Real-world Underwater Enhancement: Challenges, Benchmarks, and Solutions

Underwater image enhancement is such an important low-level vision task with many applications that numerous algorithms have been proposed in recent years. These algorithms developed upon various assumptions demonstrate successes from various aspects using different data sets and different metrics. In this work, we setup an undersea image capturing system, and construct a large-scale Real-world Underwater Image Enhancement (RUIE) data set divided into three subsets. The three subsets target at three challenging aspects for enhancement, i.e., image visibility quality, color casts, and higher-level detection/classification, respectively. We conduct extensive and systematic experiments on RUIE to evaluate the effectiveness and limitations of various algorithms to enhance visibility and correct color casts on images with hierarchical categories of degradation. Moreover, underwater image enhancement in practice usually serves as a preprocessing step for mid-level and high-level vision tasks. We thus exploit the object detection performance on enhanced images as a brand new task-specific evaluation criterion. The findings from these evaluations not only confirm what is commonly believed, but also suggest promising solutions and new directions for visibility enhancement, color correction, and object detection on real-world underwater images.

cs.CV↗

arXiv2019

Mode Collapse and Regularity of Optimal Transportation Maps

This work builds the connection between the regularity theory of optimal transportation map, Monge-Ampère equation and GANs, which gives a theoretic understanding of the major drawbacks of GANs: convergence difficulty and mode collapse. According to the regularity theory of Monge-Ampère equation, if the support of the target measure is disconnected or just non-convex, the optimal transportation mapping is discontinuous. General DNNs can only approximate continuous mappings. This intrinsic conflict leads to the convergence difficulty and mode collapse in GANs. We test our hypothesis that the supports of real data distribution are in general non-convex, therefore the discontinuity is unavoidable using an Autoencoder combined with discrete optimal transportation map (AE-OT framework) on the CelebA data set. The testing result is positive. Furthermore, we propose to approximate the continuous Brenier potential directly based on discrete Brenier theory to tackle mode collapse. Comparing with existing method, this method is more accurate and effective.

cs.LG↗

arXiv2018

High accuracy analysis of a nonconforming discrete Stokes complex over rectangular meshes

This work is devoted to the high accuracy analysis of a discrete Stokes complex over rectangular meshes with a simple structure. The 0-form in the complex is a non $C^0$ nonconforming element space for biharmonic problems. This plate element contains only 12 degrees of freedom (DoFs) over a rectangular cell with a zeroth order weak continuity for the normal derivative, therefore only the lowest convergence order can be obtained by a standard consistency error analysis. Nevertheless, we prove that, if the rectangular mesh is uniform, an $O(h^2)$ convergence rate in discrete $H^2$-norm will be achieved. Moreover, based on a duality argument, it has an $O(h^3)$ convergence order in discrete $H^1$-norm if the solution region is convex. The 1-form and 2-form constitute a divergence-free pair for incompressible flow. We also show its higher accuracy than that derived from a usual error estimate under uniform partitions, which explains the phenomenon observed in our previous work. Numerical tests verify our theoretical results.

math.NA↗

arXiv2018

Metric Based Quadrilateral Mesh Generation

This work proposes a novel metric based algorithm for quadrilateral mesh generating. Each quad-mesh induces a Riemannian metric satisfying special conditions: the metric is a flat metric with cone signualrites conformal to the original metric, the total curvature satisfies the Gauss-Bonnet condition, the holonomy group is a subgroup of the rotation group $\{e^{ikπ/2}\}$, furthermore there is cross field obtained by parallel translation which is aligned with the boundaries, and its streamlines are finite geodesics. Inversely, such kind of metric induces a quad-mesh. Based on discrete Ricci flow and conformal structure deformation, one can obtain a metric satisfying all the conditions and obtain the desired quad-mesh. This method is rigorous, simple and automatic. Our experimental results demonstrate the efficiency and efficacy of the algorithm.

cs.CG↗

arXiv2018

Two 11-node nonconforming triangular prism elements for 3D elliptic problems

This work introduces two 11-node triangular prism elements for 3D elliptic problems. The degrees of freedom (DoFs) of both elements are at the vertices and face centroids of a prism cell. The first element is $H^1$-nonconforming and works for second order problems, which achieves a second order convergence rate in discrete $H^1$-norm. The other is $H^2$-nonconforming and solves fourth order problems, with a first order convergence rate in discrete $H^2$-norm. Numerical examples verify our theoretical results.

math.NA↗

arXiv2018

A Theoretically Guaranteed Deep Optimization Framework for Robust Compressive Sensing MRI

Magnetic Resonance Imaging (MRI) is one of the most dynamic and safe imaging techniques available for clinical applications. However, the rather slow speed of MRI acquisitions limits the patient throughput and potential indi cations. Compressive Sensing (CS) has proven to be an efficient technique for accelerating MRI acquisition. The most widely used CS-MRI model, founded on the premise of reconstructing an image from an incompletely filled k-space, leads to an ill-posed inverse problem. In the past years, lots of efforts have been made to efficiently optimize the CS-MRI model. Inspired by deep learning techniques, some preliminary works have tried to incorporate deep architectures into CS-MRI process. Unfortunately, the convergence issues (due to the experience-based networks) and the robustness (i.e., lack real-world noise modeling) of these deeply trained optimization methods are still missing. In this work, we develop a new paradigm to integrate designed numerical solvers and the data-driven architectures for CS-MRI. By introducing an optimal condition checking mechanism, we can successfully prove the convergence of our established deep CS-MRI optimization scheme. Furthermore, we explicitly formulate the Rician noise distributions within our framework and obtain an extended CS-MRI network to handle the real-world nosies in the MRI process. Extensive experimental results verify that the proposed paradigm outperforms the existing state-of-the-art techniques both in reconstruction accuracy and efficiency as well as robustness to noises in real scene.

cs.CV↗

arXiv2018

A nodal type polynomial finite element exact sequence over quadrilaterals

This work proposes two nodal type nonconforming finite elements over convex quadrilaterals, which are parts of a finite element exact sequence. Both elements are of 12 degrees of freedom (DoFs) with polynomial shape function spaces selected. The first one is designed for fourth order elliptic singular perturbation problems, and the other works for Brinkman problems. Numerical examples are also provided.

math.NA↗