arXiv subjects

Marcelo Bertalmio

Publications and source records attributed to Marcelo Bertalmio.

6 recordsLinked to original sources

arXiv2022

Contrast Sensitivity Functions in Autoencoders

Three decades ago, Atick et al. suggested that human frequency sensitivity may emerge from the enhancement required for a more efficient analysis of retinal images. Here we reassess the relevance of low-level vision tasks in the explanation of the Contrast Sensitivity Functions (CSFs) in light of (1) the current trend of using artificial neural networks for studying vision, and (2) the current knowledge of retinal image representations. As a first contribution, we show that a very popular type of convolutional neural networks (CNNs), called autoencoders, may develop human-like CSFs in the spatio-temporal and chromatic dimensions when trained to perform some basic low-level vision tasks (like retinal noise and optical blur removal), but not others (like chromatic adaptation or pure reconstruction after simple bottlenecks). As an illustrative example, the best CNN (in the considered set of simple architectures for enhancement of the retinal signal) reproduces the CSFs with an RMSE error of 11\% of the maximum sensitivity. As a second contribution, we provide experimental evidence of the fact that, for some functional goals (at low abstraction level), deeper CNNs that are better in reaching the quantitative goal are actually worse in replicating human-like phenomena (such as the CSFs). This low-level result (for the explored networks) is not necessarily in contradiction with other works that report advantages of deeper nets in modeling higher-level vision goals. However, in line with a growing body of literature, our results suggests another word of caution about CNNs in vision science since the use of simplified units or unrealistic architectures in goal optimization may be a limitation for the modeling and understanding of human vision.

q-bio.NC↗

arXiv2018

Derivatives and Inverse of Cascaded Linear+Nonlinear Neural Models

In vision science, cascades of Linear+Nonlinear transforms are very successful in modeling a number of perceptual experiences [Carandini&Heeger12]. However, the conventional literature is usually too focused on only describing the input->output transform. Instead, here we present the maths of such cascades beyond the forward transform, namely the Jacobians and the inverse. The fundamental reason for this analytical treatment is that it offers useful insight into the psychophysics, the physiology, and the function of the visual system. For instance, we show how the trends of the sensitivity (discrimination regions) and the adaptation of the receptive fields can be seen in the expression of the Jacobian wrt the stimulus. This matrix also tells us which regions of the stimulus space are encoded more efficiently in multi-information terms. The Jacobian wrt the parameters shows which aspects of the model have bigger impact in the response, and hence bigger relevance. The analytic inverse implies conditions for the response and the model to ensure decoding. From an applied perspective, (a) the Jacobian wrt the stimulus is necessary in new experimental methods based on the synthesis of visual stimuli with interesting geometry, (b) the Jacobian matrices wrt the parameters are convenient to learn the model from classical experiments or alternative optimization goals, and (c) the inverse is a model-based alternative to blind machine-learning neural decoding that does not include meaningful biological information. The theory is checked by building a derivable and invertible vision model that actually follows the modular program suggested by Carandini&Heeger. To stress the generality of this modular setting we show examples where some of the canonical Divisive Normalization layers are substituted by equivalent layers such as the Wilson-Cowan model at V1, or a tone-mapping model at the retina.

q-bio.NC↗

arXiv2018

Appropriate kernels for Divisive Normalization explained by Wilson-Cowan equations

The interaction between wavelet-like sensors in Divisive Normalization is classically described through Gaussian kernels that decay with spatial distance, angular distance and frequency distance. However, simultaneous explanation of (a) distortion perception in natural image databases and (b) contrast perception of artificial stimuli requires very specific modifications in classical Divisive Normalization. First, the wavelet response has to be high-pass filtered before the Gaussian interaction is applied. Then, distinct weights per subband are also required after the Gaussian interaction. In summary, the classical Gaussian kernel has to be left- and right-multiplied by two extra diagonal matrices. In this paper we provide a lower-level justification for this specific empirical modification required in the Gaussian kernel of Divisive Normalization. Here we assume that the psychophysical behavior described by Divisive Normalization comes from neural interactions following the Wilson-Cowan equations. In particular, we identify the Divisive Normalization response with the stationary regime of a Wilson-Cowan model. From this identification we derive an expression for the Divisive Normalization kernel in terms of the interaction kernel of the Wilson-Cowan equations. It turns out that the Wilson-Cowan kernel is left- and-right multiplied by diagonal matrices with high-pass structure. In conclusion, symmetric Gaussian inhibitory relations between wavelet-like sensors wired in the lower-level Wilson-Cowan model lead to the appropriate non-symmetric kernel that has to be empirically included in Divisive Normalization to explain a wider range of phenomena.

q-bio.NC↗

arXiv2018

On the Duality Between Retinex and Image Dehazing

Image dehazing deals with the removal of undesired loss of visibility in outdoor images due to the presence of fog. Retinex is a color vision model mimicking the ability of the Human Visual System to robustly discount varying illuminations when observing a scene under different spectral lighting conditions. Retinex has been widely explored in the computer vision literature for image enhancement and other related tasks. While these two problems are apparently unrelated, the goal of this work is to show that they can be connected by a simple linear relationship. Specifically, most Retinex-based algorithms have the characteristic feature of always increasing image brightness, which turns them into ideal candidates for effective image dehazing by directly applying Retinex to a hazy image whose intensities have been inverted. In this paper, we give theoretical proof that Retinex on inverted intensities is a solution to the image dehazing problem. Comprehensive qualitative and quantitative results indicate that several classical and modern implementations of Retinex can be transformed into competing image dehazing algorithms performing on pair with more complex fog removal methods, and can overcome some of the main challenges associated with this problem.

cs.CV↗

arXiv2016

Derivatives and inverse of a linear-nonlinear multi-layer spatial vision model

Linear-nonlinear transforms are interesting in vision science because they are key in modeling a number of perceptual experiences such as color, motion or spatial texture. Here we first show that a number of issues in vision may be addressed through an analytic expression of the Jacobian of these linear-nonlinear transforms. The particular model analyzed afterwards (an extension of [Malo & Simoncelli SPIE 2015]) is illustrative because it consists of a cascade of standard linear-nonlinear modules. Each module roughly corresponds to a known psychophysical mechanism: (1) linear spectral integration and nonlinear brightness-from-luminance computation, (2) linear pooling of local brightness and nonlinear normalization for local contrast computation, (3) linear frequency selectivity and nonlinear normalization for spatial contrast masking, and (4) linear wavelet-like decomposition and nonlinear normalization for frequency-dependent masking. Beyond being the appropriate technical report with the missing details in [Malo & Simoncelli SPIE 2015], the interest of the presented analytic results and numerical methods transcend the particular model because of the ubiquity of the linear-nonlinear structure. Part of this material was presented at MODVIS 2016 (see slides of the conference talk in the appendix at the end of this document).

q-bio.NC↗

arXiv2013

A Contrario Selection of Optimal Partitions for Image Segmentation

We present a novel segmentation algorithm based on a hierarchical representation of images. The main contribution of this work is to explore the capabilities of the A Contrario reasoning when applied to the segmentation problem, and to overcome the limitations of current algorithms within that framework. This exploratory approach has three main goals. Our first goal is to extend the search space of greedy merging algorithms to the set of all partitions spanned by a certain hierarchy, and to cast the segmentation as a selection problem within this space. In this way we increase the number of tested partitions and thus we potentially improve the segmentation results. In addition, this space is considerably smaller than the space of all possible partitions, thus we still keep the complexity controlled. Our second goal aims to improve the locality of region merging algorithms, which usually merge pairs of neighboring regions. In this work, we overcome this limitation by introducing a validation procedure for complete partitions, rather than for pairs of regions. The third goal is to perform an exhaustive experimental evaluation methodology in order to provide reproducible results. Finally, we embed the selection process on a statistical A Contrario framework which allows us to have only one free parameter related to the desired scale.

cs.CV↗