arXiv subjects

Josep Lladós

Publications and source records attributed to Josep Lladós.

45 records · Page 3Linked to original sources

arXiv2020

Learning Graph Edit Distance by Graph Neural Networks

The emergence of geometric deep learning as a novel framework to deal with graph-based representations has faded away traditional approaches in favor of completely new methodologies. In this paper, we propose a new framework able to combine the advances on deep metric learning with traditional approximations of the graph edit distance. Hence, we propose an efficient graph distance based on the novel field of geometric deep learning. Our method employs a message passing neural network to capture the graph structure, and thus, leveraging this information for its use on a distance computation. The performance of the proposed graph distance is validated on two different scenarios. On the one hand, in a graph retrieval of handwritten words~\ie~keyword spotting, showing its superior performance when compared with (approximate) graph edit distance benchmarks. On the other hand, demonstrating competitive results for graph similarity learning when compared with the current state-of-the-art on a recent benchmark dataset.

cs.CV↗

arXiv2020

A Neural Model for Text Localization, Transcription and Named Entity Recognition in Full Pages

In the last years, the consolidation of deep neural network architectures for information extraction in document images has brought big improvements in the performance of each of the tasks involved in this process, consisting of text localization, transcription, and named entity recognition. However, this process is traditionally performed with separate methods for each task. In this work we propose an end-to-end model that combines a one stage object detection network with branches for the recognition of text and named entities respectively in a way that shared features can be learned simultaneously from the training error of each of the tasks. By doing so the model jointly performs handwritten text detection, transcription, and named entity recognition at page level with a single feed forward step. We exhaustively evaluate our approach on different datasets, discussing its advantages and limitations compared to sequential approaches. The results show that the model is capable of benefiting from shared features for simultaneously solving interdependent tasks.

cs.CV↗

arXiv2020

Hierarchical stochastic graphlet embedding for graph-based pattern recognition

Despite being very successful within the pattern recognition and machine learning community, graph-based methods are often unusable because of the lack of mathematical operations defined in graph domain. Graph embedding, which maps graphs to a vectorial space, has been proposed as a way to tackle these difficulties enabling the use of standard machine learning techniques. However, it is well known that graph embedding functions usually suffer from the loss of structural information. In this paper, we consider the hierarchical structure of a graph as a way to mitigate this loss of information. The hierarchical structure is constructed by topologically clustering the graph nodes, and considering each cluster as a node in the upper hierarchical level. Once this hierarchical structure is constructed, we consider several configurations to define the mapping into a vector space given a classical graph embedding, in particular, we propose to make use of the Stochastic Graphlet Embedding (SGE). Broadly speaking, SGE produces a distribution of uniformly sampled low to high order graphlets as a way to embed graphs into the vector space. In what follows, the coarse-to-fine structure of a graph hierarchy and the statistics fetched by the SGE complements each other and includes important structural information with varied contexts. Altogether, these two techniques substantially cope with the usual information loss involved in graph embedding techniques, obtaining a more robust graph representation. This fact has been corroborated through a detailed experimental evaluation on various benchmark graph datasets, where we outperform the state-of-the-art methods.

cs.CV↗

arXiv2018

Learning Cross-Modal Deep Embeddings for Multi-Object Image Retrieval using Text and Sketch

In this work we introduce a cross modal image retrieval system that allows both text and sketch as input modalities for the query. A cross-modal deep network architecture is formulated to jointly model the sketch and text input modalities as well as the the image output modality, learning a common embedding between text and images and between sketches and images. In addition, an attention model is used to selectively focus the attention on the different objects of the image, allowing for retrieval with multiple objects in the query. Experiments show that the proposed method performs the best in both single and multiple object image retrieval in standard datasets.

cs.CV↗

arXiv2018

Joint Recognition of Handwritten Text and Named Entities with a Neural End-to-end Model

When extracting information from handwritten documents, text transcription and named entity recognition are usually faced as separate subsequent tasks. This has the disadvantage that errors in the first module affect heavily the performance of the second module. In this work we propose to do both tasks jointly, using a single neural network with a common architecture used for plain text recognition. Experimentally, the work has been tested on a collection of historical marriage records. Results of experiments are presented to show the effect on the performance for different configurations: different ways of encoding the information, doing or not transfer learning and processing at text line or multi-line region level. The results are comparable to state of the art reported in the ICDAR 2017 Information Extraction competition, even though the proposed technique does not use any dictionaries, language modeling or post processing.

cs.CV↗

arXiv2018

Wikipedia in academia as a teaching tool: from averse to proactive faculty profiles

This study concerned the active use of Wikipedia as a teaching tool in the classroom in higher education, trying to identify different usage profiles and their characterization. A questionnaire survey was administrated to all full-time and part-time teachers at the Universitat Oberta de Catalunya and the Universitat Pompeu Fabra, both in Barcelona, Spain. The questionnaire was designed using the Technology Acceptance Model as a reference, including items about teachers web 2.0 profile, Wikipedia usage, expertise, perceived usefulness, easiness of use, visibility and quality, as well as Wikipedia status among colleagues and incentives to use it more actively. Clustering and statistical analysis were carried out using the k-medoids algorithm and differences between clusters were assessed by means of contingency tables and generalized linear models (logit). The respondents were classified in four clusters, from less to more likely to adopt and use Wikipedia in the classroom, namely averse (25.4%), reluctant (17.9%), open (29.5%) and proactive (27.2%). Proactive faculty are mostly men teaching part-time in STEM fields, mainly engineering, while averse faculty are mostly women teaching full-time in non-STEM fields. Nevertheless, questionnaire items related to visibility, quality, image, usefulness and expertise determine the main differences between clusters, rather than age, gender or domain. Clusters involving a positive view of Wikipedia and at least some frequency of use clearly outnumber those with a strictly negative stance. This goes against the common view that faculty members are mostly sceptical about Wikipedia. Environmental factors such as academic culture and colleagues opinion are more important than faculty personal characteristics, especially with respect to what they think about Wikipedia quality.

cs.SI↗

arXiv2017

e-Counterfeit: a mobile-server platform for document counterfeit detection

This paper presents a novel application to detect counterfeit identity documents forged by a scan-printing operation. Texture analysis approaches are proposed to extract validation features from security background that is usually printed in documents as IDs or banknotes. The main contribution of this work is the end-to-end mobile-server architecture, which provides a service for non-expert users and therefore can be used in several scenarios. The system also provides a crowdsourcing mode so labeled images can be gathered, generating databases for incremental training of the algorithms.

cs.CV↗

arXiv2017

Product Graph-based Higher Order Contextual Similarities for Inexact Subgraph Matching

Many algorithms formulate graph matching as an optimization of an objective function of pairwise quantification of nodes and edges of two graphs to be matched. Pairwise measurements usually consider local attributes but disregard contextual information involved in graph structures. We address this issue by proposing contextual similarities between pairs of nodes. This is done by considering the tensor product graph (TPG) of two graphs to be matched, where each node is an ordered pair of nodes of the operand graphs. Contextual similarities between a pair of nodes are computed by accumulating weighted walks (normalized pairwise similarities) terminating at the corresponding paired node in TPG. Once the contextual similarities are obtained, we formulate subgraph matching as a node and edge selection problem in TPG. We use contextual similarities to construct an objective function and optimize it with a linear programming approach. Since random walk formulation through TPG takes into account higher order information, it is not a surprise that we obtain more reliable similarities and better discrimination among the nodes and edges. Experimental results shown on synthetic as well as real benchmarks illustrate that higher order contextual similarities add discriminating power and allow one to find approximate solutions to the subgraph matching problem.

cs.CV↗

arXiv2010

Employing fuzzy intervals and loop-based methodology for designing structural signature: an application to symbol recognition

Motivation of our work is to present a new methodology for symbol recognition. We support structural methods for representing visual associations in graphic documents. The proposed method employs a structural approach for symbol representation and a statistical classifier for recognition. We vectorize a graphic symbol, encode its topological and geometrical information by an ARG and compute a signature from this structural graph. To address the sensitivity of structural representations to deformations and degradations, we use data adapted fuzzy intervals while computing structural signature. The joint probability distribution of signatures is encoded by a Bayesian network. This network in fact serves as a mechanism for pruning irrelevant features and choosing a subset of interesting features from structural signatures, for underlying symbol set. Finally we deploy the Bayesian network in supervised learning scenario for recognizing query symbols. We have evaluated the robustness of our method against noise, on synthetically deformed and degraded images of pre-segmented 2D architectural and electronic symbols from GREC databases and have obtained encouraging recognition rates. A second set of experimentation was carried out for evaluating the performance of our method against context noise i.e. symbols cropped from complete documents. The results support the use of our signature by a symbol spotting system.

cs.CV↗