arXiv subjects

Franck Vermet

Publications and source records attributed to Franck Vermet.

29 records · Page 2Linked to original sources

arXiv2020

Some Remarks on Replicated Simulated Annealing

Recently authors have introduced the idea of training discrete weights neural networks using a mix between classical simulated annealing and a replica ansatz known from the statistical physics literature. Among other points, they claim their method is able to find robust configurations. In this paper, we analyze this so-called "replicated simulated annealing" algorithm. In particular, we explicit criteria to guarantee its convergence, and study when it successfully samples from configurations. We also perform experiments using synthetic and real data bases.

cs.LG↗

arXiv2020

Interpretabilité des modèles : état des lieux des méthodes et application à l'assurance

Since May 2018, the General Data Protection Regulation (GDPR) has introduced new obligations to industries. By setting a legal framework, it notably imposes strong transparency on the use of personal data. Thus, people must be informed of the use of their data and must consent the usage of it. Data is the raw material of many models which today make it possible to increase the quality and performance of digital services. Transparency on the use of data also requires a good understanding of its use through different models. The use of models, even if efficient, must be accompanied by an understanding at all levels of the process that transform data (upstream and downstream of a model), thus making it possible to define the relationships between the individual's data and the choice that an algorithm could make based on the analysis of the latter. (For example, the recommendation of one product or one promotional offer or an insurance rate representative of the risk.) Models users must ensure that models do not discriminate against and that it is also possible to explain its result. The widening of the panel of predictive algorithms - made possible by the evolution of computing capacities -- leads scientists to be vigilant about the use of models and to consider new tools to better understand the decisions deduced from them . Recently, the community has been particularly active on model transparency with a marked intensification of publications over the past three years. The increasingly frequent use of more complex algorithms (\textit{deep learning}, Xgboost, etc.) presenting attractive performances is undoubtedly one of the causes of this interest. This article thus presents an inventory of methods of interpreting models and their uses in an insurance context.

stat.ML↗

arXiv2020

Towards an Intrinsic Definition of Robustness for a Classifier

The robustness of classifiers has become a question of paramount importance in the past few years. Indeed, it has been shown that state-of-the-art deep learning architectures can easily be fooled with imperceptible changes to their inputs. Therefore, finding good measures of robustness of a trained classifier is a key issue in the field. In this paper, we point out that averaging the radius of robustness of samples in a validation set is a statistically weak measure. We propose instead to weight the importance of samples depending on their difficulty. We motivate the proposed score by a theoretical case study using logistic regression, where we show that the proposed score is independent of the choice of the samples it is evaluated upon. We also empirically demonstrate the ability of the proposed score to measure robustness of classifiers with little dependence on the choice of samples in more complex settings, including deep convolutional neural networks and real datasets.

cs.CV↗

arXiv2020

Multi-group Binary Choice with Social Interaction and a Random Communication Structure -- a Random Graph Approach

We construct and analyze a random graph model for discrete choice with social interaction and several groups of equal size. We concentrate on the case of two groups of equal sizes and we allow the interaction strength within a group to differ from the interaction strength between the two groups. Given that the resulting graph is sufficiently dense we show that, with probability one, the average decision in each of the two groups is the same as in the fully connected model. In particular, we show that there is a phase transition: If the interaction among a group and between the groups is strong enough the average decision per group will either be positive or negative and the decision of the two groups will be correlated. We also compute the free energy per particle in our model.

math.PR↗

arXiv2017

Improving Accuracy of Nonparametric Transfer Learning via Vector Segmentation

Transfer learning using deep neural networks as feature extractors has become increasingly popular over the past few years. It allows to obtain state-of-the-art accuracy on datasets too small to train a deep neural network on its own, and it provides cutting edge descriptors that, combined with nonparametric learning methods, allow rapid and flexible deployment of performing solutions in computationally restricted settings. In this paper, we are interested in showing that the features extracted using deep neural networks have specific properties which can be used to improve accuracy of downstream nonparametric learning methods. Namely, we demonstrate that for some distributions where information is embedded in a few coordinates, segmenting feature vectors can lead to better accuracy. We show how this model can be applied to real datasets by performing experiments using three mainstream deep neural network feature extractors and four databases, in vision and audio.

cs.LG↗

arXiv2017

Associative Memories to Accelerate Approximate Nearest Neighbor Search

Nearest neighbor search is a very active field in machine learning for it appears in many application cases, including classification and object retrieval. In its canonical version, the complexity of the search is linear with both the dimension and the cardinal of the collection of vectors the search is performed in. Recently many works have focused on reducing the dimension of vectors using quantization techniques or hashing, while providing an approximate result. In this paper we focus instead on tackling the cardinal of the collection of vectors. Namely, we introduce a technique that partitions the collection of vectors and stores each part in its own associative memory. When a query vector is given to the system, associative memories are polled to identify which one contain the closest match. Then an exhaustive search is conducted only on the part of vectors stored in the selected associative memory. We study the effectiveness of the system when messages to store are generated from i.i.d. uniform $\pm$1 random variables or 0-1 sparse i.i.d. random variables. We also conduct experiment on both synthetic data and real data and show it is possible to achieve interesting trade-offs between complexity and accuracy.

cs.LG↗

arXiv2017

On a model of associative memory with huge storage capacity

In [7] Krotov and Hopfield suggest a generalized version of the well-known Hopfield model of associative memory. In their version they consider a polynomial interaction function and claim that this increases the storage capacity of the model. We prove this claim and take the "limit" as the degree of the polynomial becomes infinite, i.e. an exponential interaction function. With this interaction we prove that model has an exponential storage capacity in the number of neurons, yet the basins of attraction are almost as large as in the standard Hopfield model.

math.PR↗

arXiv2016

A Comparative Study of Sparse Associative Memories

We study various models of associative memories with sparse information, i.e. a pattern to be stored is a random string of $0$s and $1$s with about $\log N$ $1$s, only. We compare different synaptic weights, architectures and retrieval mechanisms to shed light on the influence of the various parameters on the storage capacity.

math.PR↗

arXiv2015

Capacity of an associative memory model on random graph architectures

We analyze the storage capacity of the Hopfield models on classes of random graphs. While such a setup has been analyzed for the case that the underlying random graph model is an Erdös-Renyi graph, other architectures, including those investigated in the recent neuroscience literature, have not been studied yet. We develop a notion of storage capacity that highlights the influence of the graph topology and give results on the storage capacity for not too irregular random graph models. The class of models investigated includes the popular power law graphs for some parameter values.

math.PR↗

arXiv2014

Large deviation upper bounds for sums of positively associated indicators

We give exponential upper bounds for $P(S \le k)$, in particular $P(S=0)$, where $S$ is a sum of indicator random variables that are positively associated. These bounds allow, in particular, a comparison with the independent case. We give examples in which we compare with a famous exponential inequality for sums of correlated indicators, the Janson inequality. Here our bound sometimes proves to be superior to Janson's bound.

math.PR↗

arXiv2014

On the capacity of a new model of associative memory based on neural cliques

Based on recent work by Gripon and Berrou, we introduce a new model of an associative memory. We show that this model has an efficiency bounded away from 0 and is therefore significantly more effective than the well known Hopfield model. We prove that the synchronous and asynchronous retrieval dynamics converge and give upper and lower bounds on the memory capacity of the model.

math.PR↗