arXiv subjects

Pascal Vincent

Publications and source records attributed to Pascal Vincent.

78 records · Page 5Linked to original sources

arXiv2012

A Generative Process for Sampling Contractive Auto-Encoders

The contractive auto-encoder learns a representation of the input data that captures the local manifold structure around each data point, through the leading singular vectors of the Jacobian of the transformation from input to representation. The corresponding singular values specify how much local variation is plausible in directions associated with the corresponding singular vectors, while remaining in a high-density region of the input space. This paper proposes a procedure for generating samples that are consistent with the local structure captured by a contractive auto-encoder. The associated stochastic process defines a distribution from which one can sample, and which experimentally appears to converge quickly and mix well between modes, compared to Restricted Boltzmann Machines and Deep Belief Networks. The intuitions behind this procedure can also be used to train the second layer of contraction that pools lower-level features and learns to be invariant to the local directions of variation discovered in the first layer. We show that this can help learn and represent invariances present in the data and improve classification error.

cs.LG↗

arXiv2011

A single NV defect coupled to a nanomechanical oscillator

A single Nitrogen Vacancy (NV) center hosted in a diamond nanocrystal is positioned at the extremity of a SiC nanowire. This novel hybrid system couples the degrees of freedom of two radically different systems, i.e. a nanomechanical oscillator and a single quantum object. The dynamics of the nano-resonator is probed through time resolved nanocrystal fluorescence and photon correlation measurements, conveying the influence of a mechanical degree of freedom given to a non-classical photon emitter. Moreover, by immersing the system in a strong magnetic field gradient, we induce a magnetic coupling between the nanomechanical oscillator and the NV electronic spin, providing nanomotion readout through a single electronic spin. Spin-dependent forces inherent to this coupling scheme are essential in a variety of active cooling and entanglement protocols used in atomic physics, and should now be within the reach of nanomechanical hybrid systems.

cond-mat.mes-hall↗

arXiv2011

Learning invariant features through local space contraction

We present in this paper a novel approach for training deterministic auto-encoders. We show that by adding a well chosen penalty term to the classical reconstruction cost function, we can achieve results that equal or surpass those attained by other regularized auto-encoders as well as denoising auto-encoders on a range of datasets. This penalty term corresponds to the Frobenius norm of the Jacobian matrix of the encoder activations with respect to the input. We show that this penalty term results in a localized space contraction which in turn yields robust features on the activation layer. Furthermore, we show how this penalty term is related to both regularized auto-encoders and denoising encoders and how it can be seen as a link between deterministic and non-deterministic auto-encoders. We find empirically that this penalty helps to carve a representation that better captures the local directions of variation dictated by the data, corresponding to a lower-dimensional non-linear manifold, while being more invariant to the vast majority of directions orthogonal to the manifold. Finally, we show that by using the learned features to initialize a MLP, we achieve state of the art classification error on a range of datasets, surpassing other methods of pre-training.

cs.AI↗

arXiv2011

Adding noise to the input of a model trained with a regularized objective

Regularization is a well studied problem in the context of neural networks. It is usually used to improve the generalization performance when the number of input samples is relatively small or heavily contaminated with noise. The regularization of a parametric model can be achieved in different manners some of which are early stopping (Morgan and Bourlard, 1990), weight decay, output smoothing that are used to avoid overfitting during the training of the considered model. From a Bayesian point of view, many regularization techniques correspond to imposing certain prior distributions on model parameters (Krogh and Hertz, 1991). Using Bishop's approximation (Bishop, 1995) of the objective function when a restricted type of noise is added to the input of a parametric function, we derive the higher order terms of the Taylor expansion and analyze the coefficients of the regularization terms induced by the noisy input. In particular we study the effect of penalizing the Hessian of the mapping function with respect to the input in terms of generalization performance. We also show how we can control independently this coefficient by explicitly penalizing the Jacobian of the mapping function on corrupted inputs.

cs.AI↗

arXiv2010

Digital and FM demodulation of a doubly-clamped single wall carbon nanotube oscillator: towards a nanotube cell phone

Electromechanical resonators are a key element in radio-frequency telecommunications devices and thus new resonator concepts from nanotechnology can readily find important industrial opportunities. In this paper, we report the successful experimental realization of AM, FM and digital demodulation with suspended single wall carbon nanotube resonators in the field effect transistor configuration. The crucial role played by the mechanical resonance in demodulation is clearly demonstrated. The FM technique is shown to lead to the suppression of unwanted background signals and the reduction of noise for a better detection of the mechanical motion of nanotubes. The digital data transfer rate of standard cell phone technology is within the reach of our devices.

cond-mat.mes-hall↗

arXiv2010

Self-oscillations in field emission nanowire mechanical resonators: a nanometric DC-AC conversion

We report the observation of self-oscillations in a bottom-up nanoelectromechanical system (NEMS) during field emission driven by a constant applied voltage. An electromechanical model is explored that explains the phenomenon and that can be directly used to develop integrated devices. In this first study we have already achieved ~50% DC/AC (direct to alternative current) conversion. Electrical self-oscillations in NEMS open up a new path for the development of high speed, autonomous nanoresonators, and signal generators and show that field emission (FE) is a powerful tool for building new nano-components.

cond-mat.mes-hall↗