arXiv subjects

Jonathan Tapson

Publications and source records attributed to Jonathan Tapson.

24 records · Page 2Linked to original sources

arXiv2014

Explicit Computation of Input Weights in Extreme Learning Machines

We present a closed form expression for initializing the input weights in a multi-layer perceptron, which can be used as the first step in synthesis of an Extreme Learning Ma-chine. The expression is based on the standard function for a separating hyperplane as computed in multilayer perceptrons and linear Support Vector Machines; that is, as a linear combination of input data samples. In the absence of supervised training for the input weights, random linear combinations of training data samples are used to project the input data to a higher dimensional hidden layer. The hidden layer weights are solved in the standard ELM fashion by computing the pseudoinverse of the hidden layer outputs and multiplying by the desired output values. All weights for this method can be computed in a single pass, and the resulting networks are more accurate and more consistent on some standard problems than regular ELM networks of the same size.

cs.NE↗

arXiv2014

Online and Adaptive Pseudoinverse Solutions for ELM Weights

The ELM method has become widely used for classification and regressions problems as a result of its accuracy, simplicity and ease of use. The solution of the hidden layer weights by means of a matrix pseudoinverse operation is a significant contributor to the utility of the method; however, the conventional calculation of the pseudoinverse by means of a singular value decomposition (SVD) is not always practical for large data sets or for online updates to the solution. In this paper we discuss incremental methods for solving the pseudoinverse which are suitable for ELM. We show that careful choice of methods allows us to optimize for accuracy, ease of computation, or adaptability of the solution.

cs.NE↗

arXiv2014

ELM Solutions for Event-Based Systems

Whilst most engineered systems use signals that are continuous in time, there is a domain of systems in which signals consist of events. Events, like Dirac delta functions, have no meaningful time duration. Many important real-world systems are intrinsically event-based, including the mammalian brain, in which the primary packets of data are spike events, or action potentials. In this domain, signal processing requires responses to spatio-temporal patterns of events. We show that some straightforward modifications to the standard ELM topology produce networks that are able to perform spatio-temporal event processing online with a high degree of accuracy. The modifications involve the re-definition of hidden layer units as synaptic kernels, in which the input delta functions are transformed into continuous-valued signals using a variety of impulse-response functions. This permits the use of linear solution methods in the output layer, which can produce events as output, if modeled as a classifier; the output classes are 'event' or 'no event'. We illustrate the method in application to a spike-processing problem.

cs.NE↗

arXiv2013

The Ripple Pond: Enabling Spiking Networks to See

In this paper we present the biologically inspired Ripple Pond Network (RPN), a simply connected spiking neural network that, operating together with recently proposed PolyChronous Networks (PCN), enables rapid, unsupervised, scale and rotation invariant object recognition using efficient spatio-temporal spike coding. The RPN has been developed as a hardware solution linking previously implemented neuromorphic vision and memory structures capable of delivering end-to-end high-speed, low-power and low-resolution recognition for mobile and autonomous applications where slow, highly sophisticated and power hungry signal processing solutions are ineffective. Key aspects in the proposed approach include utilising the spatial properties of physically embedded neural networks and propagating waves of activity therein for information processing, using dimensional collapse of imagery information into amenable temporal patterns and the use of asynchronous frames for information binding.

cs.NE↗

arXiv2012

Learning the Pseudoinverse Solution to Network Weights

The last decade has seen the parallel emergence in computational neuroscience and machine learning of neural network structures which spread the input signal randomly to a higher dimensional space; perform a nonlinear activation; and then solve for a regression or classification output by means of a mathematical pseudoinverse operation. In the field of neuromorphic engineering, these methods are increasingly popular for synthesizing biologically plausible neural networks, but the "learning method" - computation of the pseudoinverse by singular value decomposition - is problematic both for biological plausibility and because it is not an online or an adaptive method. We present an online or incremental method of computing the pseudoinverse, which we argue is biologically plausible as a learning method, and which can be made adaptable for non-stationary data streams. The method is significantly more memory-efficient than the conventional computation of pseudoinverses by singular value decomposition.

cs.NE↗

arXiv2010

Salience-Affected Neural Networks

We present a simple neural network model which combines a locally-connected feedforward structure, as is traditionally used to model inter-neuron connectivity, with a layer of undifferentiated connections which model the diffuse projections from the human limbic system to the cortex. This new layer makes it possible to model global effects such as salience, at the same time as the local network processes task-specific or local information. This simple combination network displays interactions between salience and regular processing which correspond to known effects in the developing brain, such as enhanced learning as a result of heightened affect. The cortex biases neuronal responses to affect both learning and memory, through the use of diffuse projections from the limbic system to the cortex. Standard ANNs do not model this non-local flow of information represented by the ascending systems, which are a significant feature of the structure of the brain, and although they do allow associational learning with multiple-trial, they simply don't provide the capacity for one-time learning. In this research we model this effect using an artificial neural network (ANN), creating a salience-affected neural network (SANN). We adapt an ANN to embody the capacity to respond to an input salience signal and to produce a reverse salience signal during testing. This research demonstrates that input combinations similar to the inputs in the training data sets will produce similar reverse salience signals during testing. Furthermore, this research has uncovered a novel method for training ANNs with a single training iteration.

cs.NE↗