Spiking neural networks applied to computer vision problem.

During last decade, deep convolutional neural networks have become the reference in computer vision. However, spiking neural networks bring to computer vision the double benefit of (1) performing a learning that is mainly unsupervised (thanks to bio-inspired learning rules like Spike-Timing Dependent Plasticity) – which avoids to resort to manually annotated data, and (2) allows hardware implementations that are both extremely energy efficient and computationally efficient. Circumventing the two main pitfalls of computer vision is a considerable challenge.
Given the original approach that we follow and the little amount of prior work, our early work have mainly been exploratory, following two directions:

  • study of spiking neural networks and their application to a classical computer vision problem: motion detection,
  • SNN implementation of well-established vision architectures: HMAX and convolutional architectures.

The latter direction aims at targeting modern vision problems (with over 100 classes), where multi-layer networks are likely to help capturing complex classification problems. In a multi-layer setting however, one intrinsic limitation of spiking models is the strong attenuation of output layers’ neural activity, namely because of input spike integration and neuron competition. This activity loss directly impedes learning capabilities of the network because the STDP learning rule is triggered by neural activity. In order to allow operational multi-layer SNN, we have proposed a number of techniques to help spikes propagation throughout layers, namely target frequency threshold adaptation, which forces neurons to reach a desired frequency, binary coding, which improves the performance of the network at high levels of activity, and mirrored STDP, which improves the convergence of the training.