Neuroscience Exam Questions

Hetero-Associative Memory and Auto-Associative Memory

a) Give one real-world example of hetero-associative memory and one of auto-associative-memory.

b) Which kinds of connections are typically found in neural networks that can perform auto-associative memory tasks?

c)  Learning in real and artificial neural networks can be supervised or unsupervised. For both types of learning, the data (inputs, or input-output associations) that are used for training have to have different characteristics. What are they?

d) Give two examples of artificial neural networks that perform supervised learning and two examples of artificial neural networks that perform unsupervised learning (without describing them in detail).

Question 2: Perceptrons

a) In a simple threshold linear unit (perceptron) with two inputs x1 and x2 and weights w1 and w2, how is the output y calculated?

b) Give two examples of classification tasks such a perceptron can perform.

c) What are the weights and thresholds that can be used by the perceptron to perform the two classification tasks?

d) Draw the decision lines for the two classification tasks in input space. What are the equations that describe the decision lines?

e) Simple perceptrons are able to perform a certain class of classification tasks. What are these called? Give two examples of classification tasks a simple perceptron cannot perform and draw them in input space.

f) Which type of neural network could perform these classification tasks, and why?

Question 3: Unsupervised Learning

a)  The Elastic Net Algorithm maps a 2-dimensional input space onto a 1-dimensional ring of output units. In its original application, what was represented by the 2D input space and the 1D output space? What was the original problem the Elastic Net Algorithm was designed to solve?

b) Give a simple physical analogy that is related to the name “Elastic” Net Algorithm

c) Give three examples of applications of Kohonen’s self-organising map algorithm (without describing them in detail).

Question 4: Convolutional Neural Networks

a) Convolutional neural networks (CNNs) have been widely used in image pattern recognition.  As we have introduced in the lecture, which main components are included in a conventional CNN? Draw a simple diagram to show the architecture of a conventional CNN used for image classification and explain the main function of each component in your diagram.  Be sure to indicate the role of each layer and briefly explain the role of filters (10 marks).

b) Indicate whether the following statements about convolutional neural networks (CNNs) for image analysis are TRUE or FALSE (4 marks).

(1)Filters in the earlier layers are more likely to include edge detectors.

(2)Pooling layers reduce the spatial resolution of the image.

(3)They have more parameters than fully-connected networks consisting of the same number of layers and neurons per layer.

(4)A CNN can be trained for unsupervised learning tasks, whereas in general, a neural network cannot.

Question 5: Non-linear activation functions and gradient descent (14 marks)

a)Explain briefly why is it important to include non-linear activation functions such as sigmoid, tanh, and ReLU in a neural network? What can a deep network without non-linearities learn?

b) For the sigmoid function

c) Give a brief description of the gradient descent algorithm. What are the advantages of using ReLU as an activation function compared to the sigmoid function in deep networks?

d) Indicate whether the following statements about nonlinear activation functions are TRUE or FALSE.

(1) They speed up the gradient calculation in backpropagation, as compared to linear units.

(2) They are only applied to output units.

(3) They help to learn nonlinear decision boundaries.

(4) They always output values between -1 and 1.