NN 2

9 flashcards · Computer Science · by J M COOPER

1. Describe a neural network.

   is a perceptron has inputs, layers, nodes, weights, activation function and output (scaling) function Inputs feed forward to the output layer At each stage, bias terms may be added, weighted sums are calculated and used as inputs to the activation or output functions The NN an equation of the form y = f(W, x) where y is the output (which could have multiple values), x is a vector of numerical inputs, W are the weights, and f is the overall function that determines the node structure, etc.

2. How do NNs learn from experience?

   collect many input combinations, each with a known output X is a matrix with m columns – one for each input node and n rows – one for each instance of the experiences we’re learning from y is a vector containing n outputs matching each instance of the experiences we’re learning from we supply known inputs and compare expected outputs to actual outputs The difference between the two gives us the error which can then be used to adjust the link weights in the NN rinse and repeat until acceptable e…

3. How may any machine learning technique be framed? What is the goal?

   y = f(θ,X) + e y is output, θ is an appropriate set of parameters for our predictor function f, X is the input data, e is a column vector of errors Machine learning is about finding optimal parameters θˆ so that predictions yˆ are associated with (low) errors eˆ (ˆ is supposed to be on top of the symbol in question. It just means optimal.) yˆ = f(θˆ,X) + eˆ

4. Draw as a diagram:

   1 We have two input sources: x1 and x2 2 We have an input bias source 3 There is an internal layer of three z nodes, each taking in weighted inputs and outputting tanh of the summed inputs 4 There is an internal bias source for βˆ0 5 There is an output layer with one node, producing the logistic function of the weighted sum of the internal layer outputs 6 The number output is a probability between 0 and 1

5. What is an identity function?

   Identity Function - does not alter the value of the argument. The resulting range may be ∈ R.

6. what is a sigmoid function?

   Sigmoid Functions - S-shaped functions with the logistic or hyperbolic tangent functions being common. The resulting values will be bounded - (0, 1) or (−1, 1) respectively The logistic is given by: for some argument value θ.

7. Describe back propagation

   Set some initial weights (random uniform, all -1, etc.) Calculate an initial error (based on observed versus current predicted) For each weight determine if increasing or decreasing the weight increases/decreases the error Move a bit in the correct direction. Recalculate error with new parameters. Repeat Stop when further weight alterations make no/little improvement This is a gradient search, iterating over multiple dimensions (dictated by number of parameters/weights)

8. What is happening here?

   Using the link weights and the error in output node to computer the error in each node. Here we see that the error in node f4 is the product of the link weight w45 and the error in f6.

9. What is happening here?

   The errors in all the nodes have been calculated so now we can use them to adjust the link weights. The new link weight from the input x1 to node1 (wx1,1) is the link weight plus the product of the normalised error in node1, the derivative of the activation function and the input x1. (I think.)