This post is meant to give an overview of how what neural networks are, how they work, and to give an example in the Skip-Gram Model. Most of the information in this article is taken from either this article on towards data science by Gavril Ognjanovski, or this one by Rahil Vijay, so many thanks to these two creators!
note: In this post I will be using the terms "node" and "neuron" interchangeably, they are the same thing.
Neural Networks
A neural network is an attempt to recreate the inner workings of the human/animal mind. The following Image, pulled from Gavril's article (with node labels added) shows the basic structure of a neural network.
Image 1: A neural network with 2 hidden layers. The hidden layers are the layers of "neurons" that take in the input from the aptly named input layer, and send the output to the output layer. Each node in the network is connected to every node in the subsequent layer.
Each node connection in the NN has a weight attached to it. Think of this as the importance of that node in the processes of the next layer. In general the complexity of the problem will be reflected in the number of hidden layers in the network. The hidden layers are called such because they are not directly connected to the input or output layers, and are somewhat "separate" to them. I personally think this is a bit of a weird way of describing it but there you go.
Similar to the neurons of a human brain, the nodes in a neural network send signals to each other, specifically to nodes in the subsequent layer. Each signal has a related "weight", which we will denote w(i,j), where i is the node sending the signal, and j is the node receiving the signal. In image 1 above, w(1,4) would be the strength of the signal from node 1 to node 4. The signal itself we will call x(i,j), again referring to the signal sent from node i to node j. When neuron j has received all of the signals from the previous layer, it stores the value v(j):
The neural network will then input these values into whats called an "activation function". This function helps the neural network tackle more complex problems, like problems on non-linear optimisation. An example of this that will be covered later is the sigmoid function, which is used by the logistic regression model.
留言