Basic of the computer brain - neural networks
Mother nature has spent so many million years to get humans to be able to see things and identify them correctly. Not only can we identify them we can associate specific memories with it and assess the future course of action based on previous experiences. For instance, we know what to do if somebody hands us a hammer during a craft workshop.
How do we know what to do during these situations?
Well, we have a brain of course. In our brains, we have neurons which transmit nerve impulses to process information. Basically, acting as small decision points that passes an electrical current when the threshold has been reached. A human brain has around 86 billion neurons. So, assuming we want to build a machine that is just as smart, or even smarter, than humans we would need the machine to have at least 86 billion decision points. When we are creating neural networks, we are essentially trying to translate all of this biological device into codes in machines. To fit something that took million years of evolution to create into some machine… meh not hard at all.
Neurons in our brain create a neural network the same perceptrons Developed in the 50s and 60s, perceptrons take several binary inputs, like x1 x2 and so on, and produce a single binary output. With each input, there is an associated weight is multiplied with it, the product of the weight and input is summed together and compared with a threshold value. If the sum is less than the threshold value then output is 0, if more then the output is 1.
This method is further simplified by replacing the threshold value with a negative bias value because comparing a weighted sum to 0 is more straightforward and intuitive in programming. As the neural net is trained on more data, the weights are adjusted to optimize the system and reduce error. The final weighted sum is also passed through something called an activation function. If you’d like to learn more about the nitty gritty of neural networks please check out the following link.
When you stack perceptrons together and you get a layer. When you stack layers together you form a neural network. The three major components to a neural network
As the number of hidden layers increases, you create Deep Neural Networks (DNN).
An age-old quest for artificial intelligence was for machines to recognize handwritten digits. If you wanted to a model that can recognize handwritten digits, then you would train your model with images of the handwritten digit and the number represented. The image inputted into the model is decomposed into arrays of pixels, each pixel is assigned a number from 0 to 255, run some calculations, and the model calculates a probability which then classifies the image as a number in the range [0,9]. The error rate of this model would be the probability of the model assigning the wrong number to a handwritten digit.
All this information about neural net is good and all but how do we actually create them?
Neural networks can be created in Java, C++, Python, and even MATLAB. You also have languages like R which was heavily used for projects in the past. For deep learning applications most, programmers have moved towards coding in Python with add-on libraries like Pytorch and Tensorflow. Although Caffe, written in C++ with Python interface, is still used in many research teams for deep learning models.
Perhaps due to influence from the professor I worked with, I prefer using Tensorflow with Keras on top. TensorFlow is an open source software library for numerical computation using data flow graphs. It was originally created by Google as an internal machine learning tool but the implementation of it was open sourced in 2015. Now, it has many available machine learning libraries that grow every day, and higher level APIs that work on top like Keras. Tensorflow is also used by tech companies like Uber, Airbnb, and Snapchat.