Deep Learning-101
Introduction
Artifical Intelligence is a thriving field in many practical applications and although it has rapidly tackled problems that are intellectually difficult for human beings but straight forward for computers. Problems that can be precisely described and formally listed. The true challenge for AI though has been in terms of solving problems that are easy for humans to perform but difficult to describe such as recognizing words and faces in images.
Deep Learning offers a solution to these more intuitive problems.Allowing computers to learn from experience and eliminating the need for human intervention in describing problems and rather enabling computers to gather knowledge from experience. The underlying principle being to allow computers to understand the world in terms of hierarchy of concepts where each concept is described in terms of it’s relation to simpler concepts.
Knowledge base : Several AI projects use hard coded knowledge base about the world in formal languages. The computer can reason automatically about statements in this language by logical inferences. This is known as the knowledge base approach to AI. Biggest drawback of this method can be observed in one of the most famous project in AI called the cyc(Lenant and Guha, 1989)Cyc uses a database of statements called cycL. It was observed that cyc was unable to understand a story about a person fred who shaves every morning. The engine had inferred that humans do not have electric parts and therefore asked whether Fred was still a human when shaving.
This suggests that it might be valuable to allow AI systems extraction of pattern from raw data instead of feeding information. This capability is known as Machine Learning
A simple example of Machine Learning algorithm is logistic regression it is used to determine whether to recommend cesarean delivery. This algorithm does not examine the patient but rather relies on the doctor’s diagnosis as input. So the doctor tells the system several pieces of information that help represent the patient known as features. The performance of logistic regression is heavily dependent on this features and the system has no control over these representations. So if logistic regression was inputed with an MRI report instead of doctor’s diagnosis the system would fail to predict efficiently.
Although several AI tasks can be performed by designing the right set of features to be extracted for the task and then providing these features to simple machine learning algorithm. The challenge lies in determining these features. For example in case of a speaker identification a feature would be the speakers vocal tract. This helps identify gender.
However, in case of a program to identify cars in photographs one of the features would be presence of wheel and although wheel has a simple geometric shape it is difficult to describe in terms of pixel value. As there are several factors that could complicate the image such as shadow, light reflecting off metal parts.
One solution to this problem is to use ML not only to discover mapping representation to output but also the representation itself. This approach is known as Representational learning. Learned representations offer a performance boost to the AI system over hand-designed representing. Allowing the system to adapt to new tasks with minimal human intervention.
This example indicates how learned representations can be beneficial. If we had the task of separating 2 categories of data by drawing a vertical line. The cartesian coordinate representation of these plots makes this task impossible but in turn if we were to represent it as polar coordinates the task is very simple to solve.
When designing these features we need to separate out factors of variation. Factors are unobserved objects or unobserved forces of the physical world that affect observable quantities. Constructs in human minds that help abstract concepts by simplifying explanations and inferring causes of observed data.
It is very difficult to extract such high level abstractions from raw data. For example factors of variation such as a speakers accent can only be identified by sophisticated nearly human level understanding of data.
Deep learning solves this central problem in representational learning by introducing representations that are expressed in terms of simpler representations. Deep learning enables computers to understand complex concepts by building on simpler concepts.
Example of Deep learning model is multilayer perceptron(MLP) a mathematical function that maps some set of input values to output values. Function is formed by composing many simpler functions so each application of a different mathematical function provides a new representation of the input.
