Differences Between Machine Learning & Deep Learning

Starting off, you'll learn about Artificial Intelligence and then move to machine learning and deep learning. You will further learn how machine learning is different from deep learning, the various kinds of algorithms that fall under these two domains of learning. Finally, you will be introduced to some real-life applications where machine learning and deep learning is being applied.

What is Artificial Intelligence?

Well as the name suggests, artificial intelligence commonly known as AI is a way of artificially making a computer intelligent. Can you think of a computer or maybe a robot which can do various tasks similar to humans? It is a branch of computer science in which you programmatically code the machine to become intelligent.

According to the father of Artificial Intelligence, John McCarthy, it is "The science and engineering of making intelligent machines, especially intelligent computer programs".

Intuition & Motivation behind AI

The human brain, yes you read it right. The intuition & motivation behind AI comes from the Human brain. It's the most powerful tool given to humanity. Humans are the most intelligent species on planet earth. And the secret sauce behind this powerful brain is Neocortex. Neocortex gives you the power to think, act, function, remember, etc.., it's a common myth that whales are smarter than humans which is not true since whales have 4-5 layers in their neocortex which are smaller compared to humans that have six layers in their neocortex and are more significant than whales.

Your brain is wired in such a way that the Neocortical Memory is responsible for storing not images, but instead patterns (both temporal and spatial patterns). When these patterns or memories gets recalled the brain predicts what it should see or hear and what it encounters. And if that prediction is correct, the human feels usual, but a sense of surprise is evoked when the prediction is incorrect.

Enough science right?

Anyways, as you all remember, when you were a 3-4 year old, your parent's used to make you learn the 26 English alphabet letters, which when combined form a word, words when combined form a sentence and so on. As you grow older, you start learning from the environment, like when you were playing. If you fell, your brain would remember not to make the same mistake again. Once you learned all of this, you would then be able to predict the outcome let's say what comes before Z but after X is Y, or 2 when multiplied by 12 equals to 24.

So with a similar intuition, people wanted the computer to be able to learn based on the data you feed into it and then the computer should be smart enough to make decisions based on what it has already learned.

According to Wikipedia, "Artificial intelligence (AI), sometimes called machine intelligence, is intelligence demonstrated by machines, in contrast to the natural intelligence displayed by humans and other animals. In computer science AI research is defined as the study of "intelligent agents": any device that perceives its environment and takes actions that maximize its chance of successfully achieving its goals."

Need for Intelligent Machines?

The one primary reason behind why we need AI is to automate tasks that people feel are redundant. Remember computers are far more intelligent in calculation when it comes to multiplying two or more numbers, they can give results in a fraction of seconds whereas humans struggle.

Similarly, if you make a computer learn some data or records, based on the records, it can easily predict the future outcome since it would have understood some patterns & structures from the past data or records.

For example: Let's say you have a brain MRI image that is degraded maybe because it lacks sharpness and makes it difficult for the doctors to see the scans of your brain. In these kinds of scenarios, AI can be handy, with the correct set of algorithms like performing super-resolution using a machine learning algorithm on low-resolution brain MRI images, the quality of MRI images can be improved by projecting them into a high-resolution space.

This is just one example. There are many real-life use cases where AI has been successfully deployed.

It's time to learn what exactly Machine Learning is, so without any further ado, let's get started!

Machine Learning

In the most basic sense, Machine Learning (ML) is a way to implement artificial intelligence. Similar to AI, machine learning is a branch of computer science in which you devise or study the design of algorithms that can learn.

There are various machine learning algorithms like

• Decision trees,
• Naive Bayes,
• Random forest
• Support vector machine
• K-nearest neighbor,
• K-means clustering,
• Gaussian mixture model,
• Hidden Markov model etc.

For now, understand that in machine learning you use one of the algorithms as mentioned above which provides the computer the ability to automatically learn and understand without being programmed time and again.

Now the question is how will the computer learn automatically?

Well, the answer is data. You feed in the data having different attributes or features that the algorithms have to understand and give you a decision boundary based on the data you provide it. Once the algorithm has learned and interpreted the data, meaning it has trained itself, you can then put your algorithm in the testing phase and without explicitly programming it, input a test data point and expect it to give you some results.

For example: Let's say you have to predict the price of the house, given a dataset comprising of the cost of a home and the number of rooms in the house, and 1000 houses with similar attributes. Both the price and the number of rooms are features. Now your goal is to feed these two features into let's say decision trees algorithm. Your input, in this case, will be the number of rooms, and the algorithm has to predict the price of the house. The algorithm will try to learn a relationship between the number of rooms and cost of the home.
Now, at test time you will give the algorithm let's say three (number of rooms) as input, and it should be able to predict the price of the house accurately!

Isn't that amazing?

Let's further dive into Machine Learning and see what it has underneath the hood!

As you all know by now, machine learning is a sub-category of AI. Similarly, various learning algorithms fall under Machine Learning.

From the above diagram you would have understood various algorithms that fall under machine learning, so let's quickly understand them!

• Supervised Learning: Supervised as the name suggests, is a learning technique wherein the whole learning process is governed. These learning algorithms main goal is to predict the outcome given a set of training samples along with the training labels, also known as classifying a data point. Since, you tell the algorithm at training time, what it should predict, hence it's called supervised learning. Let's understand it with an example.
  
  Let's say you have 10,000 images, 5000 cat and 5000 dog images, each cat & dog image has a label zero and one respectively. Now, the goal of this technique is to find patterns in the data given the constraint as labels. The supervised learning algorithm will try to find a boundary between the 10,000 images which divides them into two halves. Such that at test time when a new image, let's say a cat image comes as input without any label, the algorithm puts that into the cat basket having the label as 0, which means the algorithm was able to predict/classify this image as a cat image.
  
  Voila! Isn't that surprising?

• Unsupervised Learning: Unlike supervised learning, you have no training labels for the training samples. The algorithms are formulated in such a way that they can find suitable structure and patterns in the data. Once, these consistent patterns become apparent, the similar data points can be clustered together, and different data points will be in different clusters. It is mostly used to project high-dimensional data into low-dimension for visualization or analysis purposes.

• Reinforcement Learning: It is a type of machine learning that has an agent (like a robot) that learns how to behave in an environment by taking actions and quantifying the results. If the agent makes a correct response, it gets a reward point, which boosts up the agent's confidence to take more such actions. It works on Markov Decision Process also known as (MDP).
  
  For example: In the below figure, one can see that the agent (robot) takes action in the environment (a maze), interpreter being the eyes, it gets a reward for every correct move and depending on its current state it takes the next action.

Some cool applications of Reinforcement learning are playing games (Alpha Go, Chess, Mario), robotics, traffic light control system, etc.

That's all about machine learning. Now it's time for you to know a little about Deep Learning!

Deep Learning

It is a sub-category of machine learning. Similar to machine learning, deep learning also has supervised, unsupervised, and reinforcement learning in it. As discussed earlier, the idea of AI was inspired by the human brain. So, let's try to connect the dots here, deep learning was inspired by artificial neural networks and artificial neural networks commonly known as ANN were inspired by human biological neural networks. Deep learning is one of the ways of executing machine learning.

As you can see from the above figure, it is a shallow neural network which can be called as Shallow Learning Network. A neural network will always have:

• Input layer: It can be pixels of an image or a time series data
• Hidden layer: Commonly known as weights which are learned while the neural network is trained
• Output layer: Final layer mainly gives you a prediction of the input you fed into your network.

So, the neural network is an approximation function in which the network tries to learn the parameters (weights) in hidden layers which when multiplied with the input gives you a predicted output close to the desired output.

Deep Learning is nothing but stacking multiple such hidden layers between the input and the output layer, hence the name Deep learning

Deep Learning Methods

There are various methods designed to apply deep learning. Each proposed method has a specific use case like the kind of data you have, whether it is supervised or unsupervised learning you would like to apply, what type of task you would want to solve with the data. So depending on these factors, you choose one of the methods that can best solve your problem.

Some of the deep learning methods are:

• Convolutional Neural Network,
• Recurrent Neural Network,
• Long short-term memory,
• Auto Encoders
  • Denoising autoencoder
  • Stacked autoencoder
  • Variational autoencoder
  • Sparse autoencoder
• Generative Adversarial network

Deep learning can be used for:

• Classification: whether it is a task of predicting a cat or a dog and even multi-class classification,
• Feature extraction for visualizing the data in a lower dimensional space,
• Data preprocessing,
• Time series prediction: stock prediction, weather prediction, etc.,
• Regression tasks like localization,
• Object recognition which is both a classification & a regression task,
• Robotics.

Comparison between Deep Learning & Machine Learning

• Functioning: Deep learning is a subset of machine learning that takes data as an input and makes intuitive and intelligent decisions using an artificial neural network stacked layer-wise. On the other hand, machine learning being a super-set of deep learning takes data as an input, parses that data, tries to make sense of it (decisions) based on what it has learned while being trained.

• Feature Extractor: Deep learning is considered to be a suitable method for extracting meaningful features from the raw data. It does not depend on hand-crafted features like local binary patterns, a histogram of gradients, etc., and most importantly it performs a hierarchical feature extraction. It learns features layer-wise which means that in initial layers it learns low-level features and as it moves up the hierarchy it starts to learn a more abstract representation of the data (as shown in the figure below). On the other hand, machine learning is not a good method for extracting meaningful features from the data. It relies on hand-crafted features as an input to perform well.

Here features mean pixel values, shape, textures, position, color and orientation. The performance of most of the traditional machine learning algorithm depends on how accurately the features are identified and extracted. Using traditional feature extractors does not solve the significant problem since even a slight variation in the data changes the features extracted from a conventional feature extractor like local binary pattern (LBP), a histogram of oriented gradients (HOG), etc. Whereas, a deep learning network tries to learn all of these features through a combination of layers at different levels and finally combines them to form a bigger picture as an abstract representation.

• Data Dependency: Machine learning algorithms often work well even if the dataset is small, but deep learning is Data Hungry the more data you have, the better it is likely to perform. It is often said that with more data the network depth (number of layers) also increase hence more computation.

From the above figure, you can see that as the data increases the performance of deep learning algorithms increases compared to traditional machine learning algorithms in which the performance almost saturates after a while even if the data is increased.

• Computation Power: As you learned that deep learning networks are data dependent, they need more than what a CPU can offer. For the deep learning network training, you need a graphical processing unit (GPU) which have thousands of cores compared to a CPU that has very minimal cores. The computation power not only depends on the amount data but also on how deep (large) is your network, as you increase the amount of data or the number of layers, you need more and more computation power. On the other hand, a traditional machine learning algorithm can be implemented on a CPU with fairly decent specifications.

• Training and Inference Time: The training time of a deep learning network can range from anywhere between a few hours to months. Yes, you read it right! The training can often last for months. If you have a vast number of data, training a network a more significant data usually takes time. Moreover, as you increase the number of layers in your network, the number of parameters known as weights will increase, hence, resulting in slow training.
  
  Not only training but very deep neural networks can also take a lot of inference time since the input test data will pass through all the layers in your network, a lot of multiplication will take place which will consume a considerable amount of time.

Whereas traditional machine learning algorithms often train very fast ranging from few minutes to a couple of hours, but during the test time, some algorithms can also take quite a bit of time.

• Problem-solving technique: To solve a problem using machine learning, you have to divide the problem into different parts. Let's say you want to do object recognition, for that you first traverse through the complete image and find if there is an object at each location and where exactly it is present. "Then, from all the candidate objects, you apply a machine learning algorithm let's say support vector machine (SVM) with local binary patterns (LBP) as feature extractor to recognize relevant objects. On the other hand, in deep learning, you give the network both the bounding box coordinates & all the object's corresponding labels, and network learns to localize and classify on its own." -Source"

• Industry Ready: Machine learning algorithms are often easy to Decode how they worked. They are interpretable regarding what parameters it chose and why it chose those parameters, but on the other hand, deep learning algorithms are nothing but a black box. Even if the deep learning algorithms can surpass humans in performance, they are still not reliable when it comes to deploying them in the industry. Machine learning algorithms like linear regression, decision trees, random forest, etc., are widely used in industries like one of its use case is in bank sector for stock predictions.

• Output: The output of a traditional machine learning is usually a numerical value like a score or a classification. Whereas, the output of a deep learning method can be a score, an element, text, speech, etc.

Applications of Machine Learning and Deep Learning

Machine learning and deep learning are widely used in many domains to name a few:

• Medical: For cancer cell detection, brain MRI image restoration, gene printing, etc.
• Document: Super-resolving historical document images, segmenting text in document images.
• Banks: Stock prediction, financial decisions.
• Natural Language Processing: recommendation systems: Netflix uses recommendation system to suggest movies to users based on their interest, sentiment analysis, photo tagging.
• Information Retrieval: search engines, both text search, and image search like the one used by Google, Amazon, Facebook, Linkedin, etc.

Future Prospects

Both deep learning and machine learning is on the boom from quite some time, and it is there to stay for at least a decade from now. The industries are deploying deep learning and machine learning algorithms to generate more revenues; they are educating their employees to learn this skill and contribute to their firm. A lot of startups are coming up with novel deep learning solutions which can solve challenging problems.

Not only industry but also in academia a lot of groundbreaking research is taking place every day, and the way deep learning is revolutionizing the world utterly mind-boggling. The deep learning architectures have performed a lot better than the contemporary methods and have achieved state-of-the-art performances.

To survive in either industry or academia possessing deep learning and machine learning skills will most likely play a significant role in the coming years.

Conclusion

In today's post, you were introduced to a comprehensive theoretical analysis of artificial intelligence and two techniques to implement artificial intelligence namely machine learning & deep learning. You also learned about various machine learning & deep learning algorithms and how these two learning techniques are different from each other.

This post was just a beginning; there is still a lot to cover so why not take DataCamp's Deep Learning in Python course? Also, following Convolutional Neural Networks in Python with Keras tutorial could be an excellent starting point for you all!