The Anatomy of Machine Learning Algorithms


After the publication of a Harvard Business Review article titled “The world of Data Science“, the interest in this field accelerated by quite some amount. In this article, data science was termed as the most fabulous job in the 21st century. However, it is important to analyze the basics of data science in general and machine learning in particular.

This article aims to explore the anatomy of machine learning algorithms in detail.

The Crux of Machine Learning Algorithms

A basic classification that we follow to divide machine learning algorithms is supervised and unsupervised learning.

Explaining Supervised Learning with an Analogy

Supervised learning algorithms can be explained by the following analogy. Consider a teacher who trains students so that they memorize a particular task. One of the most common techniques that are used in supervised learning is the training of data which is labeled to map input variables to output functions. This allows us to predict outputs when new inputs come into the equation. Classification and regression are two main subclasses of supervised machine learning algorithms. While predicting outcomes, classification divides given samples into various categories. For instance, classification algorithms may classify spam and non-spam email in a customer inbox. Regression algorithms are similar to classification algorithms but the chief difference between the two is that classification projects a discrete quantity while regression is used to predict a continuous quantity. For instance, a regression algorithm might be used to predict the number of sales based on previous customer data of a given month. Most prominent examples of supervised learning algorithms include linear regression, Logistic regression, k-nearest neighbors, and CART.

Unsupervised Learning

We use unsupervised learning models in the cases when we have to predict patterns in the input data and we have no labeled output data. We may be able to use training data to secure patterns in the underlying data structure. Two main types of unsupervised learning algorithms are association and clustering algorithms. Association gives the probability that a certain number of items in a data set can be grouped together. The applications of association algorithms include customer segmentation, customer analysis, and market data analysis. Research-oriented applications of the association algorithms are part and parcel of any machine learning course.

In another case, we might group our customers based on the purchases they made on a particular day to predict what purchase they would make the next day corresponding to the same basket of items. We make use of clustering algorithms to group items in the same cluster and these items are more similar to each other than objects from another cluster. We also make use of dimensionality reduction to reduce the number of variables and ensure that the complexity of the problem is reduced and the same amount of information is extracted. Two main methods which we use in dimensionality reduction are feature extraction and feature selection. In feature selection, we make a subset from the original data item by selecting a certain sample of features. In feature extraction, we transform a data set from a high-dimensional space to a lower-dimensional one while preserving the characteristics. Other examples of unsupervised learning include k-means clustering and apriori algorithms.

Reinforcement Learning

Reinforcement algorithms are used for the trial and error methods. This type of machine learning algorithm enables us to decide the best course of action based on the present state. Such types of learning algorithms are usually used in human-robot interaction. In such interactions, the trial and error method is used to detect an object lying in the environs of a robot. First, the robot starts moving randomly but over a period of time, it learns when, where, and how to deal with objects in the surroundings simultaneously.

Ensemble Techniques

Ensemble techniques are treated as meta algorithms that club several machine learning algorithms together into a consolidated model. Bagging, boosting, and stacking are three simple machine learning ensemble techniques. While bagging is used to decrease variance, boosting is used to decrease bias. Predictions can be increased by means of stacking. In simple terms, ensemble learning techniques allow better prediction and analysis as compared to the use of a single algorithm. These techniques are further divided into sequential (example AdaBoost) and parallel (example random forest models). Another method of classifying ensemble techniques is that of grouping them into homogeneous and heterogeneous models.

Concluding Remarks

The anatomy of machine learning algorithms has been put to test in several industrial applications and the process of improvisation of these algorithms is continuously going on.