Machine Learning, Probability and Statistics

- Overview

Machine learning (ML) is an interdisciplinary field that uses statistics, probability, algorithms to learn from data and provide insights that can be used to build intelligent applications. 

Statistics and ML are two very closely related fields. In fact, the line between the two can be very fuzzy at times. Nevertheless, there are methods that clearly belong to the field of statistics that are not only useful, but invaluable when working on a machine learning project. 

It would be fair to say that statistical methods are required to effectively work through a ML predictive modeling project.

• Basic Statistics  —  Mean, Median, Mode, Variance, Covariance, etc.
• Basic Rules of Probability  —  Events (correlated and independent), sample space, conditional probability.
• Random variables  —  continuous and discrete, expectation, variance, distribution (joint and conditional).
• Bayes' Theorem  —  computes the validity of beliefs. Bayesian software helps machines identify patterns and make decisions.
• Maximum Likelihood Estimation (MLE)  —  Parameter estimation. Knowledge of basic probability concepts (joint probability and independence of events) is required.
• Common distributions  —  Binomial, Poisson, Bernoulli, Gaussian, Exponential. 

- Statistical Machine Learning

Statistical ML is the application of statistical methods to make predictions about unseen data. It provides mathematical tools for analyzing the behavior and performance of machine learning algorithms. 

Statistical ML is based on statistical learning theory, which is a framework for ML that draws from statistics and functional analysis. Statistical learning theory deals with the statistical inference problem of finding a predictive function based on data. 

Statistical ML is broadly the same as ML, but the main distinction between them is in the culture. ML algorithms focus on building models that can make accurate predictions on new, unseen data. Statistical learning algorithms focus on building models to make predictions or decisions based on data.

Some of the more complex ML algorithms, such as Neural Networks, have statistical principles at their core. The optimization techniques, like gradient descent, used to train these models are based on statistical theory. 

[More to come ...]