Personal tools
 

ML Algorithms

 
Vienna_Austria_Jacek_Dylag_101020A
[Vienna, Austria - Jacek Dylag]

 

- Overview

Machine learning (ML) algorithms refer to the processes and mathematical logic by which "machines" (i.e., artificial intelligence systems) learn to recognize patterns in training data and apply that pattern recognition to make accurate predictions on new data. These algorithms serve as the fundamental building blocks of modern AI and data science, spanning everything from simple linear regression models to cutting-edge deep learning (DL) techniques.

In short, applying a ML algorithm to a dataset results in a trained model. "Training" can be understood as an iterative process that continuously updates the model's parameters - the adjustable components of the mathematical logic used to make predictions or decisions—thereby generating more useful outputs.

Over the past few decades, ML systems have superseded rule-based AI as the dominant paradigm in the fields of artificial intelligence (AI) and data analysis; one reason for this is that implicit, data-driven machine learning (ML) is inherently more flexible, scalable, and accessible.

That said, it is important to note that fitting a model to training data is merely a means to an end. The fundamental premise of ML is that optimizing a model's performance on sample tasks that closely resemble real-world application scenarios will enable it to perform well on new data. 

Please refer to the following for more information:

 

- The Ultimate Goal of ML Algorithms and Models

The ultimate goal of machine learning (ML) is generalization - the ability of a model to accurately apply what it has learned from training data to new, unseen data. 

When machine learning (ML) workflows focus too narrowly on the training phase, they risk creating a myopic (nearsighted) model. This leads to a common pitfall known as overfitting, where the model perfectly memorizes the noise and specific patterns of the training dataset but fails completely when deployed in real-world applications. 

Therefore, training an effective ML model requires not only selecting and configuring the appropriate algorithm but also properly curating the training data and rigorously validating the model's performance. 

Key Pillars of an Effective ML Model: 

To prevent overfitting and ensure successful generalization, building a robust ML system requires balancing three interconnected pillars:

  • Algorithm Selection & Configuration: Choosing the right model architecture and tuning its hyperparameters to match the complexity of the problem.
  • Data Curation: Preparing, cleaning, and structuring high-quality training data that accurately reflects the real-world environment the model will encounter.
  • Rigorous Validation: Implementing strict testing methodologies (such as cross-validation or separate holdout sets) to evaluate how well the model performs on data it has never seen before.

 

- The Role of ML Algorithms

A Machine learning (ML) algorithm is a set of established steps used to train a ML model, enabling it to make useful predictions in real-world applications. This encompasses not only how the model maps input data points to their corresponding outputs but also the process of optimizing the model's predictions so that it effectively "fits" the relevant training dataset. Essentially, it is an algorithm that empowers machines to learn from data.

ML algorithms support modern computing by enabling systems to understand data and adapt to changing environments. They are widely used in numerous AI-enabled applications to enhance accuracy and efficiency while ensuring human oversight of objectives, data, and outcomes.

  • ML algorithms learn from data rather than from fixed rules. They identify patterns and use them to make predictions or adapt over time.
  • Algorithms and models serve different purposes: algorithms define the learning process, whereas models represent the trained output used within actual systems.
  • Different types of ML algorithms address different needs. Supervised learning, unsupervised learning, and reinforcement learning each employ distinct learning methods. 
  • Their value is demonstrated through practical applications; capabilities such as automation, prediction, personalization, and anomaly detection assist in managing complex data.
  • ML operates within a broader AI ecosystem. Data science, machine learning, and AI-enabled applications work collaboratively under human guidance and supervision.

 

- Three Main Types of Learning Approaches

Although there are Machine learning (ML) algorithms specifically designed to train models for particular tasks, such cases are the exception rather than the rule. Generally, each ML algorithm possesses specific mathematical or practical characteristics suited to certain types of tasks (or specific types and volumes of data). In many instances, a single ML technique can be used to train models for multiple (albeit similar) tasks. Conversely, there are almost always several ML algorithms well-suited to training a model for any given task. 

The core advantage of ML algorithms lies in their ability to enable AI models to learn implicitly from experience. This stands in stark contrast to "classical" or "rule-based" AI techniques, which require data scientists, domain experts, or ML engineers to manually and explicitly program the model's decision-making logic. 

ML algorithms are mathematical rules that computers use to find patterns in data and make predictions without fixed instructions. Building on how these systems operate, they generally fall into three main types of learning approaches: 

(A) Main Types and Applications: 

1. Supervised Learning:

  • How it works: The algorithm trains on data that already has correct labels or answers.
  • Common use: Predicting numbers or sorting categories.
  • Example: Spotting spam emails or predicting house prices based on past sales.


2. Unsupervised Learning: 

  • How it works: The algorithm looks at data that has no labels and tries to find hidden structures.
  • Common use: Finding clusters or groupings of similar items.
  • Example: Grouping online shoppers by their similar buying habits.


3. Reinforcement Learning:

  • How it works: The algorithm learns by trying actions, making mistakes, and earning rewards.
  • Common use: Making decisions in changing environments.
  • Example: Teaching a robot or a self-driving car how to navigate a path safely. 

 

- Some Examples of ML Algorithms

There are many different types of ML algorithms, depending on the goals of your ML project, how data is fed into the algorithm, and how you want the algorithm to "learn." 

Here are some ML algorithms:

  • Linear Regression: Predicts a continuous value by fitting a linear relationship between input features and the target variable. 
  • Logistic Regression: Used for binary classification by modeling the probability of a class using a logistic function. 
  • Decision Trees: Classifies or regresses by splitting data into subsets based on feature values, creating a tree-like structure. 
  • Support Vector Machines (SVM): Finds a hyperplane that best separates data into different classes in a high-dimensional space. 
  • K-Nearest Neighbors (KNN): Classifies a data point based on the majority class of its nearest neighbors. 
  • K-Means Clustering: Partitions data into K clusters by minimizing intra-cluster variance. 
  • Hierarchical Clustering: Builds a tree of clusters by iteratively merging or splitting clusters based on feature similarity. 
  • Principal Component Analysis (PCA): Reduces the dimensionality of data while retaining the most important features. 
  • Q-Learning: A reinforcement learning algorithm where an agent learns to take actions in an environment to maximize cumulative rewards. 
  • Gradient Boosting (XGBoost, LightGBM): Ensemble methods that combine weak learners (typically decision trees) to improve predictive accuracy. 
  • Random Forests: Uses multiple decision trees for classification or regression, with each tree trained on a random subset of data. 
  • Convolutional Neural Networks (CNN): Used for image classification by applying filters to extract features from images. 
  • Recurrent Neural Networks (RNN): A neural network designed for sequential data (e.g., time series, text) by maintaining internal state across time steps. 
  • Transformers: Deep learning architecture used for Natural Language Processing tasks, relying on self-attention mechanisms to process sequential data.
 
Machine Learning Cheat Sheet_121324A
[Machine Learning Cheat Sheet]

- Some ML Techniques

Some key machine learning (ML) techniques and algorithms include: linear regression, logistic regression, support vector machines (SVM), Naive Bayes, decision trees, random forests, K-Nearest Neighbors (KNN), clustering, dimensionality reduction, gradient boosting, and AdaBoost.

The choice of which algorithm to use depends on the specific data and problem you're trying to solve, factors like data size, quality, and desired outcome playing a significant role.

Here are some ML algorithms:

  • Logistic regression: A ML technique that is good for binary classification problems. It uses a logistic function at its core.
  • Decision tree: A ML technique that uses rules and conditions to solve classification problems. It involves dividing input data into two or more homogeneous data sets based on defining attributes.
  • Support vector machine: A ML method that uses statistical theory to solve fitting accuracy and generalization problems. It is used in pattern recognition, information security, and data fitting.
  • Naive Bayes: A ML algorithm that is used when the output variable is discrete. It is driven by the Bayes Theorem.
  • Random forest: A ML process that consists of many decision trees. A decision tree is a tree-like structure where each internal node represents a test on the input attribute.
  • Clustering: A ML technique that involves grouping data points. It is an unsupervised learning method and a famous technique for statistical data analysis.
  • Hyperparameters: An integral part of machine learning code that lets you control the code without directly modifying it.
  • Gradient descent: A famous optimization technique that is used in machine learning and in deep learning. Its main purpose is to minimize the cost function.
  
 
 

[More to come ...]


Document Actions