Basic Concepts of Probability

- Overview

Probability is the study of chance and how likely something is to happen. Statistics is the analysis of events governed by probability. It involves analyzing the frequency of past events and using different techniques to handle data. 

Some say there are four main types of probability: classical, empirical, subjective, and axiomatic.

- Probability and Machine Learning

Probability is a fundamental concept in machine learning (ML). It's used for:

• Modeling uncertainty: Probability provides tools for modeling uncertainty that can arise from noise in data, measurement errors, or other sources.
• Quantifying uncertainty: Probability can be used to estimate uncertainty in predictions and make decisions under uncertainty.
• Evaluating model performance: Probability can help you formulate and analyze ML problems, choose appropriate models and methods, and evaluate and improve your results.
• Designing algorithms: Algorithms like Naive Bayes are designed using probability.
• Making decisions: Machines can use models to make predictions about future data and take decisions that are rational given these predictions

- Classical Probability

Classical probability assumes that certain outcomes are equally likely. For example, when rolling a die, it's equally likely to get a 1, 2, 3, 4, 5, or 6.

The formula for classical probability is P(A)= f/N. In this formula, P(A) is classical probability, f is the number of favorable outcomes, and N is the number of total possible outcomes. 

Classical probability is based on formal reasoning. For example, the classical probability of getting a head in a coin toss is 50%. 

Classical probability is simple and easy to understand, even for laypeople. It's also easy to determine probability when the definition is applicable. 

However, dividing the number of events by the number of possible events is very simplistic, and it isn't suited to finding probabilities for a lot of situations.  

Classical probability is a method for understanding probability that assumes experiments have a set number of outcomes that are equally likely. The probability of an event is the ratio of favorable outcomes to the total number of outcomes.

- Empirical Probability

In probability theory and statistics, empirical probability is the ratio of the number of times a specific event occurs to the total number of trials. It's also known as experimental probability or relative frequency. 

Empirical probability is based on historical data and is an estimate of an event's likelihood based on its frequency during experimental trials. For example, if you want to know the probability that someone prefers cola A over cola B, you could conduct a taste test of 100 people and see that 75 prefer cola A. 

Empirical probability can be calculated using the formula: P(H) = Number of times an event occurred / Total number of trails 

For example, if the probability of getting a blue ball when a ball is drawn at random is 7/40, then the empirical probability of getting a blue ball is 0.175.

Classical probability is based on theory, while empirical probability is based on actual observations. The classical method of computing probabilities doesn't require a probability experiment. It relies on counting techniques and requires equally likely outcomes.

[Downtown Miami, Florida - Civil Engineering Discoveries]

- Subjective Probability

Empirical probability, also known as experimental probability, is based on historical data. For example, you could toss a coin 100 times to see how many heads you get.

Subjective probability is often the best option when there is no past experience or theory to use. It differs from person to person and is not based on market data or historical information.

Here are some examples of subjective probability in real life:

• "I think that the president has a 25% chance of being elected again".
• "You think that a recession is about to hit, and you want to calculate the impact to your business".
• "A friend applies for a job with three other candidates".