Please refer to the following for more information:
- Training Data and Testing Data
Machine learning (ML) algorithms learn from data in datasets. They discover patterns in data, develop a deeper understanding of the data, make decisions based on the data, and evaluate the accuracy of their choices.
In ML, datasets are typically split into two subsets: training and testing data. The training data is used to train the ML algorithm. The testing data is used to evaluate the accuracy of the trained algorithm.
Understanding the differences between these two data types ensures that ML models are reliable, accurate, and effective.
In ML, training data and testing data are subsets of a dataset:
- Training data: A subset of the original data used to train a model. Training data is typically larger than testing data. It can include photos, videos, texts, or audio files. The data is labeled with classes or tags to teach the algorithm how to make predictions.
- Testing data: A subset of the training data used to test the model's performance. Testing data is typically different from the training data and not labeled. This means the model's output is unknown for each data point. Testing data can be used to assess the progress and efficiency of algorithms' training. It can also be used to modify or optimize algorithms for better results.
The process of training and testing data in ML involves several steps:
- Data collection
- Data preprocessing
- Data splitting: Train-test split
- Data augmentation (optional)
- Model training
- Model evaluation: testing
By using training and testing data, we can ensure that the ML model can make accurate predictions on new data it has not seen before.
- Validation Data
In machine learning (ML), validation data (often called the development set or "dev set") is a distinct subset of data used to evaluate a model's performance while it is being trained.
Validation data acts as a frequent checkpoint to tune hyperparameters, select the best algorithm architecture, and prevent overfitting. Crucially, the model never directly learns from or updates its internal weights based on this data.
Core Roles of Validation Data:
- Hyperparameter Tuning: It helps engineers adjust higher-level structural configurations that the algorithm cannot learn on its own (e.g., setting the learning rate or choosing the number of layers in a neural network).
- Model Selection: When training multiple candidate models (e.g., comparing a Random Forest versus a Support Vector Machine), the validation data serves as the arena to compare their scores and select the top performer.
- Preventing Overfitting: Overfitting occurs when a model memorizes training details instead of understanding broader concepts. Monitoring validation error allows for tactics like early stopping, which halts training the moment the validation performance starts deteriorating.
