- Activation Function
- Confusion Matrix
- Convolutional Neural Networks
- Forward Propagation
- Generative Adversarial Network
- Gradient Descent
- Linear Regression
- Logistic Regression
- Machine Learning Algorithms
- Multilayer Perceptron
- Naive Bayes
- Neural Networking and Deep Learning
- RuleFit
- Stack Ensemble
- Word2Vec
- XGBoost

- Attention Mechanism
- BERT
- Binary Classification
- Classify Token ([CLS])
- Conversational Response Generation
- GLUE (General Language Understanding Evaluation)
- GPT (Generative Pre-Trained Transformers)
- Language Modeling
- Layer Normalization
- Mask Token ([MASK])
- Probability Distribution
- Probing Classifiers
- SQuAD (Stanford Question Answering Dataset)
- Self-attention
- Separate token ([SEP])
- Sequence-to-sequence Language Generation
- Sequential Text Spans
- Text Classification
- Text Generation
- Transformer Architecture
- WordPiece

- AUC-ROC
- Analytical Review
- Autoencoders
- Bias-Variance Tradeoff
- Decision Optimization
- Explanatory Variables
- Exponential Smoothing
- Level of Granularity
- Long Short-Term Memory
- Loss Function
- Model Management
- Precision and Recall
- Predictive Learning
- ROC Curve
- Recommendation system
- Stochastic Gradient Descent
- Target Leakage
- Target Variable
- Underwriting

A

C

D

G

L

M

N

P

R

S

T

X

- Activation Function
- Confusion Matrix
- Convolutional Neural Networks
- Forward Propagation
- Generative Adversarial Network
- Gradient Descent
- Linear Regression
- Logistic Regression
- Machine Learning Algorithms
- Multilayer Perceptron
- Naive Bayes
- Neural Networking and Deep Learning
- RuleFit
- Stack Ensemble
- Word2Vec
- XGBoost

- Attention Mechanism
- BERT
- Binary Classification
- Classify Token ([CLS])
- Conversational Response Generation
- GLUE (General Language Understanding Evaluation)
- GPT (Generative Pre-Trained Transformers)
- Language Modeling
- Layer Normalization
- Mask Token ([MASK])
- Probability Distribution
- Probing Classifiers
- SQuAD (Stanford Question Answering Dataset)
- Self-attention
- Separate token ([SEP])
- Sequence-to-sequence Language Generation
- Sequential Text Spans
- Text Classification
- Text Generation
- Transformer Architecture
- WordPiece

- AUC-ROC
- Analytical Review
- Autoencoders
- Bias-Variance Tradeoff
- Decision Optimization
- Explanatory Variables
- Exponential Smoothing
- Level of Granularity
- Long Short-Term Memory
- Loss Function
- Model Management
- Precision and Recall
- Predictive Learning
- ROC Curve
- Recommendation system
- Stochastic Gradient Descent
- Target Leakage
- Target Variable
- Underwriting

Cross-validation is a method used to determine if a model is accurately interpreting and predicting data. This method allows for training and testing different properties in a dataset and tuning model hyperparameters. Cross-validation evaluates and compares learning algorithms by dividing a data set into two segments: one used to train the model and the other used to validate. A model is considered accurate if it can correctly adjust to the validation set.

Cross-validation is used to assess the accuracy of a model. To evaluate the performance of any machine learning model, it must be tested on unseen data. Based on the performance of the unseen data, the model can be identified as either under-fit, over-fit, or well-fit.

Over-fitting is a problem that occurs when the machine learning algorithm performs differently on training data than on unseen data. Under-fitting refers to a model that does not perform well on the training data or generalize to the new data. When the model makes predictions without error, it is well-fit to the data.

There are several techniques to perform cross-validation, the following are the most commonly used methods.

k-fold cross-validation is a technique used to evaluate ML models on a limited data sample. k-fold refers to the number of folds the dataset is split into. This method is the most widely used as it generally results in a less biased or optimistic estimate of the models accuracy

The procedure is as follows:

Shuffle the dataset randomly

Split the dataset into k groups. The value for k can be calculated or chosen, to learn more click here.

For each unique group:

Take the group as a holdout or test data set

Take the remaining groups as a training data set

Fit the model on the training data set and evaluate it on the test set

Retain the evaluation score and discard the model

Summarize the skill of the model using the sample of model evaluation scores

The results of a k-fold cross-validation run are summarized with the mean of the model skill scores. The averaged score will determine the accuracy of the model.

**Nested k-Fold Cross-Validation** is an approach to overcome bias by nesting the hyperparameter optimization procedure under the model selection procedure.

The use of two cross-validation loops is also known as double cross-validation.

Train_test split is a model validation procedure that simulates how a model performs on new and unseen data. As previously mentioned, datasets can be split into training and testing data. The training set contains a known output and is used to train the model. The test dataset is used to validate the model’s prediction after training. Here is how the procedure works:

Arrange The Data

Make sure your data is arranged into a format acceptable for train test split.

Split the Data

Split the data set into two pieces — a training set and a testing set. This consists of random sampling without replacement, using the majority as the training set and the minority as the testing set. A general rule is 75/25.

Train the Model

Train the model on the training set.

Test the Model

Test the model on the testing set and evaluate

Leave-One-Out Cross-Validation, or LOOCV, is used to estimate the performance of a model's predictions. This procedure is most appropriate for a small dataset. LOOCV is a configuration of k-fold cross-validation where k is set to the number of examples in the dataset. It requires one model to be created and evaluated for each example in the training dataset. This provides a robust estimate of model performance as each row of data represents the test dataset. Once models have been evaluated using LOOCV and a final model and configuration are chosen, the model is then used on all available data and used to make predictions on new data.

In this approach, the p datasets are left out of the training data. If there are a total of n data points in the original input dataset, then n data points will be used as the training dataset and the p data points as the validation set. This process is repeated for all the samples, and the average error is calculated to determine the effectiveness of the model.

Some cross-validation techniques may be more appropriate in certain situations than others. In choosing a specific cross-validation procedure, one should consider both costs (eg. inefficient use of available data in estimating regression parameters) and benefits (eg. accuracy in estimation).

In complex machine learning models, the same data can inadvertently be used in different steps of the pipeline. This can lead to inaccurate results and problems within the models. The selection of the proper method and the correct use of data can ensure errors are found and corrected for a more accurate result.