Return to page

H2O.ai WIKI

Backpropagation

What is Backpropagation?

An artificial neural network’s output is improved by Backpropagation, an algorithm used in Artificial Intelligence (AI). A neural network can be considered a collection of connected input/output (I/O) nodes. Each node's accuracy is expressed as a loss function (error rate).

Backpropagation calculates the mathematical gradient of a loss function about the other weights in the neural network. Using the calculations, artificial network nodes with high error rates are given less weight than nodes with lower error rates.

Machine learning and data mining use Backpropagation (backward propagation) to improve prediction accuracy because Backpropagation is an algorithm used to calculate derivatives quickly. 

Training a neural network is a process of fine-tuning its weights based on its error rate (i.e., loss) during the previous epoch (i.e., iteration). As a result of proper tuning of the weights, the model is more reliable by increasing its generalization.

 

Examples of Backpropagation

Examples of Backpropagation are:

  • Neural Network enunciating each letter of words and sentences

  • Speech recognition 

  • Identifying faces and characters, e.g., face recognition.

 

What are the types of Backpropagation?

There are two kinds of Backpropagation networks. It is categorized as below:

Static Backpropagation

A static backpropagation network is one kind of network that aims to produce a mapping of a static input to a fixed output. This type can solve static classification problems such as optical character recognition (OCR).

Recurrent Backpropagation

In data mining, recurrent Backpropagation is used until a fixed value is achieved. Then, the error is computed and propagated backward.

In static Backpropagation, the mapping is rapid, while in recurrent Backpropagation, it is nonstatic.

 

Why is Backpropagation important?

In data mining and machine learning, Backpropagation (backward propagation) is a powerful mathematical tool that increases the accuracy of predictions. In essence, Backpropagation is an algorithm that calculates derivatives quickly. The Backpropagation algorithm is intended to develop a learning algorithm for multilayer feedforward neural networks to be trained to capture mapping implicitly.

 

 

Backpropagation vs.Other Technologies & Methodologies

Backpropagation vs. Gradient descent

The process of back-propagation involves calculating the derivatives. In contrast, gradient descent consists of descending through the gradient, i.e., setting the model’s parameters to fall through the loss function.

In summary:

  • Back-Propagation: An algorithm for calculating the gradient of a loss function concerning a model's variable

  • Gradient: An array of partial derivatives of specific input values concerning a target function.