- Logistic regression models the probability of a specific outcome given input variables.
- The most common form models a binary outcome, such as yes/no or true/false.
- Multinomial logistic regression models more than two possible outcomes.
- Typical uses include credit card fraud detection and email spam detection.
- Logistic regression is easy to implement, interpret and train, which makes it one of the most widely used algorithms in machine learning.
What is Logistic Regression?
Logistic regression is the process of modeling probabilities of a specific outcome given input variables. The most common logistic regression models a binary outcome that can take two values such as healthy/not healthy, yes/no, true/false, and so on. Multinomial logistic regression can model more than two possible outcomes.
Examples of Logistic Regression
There are many practical examples of logistic regression used in everyday life such as:
Credit Card Fraud Detection: When a credit card transaction happens, a bank takes note of several things that are happening at the time of the transaction; transaction date, transaction amount, location, type of purchase, and so on. These factors are used to develop a logistic regression model to predict an outcome of whether or not the credit card transaction was fraudulent.
Email Spam Detection: Spam detection is a binary classification problem that tries to classify whether or not an email is spam. Email features such as; the sender of the email, number of typos, and frequent word occurrences like “free gift,” “offer,” “prize,” and so on, are extracted to produce a feature vector that is used to train a logistic classifier. The email then receives a score based on binary classification, and a programmed score threshold then determines if the email should be marked as spam.
Logistic Regression vs Other Technologies & Methodologies
Logistic regression vs multiple regression
Logistic regression is used to determine one dependent variable that can only have two outcomes, e.g. pass/fail, yes/no. Much like classification, it is best used in situations where the outcome is binary. The model can have one or more independent variables that it depends on. The model relies on these independent variables for a certain event to occur or not. If the model has three or more outcomes, it is called multinomial logistic regression.
Multiple linear regression models are used in situations where the user needs to find values of two or more dependent variables, the outcome of which can depend on any number of independent variables. The individual dependent variables can be binomial or multinomial, i.e. it could have just two outcomes or more than two.
Logistic regression vs decision tree
Logistic regression divides a given dataset into two, assuming the plane is 2-dimensional. Examples could be yes/no, black/white, etc. Decision tree, on the other hand, divides the space into smaller areas akin to branches coming out of the main trunk. Depending upon the distribution of data, logistic regression could be seen as limiting whereas a decision tree could seem overly fitting. Selection of one or the other should be made after having a good understanding of the spread of data.
Logistic regression vs classification
Logistic regression in statistics can be thought of as a regression technique that helps a data scientist derive the probability of an event, depending upon one or more independent variables. The output variable, for a binomial logistic regression as an example, is a binary value that is derived based on a pre-configured precision-recall curve.
Classification, on the other hand, is the task of predicting a discrete class label depending upon one or more discrete input variables. There is usually no precision-recall curve involved.
Logistic Regression in H2O AI
Open-source H2O-3 fits logistic regression with its generalized linear model (GLM) algorithm: the binomial family for two outcomes and the multinomial family for more than two. H2O-3 logistic regression documentation.