Table of Contents
Logistic Regression is a type of supervised learning algorithm used to predict the probability of a categorical outcome. It is a type of regression analysis used when the dependent variable (target) is categorical. It produces a probability score between 0 and 1. There are three types of logistic regression: binary, multinomial, and ordinal. Binary logistic regression is used to predict a binary outcome (e.g. yes/no, 0/1), multinomial logistic regression is used to predict more than two categorical outcomes, and ordinal logistic regression is used to predict an ordinal outcome (e.g. low, medium, high).
Logistic regression refers to any regression model in which the is categorical.
There are three types of logistic regression models:
- Binary logistic regression: The response variable can only belong to one of two categories.
- Multinomial logistic regression: The response variable can belong to one of three or more categories and there is no natural ordering among the categories.
- Ordinal logistic regression: The response variable can belong to one of three or more categories and there is a natural ordering among the categories.
The following table summarizes these differences:
This tutorial provides a brief explanation of each type of logistic regression model along with examples of each.
Type #1: Binary Logistic Regression
Binary logistic regression models are a type of logistic regression in which the response variable can only belong to two categories.
Here are a couple examples:
Example 1: NBA Draft
Suppose a sports data scientist wants to use the predictor variables (1) points, (2) rebounds, and (3) assists to predict the probability that a given college basketball player gets drafted into the NBA.
Since there are only two possible outcomes (drafted or not drafted) for the response variable, the data scientist would use a binomial logistic regression model.
Example 2: Spam Detection
Suppose a business wants to use the predictor variables (1) word count and (2) country of origin to predict the probability that a given email is spam.
Since there are only two possible outcomes (spam or not spam) for the response variable, the business would use a binomial logistic regression model.
Type #2: Multinomial Logistic Regression
Multinomial logistic regression models are a type of logistic regression in which the response variable can belong to one of three or more categories and there is no natural ordering among the categories.
Example 1: Political Preference
Suppose a political scientist wants to use the predictor variables (1) annual income and (2) years of education to predict the probability that an individual will vote for one of four different presidential candidates.
Since there are more than two possible outcomes (there are four potential candidates) for the response variable and there is no natural ordering among the outcomes, the political scientist would use a multinomial logistic regression model.
Example 2: Sports Preference
Suppose a sports analyst wants to use the predictor variables (1) TV hours viewed per week and (2) age to predict the probability that an individual will pick either basketball, football, or baseball as their preferred sport.
Since there are more than two possible outcomes (there are three sports) for the response variable, the sports analyst would use a multinomial logistic regression model.
Type #3: Ordinal Logistic Regression
Ordinal logistic regression models are a type of logistic regression in which the response variable can belong to one of three or more categories and there is a natural ordering among the categories.
Here are a couple examples:
Example 1: School Ratings
Suppose an academic advisor wants to use the predictor variables (1) GPA, (2) ACT score, and (3) SAT score to predict the probability that an individual will get into a university that can be categorized into “bad”, “mediocre”, “good”, or “great.”
Since there are more than two possible outcomes (there are four classifications of school quality) for the response variable and there is a natural ordering among the outcomes, the academic advisor would use an ordinal logistic regression model.
Example 2: Movie Ratings
Suppose a movie critic wants to use the predictor variables (1) total run time and (2) genre to predict the probability that a given movie will receiving a rating between 1 and 10.
Since there are more than two possible outcomes (there are 10 possible ratings) for the response variable and there is a natural ordering among the outcomes, the movie critic would use an ordinal logistic regression model.
The following tutorials provide more details on logistic regression models: