How to obtain the MLE for a Poisson Distribution?

Introduction to Maximum Likelihood Estimation

Maximum Likelihood Estimation (MLE) is a powerful statistical framework used to estimate the unknown parameters of a assumed probability distribution. The core idea is to find the parameter values that make the observed data most probable. If we assume our data $X = (x_1, x_2, dots, x_n)$ was generated by a specific probability distribution, the MLE method seeks the specific distribution parameters that maximize the likelihood of having generated that exact set of observations.

This method is highly favored in statistical practice due to its desirable asymptotic properties, such as consistency, asymptotic normality, and asymptotic efficiency (meaning the MLE often achieves the lowest possible variance among all estimators as the sample size increases). However, applying MLE requires careful formulation of the likelihood function, which is fundamentally the joint probability of observing all data points, treated as a function of the unknown parameters.

Before proceeding with the derivation, we must first confirm the assumptions underlying the data generation process. We assume that the observations $x_1, dots, x_n$ are independent and identically distributed (i.i.d.) random variables, all following the Poisson distribution with the same rate parameter $lambda$. This is the foundation upon which the entire likelihood calculation rests. If the i.i.d. assumption is violated, the resulting estimator may be biased or inconsistent.

This detailed tutorial provides a step-by-step mathematical derivation of the MLE for the parameter $lambda$ of the Poisson distribution, leading us to the famous and practical result that $hat{lambda}$ equals the sample mean, $bar{x}$.

Understanding the Poisson Distribution and Its Parameter

The Poisson distribution is a discrete probability distribution that expresses the probability of a given number of events occurring in a fixed interval of time or space if these events occur with a known constant mean rate and independently of the time since the last event. It is widely used to model counts, such as the number of website clicks per minute, the number of defects in a manufactured item, or the number of phone calls received by a call center per hour.

The distribution is entirely governed by a single parameter, $lambda$ (lambda), which must be strictly positive ($lambda > 0$). Crucially, $lambda$ is equal to both the expected value $E[X]$ and the variance $Var[X]$ of the distribution. This equality of mean and variance is a defining characteristic of the Poisson process and is key to validating whether real-world count data is truly Poisson distributed. If the sample mean and sample variance of observed data are significantly different, the Poisson model may be inappropriate.

The MLE process seeks to estimate this specific rate $lambda$. When we collect a sample of data, we are observing realizations of this underlying Poisson process. The objective is to reverse-engineer the process: what value of $lambda$ makes the specific sequence of observed counts we recorded most probable? The resulting estimate $hat{lambda}$ will represent our best guess of the true underlying mean rate.

The calculation involves five distinct steps, moving from the initial definition of the distribution to the final algebraic solution. Each step employs established mathematical techniques designed to isolate the parameter and find its maximizing value through differential calculus. This rigorous approach ensures that the resulting estimator is mathematically optimal under the defined likelihood criterion.

Step 1: Defining the Probability Mass Function (PMF)

The first step in any MLE derivation is to state the probability function for a single observation. Since the Poisson distribution deals with count data, we use the Probability Mass Function (PMF), $P(X=x)$, rather than the Probability Density Function (PDF).

The PMF defines the probability of observing exactly $x$ events, given the rate parameter $lambda$. For a single observation $x_i$, the PMF is written as:

Where:

• $P(X=x_i ; lambda)$ is the probability of observing $x_i$.
• $lambda$ is the mean rate of occurrence (the parameter we want to estimate).
• $e$ is Euler’s number (the base of the natural logarithm).
• $x_i!$ is the factorial of the observed count $x_i$.

This function provides the probability of a specific outcome $x_i$ for any given choice of $lambda$. It forms the basic building block for the overall likelihood function.

Step 2: Constructing the Joint Likelihood Function

Since we assume our $n$ observations ($x_1, x_2, dots, x_n$) are independent, the joint probability of observing the entire sample is the product of the individual probabilities defined by the PMF in Step 1. This joint probability, when viewed as a function of $lambda$, is called the Likelihood Function, denoted $L(lambda | x)$.

The likelihood function is constructed by multiplying the PMF for each of the $n$ observations:

$$L(lambda | x) = prod_{i=1}^{n} P(X=x_i ; lambda)$$

Substituting the Poisson PMF into the product notation yields the full likelihood function for the observed data set:

Maximizing this function directly can be mathematically complex due to the presence of exponentiations and products. The multiplication of $n$ terms, many of which contain exponents of $lambda$, suggests that algebraic simplification is necessary before attempting to find the maximum through derivative calculation. This leads directly to the introduction of the log-likelihood.

Step 3: Utilizing the Natural Log-Likelihood Function

To simplify the mathematical optimization process, we maximize the natural logarithm of the likelihood function, known as the Log-Likelihood Function, $l(lambda | x) = ln(L(lambda | x))$. Because the natural logarithm function is a strictly increasing monotonic transformation, maximizing $L(lambda | x)$ is equivalent to maximizing $l(lambda | x)$. The resulting MLE, $hat{lambda}$, will be identical.

The primary benefit of using the logarithm is that it converts the product ($prod$) in the likelihood function into a sum ($sum$) of terms, which is much easier to differentiate. Applying the properties of logarithms ($ln(ab) = ln(a) + ln(b)$ and $ln(a^b) = b ln(a)$), we transform the complex expression:

$$l(lambda | x) = ln left( prod_{i=1}^{n} frac{e^{-lambda} lambda^{x_i}}{x_i!} right)$$

This transformation expands into three main summations, allowing us to separate the terms involving the parameter $lambda$ from those that are constant with respect to $lambda$. The resulting natural log likelihood function is:

In this expanded form, we identify the terms relevant for optimization. Specifically, the third term ($sum_{i=1}^{n} ln(x_i!)$) is constant with respect to $lambda$ and will therefore vanish when we take the derivative. The first two terms, $ -nlambda$ and $(sum_{i=1}^{n} x_i) ln(lambda)$, are the ones we must differentiate to find the maximum.

Step 4: Calculating the Score Function (Derivative)

To find the maximum of the log-likelihood function, we must use calculus. The maximum occurs where the slope of the function is zero. We calculate the first derivative of the log-likelihood function with respect to the parameter $lambda$. This derivative is formally known as the Score Function.

The log-likelihood function we are differentiating is:

$$l(lambda | x) = -nlambda + left(sum_{i=1}^{n} x_iright) ln(lambda) – sum_{i=1}^{n} ln(x_i!)$$

We calculate the derivative $frac{partial l}{partial lambda}$ term by term:

1. The derivative of the first term, $-nlambda$, with respect to $lambda$ is $-n$.
2. The derivative of the second term, $left(sum_{i=1}^{n} x_iright) ln(lambda)$, is $left(sum_{i=1}^{n} x_iright) cdot frac{1}{lambda}$, since $sum x_i$ is treated as a constant factor.
3. The derivative of the final term, $-sum_{i=1}^{n} ln(x_i!)$, is 0, as it does not depend on $lambda$.

Combining these results, the Score Function is:

This simplified expression is essential because it is now linear in terms of the constant sum of observations ($sum x_i$) and the parameter $lambda$, making the final algebraic solution straightforward.

Step 5: Solving for the Maximum Likelihood Estimator ($hat{lambda}$)

The final step in the optimization process is to set the Score Function equal to zero and solve for $lambda$. The value of $lambda$ that satisfies this equation is the MLE, often denoted $hat{lambda}$ (lambda-hat), as it represents the estimated value.

Setting the derivative to zero:

$$frac{partial l}{partial lambda} = -n + frac{sum_{i=1}^{n} x_i}{lambda} = 0$$

We then isolate the term involving $lambda$. First, move the term $-n$ to the right side of the equation:

Finally, we multiply both sides by $lambda$ and divide by $n$ to solve for $hat{lambda}$:

$$hat{lambda} = frac{sum_{i=1}^{n} x_i}{n}$$

Conclusion: The Sample Mean as the Optimal Estimator

The result of the Maximum Likelihood Estimation process for the Poisson distribution is that the optimal estimator $hat{lambda}$ is mathematically equivalent to the sample mean ($bar{x}$) of the $n$ observations in the sample. This is an extremely intuitive and convenient result for practitioners, as calculating the MLE requires only the most basic statistical operation—averaging the observations.

The MLE for the Poisson parameter $lambda$ being the sample mean confirms its consistency with the defining property of the distribution, where $lambda$ represents the theoretical mean $E[X]$. Thus, when applying the Poisson model to real-world data, the best estimate for the rate parameter is simply the average number of events observed in the sample. This simplicity contributes significantly to the Poisson distribution’s practical utility across fields like quality control, epidemiology, and finance.

Furthermore, because the sample mean possesses desirable statistical properties (it is unbiased and consistent), the derived MLE $hat{lambda}$ inherits these strengths. This rigorous mathematical proof validates the use of the sample mean as the statistically optimal estimator for the rate parameter under the maximum likelihood criterion.

An Introduction to the Poisson Distribution
Poisson Distribution Calculator
How to Use the Poisson Distribution in Excel