How to Determine if Your Project Delivers Value

Establishing Assumptions through Statistical Hypotheses

A statistical hypothesis is an initial assumption or conjecture regarding a characteristic of a large group, known as the population parameter. This assumption forms the basis for structured statistical testing.

For example, we may assume that the mean height of a male in the U.S. is 70 inches. This specific assumption provides a testable claim about the population characteristic.

The assumption about the height is the statistical hypothesis under investigation, and the true mean height of a male in the U.S. that we are trying to infer or measure is the true population parameter.

A hypothesis test is a formal statistical procedure rigorously applied to sample data to gather evidence required to either reject or fail to reject a statistical hypothesis.

The Two Fundamental Types of Statistical Hypotheses

To test the validity of a statistical hypothesis about a population parameter, analysts obtain a representative sample from the population and subsequently perform a detailed hypothesis test on the collected sample data.

There are two necessary and mutually exclusive types of statistical hypotheses in every formal test:

• The null hypothesis, denoted as H₀, is the hypothesis that the sample data occurs purely from random chance. It is the statement of "no effect" or "no difference" that the researcher attempts to find evidence against.
• The alternative hypothesis, denoted as H₁ or H_a, is the contrasting hypothesis that asserts the sample data is influenced by some genuine, non-random cause or underlying effect. This hypothesis is accepted if the evidence strongly leads to the rejection of H₀.

The Five Essential Steps of Hypothesis Testing

A rigorous hypothesis test methodology consists of five well-defined steps designed to guide the statistical decision-making process:

1. State the hypotheses.

   Precisely state the null and alternative hypotheses. These two statements must be logically designed to be mutually exclusive, meaning that the truth of one automatically implies the falsehood of the other.

2. Determine a significance level (α) to use for the hypothesis.

   Decide on a significance level, symbolized by alpha (α). This value dictates the threshold for statistical significance and represents the maximum allowable risk of making a Type I error. Common choices for α are .01, .05, and .1.

3. Find the test statistic and p-value.

   Calculate the appropriate test statistic based on the sample data. This statistic quantifies how far the sample result lies from the hypothesized population parameter. The general formula to find the test statistic is often simplified as: (sample statistic – population parameter) / (standard deviation of statistic).

4. Make the decision.

   Using the test statistic or the derived p-value, determine if there is sufficient evidence to reject or if you must fail to reject the null hypothesis based on the established significance level (α).

5. Interpret the results.

   The final step involves translating the statistical conclusion back into the real-world context of the original project or research question, communicating the implications clearly to stakeholders.

Decision Criteria: The P-Value and Significance

The p-value is a measure of the strength of evidence supporting a null hypothesis. Technically, the p-value represents the probability of observing sample data as extreme as, or more extreme than, the current data, assuming the null hypothesis is entirely true.

The definitive rule for decision-making is: If the p-value is less than the chosen significance level (α), we conclude that the data is highly unlikely under H₀ and therefore we must reject the null hypothesis.

Understanding Potential Decision Errors (Type I and Type II)

When conducting a hypothesis test, there are two types of decision errors related to the true state of the population that one must acknowledge and manage:

• Type I error: You mistakenly reject the null hypothesis when it is actually true (a false positive). The probability of committing a Type I error is precisely equal to the significance level, often called alpha, and denoted as α.
• Type II error: You fail to reject the null hypothesis when it is actually false (a false negative). The probability of committing a Type II error is called beta, denoted as β, and the power of the test is defined as (1 – β).

Directional vs. Non-Directional Tests

Statistical hypotheses can be categorized as either one-tailed (directional) or two-tailed (non-directional), depending on the claim made by the alternative hypothesis.

A one-tailed hypothesis involves making a specific directional claim, such as a "greater than" or "less than" statement. The rejection region is focused entirely on one end of the distribution.

For example, if we hypothesize the mean height of a male in the U.S. is greater than or equal to 70 inches, the null hypothesis would be H0: µ ≥ 70 inches and the alternative hypothesis would be Ha: µ < 70 inches.

A two-tailed hypothesis involves making a non-directional claim, such as an "equal to" or "not equal to" statement. This test is used when a deviation in either direction from the hypothesized value is considered significant.

For instance, suppose we assume the mean height of a male in the U.S. is equal to 70 inches. The competing hypotheses would be H0: µ = 70 inches and the alternative hypothesis would be Ha: µ ≠ 70 inches.

Note: The "equal" sign (whether it is =, ≥, or ≤) is always included in the null hypothesis (H₀) formulation.

Selecting the Right Type of Hypothesis Test

There are many different types of statistical hypothesis tests available, and the selection depends fundamentally on the type of data you are analyzing (e.g., continuous, discrete, categorical) and the specific analytical goal.

The choice of test must align with the nature of the population parameter being examined and the characteristics of the sample distribution. Misapplying a test can lead to invalid statistical conclusions, impacting the perceived value of the project.

The following tutorials provide a deeper explanation of the most common types of hypothesis tests used in contemporary data analysis: