top of page

Hypothesis Testing

REFLECTION: FOR STUDENTS: Always remember to use data and analysis to make decisions. Rely upon your own critical thinking skills, and do not allow your team to be derailed by groupthink.

FOR ACADEMICS: Teach critical thinking skills first, not expected behaviors. Without each individual able to independently and confidently voice objections to the group view, there can be no growth.

FOR PROFESSIONALS/PRACTITIONERS: Choosing the right hypothesis test can be daunting. It's always best to understand, but if you are using statistical software like Minitab, never hesitate to use the software to choose your path based on the data and then double-check with a statistician to be certain you are taking the correct path.

Basic Terminology

H0: is a test of statistical significance called the null hypothesis. The test of significance is designed to assess the strength of the evidence against the null hypothesis. Usually, the null hypothesis is a statement of “no effect” or “no difference” symbolized as H0. (The one we hope to disprove and usually commonly accepted.)

Ha: This symbol represents the alternative hypothesis- the one for which we want to develop supporting evidence and prove and should usually be the opposite (inverse) of the null hypothesis.

α-value: alpha level or “significance level”- By definition, the alpha level is the probability of rejecting the null hypothesis when the null hypothesis is correct. Translation: It’s the probability of making a wrong decision.

Confidence Interval (also CI): CI provides the boundaries for an unknown parameter of a population with a specified degree of confidence that the parameter falls within the interval. CI is equal to 1-α, and the typical levels of confidence used to test a hypothesis are 0.99, 0.95, and 0.90.

Parameter: summary description of a fixed characteristic or measure of the target population. A Parameter denotes the actual value that would be obtained if a census rather than a sample were undertaken.

[Ex: Mean (μ), Variance (σ²), Standard Deviation (σ), Proportion (π)]

Population: Population is a collection of objects that we want to study/test. The collection of objects could be Cities, Students, Factories, Parts, etc. It depends on the study at hand.

It isn’t effortless to get complete information about a population in the real world. Therefore, we draw a sample out of that population and derive the same statistical measures mentioned above. These measures are called Sample Statistics.

Statistic- a summary description of a characteristic or measure of the sample. The Sample Statistic is used as an estimate of the population parameter.

[Ex: Sample Mean (x̄), Sample Variance (S²), Sample Standard Deviation (S), Sample Proportion (p)]

p-value: Probability of obtaining a result as extreme as, or more extreme than, the result obtained when the null hypothesis is correct- Ranges from 0 to 1 (obtained as a result of several different types of hypothesis tests) (Kubiak, 2017) (Crossley, 2008)(Minitab Editor, 2012)


Fundamentally, Hypothesis testing is a test of significance and tests whether events occur by chance or not. Statistically, a sample is drawn from a population, and a statistic is computed from that sample. If that statistic is a mean, the hypothesis tests whether the mean occurred by chance at some specified level of significance. There are many different testing methods available based on the data available. Still, there is always a chance that even with a flawless analysis of a sample, the conclusion will yield a false result relative to the population. There are two types of errors that can arise when testing a hypothesis-

Type I Error: Occurs when we reject the null hypothesis that is true (probability of Type I Error is equal to α).

Type II Error: Occurs when we fail to reject a false null hypothesis (probability of Type II Error is equal to 1-α).

H0 is "true" but rejected: Type I or α error

H0 is "false but not rejected Type II or ß error

Interpreting Hypothesis Test Statistics

Confidence level + alpha = 1

As you increase alpha, you both increase the probability of incorrectly rejecting the null hypothesis and decrease your confidence level.

If the p-value is low, the null must go.

If the p-value is below the alpha—the risk you’re willing to take of making a wrong decision—then you reject the null hypothesis "if the p-value is low, the null must go." If the p-value exceeds alpha, we fail to reject the null hypothesis. Another way to remember it is, “if the p-value is high, the null will fly.”

The confidence interval and p-value will always lead you to the same conclusion-

If the p-value is less than alpha (it is significant), then the confidence interval will NOT contain the hypothesized mean/variance; however, if the p-value is greater than alpha (it is not significant), then the confidence interval will include the hypothesized mean/variance. (Kubiak, 2017) (Crossley, 2008)(Minitab Editor, 2012)

Deciding upon the correct test method:

It is always best to understand the potentially daunting task of hypothesis testing, and sometimes critical, cut never fear. Most modern statistical software (even many Excel add-ons) will help guide you down the proper path as long as you have the data, know what kind of data you have, and have determined if it is normal or non-normal.


As promised, this was not a deep dive. The more you know about statistics, the more likely you will draw the correct conclusion when you evaluate your test statistics against your hypothesis. It is critical to remember that while you are doing mathematical gymnastics or navigating Minitab, the hypotheses are not really about the data; instead, you should think about the processes producing the data. Always understand the implications of the hypothesis test on the associated process(es) in order to take the correct actions.


Crossley, M. L. (2008). The Desk Reference of Statistical Quality Methods (2nd Ed). Milwaukee, WI: ASQ Quality Press.

CSSBB Primer. (2014). West Terre Haute, Indiana: Quality Council of Indiana.

Kubiak, T. a. (2017). The Certified Six Sigma Black Belt Handbook Third Edition. Milwaukee: ASQ Quality Press.

Minitab Editor. (2012, October 01). Retrieved from Minitab Blog:


IASSC Certified Lean Six Sigma Training at the Yellow Belt, Green Belt, and Black Belt levels, $185.00

51 views0 comments


Post: Blog2_Post
bottom of page