Seven ways to remember the difference between Type 1 and Type 2 errors in hypothesis testing

It’s one thing to understand the difference between Type 1 and Type 2 errors. And another to remember the difference between Type 1 and Type 2 errors!  If the man who put a rocket in space finds this challenging, how do you expect students to find this easy!?

Elon Musk’s tweet: Is this a type 1 or Type 2 error?

If you teach statistics, you already know this! Students from all walks of life, MBAs, medicine, arts, STEM, etc., are troubled when asked to differentiate between Type 1 and Type 2 errors! And don’t get me started on type 3 or type 4 errors! Here are Tamsin and Caitlin on Twitter ranting about Type 1 and Type 2 errors!

Tamsin, @anorakgirl, says “Statistics people! How am I supposed to remember anything for my exam when your naming is SO bad? Type 1 and Type 2 errors are just the last straw…”

Caitlin, @Saiki_36, “I’ve read 2 chapters and watched the videos on type 1 and 2 errors for stats and retained nothing. Goodbye brain.”

In fact, @stianwestlake, CEO of nonother than “Royal Statistical Society,” concludes.

To help, we address on this page:

What are Type 1 and Type 2 Errors?

Before we try and remember Type 1 and Type 2 errors, it is good first to understand the concept.

In statistics (and in life), we often start with a null hypothesis (a belief we hold) and test it using data (or evidence). Then, we evaluate the sample data (or evidence gathered) to conclude if we continue with the null hypothesis (or change belief we hold). It is always possible we make the wrong conclusion because the data is a sample only, or evidence may be insufficient. There are two ways of making this error. The two ways were named Type 1 error and Type 2 error.

  • A type I error occurs when we reject a null hypothesis that is actually true in the population.
    • This is also referred to as a false-positive.
    • Probability of a type I error is measured by Alpha.
    • This is also referred to as the level of significance.
  • A type II error is when we fail to reject a null hypothesis that is actually false in the population.
    • This is also referred to as false-negative.
    • Measured by Beta.
Type 1 & Type 2 Matrix Source: @tryextro

Students usually ask why we do not drop the type 1 and type 2 error terminology and call it a false positive and false negative! Technically, we use the type 1 and type 2 error terminology for hypothesis testing in statistics. Clearly, there could have been better ways to name these errors!

If you are looking for a primer on hypothesis testing, you can read the Industrial Psychiatry Journal has one here or read our step by step guide on hypothesis testing here.

Trade off between Type 1 and Type 2 errors

We want to ideally reduce both types of errors. But remember that there is always a trade-off between Type 1 and Type 2 errors (for a given type of test and fixed sample size). When you try and reduce Type 1 errors, you increase the odds of Type 2 errors. Therefore, controlling for Type 1 and Type 2 errors is based on the situation you are in.

Statistically speaking a reducing α risk (type 1) makes it harder to reject the null hypothesis. This increases the β risk (type 2). Both α and β can be reduced simultaneously only by increasing the sample size.

@ProfJessieSmith points to The Texas Tribune article on bail reform to illustrate the trade-off between type 1 and type 2 errors.

Another good example for to show that controlling for Type 1 vs. Type 2 errors is based on the situation you are in is the example of screws. Assume you want to set the quality cut-off for boxes of screws. If the screws are used in airplanes, you want to minimize Type 2 errors as the cost of an error is an airplane failure – lethal and expensive. On the other hand, if the screws are used in a wooden table, you want to minimize Type 1 errors as the cost of an error is minimal.

PS: Please run if anyone mentions type 3 or type 4 errors! Don’t even reply!

Minimizing Errors: Type 1 and Type 2 Errors

Errors can be expensive. The cost of Type I errors are usually borne by an individual. However, the costs of Type II errors are usually borne by society. How can we minimize Type 1 and Type 2 errors?

  • We can minimize the chance of type 1 by setting a smaller alpha or level of significance.
  • We can minimize the chance of both type 1 and type 2 errors by collecting as much sample evidence (increasing sample size) as possible as well has using proper sampling and testing procedures.

Easy Ways to Remember Type 1 and Type 2 Errors

We have searched the internet and submit different ways that people have tried to remember the difference between Type 1 and Type 2 errors.

1) The Wolf Story to remember the difference between Type 1 and Type 2 errors

This is the most popular method to remember the difference between Type 1 and Type 2 errors.

Everyone probably knows the story of the boy who cried wolf! You only need to remember that the villagers committed both Type 1 and Type 2 errors in that order.

Type I vs. Type II Errors
  • When the boy cries wolf the 1st time, the villagers believe it to be true and rush to the scene but there’s no wolf. This is a false positive or Type 1 error.
  • Then the boy cries wolf the 2nd time when a wolf is encountered and the villagers ignore or don’t believe there wolf.  This is a false negative or Type 2 error.

2) Pregnancy & the difference between Type 1 and Type 2 errors

The close second is the pregnancy image doing the rounds on the internet.

3) The Rain and Umbrella & the difference between Type 1 and Type 2 errors

4) The DIY Calligrapher & the difference between Type 1 and Type 2 errors

If you are a calligrapher and want to make your own image -something like this is good.

DIY Drawing to remember Type 1 and Type 2 Errors – Source: @psysocatsheldon

5) Creating Your Own Memory Aid

@manishaaaaxo tried the approach here! 

  • Type 1… 
  • another word for 1 is First
  • First begins with the letter F
  • F = false (actual) hypothesis which was accepted Then you know that for the Null Hypothesis it’s just the opposite i.e. it was true but it was rejected
  • Therefore, you just need to remember “First False” and remember that the false is referring to the ACTUAL hypothesis
  • Type 2… think “too true” = 2 true The (actual) hypothesis is TRUE but it was rejected. Again the Null would just be the opposite- it was false but accepted
  • So if you remember either First False or Too True you’ll be fine bc you just know that it’d be the opposite

6) Number of Falses’ Method

Here is another memory aid from Professor Kelsey-Jo.

Source: Prof’s Kelsey’s

7) The Tradition Matrix Showing Type 1 and Type 2 Errors

And of course, you have the traditional method to understand and remember the difference between Type 1 and Type 2 errors in Hypothesis testing. Sometimes, the traditional way works best!

Type 1 & Type 2 Matrix Source: @tryextro

@MoeMKiss and @MarqJacobs prefer this matrix. Note the change in the x-axis and y-axis.

You now have at least seven ways to remember and distinguish between Type 1 and Type 2 errors. Pick whatever suits you best

Examples of Misunderstanding Type 1 and Type 2 Errors

Elon Musk’s response to his COVID-19 Tests

Cant Call Type 1 & 2 errors “positively toward negative, right”

Unless you are a sitting president. Watch Trump’s Response on His COVID-19 Tests

Trump on his latest coronavirus test: “I tested very positively in another sense, so, this morning, yeah. I tested positively toward negative, right? So, no. I tested perfectly this morning, meaning I tested negative. But that’s a way of saying it. Positively toward the negative”

On Fox News

Another Trump response on his colleague’s COVID-19 test results.

Trump: Katie, she tested very good for a long period of time and then all of the sudden today she tested positive… This is why the whole concept of tests aren’t necessarily great

Covid-19 Response

It is not just Donald Trump; the White House response was also unclear on potential Type I and Type II errors!

@MattGertz, Senior Fellow at @mmfa (and one who takes care to clarify that he is definitely not the GOP congressman) said on Twitter, “Dawning realization that nine months into the pandemic. senior WH officials think that if you test negative, it means that you definitely have not caught the virus and can do whatever you want and don’t need to quarantine if you’ve been in contact with someone with COVID.”

Enjoy learning! Let us know if we can help you with tutoring, and one of our statistics tutors will be glad to assist you.

Sign Up Now