Types of Power of a Hypothesis Test AP Statistics - Chapter 21
We make decisions based on a probability but what if we re WRONG?!?
When we perform a hypothesis test: In real life... In our hypothesis... Reject Fail to reject true Type I α false Type II β
When we perform a hypothesis test: In our hypothesis... Reject false Fail negative to reject In real life... In medical testing, we call this a false true positive false Type I α Type II β
Consider a test for a serious disease... What are the hypotheses? Type I? Consequence? Ho: the person is healthy Ha: the person is NOT healthy/has the disease We decide the person has the disease when in reality, they re healthy. We scare the person for no reason. They unnecessarily go in for further testing or treatment..
Consider a test for a serious disease... What are the hypotheses? Type II? Ho: the person is healthy Ha: the person is NOT healthy/has the disease We say there s not enough evidence to conclude the person has the disease (in other words, we think they re healthy), when in reality, they actually do have it. Consequence? The person fails to get treatment for a disease they have..
Lay s chip company tests a sample of potatoes from a truckload for E-coli to determine whether or not to accept the truckload. Type I? Ho: the potatoes are good Ha: the potatoes have E-coli We decide the potatoes have E-coli, when they really don t. Type II? We decide the potatoes are good, when they actually have E-coli. Which error is worse? Which error is more concerning? What if you re the potato farmer? What if you re the CEO of Lay s chip company? What if you re a person buying potato chips?
When we perform a hypothesis test: If we decrease P(Type I error), the probability of Type II error increases by default. In our hypothesis... Statisticians have to decide which error they want to avoid more, knowing that decreasing Reject one will increase the other! Fail to reject Type I In real life... true α false Type II β
When we perform a hypothesis test: In real life... In our hypothesis... Reject And it works the other way around as well: if we decrease P(Type II error), the probability of Type I error Fail to reject increases by default. true Type I α false Type II β
A school district is considering purchasing laptops for all of high school students, in hopes that using the devices will improve achievement on end-of-year exams. What are the hypotheses? Is this a one-tailed or two-tailed test? Ho: student achievement stays the same/does not improve Ha: student achievement improves One-tailed. The district wants to show an improvement in achievement. (They re not going to test for just a change in achievement, because they wouldn t buy laptops if achievement got worse!)
A school district is considering purchasing laptops for all of high school students, in hopes that using the devices will improve achievement on end-of-year exams. What are the hypotheses? Type I? The district decides that laptops do improve student achievement, when they actually don t. Consequence? District spends $$$ on laptops Is this a one-tailed or two-tailed test? Type II? The district decides that laptops don t change student achievement, when they actually do. Consequence? Students don t get access to Ho: student achievement stays the same/does not improve Ha: student achievement improves One-tailed. The district wants to show an improvement in achievement. (They re that aren t not going actually to test helping for just a change in achievement, laptops that because would they actually wouldn t buy laptops if achievement got worse!) anything. help them be more successful.
Ho: the defendant is innocent Ha: the defendant is guilty (let s think about a criminal trial) Let s pretend we have a defendant that we K N O W is guilty of a crime. What do we NEED in order to convict the criminal? (in other words, to reject the null hypothesis?) STRONG evidence! Without it, we risk committing a Type II error.
Or what if I ve developed a new medication that I KNOW is better than the previous one? What do I need if I want to prove that the new medication is better? STRONG evidence!
Or what if I ve developed a new medication that I KNOW is better than the previous one? STRONG evidence! In hypothesis testing, that What do I need if I want to prove that the new medication is better? evidence we need is called POWER.
POWER the probability of rejecting Ho, when Ho is false. (or, the probability of concluding Ha is true, when Ha IS true!) POWER = 1 - β
When we perform a hypothesis test: In real life... In our hypothesis... POWER! Reject Fail to reject true Type I α false Type II β
When we have a Ha that we KNOW is true, but still need the hypothesis test to PROVE it s true - we need POWER to be as big as it can possibly be.
Here are 3 ways to increase power: 1. Increase α - Increasing alpha lowers P(Type II) or β, thus increasing power. But this will also increase P(Type I), which may not be ideal. 2. Increase n - Increasing sample size will lower both P(Type I) & P(Type II), and increase power. But taking a bigger sample size is not always possible or realistic. 3. Increase effect size - This means make the new thing REALLY better, by a lot. This would lower P(Type I) and P(Type II) and increase power. It s just not something statisticians usually have any control over.