background-image: url("img/DAW.png") background-position: left background-size: 50% class: middle, center, .pull-right[ ## .base-blue[Theory-Based Inference] <br> ### .purple[Kelly McConville] #### .purple[ Stat 100 | Week 12 | Fall 2022] ] --- ### Announcements * P-Set 9 (the LAST p-set) is due Th, Dec 1st! * Project Assignment 3 is due Tu, Dec 6th. * There will be a **surprise** for those in lecture on Monday... **************************** -- ### Goals for Today .pull-left[ * Theory-Based Inference ] .pull-right[ * Learn about a new group of test statistics based on **z-scores** * Cover theory-based confidence intervals ] --- ### Recap * Sample statistics ARE random variables! -- * We can often **approximate** the probability function (i.e. sampling distribution) of a statistic with the probability function of a named random variable (i.e., Normal, t, ...). -- * `\(\hat{p}\)` = sample proportion of correct receiver guesses out of 329 trials * Null Distribution and the probability function of a N(0.25, 0.024): .pull-left[ <img src="stat100_wk12wed_files/figure-html/unnamed-chunk-1-1.png" width="576" style="display: block; margin: auto;" /> ] -- .pull-right[ * The CLT helps us find a good approximate distribution based on the probability function of a random variable. * The CLT does require the sample size to be large and the definition of large depends on the sample statistic. ] --- ### CLT in General For random samples and a large sample size `\((n)\)`, the sampling distribution of many sample statistics is approximately normal. -- ### Approximating the Sampling Distribution of a Sample Mean * Here's the version of the CLT for the sample mean. * For the sample mean, the CLT says: For large `\(n\)` (At least 30 observations), $$ \bar{x} \sim N \left(\mu, \frac{\sigma}{\sqrt{n}} \right) $$ --- ### Z-score Test Statistics * All of our test statistics so far have been sample statistics. -- * Another commonly used test statistic takes the form of a **z-score**: $$ \mbox{Z-score} = \frac{X - \mu}{\sigma} $$ * Standardized version of the sample statistic. -- * Z-score measures how many **standard deviations** the sample statistic is away from its **mean**. -- * Allows us to quickly (but roughly) classify results as unusual or not. + `\(|\)` Z-score `\(|\)` > 2 → results are unusual/p-value will be smallish -- * Commonly used because if `\(X \sim N(\mu, \sigma)\)`, then $$ \mbox{Z-score} = \frac{X - \mu}{\sigma} \sim N(0, 1) $$ --- class: middle, center ## Let's consider theory-based inference for a population proportion. --- background-image: url("img/hyp_testing_diagram.png") background-position: contain background-size: 80% ### Statistical Inference Zoom Out -- Testing --- background-image: url("img/ci_diagram.png") background-position: contain background-size: 70% ### Statistical Inference Zoom Out -- Estimation --- ### Inference for a Single Proportion -- Testing Let's consider conducting a hypothesis test for a single proportion: `\(p\)` -- Need: * Hypotheses + Same as with the simulation-based methods -- * Test statistic and its null distribution + Use a z-score test statistic and a standard normal distribution -- * P-value + Compute from the standard normal distribution directly --- ### Inference for a Single Proportion -- Testing Let's consider conducting a hypothesis test for a single proportion: `\(p\)` -- `\(H_o: p = p_o\)` where `\(p_o\)` = null value -- `\(H_a: p > p_o\)` or `\(H_a: p < p_o\)` or `\(H_a: p \neq p_o\)` -- By the CLT, under `\(H_o\)`: $$ \hat{p} \sim N \left(p_o, \sqrt{\frac{p_o(1-p_o)}{n}} \right) $$ -- * Z-score test statistic: $$ Z = \frac{\hat{p} - p_o}{\sqrt{\frac{p_o(1-p_o)}{n}}} $$ -- * Use `\(N(0, 1)\)` to find the p-value once you have computed the test statistic. --- ### Inference for a Single Proportion -- Testing Let's consider conducting a hypothesis test for a single proportion: `\(p\)` **Example**: Bern and Honorton's (1994) extrasensory perception (ESP) studies ```r # Construct data frame of sample results esp <- data.frame(guess = c(rep("correct", 106), rep("incorrect", 329 - 106))) ``` --- ### Inference for a Single Proportion -- Testing Let's consider conducting a hypothesis test for a single proportion: `\(p\)` **Example**: Bern and Honorton's (1994) extrasensory perception (ESP) studies .pull-left[ ```r library(infer) # Compute observed test statistic test_stat <- esp %>% specify(response = guess, success = "correct") %>% hypothesize(null = "point", p = 0.25) %>% calculate(stat ="z") test_stat ``` ``` ## Response: guess (factor) ## Null Hypothesis: point ## # A tibble: 1 × 1 ## stat ## <dbl> ## 1 3.02 ``` ] -- .pull-right[ ```r # Use N(0,1) to find p-value pnorm(q = test_stat$stat, mean = 0, sd = 1, lower.tail = FALSE) ``` ``` ## [1] 0.001247763 ``` ```r # Or 1 - pnorm(q = test_stat$stat, mean = 0, sd = 1) ``` ``` ## [1] 0.001247763 ``` ] --- **Example**: Bern and Honorton's (1994) extrasensory perception (ESP) studies ```r # Use probability model to approximate null distribution prop.test(x = 106, n = 329, p = 0.25, alternative = "greater", correct = FALSE) ``` ``` ## ## 1-sample proportions test without continuity correction ## ## data: 106 out of 329, null probability 0.25 ## X-squared = 9.1439, df = 1, p-value = 0.001248 ## alternative hypothesis: true p is greater than 0.25 ## 95 percent confidence interval: ## 0.2814092 1.0000000 ## sample estimates: ## p ## 0.3221884 ``` ```r # This R function does first square the z-score test statistic sqrt(9.1439) ``` ``` ## [1] 3.023888 ``` --- ### Theory-Based Confidence Intervals Suppose statistic `\(\sim N(\mu = \mbox{parameter}, \sigma = SE)\)`. -- #### 95% CI for parameter: $$ \mbox{statistic} \pm 2 SE $$ -- Can generalize this formula! -- #### P% CI for parameter: $$ \mbox{statistic} \pm z^* SE $$ ```r # Find z-star qnorm(p = 0.975, mean = 0, sd = 1) ``` ``` ## [1] 1.959964 ``` ```r qnorm(p = 0.95, mean = 0, sd = 1) ``` ``` ## [1] 1.644854 ``` --- ### Theory-Based CIs in Action Let's consider constructing a confidence interval for a single proportion: `\(p\)` -- By the CLT, $$ \hat{p} \sim N \left(p, \sqrt{\frac{p(1-p)}{n}} \right) $$ -- #### P% CI for parameter: `\begin{align*} \mbox{statistic} \pm z^* SE \end{align*}` --- ### Theory-Based CIs in Action Let's consider constructing a confidence interval for a single proportion: `\(p\)` **Example**: Bern and Honorton's (1994) extrasensory perception (ESP) studies ```r # Use probability model to approximate null distribution prop.test(x = 106, n = 329, p = 0.25, alternative = "two.sided", conf.level = 0.95) ``` ``` ## ## 1-sample proportions test with continuity correction ## ## data: 106 out of 329, null probability 0.25 ## X-squared = 8.7629, df = 1, p-value = 0.003074 ## alternative hypothesis: true p is not equal to 0.25 ## 95 percent confidence interval: ## 0.2725541 0.3760499 ## sample estimates: ## p ## 0.3221884 ``` --- ### Theory-Based CIs #### P% CI for parameter: $$ \mbox{statistic} \pm z^* SE $$ Notes: -- * Didn't construct the bootstrap distribution. -- * Need to check that `\(n\)` is large and that the sample is random/representative. -- * Interpretation of the CI doesn't change. -- * For some parameters, the critical value comes from a `\(t\)` distribution. -- * Now we have a formula for the Margin of Error. + That will prove useful for **sample size calculations**. --- class: , middle, center ## Now let's explore how to do inference for a single mean. --- ### Inference for a Single Mean **Example:** *Are lakes in Florida more acidic or alkaline?* The pH of a liquid is the measure of its acidity or alkalinity where pure water has a pH of 7, a pH greater than 7 is alkaline and a pH less than 7 is acidic. The following dataset contains observations on 53 lakes in Florida. Use these data to answer our question. ```r library(tidyverse) FloridaLakes <- read_csv("https://www.lock5stat.com/datasets1e/FloridaLakes.csv") ``` * **Cases**: <br> * **Variable of interest**: <br> * **Parameter of interest:** <br> * **Hypotheses:** <br> --- ### Inference for a Single Mean Let's consider conducting a hypothesis test for a single mean: `\(\mu\)` -- Need: * Hypotheses + Same as with the simulation-based methods -- * Test statistic and its null distribution + Use a z-score test statistic and a t distribution -- * P-value + Compute from the t distribution directly --- ### Inference for a Single Mean Let's consider conducting a hypothesis test for a single mean: `\(\mu\)` -- `\(H_o: \mu = \mu_o\)` where `\(\mu_o\)` = null value -- `\(H_a: \mu > \mu_o\)` or `\(H_a: \mu < \mu_o\)` or `\(H_a: \mu \neq \mu_o\)` -- By the CLT, under `\(H_o\)`: $$ \bar{x} \sim N \left(\mu_o, \frac{\sigma}{\sqrt{n}} \right) $$ -- * Z-score test statistic: $$ Z = \frac{\bar{x} - \mu_o}{\frac{\sigma}{\sqrt{n}}} $$ -- * **Problem:** Don't know `\(\sigma\)`: the population standard deviation of our response variable! --- ### Inference for a Single Mean Let's consider conducting a hypothesis test for a single mean: `\(\mu\)` `\(H_o: \mu = \mu_o\)` where `\(\mu_o\)` = null value `\(H_a: \mu > \mu_o\)` or `\(H_a: \mu < \mu_o\)` or `\(H_a: \mu \neq \mu_o\)` By the CLT, under `\(H_o\)`: $$ \bar{x} \sim N \left(\mu_o, \frac{\sigma}{\sqrt{n}} \right) $$ * Z-score test statistic: $$ t = \frac{\bar{x} - \mu_o}{\frac{s}{\sqrt{n}}} $$ * **Problem:** Don't know `\(\sigma\)`: the population standard deviation of our response variable! * **Solution:** Plug in `\(s\)`: the sample standard deviation of our response variable! * Use `\(t(\mbox{df} = n - 1)\)` to find the p-value --- ### Inference for a Single Mean ```r library(infer) #Compute obs stat t_obs <- FloridaLakes %>% specify(response = pH) %>% hypothesize(null = "point", mu = 7) %>% calculate(stat = "t") t_obs ``` ``` ## Response: pH (numeric) ## Null Hypothesis: point ## # A tibble: 1 × 1 ## stat ## <dbl> ## 1 -2.31 ``` -- ```r # Generate null distribution null_dist <- FloridaLakes %>% specify(response = pH) %>% hypothesize(null = "point", mu = 7) %>% generate(reps = 1000, type = "bootstrap") %>% calculate(stat = "t") ``` --- ```r # Graph the null distribution null_dist %>% visualize(bins = 30) + geom_vline(xintercept = t_obs$stat, color = "deeppink", size = 2) + geom_vline(xintercept = abs(t_obs$stat), color = "deeppink", size = 2) ``` <img src="stat100_wk12wed_files/figure-html/unnamed-chunk-11-1.png" width="576" style="display: block; margin: auto;" /> --- ### Inference for a Single Mean What probability function is a good approximation to the null distribution? <img src="stat100_wk12wed_files/figure-html/unnamed-chunk-12-1.png" width="576" style="display: block; margin: auto;" /> --- ### Inference for a Single Mean What probability function is a good approximation to the null distribution? <img src="stat100_wk12wed_files/figure-html/unnamed-chunk-13-1.png" width="576" style="display: block; margin: auto;" /> --- .pull-left[ #### P-value using the generated null distribution: ```r pvalue <- null_dist %>% get_p_value(obs_stat = t_obs, direction = "both") pvalue ``` ``` ## # A tibble: 1 × 1 ## p_value ## <dbl> ## 1 0.022 ``` ] -- .pull-right[ #### P-value using an approximate probability function: ```r # Using t distribution pt(q = t_obs$stat, df = 52)*2 ``` ``` ## t ## 0.02468707 ``` ] -- #### Do-it-all function: ```r t.test(FloridaLakes$pH, mu = 7, conf.level = .90, alternative = "two.sided") ``` ``` ## ## One Sample t-test ## ## data: FloridaLakes$pH ## t = -2.3134, df = 52, p-value = 0.02469 ## alternative hypothesis: true mean is not equal to 7 ## 90 percent confidence interval: ## 6.294176 6.886956 ## sample estimates: ## mean of x ## 6.590566 ``` --- ### Statistical Inference using Probability Models * We went through theory-based inference for `\(p\)` and for `\(\mu\)`. -- * There are similar results for other parameters. But the specific named RV and its probability function change! -- * Can find some common test statistics, confidence interval formulae, and useful R functions [on the course website](https://mcconvil.github.io/stat100f22/inference_procedures.html). + In our last p-set, will explore inference for the correlation coefficient.