Hypothesis Testing




Kelly McConville

Stat 100
Week 9 | Fall 2023

Announcements

  • Don’t forget that the midterm exam rewrites are due on Thursday at 5pm on Gradescope.
    • Make sure to use the Quarto doc in the Midterm Exam (Rewrites) project on Posit Cloud.
  • 🎉 We are now accepting Course Assistant/Teaching Fellow applications for Stat 100 for next semester. To apply, fill out this application by Nov 15th.
    • About 10-12 hours of work per week.
    • Primary responsibilities: Attend weekly team meetings, lead a discussion section, hold office hours, grade assessments.

Goals for Today

  • Confidence interval interpretations
  • Set-up the structure of hypothesis testing
  • Determine if Harvard students have ESP!

What do we mean by confidence?

  • Confidence level = success rate of the method under repeated sampling

  • How do I know if my ONE CI successfully contains the true value of the parameter?

  • As we increase the confidence level, what happens to the width of the interval?

  • As we increase the sample size, what happens to the width of the interval?

  • As we increase the number of bootstrap samples we take, what happens to the width of the interval?

Interpreting Confidence Intervals

Example: Estimating average household income before taxes in the US

SE Method Formula:

\[ \mbox{statistic} \pm{\mbox{ME}} \]

# A tibble: 1 × 3
     ME  lower  upper
  <dbl>  <dbl>  <dbl>
1 1968. 60512. 64448.

“The margin of [sampling] error can be described as the ‘penalty’ in precision for not talking to everyone in a given population. It describes the range that an answer likely falls between if the survey had reached everyone in a population, instead of just a sample of that population.” – Courtney Kennedy, Director of Survey Research at Pew Research Center

CI = interval of plausible values for the parameter

Safe interpretation: I am P% confident that {insert what the parameter represents in context} is between {insert lower bound} and {insert upper bound}.

Caution: Confidence intervals in the wild

Statement in an article for The BMJ (British Medical Journal):

Statistical Inference

Goal: Draw conclusions about the population based on a sample.

Main Flavors:

  • Estimating numerical quantities.
  • Testing conjectures.

Example: Does Extrasensory Perception (ESP) exist?

Daryl Bem and Ben Honorton

Bem and Honorton conducted extrasensory perception studies:

  • A “sender” randomly chooses an object out of 4 possible objects and sends that information to a “receiver”.
  • The “receiver” is then given a set of 4 possible objects and they must decide which one most resembles the object sent to them.

Out of 329 trials, the “receivers” correctly identified the object 106 times.

ESP Example

Let’s consider the following questions:

  1. If ESP does not exist and the “receivers” are guessing, how often would we expect them to be correct?

  2. For each sample (set of 329 trials), do we expect the proportion of correct guesses to be equal? Why or why not?

  3. Is it possible to randomly guess correctly 106 out of 329 times (i.e., 32% of the time)?

  4. How unusual is it to guess correctly 106 out of 329 times if ESP doesn’t exist?

To help us answer d., we need a sampling distribution for the sample proportion where we assume the “receivers” were purely guessing!

Sampling Distribution of a Statistic

Steps for (Approximate) Distribution:

  1. Decide on a sample size, \(n\).

  2. Randomly select a sample of size \(n\) from the population.

  3. Compute the sample statistic.

  4. Put the sample back in.

  5. Repeat Steps (2) - (4) many (1000+) times.

Sampling Distribution of a Statistic

Steps for (Approximate) Distribution:

  1. Decide on a sample size, \(n\).

  2. Randomly select a sample of size \(n\) from the population.

  3. Compute the sample statistic.

  4. Put the sample back in.

  5. Repeat Steps (2) - (4) many (1000+) times.

Sampling Distribution Under No ESP

Steps for (Approximate) Distribution:

  1. Decide on a sample size, \(n\).

  2. Randomly select a sample of size \(n\) from the population.

library(mosaic)
rflip(n = 329, prob = 0.25)

Flipping 329 coins [ Prob(Heads) = 0.25 ] ...

T T T H T T T T T H T T T T T T T T T T T T T T T T T H T T T H T T T T
T T T T H H H T T T T T T T T T T H T T T T T H T T T T T T T H T H T H
H H T T T H H T T H T H T T T T T T H H T T H T T T T T T T T T T H H H
H H T H H H T T T T T H T H T T H T T T T T H T T T T T T T T T H H T T
H T T T T T H H T T T T T T H H T T H T T T T T T T T T T T H T H T T T
T T H H T T T T H T H H T H H T T T T H T T H T H T H T T T H T T T T T
T T T H T T T T T T T H T T T T H H T H T T H T T T T T H T T T H T H T
H T H H T H T T T T T T T T T T H T T T T T T T T H T H T T T T T H T T
T H T T H T T T H T T H T H T T H H H T T T T T T T T T T H H T H T T T
T T T T T

Number of Heads: 83 [Proportion Heads: 0.252279635258359]

Sampling Distribution Under No ESP

  1. Compute the sample statistic.
rflip(n = 329, prob = 0.25, summarize = TRUE)
    n heads tails prob
1 329    64   265 0.25

  1. Put the sample back in.

  2. Repeat Steps (2) - (4) many (1000+) times.

guess_sampling_dist <- do(1000)*rflip(n = 329, prob = 0.25)
guess_sampling_dist
       n heads tails      prop
1    329    86   243 0.2613982
2    329    84   245 0.2553191
3    329    83   246 0.2522796
4    329    84   245 0.2553191
5    329    70   259 0.2127660
6    329    81   248 0.2462006
7    329    92   237 0.2796353
8    329    70   259 0.2127660
9    329    82   247 0.2492401
10   329    75   254 0.2279635
11   329    73   256 0.2218845
12   329    71   258 0.2158055
13   329    71   258 0.2158055
14   329    75   254 0.2279635
15   329    89   240 0.2705167
16   329    86   243 0.2613982
17   329    72   257 0.2188450
18   329    77   252 0.2340426
19   329    91   238 0.2765957
20   329    77   252 0.2340426
21   329    87   242 0.2644377
22   329    75   254 0.2279635
23   329    82   247 0.2492401
24   329    89   240 0.2705167
25   329    91   238 0.2765957
26   329    89   240 0.2705167
27   329    77   252 0.2340426
28   329    82   247 0.2492401
29   329    91   238 0.2765957
30   329    86   243 0.2613982
31   329    77   252 0.2340426
32   329    71   258 0.2158055
33   329    76   253 0.2310030
34   329    83   246 0.2522796
35   329    83   246 0.2522796
36   329    94   235 0.2857143
37   329    84   245 0.2553191
38   329    83   246 0.2522796
39   329    88   241 0.2674772
40   329    86   243 0.2613982
41   329    77   252 0.2340426
42   329    82   247 0.2492401
43   329    84   245 0.2553191
44   329    94   235 0.2857143
45   329    78   251 0.2370821
46   329    62   267 0.1884498
47   329    72   257 0.2188450
48   329    84   245 0.2553191
49   329    82   247 0.2492401
50   329    73   256 0.2218845
51   329    89   240 0.2705167
52   329    92   237 0.2796353
53   329    75   254 0.2279635
54   329    75   254 0.2279635
55   329    79   250 0.2401216
56   329    75   254 0.2279635
57   329    81   248 0.2462006
58   329    76   253 0.2310030
59   329    79   250 0.2401216
60   329    75   254 0.2279635
61   329   100   229 0.3039514
62   329    85   244 0.2583587
63   329    83   246 0.2522796
64   329    89   240 0.2705167
65   329    78   251 0.2370821
66   329    81   248 0.2462006
67   329    67   262 0.2036474
68   329    94   235 0.2857143
69   329    83   246 0.2522796
70   329    76   253 0.2310030
71   329    87   242 0.2644377
72   329    68   261 0.2066869
73   329    79   250 0.2401216
74   329    91   238 0.2765957
75   329    83   246 0.2522796
76   329    93   236 0.2826748
77   329    89   240 0.2705167
78   329    98   231 0.2978723
79   329    69   260 0.2097264
80   329    80   249 0.2431611
81   329    82   247 0.2492401
82   329    72   257 0.2188450
83   329    79   250 0.2401216
84   329    85   244 0.2583587
85   329    90   239 0.2735562
86   329    88   241 0.2674772
87   329    78   251 0.2370821
88   329    73   256 0.2218845
89   329    72   257 0.2188450
90   329    83   246 0.2522796
91   329    78   251 0.2370821
92   329    76   253 0.2310030
93   329    90   239 0.2735562
94   329    79   250 0.2401216
95   329    68   261 0.2066869
96   329    90   239 0.2735562
97   329    92   237 0.2796353
98   329    93   236 0.2826748
99   329    98   231 0.2978723
100  329    90   239 0.2735562
101  329    92   237 0.2796353
102  329    78   251 0.2370821
103  329    69   260 0.2097264
104  329    83   246 0.2522796
105  329    88   241 0.2674772
106  329    84   245 0.2553191
107  329    68   261 0.2066869
108  329    77   252 0.2340426
109  329    85   244 0.2583587
110  329    86   243 0.2613982
111  329    91   238 0.2765957
112  329    84   245 0.2553191
113  329    91   238 0.2765957
114  329    90   239 0.2735562
115  329    69   260 0.2097264
116  329    70   259 0.2127660
117  329    81   248 0.2462006
118  329    84   245 0.2553191
119  329    79   250 0.2401216
120  329    81   248 0.2462006
121  329   101   228 0.3069909
122  329    74   255 0.2249240
123  329    66   263 0.2006079
124  329    78   251 0.2370821
125  329    77   252 0.2340426
126  329    94   235 0.2857143
127  329    97   232 0.2948328
128  329    73   256 0.2218845
129  329    77   252 0.2340426
130  329    83   246 0.2522796
131  329    76   253 0.2310030
132  329    80   249 0.2431611
133  329    87   242 0.2644377
134  329    86   243 0.2613982
135  329    74   255 0.2249240
136  329    89   240 0.2705167
137  329    80   249 0.2431611
138  329    69   260 0.2097264
139  329    91   238 0.2765957
140  329    73   256 0.2218845
141  329    91   238 0.2765957
142  329    81   248 0.2462006
143  329    82   247 0.2492401
144  329    76   253 0.2310030
145  329    91   238 0.2765957
146  329    84   245 0.2553191
147  329    83   246 0.2522796
148  329    82   247 0.2492401
149  329    92   237 0.2796353
150  329    94   235 0.2857143
151  329    92   237 0.2796353
152  329    79   250 0.2401216
153  329    82   247 0.2492401
154  329    83   246 0.2522796
155  329    66   263 0.2006079
156  329    85   244 0.2583587
157  329    88   241 0.2674772
158  329   101   228 0.3069909
159  329    75   254 0.2279635
160  329    81   248 0.2462006
161  329    77   252 0.2340426
162  329    93   236 0.2826748
163  329    79   250 0.2401216
164  329    87   242 0.2644377
165  329    90   239 0.2735562
166  329    76   253 0.2310030
167  329    71   258 0.2158055
168  329    80   249 0.2431611
169  329    80   249 0.2431611
170  329    84   245 0.2553191
171  329    94   235 0.2857143
172  329    87   242 0.2644377
173  329    84   245 0.2553191
174  329    69   260 0.2097264
175  329    79   250 0.2401216
176  329    78   251 0.2370821
177  329    85   244 0.2583587
178  329    90   239 0.2735562
179  329    80   249 0.2431611
180  329    85   244 0.2583587
181  329    90   239 0.2735562
182  329    83   246 0.2522796
183  329    80   249 0.2431611
184  329    73   256 0.2218845
185  329    70   259 0.2127660
186  329    79   250 0.2401216
187  329    93   236 0.2826748
188  329    80   249 0.2431611
189  329    85   244 0.2583587
190  329    85   244 0.2583587
191  329    85   244 0.2583587
192  329    80   249 0.2431611
193  329    84   245 0.2553191
194  329    71   258 0.2158055
195  329    86   243 0.2613982
196  329    92   237 0.2796353
197  329    77   252 0.2340426
198  329    93   236 0.2826748
199  329    78   251 0.2370821
200  329    83   246 0.2522796
201  329    97   232 0.2948328
202  329    82   247 0.2492401
203  329    82   247 0.2492401
204  329    78   251 0.2370821
205  329    93   236 0.2826748
206  329    81   248 0.2462006
207  329    80   249 0.2431611
208  329    72   257 0.2188450
209  329    72   257 0.2188450
210  329    86   243 0.2613982
211  329    80   249 0.2431611
212  329    89   240 0.2705167
213  329    79   250 0.2401216
214  329    84   245 0.2553191
215  329    78   251 0.2370821
216  329    74   255 0.2249240
217  329    74   255 0.2249240
218  329    90   239 0.2735562
219  329    74   255 0.2249240
220  329    83   246 0.2522796
221  329    85   244 0.2583587
222  329    71   258 0.2158055
223  329    72   257 0.2188450
224  329    94   235 0.2857143
225  329    86   243 0.2613982
226  329    74   255 0.2249240
227  329    83   246 0.2522796
228  329    78   251 0.2370821
229  329    75   254 0.2279635
230  329    86   243 0.2613982
231  329    75   254 0.2279635
232  329    83   246 0.2522796
233  329    85   244 0.2583587
234  329    80   249 0.2431611
235  329    83   246 0.2522796
236  329    86   243 0.2613982
237  329    78   251 0.2370821
238  329    88   241 0.2674772
239  329    73   256 0.2218845
240  329    75   254 0.2279635
241  329    83   246 0.2522796
242  329    82   247 0.2492401
243  329    79   250 0.2401216
244  329    78   251 0.2370821
245  329    79   250 0.2401216
246  329    76   253 0.2310030
247  329    73   256 0.2218845
248  329    71   258 0.2158055
249  329    87   242 0.2644377
250  329    86   243 0.2613982
251  329    78   251 0.2370821
252  329    82   247 0.2492401
253  329    76   253 0.2310030
254  329    85   244 0.2583587
255  329    81   248 0.2462006
256  329    76   253 0.2310030
257  329    84   245 0.2553191
258  329    86   243 0.2613982
259  329    94   235 0.2857143
260  329    73   256 0.2218845
261  329    82   247 0.2492401
262  329    82   247 0.2492401
263  329    83   246 0.2522796
264  329    91   238 0.2765957
265  329    63   266 0.1914894
266  329    87   242 0.2644377
267  329    74   255 0.2249240
268  329    83   246 0.2522796
269  329    82   247 0.2492401
270  329    73   256 0.2218845
271  329    73   256 0.2218845
272  329    74   255 0.2249240
273  329    90   239 0.2735562
274  329    72   257 0.2188450
275  329    76   253 0.2310030
276  329    92   237 0.2796353
277  329    87   242 0.2644377
278  329    72   257 0.2188450
279  329    82   247 0.2492401
280  329    70   259 0.2127660
281  329    88   241 0.2674772
282  329    93   236 0.2826748
283  329    79   250 0.2401216
284  329    84   245 0.2553191
285  329    86   243 0.2613982
286  329    76   253 0.2310030
287  329    92   237 0.2796353
288  329    88   241 0.2674772
289  329    88   241 0.2674772
290  329    85   244 0.2583587
291  329    91   238 0.2765957
292  329    89   240 0.2705167
293  329    73   256 0.2218845
294  329    79   250 0.2401216
295  329    87   242 0.2644377
296  329    93   236 0.2826748
297  329    62   267 0.1884498
298  329    90   239 0.2735562
299  329    72   257 0.2188450
300  329    86   243 0.2613982
301  329    90   239 0.2735562
302  329    83   246 0.2522796
303  329    85   244 0.2583587
304  329    77   252 0.2340426
305  329    86   243 0.2613982
306  329    79   250 0.2401216
307  329    84   245 0.2553191
308  329    92   237 0.2796353
309  329    82   247 0.2492401
310  329    87   242 0.2644377
311  329    90   239 0.2735562
312  329    75   254 0.2279635
313  329    79   250 0.2401216
314  329    61   268 0.1854103
315  329    90   239 0.2735562
316  329    87   242 0.2644377
317  329    76   253 0.2310030
318  329    67   262 0.2036474
319  329    75   254 0.2279635
320  329    85   244 0.2583587
321  329    72   257 0.2188450
322  329    66   263 0.2006079
323  329    77   252 0.2340426
324  329    85   244 0.2583587
325  329    91   238 0.2765957
326  329    83   246 0.2522796
327  329    95   234 0.2887538
328  329    90   239 0.2735562
329  329    67   262 0.2036474
330  329    87   242 0.2644377
331  329    81   248 0.2462006
332  329    85   244 0.2583587
333  329    77   252 0.2340426
334  329    87   242 0.2644377
335  329    78   251 0.2370821
336  329    86   243 0.2613982
337  329    77   252 0.2340426
338  329    80   249 0.2431611
339  329    87   242 0.2644377
340  329    76   253 0.2310030
341  329    84   245 0.2553191
342  329    93   236 0.2826748
343  329    80   249 0.2431611
344  329   102   227 0.3100304
345  329    84   245 0.2553191
346  329    95   234 0.2887538
347  329    84   245 0.2553191
348  329    91   238 0.2765957
349  329    68   261 0.2066869
350  329    77   252 0.2340426
351  329    77   252 0.2340426
352  329    86   243 0.2613982
353  329    85   244 0.2583587
354  329    88   241 0.2674772
355  329    75   254 0.2279635
356  329    96   233 0.2917933
357  329    78   251 0.2370821
358  329    90   239 0.2735562
359  329    94   235 0.2857143
360  329    86   243 0.2613982
361  329    69   260 0.2097264
362  329    86   243 0.2613982
363  329    88   241 0.2674772
364  329    95   234 0.2887538
365  329    80   249 0.2431611
366  329    72   257 0.2188450
367  329    79   250 0.2401216
368  329    85   244 0.2583587
369  329    90   239 0.2735562
370  329    91   238 0.2765957
371  329    78   251 0.2370821
372  329    87   242 0.2644377
373  329    80   249 0.2431611
374  329    83   246 0.2522796
375  329    78   251 0.2370821
376  329    86   243 0.2613982
377  329    94   235 0.2857143
378  329    76   253 0.2310030
379  329    87   242 0.2644377
380  329    94   235 0.2857143
381  329    78   251 0.2370821
382  329    78   251 0.2370821
383  329    69   260 0.2097264
384  329    86   243 0.2613982
385  329    83   246 0.2522796
386  329    72   257 0.2188450
387  329    76   253 0.2310030
388  329    79   250 0.2401216
389  329    74   255 0.2249240
390  329    74   255 0.2249240
391  329    95   234 0.2887538
392  329    86   243 0.2613982
393  329    85   244 0.2583587
394  329    80   249 0.2431611
395  329    85   244 0.2583587
396  329    90   239 0.2735562
397  329    79   250 0.2401216
398  329    65   264 0.1975684
399  329    87   242 0.2644377
400  329    79   250 0.2401216
401  329    89   240 0.2705167
402  329    89   240 0.2705167
403  329    80   249 0.2431611
404  329    82   247 0.2492401
405  329    96   233 0.2917933
406  329    76   253 0.2310030
407  329    77   252 0.2340426
408  329    78   251 0.2370821
409  329    95   234 0.2887538
410  329    86   243 0.2613982
411  329    89   240 0.2705167
412  329    82   247 0.2492401
413  329    85   244 0.2583587
414  329    83   246 0.2522796
415  329    88   241 0.2674772
416  329    84   245 0.2553191
417  329    88   241 0.2674772
418  329    76   253 0.2310030
419  329    64   265 0.1945289
420  329    75   254 0.2279635
421  329    86   243 0.2613982
422  329    82   247 0.2492401
423  329    95   234 0.2887538
424  329    88   241 0.2674772
425  329    64   265 0.1945289
426  329    86   243 0.2613982
427  329    75   254 0.2279635
428  329    91   238 0.2765957
429  329    89   240 0.2705167
430  329    88   241 0.2674772
431  329    84   245 0.2553191
432  329    81   248 0.2462006
433  329    93   236 0.2826748
434  329    78   251 0.2370821
435  329    82   247 0.2492401
436  329    77   252 0.2340426
437  329    82   247 0.2492401
438  329    95   234 0.2887538
439  329    90   239 0.2735562
440  329    85   244 0.2583587
441  329    82   247 0.2492401
442  329    80   249 0.2431611
443  329    66   263 0.2006079
444  329    83   246 0.2522796
445  329    87   242 0.2644377
446  329    80   249 0.2431611
447  329    80   249 0.2431611
448  329    72   257 0.2188450
449  329    69   260 0.2097264
450  329    84   245 0.2553191
451  329    68   261 0.2066869
452  329    68   261 0.2066869
453  329    72   257 0.2188450
454  329    90   239 0.2735562
455  329    80   249 0.2431611
456  329    73   256 0.2218845
457  329    89   240 0.2705167
458  329    94   235 0.2857143
459  329    88   241 0.2674772
460  329    86   243 0.2613982
461  329    82   247 0.2492401
462  329    67   262 0.2036474
463  329    82   247 0.2492401
464  329    96   233 0.2917933
465  329    84   245 0.2553191
466  329    63   266 0.1914894
467  329    78   251 0.2370821
468  329    79   250 0.2401216
469  329    82   247 0.2492401
470  329    79   250 0.2401216
471  329    79   250 0.2401216
472  329    72   257 0.2188450
473  329    92   237 0.2796353
474  329    71   258 0.2158055
475  329    80   249 0.2431611
476  329    84   245 0.2553191
477  329    88   241 0.2674772
478  329    86   243 0.2613982
479  329    82   247 0.2492401
480  329    82   247 0.2492401
481  329    80   249 0.2431611
482  329    78   251 0.2370821
483  329    88   241 0.2674772
484  329    93   236 0.2826748
485  329    79   250 0.2401216
486  329    79   250 0.2401216
487  329    83   246 0.2522796
488  329    86   243 0.2613982
489  329    65   264 0.1975684
490  329    78   251 0.2370821
491  329    83   246 0.2522796
492  329    77   252 0.2340426
493  329    86   243 0.2613982
494  329    85   244 0.2583587
495  329    81   248 0.2462006
496  329    89   240 0.2705167
497  329    79   250 0.2401216
498  329    76   253 0.2310030
499  329    92   237 0.2796353
500  329    72   257 0.2188450
501  329    86   243 0.2613982
502  329    87   242 0.2644377
503  329    78   251 0.2370821
504  329    79   250 0.2401216
505  329    71   258 0.2158055
506  329    61   268 0.1854103
507  329    81   248 0.2462006
508  329    89   240 0.2705167
509  329    80   249 0.2431611
510  329    95   234 0.2887538
511  329    75   254 0.2279635
512  329    86   243 0.2613982
513  329    66   263 0.2006079
514  329    82   247 0.2492401
515  329    86   243 0.2613982
516  329    73   256 0.2218845
517  329    77   252 0.2340426
518  329    86   243 0.2613982
519  329    90   239 0.2735562
520  329    65   264 0.1975684
521  329    82   247 0.2492401
522  329    89   240 0.2705167
523  329    86   243 0.2613982
524  329    75   254 0.2279635
525  329    75   254 0.2279635
526  329    91   238 0.2765957
527  329    81   248 0.2462006
528  329    68   261 0.2066869
529  329    80   249 0.2431611
530  329    90   239 0.2735562
531  329    70   259 0.2127660
532  329    79   250 0.2401216
533  329    86   243 0.2613982
534  329    92   237 0.2796353
535  329    88   241 0.2674772
536  329    73   256 0.2218845
537  329    81   248 0.2462006
538  329    65   264 0.1975684
539  329    83   246 0.2522796
540  329    86   243 0.2613982
541  329    71   258 0.2158055
542  329    83   246 0.2522796
543  329    96   233 0.2917933
544  329    78   251 0.2370821
545  329    82   247 0.2492401
546  329    89   240 0.2705167
547  329    72   257 0.2188450
548  329    74   255 0.2249240
549  329    73   256 0.2218845
550  329    78   251 0.2370821
551  329    75   254 0.2279635
552  329    81   248 0.2462006
553  329    73   256 0.2218845
554  329    75   254 0.2279635
555  329    95   234 0.2887538
556  329    75   254 0.2279635
557  329    79   250 0.2401216
558  329    69   260 0.2097264
559  329    74   255 0.2249240
560  329    90   239 0.2735562
561  329    99   230 0.3009119
562  329    88   241 0.2674772
563  329    84   245 0.2553191
564  329    91   238 0.2765957
565  329    86   243 0.2613982
566  329    82   247 0.2492401
567  329    91   238 0.2765957
568  329    84   245 0.2553191
569  329    77   252 0.2340426
570  329    84   245 0.2553191
571  329    84   245 0.2553191
572  329    78   251 0.2370821
573  329    80   249 0.2431611
574  329    92   237 0.2796353
575  329    65   264 0.1975684
576  329    89   240 0.2705167
577  329    70   259 0.2127660
578  329    93   236 0.2826748
579  329    88   241 0.2674772
580  329    83   246 0.2522796
581  329    74   255 0.2249240
582  329    96   233 0.2917933
583  329    73   256 0.2218845
584  329    87   242 0.2644377
585  329    70   259 0.2127660
586  329    96   233 0.2917933
587  329    79   250 0.2401216
588  329    76   253 0.2310030
589  329    88   241 0.2674772
590  329    79   250 0.2401216
591  329    78   251 0.2370821
592  329    88   241 0.2674772
593  329    92   237 0.2796353
594  329    78   251 0.2370821
595  329    70   259 0.2127660
596  329    79   250 0.2401216
597  329    79   250 0.2401216
598  329    75   254 0.2279635
599  329    82   247 0.2492401
600  329    82   247 0.2492401
601  329    81   248 0.2462006
602  329    85   244 0.2583587
603  329    90   239 0.2735562
604  329    89   240 0.2705167
605  329    82   247 0.2492401
606  329    93   236 0.2826748
607  329    78   251 0.2370821
608  329    75   254 0.2279635
609  329    73   256 0.2218845
610  329    80   249 0.2431611
611  329    89   240 0.2705167
612  329    91   238 0.2765957
613  329    65   264 0.1975684
614  329    86   243 0.2613982
615  329    74   255 0.2249240
616  329    59   270 0.1793313
617  329    81   248 0.2462006
618  329    81   248 0.2462006
619  329    93   236 0.2826748
620  329    74   255 0.2249240
621  329    79   250 0.2401216
622  329    78   251 0.2370821
623  329    91   238 0.2765957
624  329    80   249 0.2431611
625  329    85   244 0.2583587
626  329    86   243 0.2613982
627  329    85   244 0.2583587
628  329    86   243 0.2613982
629  329    73   256 0.2218845
630  329    83   246 0.2522796
631  329    73   256 0.2218845
632  329    92   237 0.2796353
633  329    74   255 0.2249240
634  329    84   245 0.2553191
635  329    83   246 0.2522796
636  329    77   252 0.2340426
637  329    93   236 0.2826748
638  329    78   251 0.2370821
639  329    74   255 0.2249240
640  329    88   241 0.2674772
641  329    93   236 0.2826748
642  329    86   243 0.2613982
643  329    86   243 0.2613982
644  329    79   250 0.2401216
645  329    80   249 0.2431611
646  329    68   261 0.2066869
647  329    85   244 0.2583587
648  329    69   260 0.2097264
649  329    91   238 0.2765957
650  329    82   247 0.2492401
651  329    85   244 0.2583587
652  329    69   260 0.2097264
653  329    93   236 0.2826748
654  329    82   247 0.2492401
655  329    88   241 0.2674772
656  329    85   244 0.2583587
657  329    88   241 0.2674772
658  329    76   253 0.2310030
659  329    91   238 0.2765957
660  329    91   238 0.2765957
661  329    87   242 0.2644377
662  329    78   251 0.2370821
663  329    70   259 0.2127660
664  329    79   250 0.2401216
665  329    77   252 0.2340426
666  329    78   251 0.2370821
667  329    80   249 0.2431611
668  329    93   236 0.2826748
669  329    82   247 0.2492401
670  329    86   243 0.2613982
671  329    89   240 0.2705167
672  329    76   253 0.2310030
673  329    87   242 0.2644377
674  329    98   231 0.2978723
675  329    77   252 0.2340426
676  329   100   229 0.3039514
677  329    81   248 0.2462006
678  329    74   255 0.2249240
679  329    93   236 0.2826748
680  329    71   258 0.2158055
681  329    86   243 0.2613982
682  329    81   248 0.2462006
683  329    86   243 0.2613982
684  329    79   250 0.2401216
685  329    73   256 0.2218845
686  329    84   245 0.2553191
687  329    84   245 0.2553191
688  329    81   248 0.2462006
689  329    80   249 0.2431611
690  329    79   250 0.2401216
691  329    82   247 0.2492401
692  329    85   244 0.2583587
693  329    83   246 0.2522796
694  329    82   247 0.2492401
695  329    81   248 0.2462006
696  329    92   237 0.2796353
697  329    86   243 0.2613982
698  329    74   255 0.2249240
699  329    88   241 0.2674772
700  329    80   249 0.2431611
701  329    82   247 0.2492401
702  329    79   250 0.2401216
703  329    76   253 0.2310030
704  329    75   254 0.2279635
705  329    87   242 0.2644377
706  329    76   253 0.2310030
707  329    86   243 0.2613982
708  329    82   247 0.2492401
709  329    86   243 0.2613982
710  329    83   246 0.2522796
711  329    82   247 0.2492401
712  329    96   233 0.2917933
713  329    77   252 0.2340426
714  329    78   251 0.2370821
715  329    98   231 0.2978723
716  329    84   245 0.2553191
717  329    78   251 0.2370821
718  329    93   236 0.2826748
719  329    83   246 0.2522796
720  329    87   242 0.2644377
721  329    88   241 0.2674772
722  329    81   248 0.2462006
723  329    75   254 0.2279635
724  329    72   257 0.2188450
725  329    95   234 0.2887538
726  329    92   237 0.2796353
727  329    85   244 0.2583587
728  329    82   247 0.2492401
729  329    87   242 0.2644377
730  329    73   256 0.2218845
731  329    85   244 0.2583587
732  329    80   249 0.2431611
733  329   105   224 0.3191489
734  329    66   263 0.2006079
735  329    88   241 0.2674772
736  329    77   252 0.2340426
737  329    69   260 0.2097264
738  329    89   240 0.2705167
739  329    86   243 0.2613982
740  329    71   258 0.2158055
741  329    73   256 0.2218845
742  329    87   242 0.2644377
743  329    79   250 0.2401216
744  329    84   245 0.2553191
745  329    90   239 0.2735562
746  329    83   246 0.2522796
747  329    84   245 0.2553191
748  329    96   233 0.2917933
749  329    79   250 0.2401216
750  329    78   251 0.2370821
751  329    84   245 0.2553191
752  329    85   244 0.2583587
753  329    76   253 0.2310030
754  329    82   247 0.2492401
755  329    86   243 0.2613982
756  329    84   245 0.2553191
757  329    80   249 0.2431611
758  329    85   244 0.2583587
759  329    93   236 0.2826748
760  329    77   252 0.2340426
761  329    77   252 0.2340426
762  329    85   244 0.2583587
763  329    80   249 0.2431611
764  329    69   260 0.2097264
765  329    91   238 0.2765957
766  329    73   256 0.2218845
767  329    84   245 0.2553191
768  329    95   234 0.2887538
769  329    89   240 0.2705167
770  329    75   254 0.2279635
771  329    94   235 0.2857143
772  329    84   245 0.2553191
773  329    84   245 0.2553191
774  329    78   251 0.2370821
775  329    80   249 0.2431611
776  329    88   241 0.2674772
777  329    72   257 0.2188450
778  329    85   244 0.2583587
779  329    82   247 0.2492401
780  329    89   240 0.2705167
781  329    70   259 0.2127660
782  329    72   257 0.2188450
783  329    86   243 0.2613982
784  329    73   256 0.2218845
785  329    84   245 0.2553191
786  329    72   257 0.2188450
787  329    78   251 0.2370821
788  329    85   244 0.2583587
789  329    84   245 0.2553191
790  329    70   259 0.2127660
791  329    83   246 0.2522796
792  329    80   249 0.2431611
793  329    79   250 0.2401216
794  329    93   236 0.2826748
795  329    97   232 0.2948328
796  329    83   246 0.2522796
797  329    87   242 0.2644377
798  329    82   247 0.2492401
799  329    73   256 0.2218845
800  329    77   252 0.2340426
801  329    77   252 0.2340426
802  329    87   242 0.2644377
803  329    71   258 0.2158055
804  329    77   252 0.2340426
805  329    60   269 0.1823708
806  329    75   254 0.2279635
807  329    84   245 0.2553191
808  329    76   253 0.2310030
809  329    79   250 0.2401216
810  329    80   249 0.2431611
811  329    79   250 0.2401216
812  329    82   247 0.2492401
813  329    93   236 0.2826748
814  329    70   259 0.2127660
815  329    76   253 0.2310030
816  329   102   227 0.3100304
817  329    77   252 0.2340426
818  329    83   246 0.2522796
819  329    82   247 0.2492401
820  329    78   251 0.2370821
821  329    85   244 0.2583587
822  329    96   233 0.2917933
823  329    92   237 0.2796353
824  329    67   262 0.2036474
825  329   101   228 0.3069909
826  329    85   244 0.2583587
827  329    72   257 0.2188450
828  329    83   246 0.2522796
829  329    93   236 0.2826748
830  329    79   250 0.2401216
831  329    82   247 0.2492401
832  329   100   229 0.3039514
833  329    87   242 0.2644377
834  329    81   248 0.2462006
835  329    90   239 0.2735562
836  329    87   242 0.2644377
837  329    73   256 0.2218845
838  329    91   238 0.2765957
839  329    83   246 0.2522796
840  329    95   234 0.2887538
841  329    80   249 0.2431611
842  329    82   247 0.2492401
843  329    77   252 0.2340426
844  329    91   238 0.2765957
845  329    88   241 0.2674772
846  329    83   246 0.2522796
847  329    79   250 0.2401216
848  329    75   254 0.2279635
849  329    88   241 0.2674772
850  329    80   249 0.2431611
851  329    89   240 0.2705167
852  329    86   243 0.2613982
853  329    76   253 0.2310030
854  329    85   244 0.2583587
855  329    79   250 0.2401216
856  329    71   258 0.2158055
857  329    81   248 0.2462006
858  329    83   246 0.2522796
859  329    79   250 0.2401216
860  329    79   250 0.2401216
861  329    68   261 0.2066869
862  329    87   242 0.2644377
863  329    70   259 0.2127660
864  329    81   248 0.2462006
865  329    68   261 0.2066869
866  329    85   244 0.2583587
867  329    72   257 0.2188450
868  329    77   252 0.2340426
869  329    79   250 0.2401216
870  329    91   238 0.2765957
871  329    76   253 0.2310030
872  329    77   252 0.2340426
873  329    92   237 0.2796353
874  329    86   243 0.2613982
875  329    78   251 0.2370821
876  329    96   233 0.2917933
877  329    83   246 0.2522796
878  329    84   245 0.2553191
879  329    79   250 0.2401216
880  329    90   239 0.2735562
881  329    78   251 0.2370821
882  329    75   254 0.2279635
883  329    89   240 0.2705167
884  329    93   236 0.2826748
885  329    78   251 0.2370821
886  329    75   254 0.2279635
887  329    81   248 0.2462006
888  329    75   254 0.2279635
889  329    77   252 0.2340426
890  329    93   236 0.2826748
891  329    92   237 0.2796353
892  329    91   238 0.2765957
893  329    89   240 0.2705167
894  329    92   237 0.2796353
895  329    89   240 0.2705167
896  329    96   233 0.2917933
897  329    86   243 0.2613982
898  329    81   248 0.2462006
899  329    75   254 0.2279635
900  329    82   247 0.2492401
901  329    83   246 0.2522796
902  329    77   252 0.2340426
903  329    85   244 0.2583587
904  329    79   250 0.2401216
905  329    76   253 0.2310030
906  329    83   246 0.2522796
907  329    67   262 0.2036474
908  329    67   262 0.2036474
909  329    94   235 0.2857143
910  329    78   251 0.2370821
911  329    74   255 0.2249240
912  329    81   248 0.2462006
913  329    83   246 0.2522796
914  329    92   237 0.2796353
915  329    79   250 0.2401216
916  329    78   251 0.2370821
917  329    90   239 0.2735562
918  329    99   230 0.3009119
919  329    74   255 0.2249240
920  329    94   235 0.2857143
921  329    92   237 0.2796353
922  329    78   251 0.2370821
923  329    95   234 0.2887538
924  329    68   261 0.2066869
925  329    81   248 0.2462006
926  329    99   230 0.3009119
927  329    84   245 0.2553191
928  329    88   241 0.2674772
929  329    86   243 0.2613982
930  329    68   261 0.2066869
931  329    85   244 0.2583587
932  329    81   248 0.2462006
933  329    97   232 0.2948328
934  329    89   240 0.2705167
935  329    79   250 0.2401216
936  329    87   242 0.2644377
937  329    90   239 0.2735562
938  329    92   237 0.2796353
939  329    82   247 0.2492401
940  329    86   243 0.2613982
941  329    80   249 0.2431611
942  329    75   254 0.2279635
943  329    79   250 0.2401216
944  329    71   258 0.2158055
945  329    94   235 0.2857143
946  329    77   252 0.2340426
947  329    93   236 0.2826748
948  329    84   245 0.2553191
949  329    79   250 0.2401216
950  329    87   242 0.2644377
951  329    80   249 0.2431611
952  329    84   245 0.2553191
953  329    78   251 0.2370821
954  329    97   232 0.2948328
955  329    71   258 0.2158055
956  329    74   255 0.2249240
957  329    93   236 0.2826748
958  329    86   243 0.2613982
959  329    81   248 0.2462006
960  329    96   233 0.2917933
961  329    79   250 0.2401216
962  329    78   251 0.2370821
963  329    84   245 0.2553191
964  329    99   230 0.3009119
965  329    78   251 0.2370821
966  329    77   252 0.2340426
967  329    84   245 0.2553191
968  329    84   245 0.2553191
969  329    85   244 0.2583587
970  329    88   241 0.2674772
971  329    86   243 0.2613982
972  329    78   251 0.2370821
973  329    89   240 0.2705167
974  329    76   253 0.2310030
975  329    82   247 0.2492401
976  329    86   243 0.2613982
977  329    80   249 0.2431611
978  329    95   234 0.2887538
979  329    73   256 0.2218845
980  329    81   248 0.2462006
981  329    78   251 0.2370821
982  329    71   258 0.2158055
983  329    86   243 0.2613982
984  329    77   252 0.2340426
985  329    86   243 0.2613982
986  329    93   236 0.2826748
987  329    77   252 0.2340426
988  329    82   247 0.2492401
989  329    80   249 0.2431611
990  329    92   237 0.2796353
991  329    73   256 0.2218845
992  329    89   240 0.2705167
993  329    84   245 0.2553191
994  329    86   243 0.2613982
995  329    85   244 0.2583587
996  329    70   259 0.2127660
997  329    84   245 0.2553191
998  329    85   244 0.2583587
999  329    75   254 0.2279635
1000 329    82   247 0.2492401

Sampling Distribution Under No ESP

ggplot(data = guess_sampling_dist,
       mapping = aes(x = prop)) +
  geom_histogram(color = "white",
                 bins = 20)

What value should our sampling distribution be centered around if the receivers are just guessing?

Sampling Distribution Under No ESP

  • How do the study results compare to the sampling distribution under no ESP?
    • How unusual is it to guess correctly 106 out of 329 times if ESP doesn’t exist?
p_hat <- 106/329
ggplot(data = guess_sampling_dist,
       mapping = aes(x = prop)) +
  geom_histogram(color = "white",
                 bins = 20) +
  geom_vline(xintercept = p_hat,
             color = "orange",
             size = 2)

  • Do Bem and Honorton have evidence that ESP exists?

Do Harvardians Have ESP?

In pairs:

  • Decide who is going to be the sender and who is going to be the receiver.

  • Sender: Think of one of these images.

  • Receiver: Guess which image the sender was thinking of.

  • Now switch roles and do it again!

  • Once you have both played each role, each person should add a tally mark on the chalkboard.

Do Harvardians Have ESP?

What do we need to modify in the code to answer the question?

guess_sampling_dist <- do(1000)*rflip(n = 80, prob = 0.25)
p_hat <- 27/80
ggplot(data = guess_sampling_dist, mapping = aes(x = prop)) +
  geom_histogram(color = "white", bins = 20) +
  geom_vline(xintercept = p_hat, color = "orange", size = 2)

Hypothesis Testing

Big Idea:

  • Make an assumption about the population parameter.

  • Generate a sampling distribution for a test statistic based on that assumption.

    • Called a null distribution

  • See if the test statistic based on the observed sample aligns with the generated sampling distribution or not.

  • If it does, then we didn’t learn much.

    • (Didn’t prove the parameter equals the assumed value but it is still plausible)

  • If it doesn’t, then we have evidence that our assumption about the parameter was wrong.

ESP Example

Big Idea:

  • Make an assumption about the population parameter.
    • Ex: ESP doesn’t exist. p, probability of guessing correctly, equals 0.25.
  • Generate a sampling distribution for a test statistic based on that assumption.
    • Called a null distribution

ESP Example

Big Idea:

  • See if the test statistic based on the observed sample aligns with the generated sampling distribution or not.
    • Ex: It is in the center-ish of the distribution. It isn’t an unusual value.
  • If it does, then we didn’t learn much. (Didn’t prove the parameter equals the assumed value but it is still plausible)
    • It is still possible that ESP doesn’t exist.

ESP Example

Big Idea:

  • See if the test statistic based on the observed sample aligns with the generated sampling distribution or not.
    • It is far in the tails of the distribution. It is an unusual value.
  • If it doesn’t, then we have evidence that our assumption about the parameter was wrong.
    • We have evidence ESP exists.

Let’s Take a Step Back from Our Last Statement…

  • Two important words in data analysis:

    • Reproducibility
    • Replicability

  • Reproducibility: If I give you the raw data and my write-up, you will get to the exact same final numbers that I did.

  • By using Quarto Documents, we are learning a reproducible workflow.

  • Replicability: If you follow my study design but collect new data (i.e. repeat my study on new subjects), you will come to the same conclusions that I did.

Replication Crisis

Reminders:

  • Don’t forget that the midterm exam rewrites are due on Thursday at 5pm on Gradescope.
    • Make sure to use the Quarto doc in the Midterm Exam (Rewrites) project on Posit Cloud.
  • 🎉 We are now accepting Course Assistant/Teaching Fellow applications for Stat 100 for next semester. To apply, fill out this application by Nov 15th.
    • About 10-12 hours of work per week.
    • Primary responsibilities: Attend weekly team meetings, lead a discussion section, hold office hours, grade assessments.