Real-time collaboration for Jupyter Notebooks, Linux Terminals, LaTeX, VS Code, R IDE, and more,
all in one place. Commercial Alternative to JupyterHub.

| Download

Project: Serena Huang - LS 40 Lab 1B

Path: Midterm/Midterm.ipynb

Views: ²²⁷
Image: ubuntu2004

Kernel: Python 3 (system-wide)

Life Sciences 40

Midterm Part 2, Winter 2022

Below, type the full names of all members of your group below. By submitting these names, you affirm that you have neither given nor received unauthorized help on this exam.

Eliza Parry, India Davis, Leonides Cigarroa, Serena Huang

Instructions:

This is Part II of the midterm. It consists of 26 parts lumped into two questions. The total point value is 100 points (43% of midterm grade).
Please enter all of your answers either as code or markdown text into the appropriate place a copy of this notebook on CoCalc (similar to homework). Some problems will require Python programming.
Then, you will only submit one version of the midterm, for your entire group, via Gradescope. The Gradescope assignment for the Midterm allows group participation, so make sure you select and enter all members of your group.
Since you are completing this on CoCalc but uploading to Gradescope, we recommend that you wait to upload until you have completed the full exam on CoCalc. At that point, on CoCalc, select File / Download as… / PDF. CoCalc will convert to a PDF that you can download. This entire PDF can then be easily uploaded to Gradescope, and you can manually select where each answer is for each question.
Please be aware, when uploading PDFs and your answers to Gradescope, that page-breaks can accidentally hide text. Additionally, if you answer questions via code comments (as opposed to markdown text), unless you manually create line breaks in your comment, the full text answer may not be legible. After uploading to Gradescope, please double-check all of your answers to make sure that they are legible and clear.
While screen shots are an acceptable alternative to uploading PDF, due to low resolution, we do not recommend taking photos of your computer screen with your phone, unless absolutely necessary.
Again, we will not grade any material left on CoCalc. In order to receive a grade, you must submit – as a group – to Gradescope. Don’t forget to add group members to your submission on Gradescope!
You may use your notes, assignments, slides, readings, solutions, and other resources on our LS 40 BruinLearn site and your CoCalc project (but not elsewhere on the internet).
However, as always, you must show all of your work to receive full credit for each problem.
If you have a clarifying question about the exam at any point during the exam period, post a question on Campuswire “To Instructors and TAs", so our instructional team can help you as quickly as possible. Questions about content or your own progress will not be answered.
For technical glitches with Python, try "Kernel menu > Restart kernel" or Backups in files view first.
Gradescope will forbid uploads after 6:00 pm Pacific Time on Thursday, February 10. Please plan accordingly.

We recommend running the following code:

In [2]:

import pandas as pd
import numpy as np
import seaborn as sns
import matplotlib.pyplot as plt

1. Warming Up

UCLA scientists are studying the effects of climate change on the rate at which glaciers are disappearing. Over 10 years, they carefully monitor 25 glaciers sampled across North and South America, and find that 14 showed retreats of more than 5 meters. You are recruited to help them analyze the data.

a.) What is the effect size that the scientists should report for this study? (3 points)

The percentage of glaciers that showed retreats of more than 5 meters over 10 years

b.) What is the “population” upon which inference from the sample is being studied? (3 points)

All of the glaciers across North and South America

c.) It is hard to attribute any given phenomenon to ongoing climate change since there is no “control”. However, global climate models can simulate how glaciers might be melting in the absence of climate change. These models suggest that without climate change, we might expect 40% of glaciers, on average, to retreat more than 5 meters per decade. Given this expectation under an absence of climate change, what is the probability of the scientists’ observed finding (or greater)? (5 points)

In [24]:

sample = 14/25
null = ["retreat"]*4+["none"]*6
H0 = []

for i in range(10000):
    selection = np.random.choice(np.array(null),25)
    H0.append(np.sum(selection == "retreat")/25)

sns.displot(data = np.array(H0), bins = 20)
plt.axvline(sample, color = "orange")

pvalue = np.sum(np.array(H0)>=14/25)/10000
print(pvalue)

0.0755

d.) The scientists are excited to hear that the p < 0.05. You tell them that they really should use an alpha of 0.01. Give two reasons that will be convincing. (4 points)

Reduction of type I error, or a false positive, which ensures that the expected outcome is less likely to be due to random chance.
If multiple testing was used, using a p-value of 0.05 would give a higher number of false positives compared to the number of false positives obtained in a p-value of 0.01

e.) If you rejected the null hypothesis based on this result, what is the expected probability that you are making a false positive? (3 points)

For a p-value of 0.05, we would expect a 5% false positive rate. However, because the question doesn't explicitly state whether we are rejecting the null hypothesis based on "0.01 or 0.05" (mentions only "this"), then we could expect a 1% false positive rate.

f.) In this specific example, what is the interpretation of a false positive? (3 points)

The results in this experiment were due to a 5% random likelihood that our observed value could be gathered from the null hypothesis data distribution. In other words, there's a 5% possibility that 14/25 glaciers could retreat given that 40% retreat on average.

g.) The problem with models is that when you make a small adjustment to your assumptions, sometimes it can strongly change your answers. After better accounting for ocean circulation, the climate models now suggest that without climate change, we would only expect 30% of glaciers, on average, to retreat more than 5 meters per decade. Given this new expectation, what is the probability of the scientists’ observed finding (or greater)? (5 points)

In [25]:

null = ["retreat"]*3+["none"]*7
H0 = []

for i in range(10000):
    selection = np.random.choice(np.array(null),25)
    H0.append(np.sum(selection == "retreat")/25)

sns.displot(data = np.array(H0), bins = 20)
plt.axvline(sample, color = "orange")

pvalue = np.sum(np.array(H0)>=14/25)/10000
print(pvalue)

0.0074

h.) Describe how the interpretation of results for (g) differs from (c). (3 points)

Results for (g) shows a p-value of 0.0074 while the results for (c) shows a p-value of 0.0755. For the alpha-threshold of 0.05, we would fail to reject the null hypothesis for (g) but reject the null hypothesis for (c). Even for a smaller p-value of 0.01, the actions we would take for each of the results would stay the same. Therefore, when the null hypothesis changed from 40% to 30% of glaciers retreat 5m, the observed results become more surprising.

i.) Between (g) and (c), which of the following changed: effect size, sample size, null hypothesis, alternative hypothesis. Specifically describe any changes. (6 points)

Effect size: stayed the same

Sample size: stayed the same

Null hypothesis: changed for g and c, where the null hypothesis for c states that given a group of measured glaciers measured in North and South America, 40% of these glaciers will show a 5 meter or more retreat. Meanwhile, the null hypothesis for g states that given a group of measured glaciers measured in North and South America, 30% of these glaciers will show a 5 meter or more retreat

Alternative hypothesis: stayed the same (having 14/25 glaciers show a retreat was statistically significant compared to the null hypothesis; the only difference is that in c, we failed to reject the null hypothesis, while in g, we rejected the null hypothesis)

2. No Belching Matter

A friend who has not taken any statistics shows you an online article that claims new research shows that organic vegetables cause you to burp. The article says there is only a 5% chance the finding is false.

a.) Explain to your friend where that 5% figure likely comes from and what it actually means. Your explanation should be understandable to someone with no statistics background. (3 points)

The 5% figure likely is a p-value of the study, which states the likelihood that the observed result would be found given the null hypothesis is true. This does not mean that the finding has a 5% of being false. A p-value is just an indicator of surprise of seeing an observed value compared to a null hypothesis dataset.

b.) Moreover, should you believe this finding? What other information would you like to know? (3 points)

We shouldn't believe this finding because we need to know:

The sample size, effect size and confidence intervals, the statistical test carried out and whether or not the test made any assumptions, whether the sample is representative, and how the null hypothesis was obtained or what it is

Inspired by your friend’s claim, you team up with UCLA researchers to try to replicate the finding. The scientists enlist 40 participants, 20 of which ate only organic vegetables for two weeks, and 20 of which ate conventional vegetables. Over the two weeks, they counted the number of times they burped. The data for this study is available in `burps.csv`.

In [3]:

burps = pd.read_csv("burps.csv")
burps

c.) Visualize the data using an appropriate graph. (3 points)

In [10]:

sns.violinplot(data=burps, palette = "twilight").set(ylabel="Number of burps")
sns.swarmplot(data=burps, palette = "twilight_shifted_r")

<AxesSubplot:ylabel='Number of burps'>

The graph for 'participants' isn't actually part of the data, but we don't know how to code it out of the plot so have to keep it there. Otherwise, it shouldn't be there.

d.) Describe the data distributions for both groups. (3 points)

The organic group has much a wider variability than the nonorganic group with roughly the same modality and shape (slightly unimodal and asymmetric). The right skew on the organic group is larger and both have thick left tails. The IQRs and the medians are nearly identical.

e.) Define and justify an appropriate metric (i.e., test statistic) to compare the centers of the two groups. (3 points)

Median; since this data is not bell-shaped, the mean would be greatly affected by extreme values (such as the right skew).

f.) You prepare to conduct a statistical test comparing the groups. What is the null hypothesis that you will test? (3 points)

The null hypothesis that I will test is that there is no difference in the median of burps for the organic vegetable group and conventional vegetable group. We will use a big-box null hypothesis significance test to compare the groups.

g.) What is the observed test statistic? Interpret this observed test statistic in the context of the study. (3 points)

In [12]:

morg = np.median(burps["organic"])
mnon = np.median(burps["non.organic"])

print(morg, mnon)
obs = morg-mnon
print(obs)

9.5 9.0
0.5

The observed test statistic is the difference in medians of the number of burps in the organic group and the median number of burps of the inorganic group. 0.5 is our observed effect size (difference in medians).

h.) Using an appropriate method, simulate the null hypothesis and calculate an associated p-value. (6 points)

In [13]:

# Simulating a big-box comparison NHST test

bigbox = np.concatenate([burps["organic"],burps["non.organic"]])
null = []

for i in range(10000):
    a = np.random.choice(bigbox,len(burps["organic"]))
    b = np.random.choice(bigbox,len(burps["non.organic"]))
    null.append(np.median(a)-np.median(b)) # Calculating the difference in medians of the two null groups

sns.displot(data = null, bins = 30).set(xlabel = "Difference in medians", ylabel = "Number of simulations")
plt.axvline(obs, color = "orange")

<matplotlib.lines.Line2D at 0x7f19d74a54f0>

In [14]:

pvalue = (np.sum(null>=obs)+np.sum(null<=-obs))/10000
pvalue

0.9645

i.) Is your p-value one-sided or two-sided? Make sure to justify your choice. (4 points)

Two-sided; while we are investigating how organic vegetables change the number of burps compared to inorganic vegetables, we cannot make the assumption that it will increase the number of burps or decrease them. A two-sided p-value calculates both the possibility that inorganic vegetables increase but also decrease the number of burps compared to organic vegetables. In other words, we would be just as surprised if we found that the change in the type of vegetable decreased the frequency of burps.

j.) Given your null hypothesis significance test, state your conclusions from the study. (3 points)

Given a two-tailed p-value of 0.9645, there is no significant difference in the median number of burps for eating organic versus eating inorganic vegetables. Any deviations from a difference of 0 was due to chance; we failed to reject the null hypothesis.

It turns out that when the UCLA researchers gave you the data, they forgot a key detail: rather than being a study of 40 different people, it was actually only 20 people! Each participant counted their burps for two weeks while randomized into either an organic or conventional diet. After two weeks, they switched to the other diet. The data are still the same, but this changes everything.

k.) Given this new study design knowledge, you prepare to conduct a new statistical test. What is the new null hypothesis that you will test? (3 points)

The new null hypothesis is that the median of the differences between the pairs of data is 0. In other words, there are no systematic differences in the number of burps on each individual. Thus, the median of the differences for each individual is 0.

l.) What is the observed test statistic? Interpret this observed test statistic in the context of the study. (3 points)

In [4]:

organic=np.array(burps["organic"])
nonorganic=np.array(burps["non.organic"])
med=np.median(organic-nonorganic)
print(med)

2.0

The observed test statistic is 2

m.) Using an appropriate method, simulate the null hypothesis and calculate an associated p-value. (6 points)

In [5]:

diff=organic-nonorganic
ran=[-1,1]
a=np.array(ran)
ranarray=np.random.choice(a,20)
null=diff*ranarray

masterlist=[]
for i in range(10000):
    ranarray=np.random.choice(a,20)
    null=diff*ranarray
    masterlist.append(np.median(null))
nullarray=np.array(masterlist)
nullplot=sns.displot(data=nullarray,bins=15)
nullplot.set(xlabel="difference in median",ylabel="Number of simulations")
plt.axvline(((med)), color="red")
plt.axvline(((-med)), color="red")
resultsp1=sum(nullarray>=med)
otherlim=-1*med
resultsp2=sum(nullarray<=otherlim)
pvalue=((resultsp1+resultsp2)/10000)
print(pvalue)

0.021

n.) Given your null hypothesis significance test, state your conclusions from the study. (4 points)

Given the data of the null hypothesis and an alpha of .05, we have found that this data is statistically significant with a p-value of .0238. The paired samples showed a statistically significant difference of burps in the organic vegetable group compared to the

o.) Do your results differ substantially in your re-analysis of the data? If so, explain why a simple issue of knowing the study design can have such a substantial effect. If not, explain why not. (5 points)

This result does differ substantially between analyses. In the first design, the p-value was high enough to indicate nonsignificance. In the paired design, we found a p-value small enough to indicate significance at an alpha of .05. The reason why we were able to change inferences from the data so heavily is because paired experimental designs have a much greater control on individual variability between two groups. We can better see differences between two groups because they occur within the same person rather than across multiple people.

p.) Finally, your friend wants to know, from your study, just how much it might truly help his burping if he stops eating organic vegetables. Calculate an appropriate estimate with 95% confidence. (6 points)

In [6]:

CIlist=[]
for i in range(10000):
    randif=np.random.choice(diff,20)
    CIlist.append(np.median(randif))
CIlist.sort()

lowerbound=2*med-CIlist[9749]
higherbound=2*med-CIlist[249]
print(lowerbound)
print(higherbound)

-0.5
3.5

q.) Provide an interpretation of your estimate from part (p) in terms your friend might comprehend. (5 points)

The pivotal bootstrap confidence interval of 95% means that we can be 95% sure that the data range given would contain the true differences in values. Since we cannot sample the real population, we have to simulate the statistical significances of multiple theoretical smaller data samples.

Thus, if we sampled a lot of data from the real population, and found all of their statistical significances using the method we used (two-box paired NHST), and found a range of the effect sizes, 95% of these values contain the true population effect size. We can be 95% confident that the degree of change in burping when eating organix versus inorganic vegetables is some value between (-0.5 and 3.5).

Real-time collaboration for Jupyter Notebooks, Linux Terminals, LaTeX, VS Code, R IDE, and more,
all in one place. Commercial Alternative to JupyterHub.

Life Sciences 40

Midterm Part 2, Winter 2022

Below, type the full names of all members of your group below. By submitting these names, you affirm that you have neither given nor received unauthorized help on this exam.

Instructions:

1. Warming Up

a.) What is the effect size that the scientists should report for this study? (3 points)

b.) What is the “population” upon which inference from the sample is being studied? (3 points)

d.) The scientists are excited to hear that the p < 0.05. You tell them that they really should use an alpha of 0.01. Give two reasons that will be convincing. (4 points)

e.) If you rejected the null hypothesis based on this result, what is the expected probability that you are making a false positive? (3 points)

f.) In this specific example, what is the interpretation of a false positive? (3 points)

h.) Describe how the interpretation of results for (g) differs from (c). (3 points)

i.) Between (g) and (c), which of the following changed: effect size, sample size, null hypothesis, alternative hypothesis. Specifically describe any changes. (6 points)

2. No Belching Matter

A friend who has not taken any statistics shows you an online article that claims new research shows that organic vegetables cause you to burp. The article says there is only a 5% chance the finding is false.

a.) Explain to your friend where that 5% figure likely comes from and what it actually means. Your explanation should be understandable to someone with no statistics background. (3 points)

b.) Moreover, should you believe this finding? What other information would you like to know? (3 points)

c.) Visualize the data using an appropriate graph. (3 points)

d.) Describe the data distributions for both groups. (3 points)

e.) Define and justify an appropriate metric (i.e., test statistic) to compare the centers of the two groups. (3 points)

f.) You prepare to conduct a statistical test comparing the groups. What is the null hypothesis that you will test? (3 points)

g.) What is the observed test statistic? Interpret this observed test statistic in the context of the study. (3 points)

h.) Using an appropriate method, simulate the null hypothesis and calculate an associated p-value. (6 points)

i.) Is your p-value one-sided or two-sided? Make sure to justify your choice. (4 points)

j.) Given your null hypothesis significance test, state your conclusions from the study. (3 points)

k.) Given this new study design knowledge, you prepare to conduct a new statistical test. What is the new null hypothesis that you will test? (3 points)

l.) What is the observed test statistic? Interpret this observed test statistic in the context of the study. (3 points)

m.) Using an appropriate method, simulate the null hypothesis and calculate an associated p-value. (6 points)

n.) Given your null hypothesis significance test, state your conclusions from the study. (4 points)

o.) Do your results differ substantially in your re-analysis of the data? If so, explain why a simple issue of knowing the study design can have such a substantial effect. If not, explain why not. (5 points)

p.) Finally, your friend wants to know, from your study, just how much it might truly help his burping if he stops eating organic vegetables. Calculate an appropriate estimate with 95% confidence. (6 points)

q.) Provide an interpretation of your estimate from part (p) in terms your friend might comprehend. (5 points)

Congratulations, you are done! When you are ready to submit, please choose File > Save and Download As > PDF and then upload to Gradescope.

Product

Resources

Company

Real-time collaboration for Jupyter Notebooks, Linux Terminals, LaTeX, VS Code, R IDE, and more, all in one place. Commercial Alternative to JupyterHub.

Life Sciences 40

Midterm Part 2, Winter 2022

Below, type the full names of all members of your group below. By submitting these names, you affirm that you have neither given nor received unauthorized help on this exam.

Instructions:

1. Warming Up

a.) What is the effect size that the scientists should report for this study? (3 points)

b.) What is the “population” upon which inference from the sample is being studied? (3 points)

d.) The scientists are excited to hear that the p < 0.05. You tell them that they really should use an alpha of 0.01. Give two reasons that will be convincing. (4 points)

e.) If you rejected the null hypothesis based on this result, what is the expected probability that you are making a false positive? (3 points)

f.) In this specific example, what is the interpretation of a false positive? (3 points)

h.) Describe how the interpretation of results for (g) differs from (c). (3 points)

i.) Between (g) and (c), which of the following changed: effect size, sample size, null hypothesis, alternative hypothesis. Specifically describe any changes. (6 points)

2. No Belching Matter

A friend who has not taken any statistics shows you an online article that claims new research shows that organic vegetables cause you to burp. The article says there is only a 5% chance the finding is false.

a.) Explain to your friend where that 5% figure likely comes from and what it actually means. Your explanation should be understandable to someone with no statistics background. (3 points)

b.) Moreover, should you believe this finding? What other information would you like to know? (3 points)

c.) Visualize the data using an appropriate graph. (3 points)

d.) Describe the data distributions for both groups. (3 points)

e.) Define and justify an appropriate metric (i.e., test statistic) to compare the centers of the two groups. (3 points)

f.) You prepare to conduct a statistical test comparing the groups. What is the null hypothesis that you will test? (3 points)

g.) What is the observed test statistic? Interpret this observed test statistic in the context of the study. (3 points)

h.) Using an appropriate method, simulate the null hypothesis and calculate an associated p-value. (6 points)

i.) Is your p-value one-sided or two-sided? Make sure to justify your choice. (4 points)

j.) Given your null hypothesis significance test, state your conclusions from the study. (3 points)

k.) Given this new study design knowledge, you prepare to conduct a new statistical test. What is the new null hypothesis that you will test? (3 points)

l.) What is the observed test statistic? Interpret this observed test statistic in the context of the study. (3 points)

m.) Using an appropriate method, simulate the null hypothesis and calculate an associated p-value. (6 points)

n.) Given your null hypothesis significance test, state your conclusions from the study. (4 points)

o.) Do your results differ substantially in your re-analysis of the data? If so, explain why a simple issue of knowing the study design can have such a substantial effect. If not, explain why not. (5 points)

p.) Finally, your friend wants to know, from your study, just how much it might truly help his burping if he stops eating organic vegetables. Calculate an appropriate estimate with 95% confidence. (6 points)

q.) Provide an interpretation of your estimate from part (p) in terms your friend might comprehend. (5 points)

Congratulations, you are done! When you are ready to submit, please choose File > Save and Download As > PDF and then upload to Gradescope.

Real-time collaboration for Jupyter Notebooks, Linux Terminals, LaTeX, VS Code, R IDE, and more,
all in one place. Commercial Alternative to JupyterHub.