Homework 4: DataFrames, Control Flow, and Probability
Due Tuesday, October 25th at 11:59PM
Welcome to Homework 4! This homework will cover lots of different topics:
Instructions
This assignment is due Tuesday, October 25th at 11:59PM. You are given six slip days throughout the quarter to extend deadlines. See the syllabus for more details. With the exception of using slip days, late work will not be accepted unless you have made special arrangements with your instructor.
Remember to start early and submit often.
Important: For homeworks, the otter tests don't usually tell you that your answer is correct. More often, they help catch careless mistakes. It's up to you to ensure that your answer is correct. If you're not sure, ask someone (not for the answer, but for some guidance about your approach). These are great questions for office hours (the schedule can be found here) or EdStem. Directly sharing answers is not okay, but discussing problems with the course staff or with other students is encouraged.
Supplemental Video on DataHub and Jupyter Notebooks
In Lab 1, we linked you to a video that walks you through key ideas you should be aware of when working on DataHub and in Jupyter Notebooks, including
how files are organized on DataHub
what it means to "restart the kernel"
how to use keyboard shortcuts (most important: use
SHIFT + ENTERto run a cell!)
We're linking it here again for your convenience. If you feel a little shaky on how to work your way around a notebook or troubleshoot issues, we recommend you give it another watch. (When troubleshooting, make sure to always check the Debugging tab on the course website as well.)
The video is quite long, but if you open the video directly on YouTube (which you can do by clicking the video's title after it loads in the next cell) you'll see timestamps in the description which you can use to jump to different parts of the video depending on what you'd like to learn more about.
0. DSC Student Rep Survey 🙏🏼
The Data Science Student Representatives and the DSC 10 couse staff created a survey for you to voice your opinion about what you love in DSC 10, and how we can improve your class and the department!
Feel free to talk about the topics covered in this course, the lectures and staff, resources, course structure, or anything else you feel is relevant and constructive.
Please click here to complete the survey. If you aren't able to access it, make sure you're logged into your UCSD Google account.
At the very bottom of the survey, you are given a codeword. Set completed_survey to that codeword.
1. DSC 40A Homeworks 📝
In this problem, we'll look at a dataset consisting of the point breakdowns of all homework assignments in the Fall 2021 offering of DSC 40A, another DSC course that covers the mathematical foundations of data science and machine learning. The homework assignments in DSC 40A consist of long-answer open-ended questions that are often broken into several parts. Each part is worth a different amount of points, depending on the difficulty.
Run the cell below to load this dataset as a DataFrame and store it in the hw variable, and answer the questions that follow.
Question 1.1. How many homework questions were assigned in DSC 40A throughout the quarter? Store the result in the variable num_questions.
Note: We are asking for the number of homework questions, not the number of parts.
Question 1.2. How many parts were there in each homework question? Create a DataFrame named num_parts with one row for each homework question and columns 'Homework', 'Question', and 'Num_Parts', which describes the total number of parts that homework question had. The first few rows of num_parts are shown below.
| Homework | Question | Num_Parts | |
|---|---|---|---|
| 0 | 1 | 1 | 4 |
| 1 | 1 | 2 | 12 |
| 2 | 1 | 3 | 3 |
| 3 | 1 | 4 | 2 |
| 4 | 1 | 5 | 5 |
Hint: Don't forget to use .reset_index().
Question 1.3. How many points was each homework question worth? Create a DataFrame named num_points that has one row for each homework question and columns 'Homework', 'Question', and 'Num_Points', which is the total number of points that homework question was worth. The first few rows of num_points are shown below.
| Homework | Question | Num_Points | |
|---|---|---|---|
| 0 | 1 | 1 | 10 |
| 1 | 1 | 2 | 17 |
| 2 | 1 | 3 | 8 |
| 3 | 1 | 4 | 6 |
| 4 | 1 | 5 | 15 |
2. Death and Taxes 💰💸
The United States, like many countries, uses a progressive tax bracket system. This means that as your earnings increase, the percentage of your earnings you owe in tax also increases. In addition, the US tax system uses marginal tax brackets – what this means is that US taxpayers pay different tax percentages on different "chunks" of their earnings.
Let's suppose the tax brackets for the 2023 tax year for single filers are defined by the table below. These are pretty close to the actual brackets, but for simplicity's sake we'll use 5 brackets instead of 7.
| Tax Bracket | Taxable Income |
|---|---|
| 10% | [$0, $11,000] |
| 12% | ($11,000, $44,725] |
| 22% | ($44,725, $95,375] |
| 24% | ($95,375, $182,100] |
| 32% | Over $182,100 |
You will need to use these numbers throughout this question.
A few notes:
The notation means "greater than and less than or equal to ". For example, someone with a taxable income of $44,725 is in the 12% bracket, but someone with a taxable income of $44,725.01 is in the 22% bracket.
"Taxable income" refers to the part of one's income that is taxable; in the US there is what's known as a "standard deduction" which can be thought of as a discount on your taxes. In this question, we won't worry about deductions.
If someone has a taxable income of $75,000, we say they are in the 22% tax bracket. However, such an individual doesn't owe 22% of $75,000 in taxes. Instead, they owe:
10% of $11,000, plus
12% of $33,725 (which is $44,725 - $11,000), plus
22% of $30,275 (which is $75,000 - $44,725).
More concretely, their tax owed is $$0.1 \cdot \$11{,}000 + 0.12 \cdot \$33{,}725 + 0.22 \cdot \$30{,}275$$
which adds up to $11,807.50. This makes their effective tax rate , or 15.743%, because they wound up having to pay 15.743% of their taxable income in taxes.
If you want to read more about the US federal income tax system, click here.
Question 2.1. Complete the implementation of the function tax_bracket, which takes in a taxable income (income) and returns the tax bracket it is in, as a proportion. For instance, tax_bracket(75000) should evaluate to 0.22 and tax_bracket(402150) should evaluate to 0.32.
For your convenience, the tax bracket table is shown again below.
| Tax Bracket | Taxable Income |
|---|---|
| 10% | [$0, $11,000] |
| 12% | ($11,000, $44,725] |
| 22% | ($44,725, $95,375] |
| 24% | ($95,375, $182,100] |
| 32% | Over $182,100 |
Hint: Use elif.
Question 2.2. Complete the implementation of the function tax_owed, which takes in a taxable income (income) and returns the amount of tax owed by an individual with that taxable income. For instance, tax_owed(75000) should evaluate to 11807.50 as in the example at the start of Question 2.
For your convenience, the tax bracket table is shown again below.
| Tax Bracket | Taxable Income |
|---|---|
| 10% | [$0, $11,000] |
| 12% | ($11,000, $44,725] |
| 22% | ($44,725, $95,375] |
| 24% | ($95,375, $182,100] |
| 32% | Over $182,100 |
Hint: Below are some suggested steps to break down this problem into manageable chunks.
Make sure you understand the example calculating the tax owed on $75,000 of taxable income.
Once you're sure you understand the given example calculation, then calculate the tax owed for a different taxable income, say $152,000, by hand.
In a blank cell in your notebook, write Python code to compute the tax owed for $152,000 in taxable income and verify you get the same answer as when you did it by hand.
Once you have that working, write the more general function and test it on a variety of inputs.
Question 2.3. Finally, complete the implementation of effective_tax_rate, which takes in a taxable income (income) and returns the effective tax rate for an individual with that taxable income, as a percentage. For instance, effective_tax_rate(75000) should evaluate to 15.743 because someone who earns $75,000 of taxable income winds up having to pay 15.742% of that amount in taxes.
If income is less than or equal to 0, effective_tax_rate should evaluate to 0.
Hint: You should use your tax_owed function. Our entire solution is only four lines, and two of them are for handling the case when income <= 0.
Fun Demo
You just did all of the math necessary to build an income tax calculator, like the ones you see on many financial websites. Run the following cell once you've completed the rest of this question!
3. Trick or Treat 🍭🎃
Halloween is fast approaching, and candy sales are skyrocketing as people stock up for trick-or-treating. For this question, we'll be exploring some data on the most popular Halloween candies in each state, from this article.
Run the cell below to see a fun interactive data visualization from the same article.
In the states DataFrame above, each state's 'Most_Popular_Candy' is recorded, based on candy sales in that state. 'Pounds_Of_Candy' refers to the total pounds of that specific candy sold in that state.
The states DataFrame does not contain any information about the candies themselves, e.g. which candies are chocolate and which candies are fruity. For this information, we can refer to a dataset curated by FiveThirtyEight for their article The Ultimate Halloween Candy Power Ranking. Run the cell below to load in a dataset containing information about many varieties of candy and save it as a DataFrame named varieties.
Note: The column in varieties that contains the names of the candies is 'competitorname', because these candies were all competing against each other in FiveThirtyEight's Halloween Candy Power Ranking.
Question 3.1. Using the merge method, combine the states and varieties DataFrames, and assign the resulting DataFrame to the variable states_and_varieties.
states_and_varietiesshould contain all of the columns in bothstatesandvarieties, minus the'competitorname'column fromvarieties, which is redundant with the'Most_Popular_Candy'column fromstates.Sort
states_and_varietiesby'State'in ascending order.Reset the index of
states_and_varietiesso that it is0,1,2, and so on. Make sure the old index isn't also included as a column.
Question 3.2. If you completed Question 3.1 correctly, you'll notice that states_and_varieties has fewer rows than both states and varieties. This is because there are some candies that are in states and not in varieties, and other candies that are in varieties and not in states.
Below, assign states_not_varieties to the number of different candies that are in states and not in varieties. Similarly, assign varieties_not_states to the number of different candies that are in varieties and not in states.
Hint: There are two ways to find the number of unique values in a column.
Group by that column. On the resulting DataFrame, use
.shape[0].Use the
.unique()method on the Series corresponding to that column. Uselenon the resulting array.
You'll need to do this three times – once each for the columns that contain candy names in states, varieties, and states_and_varieties.
Now that we better understand how states_and_varieties came to be, let's use it to learn more about states' candy preferences.
The 0s and 1s in the columns 'chocolate', 'fruity', 'caramel', etc. can be interpreted as Boolean values. For instance, since the state 'CA' has a 1 in its 'chocolate' column, it means that Calfornia's most popular candy is a chocolate candy.
Question 3.3. Among just the states in states_and_varieties where the most popular candy is a chocolate, what proportion of these states have a most popular candy that is also caramel? Assign your answer to p_caramel_given_chocolate. It should be a decimal between 0 and 1.
4. Wordle 🟨 ⬛ 🟨 🟩 ⬛
Wordle is a word-guessing game that became extremely popular at the end of 2021. Players have six tries to guess a five-letter word. To play, you first enter a guess for the five-letter word. After you make your guess, each letter of your guess will be highlighted with a color-coded square as follows:
A black square ⬛ means that this letter is not in the word at all.
A yellow square 🟨 means that this letter is in the word, but in a different position.
A green square means 🟩 the letter is in the correct position.
In this question, you will replicate some of that behavior using Python dictionaries and for-loops. We'll make a simplifying assumption that's not present in the real Wordle game: the answer word will always have five different letters and every guess must also have five different letters.
We'll start by refreshing our memories on Python dictionaries. Dictionaries provide a convenient way to store data in key: value pairs. You worked with a dictionary in Homework 3, Question 3 (Ramen 🍜). Here's an example dictionary you also saw in Homework 3.
We can access the value corresponding to each key using bracket notation.
Here, 'DSC 30' is the key and 'Data Structures and Algorithms for Data Science' is the value.
Question 4.1 In the cell below, create a dictionary called colors that has the following three keys:
'green''yellow''black'
The values corresponding to each of these three keys should be the colored square emojis '🟩', '🟨' , and '⬛'.
Note: Emojis can be copy-pasted and included in strings, just like letters, numbers, and punctuation.
Great! Now, if we want to create strings that use the green, yellow, and black square emojis, we don't need to copy-paste the emojis themselves – we can just access them from the colors dictionary.
Recall, in Lecture 11, we introduced the accumulator pattern. In the coin-flipping example, we started an empty array, and added to it in each iteration of our for-loop.
We can also use the accumulator pattern for strings, and we will do just that in the next question. Here's an example:
In the above example, we started with an empty string, output. For each character of the string 'good boy', we added a single new character to output, depending on whether we saw 'g', 'b', 'y', or something else.
Question 4.2. Now, complete the implementation of the function emojify, which takes as input two five-letter strings, each having no repeated letters. The first input string, guess, should be cross-checked with each letter of the second input string, answer. The function should return a new string, formed entirely of emojis from the colors dictionary, that indicates the accuracy of each letter in the guess, as described in the rules above. For example, let's say the answer string is "shark". Here is how the function would work on various example guesses.
Note: As we did in the preceeding example, use the dictionary colors to access the emojis – don't actually write any emojis within your function
Hints:
Look at the slide titled Ranges in Lecture 11 for guidance.
You'll need to use the
inkeyword, which we also introduced in Lecture 11.
Fun Demo
Run the cell below to produce a text box (don't worry about the code itself). Type a five-letter word with no repeated letters in the text box and play Wordle with a secret answer word. You can change your guess by backspacing and typing in a different guess. You can play more than once by running the cell again to generate a new answer word.
5. Alternating Means 🔁
In this problem, we'll define two functions that compute some sort of "alternating mean" of a sequence of values.
Question 5.1. Complete the implementation of the function alternating_mean, which takes in an array of numbers, values, and computes the mean of every other element in the original array starting with the element at position 0.
Note: Once you've written your function, you should test it out on several arrays yourself to make sure it works as intended.
Hint: Compute the mean by finding the sum of every other element in the array and dividing it by the number of elements that were summed.
Question 5.2. In math, the word "alternating" is sometimes used to indicate that the signs of the elements of a sequence oscillate back and forth between positive and negative. Complete the implementation of the function alternating_sign_mean, which takes in an array of positive numbers, values, and computes the mean of every element in the original array with alternating signs, starting with a positive sign for the element at position 0, a negative sign for the element at position 1, and so on. Example behavior is shown below.
Hint: If x is an integer, x % 2 evaluates to 0 when x is even and to 1 when x is odd. Use this to help you figure out the signs.
6. Lucky Triton Lotto 🔱 🎱
Suppose UCSD holds an annual lottery called the Lucky Triton Lotto, where students can enter to win Triton Cash, or even free housing! Here's how the Lucky Triton Lotto works:
First, you pick five different numbers, one at a time, from 1 to 62, representing the 62 years UCSD has been established. Let's say you select (15, 1, 13, 3, 61), in that order.
Then, you separately pick a number from 1 to 16. This is because 16 Nobel laureates have taught or are teaching at UCSD. Let's say you select 8.
The six numbers you have selected, or your numbers, can be represented all together as (15, 1, 13, 3, 61, 8). This is a sequence of six numbers - order matters!
The winning numbers are chosen by King Triton drawing five balls, one at a time, without replacement, from a pot of white balls numbered 1 to 62. Then, he draws a gold ball, the Tritonball, from a pot of gold balls numbered 1 to 16. Both pots are completely separate, hence the different ball colors. For example, maybe the winning numbers are (13, 15, 62, 3, 5, 8).
We’ll assume for this problem that in order to win the grand prize (free housing), all six of your numbers need to match the winning numbers and be in the exact same positions. In other words, your entire sequence of numbers must be exactly the same as the sequence of winning numbers. However, if some numbers in your sequence match up with the corresponding number in the winning sequence, you will still win some Triton Cash.
Suppose again that you select (15, 1, 13, 3, 61, 8) and the winning numbers are (13, 15, 62, 3, 5, 8). In this case, two of your numbers are considered to match two of the winning numbers.
Your numbers: (15, 1, 13, 3, 61, 8)
Winning numbers: (13, 15, 62, 3, 5, 8)
You won't win free housing, but you will win some Triton Cash. Note that although both sequences include the number 15 within the first five numbers (representing a white ball), since they are in different positions, that's not considered a match.
Question 6.1. What is the probability that your Tritonball number (the last number in your sequence) matches the winning Tritonball number? Calculate your answer and assign it to tritonball_chance. If you need to do any calculations (e.g. multiplication or division), make Python do it; don't use a separate calculator. Your result should be a decimal number between 0 and 1.
Question 6.2. What is the probability that your first two numbers match the first two winning numbers? Calculate your answer and assign it to first_two_chance. If you need to do any calculations (e.g. multiplication or division), make Python do it; don't use a separate calculator. Your result should be a decimal number between 0 and 1.
Hint: You need both of the first two numbers to match. What probability rule should you use?
Question 6.3. What is the probability that you win the grand prize, free housing? Calculate your answer and assign it to free_housing_chance. If you need to do any calculations (e.g. multiplication or division), make Python do it; don't use a separate calculator. Your result should be a decimal number between 0 and 1.
Hint: When you select a ball without replacement, what happens to the total number of balls you can select next time?
Question 6.4. What is the probability that you do not win free housing? Calculate your answer and assign it to no_free_housing_chance. If you need to do any calculations (e.g. multiplication or division), make Python do it; don't use a separate calculator. Your result should be a decimal number between 0 and 1.
Finish Line 🏁
To submit your assignment:
Select
Kernel -> Restart & Run Allto ensure that you have executed all cells, including the test cells.Read through the notebook to make sure everything is fine and all tests passed.
Run the cell below to run all tests, and make sure that they all pass.
Download your notebook using
File -> Download as -> Notebook (.ipynb), then upload your notebook to Gradescope.