1
CptS 475/575: Data Science, Fall 2020
Assignment 3: Data Transformation and Tidying
Release Date: September 16, 2020 Due Date: September 23, 2020 (11:59 pm)
This assignment has two questions. You are encouraged to use R Markdown to generate your
report (in PDF).
Question 1. (60 pts total) For this question you will be using either the dplyr package from R or
the Pandas library in python to manipulate and clean up a dataset called msleep (mammals sleep)
that is available on the course webpage at
https://scads.eecs.wsu.edu/wp-content/uploads/2017/10/msleep_ggplot2.csv
The dataset contains the sleep times and weights for a set of mammals. It has 83 rows and 11
variables. Here is a description of the variables:
Name Description
name common name
genus taxonomic rank
vore carnivore, omnivore or herbivore
order taxonomic rank
conservation the conservation status of the mammal
sleep_total total amount of sleep, in hours
sleep_rem rem sleep, in hours
sleep_cycle length of sleep cycle, in hours
awake amount of time spent awake, in hours
brainwt brain weight in kilograms
bodywt body weight in kilograms
Load the data into R or Python, and check the first few rows for abnormalities. You will likely
notice several. All of the tasks in this assignment can be hand coded, but the goal is to use the
functions built into dplyr or Pandas to complete the tasks. Suggested functions for Python will be
shown in blue while suggested R functions are shown in red. Note: if you are using Python, be
sure to load the data as a Pandas DataFrame.
Below are the tasks to perform. Before you begin, print the first few values of the columns with a
header including “sleep”. (head(), head())
a) (10 pts) Count the number of animals which weigh under 1 kilogram and sleep more than 14
hours a day. (filter(), query())
b) (10 pts) Print the name, order, sleep time and bodyweight of the animals with the 6 longest
sleep times, in order of sleep time. (select(), arrange(), loc(), sort_values())
c) (10 pts) Add two new columns to the dataframe; wt_ratio with the ratio of brain size to body
weight, rem_ratio with the ratio of rem sleep to sleep time. If you think they might be useful,
feel free to extract more features than these, and describe what they are. (mutate(), assign())
2
d) (14 pts) Display the average, min and max sleep times for each order. (group_by(),
summarise(), groupby(), agg())
e) (16 pts) Impute the missing brain weights as the average wt_ratio for that animal’s order
times the animal’s weight. Make a second copy of your dataframe, but this time impute
missing brain weights with the average brain weight for that animal’s order. What
assumptions do these data filling methods make? Which is the best way to impute the data, or
do you see a better way, and why? You may impute or remove other variables as you find
appropriate. Briefly explain your decisions. (group_by(), mutate(), groupby(),assign())
Question 2. (40 pts total) For this question, you will first need to read section 12.6 in the R for
Data Science book, here (http://r4ds.had.co.nz/tidy-data.html#case-study). Grab the dataset from
the tidyr package (tidyr::who), and tidy it as shown in the case study before answering the
following questions. Note: if you are using pandas you can perform these same operations, just
replace the pivot_longer() function with melt() and the pivot_wider() function with pivot().
However, you may prefer to use R for this question, as the dataset is from an R package.
a) (5 pts) Explain why this line
> mutate(key = stringr::str_replace(key, "newrel", "new_rel"))
is necessary to properly tidy the data. What happens if you skip this line?
b) (5 pts) How many entries are removed from the dataset when you set values_drop_na to
true in the pivot_longer command (in this dataset)?
c) (5 pts) Explain the difference between an explicit and implicit missing value, in general.
Can you find any implicit missing values in this dataset, if so where?
d) (5 pts) Looking at the features (country, year, var, sex, age, cases) in the tidied data, are
they all appropriately typed? Are there any features you think would be better suited as a
different type? Why or why not?
e) (10 pts) Generate an informative visualization, which shows something about the data.
Give a brief description of what it shows, and why you thought it would be interesting to
investigate.
f) (10 pts) Suppose you have the following dataset called siteDemo:
Site U30.F U30_M O30.F O30.M
facebook 30 35 66 58
myspace 1 2 3 6
snapchat 6 5 3 2
twitter 18 23 12 28
tiktok 44 60 2 7

You know that the U30.F column is the number of female users under 30 on the site,
O30.M denotes the number of male users 30 or older on the site, etc. Construct this table,
and show the code you would use to tidy this dataset (using gather()/pivot_longer() and
separate()/pivot_wider() or melt() and pivot()) such that the columns are organized as:
Site, AgeGroup, Gender and Count.