```{r setup, include=FALSE}
options(htmltools.dir.version = FALSE)
```

# Solution to the test case

The dataset `brfss_iowa.csv` (linked from website) contains 6227 records from the Behavioral Risk Factor Surveillance System (BRFSS) for Iowans.

> The Behavioral Risk Factor Surveillance System (BRFSS) is the nation's premier system of health-related telephone surveys that collect state data about U.S. residents regarding their health-related risk behaviors.

It is conducted annually by the Center for Disease Control and Prevention (CDC).

Codebook with detailed explanations of variables is [available here](https://www.cdc.gov/brfss/annual_data/2015/pdf/codebook15_llcp.pdf).

## The Problem

0.  Complete the following tasks and **write instructions / documentation** for a collaborator to be able to reproduce your findings.

1.  Verify that there are 6227 cases (= number of interviews) in the data.

2.  Verify that there are variables `WEIGHT2` and `HEIGHT3` in the data and read the description of the variables in the codebook.

3.  How are height and weight related? Find correlations of weight and height by gender (`SEX`). How many values are the correlations based on for each gender?

4.  Write a short report of your findings. Address potential problems in the data.

### Read the data and check dimensions

```{r message = FALSE}
library(tidyverse)
iowa <- readr::read_csv("https://github.com/ds202-at-ISU/materials/raw/master/01_collaborative-environment/data/brfss_mn.csv") 
#iowa <- read_csv("~/Documents/Data/BRFSS/brfss-iowa-2020.csv")
# the read_csv function from the readr package is faster than the standard read.csv
# but the output is a tibble (we'll come back to that)

dim(iowa)
# Yes, there are 6227 observations (rows) in the data set and a lot of columns/variables
head(iowa)
```

### Variables `HEIGHT3` and `WEIGHT2`

```{r, message = FALSE}
library(tidyverse)
iowa %>% select(HEIGHT3, WEIGHT2) %>% head()
```

Variables are there ... but ... some data values look odd.

Let's follow up by having a look at the Codebook.

From the [codebook](https://www.cdc.gov/brfss/annual_data/2015/pdf/codebook15_llcp.pdf): HEIGHT3 and WEIGHT2 are the originally recorded variables.

<img src="https://github.com/ds202-at-ISU/materials/raw/master/01_collaborative-environment/images/codebook-height3.png" width="750"/>

Let's plot that ...

Load the `ggplot2` package to plot weight versus height:

```{r message = FALSE, fig.width=3.5, fig.height = 3.5, warning = FALSE}
library(ggplot2)
library(tidyverse)
iowa %>% 
  ggplot(aes(x = WEIGHT2, y = HEIGHT3)) + geom_point()
```

Height and weight should show a somewhat positive correlation. These clusters are an indication of the data coding.

### Next steps

Obviously, we need to spend some time cleaning these variables before we can make use of them.

We have a couple of choices:

1.  We can buckle down and do that clean-up and then move on to the calculation.

2.  We can take another look at the variables and investigate what `HTIN4`, `HTM4`, and `WTKG3` are ...

## Route 2

Let's take the second route first and draw a picture of height and weight in metric units:

```{r message = FALSE, warning = FALSE, fig.width = 5.5, fig.height = 2.5}
iowa %>% 
  ggplot(aes(x = HTM4, y = WTKG3)) + geom_point(alpha = 0.2) + 
  facet_grid(.~SEX) +
  geom_smooth()
```

These charts look much better! We see a general increase in weight as height increases. The variability in weight is huge, though. On average, women (`SEX = 2`) are shorter and lighter.

### Correlations of height and weight by gender (Route 2)

Based on variables `HTM4` and `WTKG3`

```{r}
iowa %>% 
  group_by(SEX) %>%
  summarize(
    n = n(),
    cor = cor(HTM4, WTKG3, use="pairwise.complete"),
    non_missing = sum(!is.na(HTM4) & !is.na(WTKG3)),
    non_missing_perc = non_missing/n*100
  )

```

## Route 1

This is a bit more work.

```{r}
# first of all, replace all missing values by NAs
iowa <- iowa %>% mutate(
  HEIGHT3 = replace(HEIGHT3, HEIGHT3 %in% c(7777,9999), NA),
  WEIGHT2 = replace(WEIGHT2, WEIGHT2 %in% c(7777,9999), NA)
)
```

Let's take on height first:

```{r}
iowa <- iowa %>% mutate(
  feet = HEIGHT3 %/% 100, # feet and inch only make sense for standard values,
  inch = HEIGHT3 %% 100,  # but we can calculated them for all
  height = ifelse(HEIGHT3 >= 9000, 
                  HEIGHT3 - 9000, # transform all metric values by subtracting 9000
                  feet *30.48 + 2.54*inch # add feet and inch converted to centimeter
                  )
)
```

### Now plot!

```{r, fig.height = 5}
iowa %>% filter(!is.na(height)) %>%
  ggplot(aes(x = height)) + geom_histogram(binwidth = 2.5) +
  facet_grid(SEX~.)
```

We get similar findings - for height at least: women are on average shorter than men.

Now we do the same thing for weight as we did for height before: 1 pound is 0.454 kg

```{r}
iowa <- iowa %>% mutate(
  weight = ifelse(WEIGHT2 >= 9000, 
                  WEIGHT2 - 9000, # transform all metric values by subtracting 9000
                  WEIGHT2*.454 # 1 lbs = 0.454 kg
                  )
)
```

### Correlations of weight and height by gender (Route 1)

```{r warning = FALSE}
iowa %>% 
  group_by(SEX) %>%
  summarize(
    n = n(),
    cor = cor(height, weight, use="pairwise.complete"),
    non_missing = sum(!is.na(height) & !is.na(weight)),
    non_missing_perc = non_missing/n*100
  )
```

## Recap 1

-   Route 1 and 2 give identical solutions

-   Read the codebook, visualize the data, even if the question does not ask for it.

## Recap 2

1.  the solution is disseminated from an R Markdown document, located at [a github repo](https://github.com/Stat585-at-ISU/materials-2019/tree/master/01_collaborative-environment).
2.  code and text/documentation are interwoven: reproducible and self-documenting.
3.  extend or refine analyses by copying and modifying code blocks.
4.  disseminate your work by sharing the RMarkdown file
