04-intro-to-data-frames.Rmd

---
title: "Introduction to Data Frames"
subtitle: "Stat 133"
author: "Gaston Sanchez"
output: github_document
fontsize: 11pt
urlcolor: blue
---

> ### Learning Objectives
>
> - Understand Data Frames
> - Basic Manipulation with brackets `[ , ]`

------

## Manipulating Data Frames

The most common format/structure for a data set is a tabular format: 
with row and columns (like a spreadsheet). When your data is in this shape, 
most of the time you will work with R __data frames__ (or similar rectangular 
structures like a `"matrix"`, `"table"`, etc).

Learning how to manipulate data tables is among the most important
_data computing_ basic skills. The traditional way of manipulating data frames 
in R is based on bracket notation, e.g. `dat[ , ]`, to select specific 
rows, columns, or cells. Also, the use of the dollar `$` operator to handle
columns is fundamental. In this part of the lab, you will practice a wide
array of data wrangling tasks with the so-called bracket notation, and the 
dollar operator.

I should say that there are alternative ways for manipulating tables in R.
Among the most recent paradigms, there is the __plying__ framework devised
by Hadley Wickham. From his doctoral research, the first _plyr_ tools were 
available in the packages `"plyr"` and `"reshape"`. Nowadays we have the 
`"reshape2"`package, and the extremely popular package `"dplyr"` 
(among other packages). You will have time to learn more about `"dplyr"` in the 
next weeks. In the meantime, take some time to understand more about the 
bracket notation.



## R Data Frames

A data frame is a special type of R list, in which each column is an R vector 
(or a factor).

When working with data frames, you should always spend some time inspecting 
the contents, and checking how R is handling the data types. It is in these 
early stages of data exploration that you can catch potential issues in order 
to avoid disastrous consequences or bugs in subsequent stages.

What `str()` returns is a display 
of the dimensions of the data frame, and then a list with the name of all the 
variables, and their data types (e.g. `chr` character, `num` real, etc). 
The argument `vec.len = 1` indicates that just the first element in each 
column should be displayed.



## Creating data frames

Most of the (raw) data tables you will be working with will already be in 
some data file. However, from time to time you will face the need of creating 
some sort of data table in R. In these situations, you will likely have to 
create such table with a data frame. So let's look at various ways to 
"manually"" create a data frame.

__Option 1__: The primary option to build a data frame is with `data.frame()`. 
You pass a series of vectors (or factors), of the same length, separated by commas.
Each vector (or factor) will become a column in the generated data frame. 
Preferably, give names to each column, like `col1`, `col2`, and `col3`, in the 
example below:

```{r create_data_frame1}
# creating a basic data frame
my_table1 <- data.frame(
  col1 = LETTERS[1:5],
  col2 = seq(from = 10, to = 50, by = 10),
  col3 = c(TRUE, TRUE, FALSE, TRUE, FALSE)
)

my_table1
```


__Option 2__: Another way to create data frames is with a `list` containing 
vectors or factors (of the same length), which then you convert to a data.frame 
with `data.frame()`: 

```{r create_data_frame2}
# another way to create a basic data frame
my_list <- list(
  col1 = LETTERS[1:5],
  col2 = seq(from = 10, to = 50, by = 10),
  col3 = c(TRUE, TRUE, FALSE, TRUE, FALSE)
)

my_table2 <- data.frame(my_list)

my_table2
```

Remember that a `data.frame` is nothing more than a `list`. So as long as the 
elements in the list (vectors or factors) are of the same length, we can simply
convert the list into a data frame.

By default, `data.frame()` converts character vectors into factors. You can 
check that by exmining the structure of the data frame with `str()`:

```{r}
str(my_table2)
```

To prevent `data.frame()` from converting strings into factors, you must use
the argument `stringsAsFactors = FALSE`

```{r}
# strings as strings (not as factors)
my_table3 <- data.frame(
  col1 = LETTERS[1:5],
  col2 = seq(from = 10, to = 50, by = 10),
  col3 = c(TRUE, TRUE, FALSE, TRUE, FALSE),
  stringsAsFactors = FALSE
)

str(my_table3)
```



## Basic Operations with Data Frames

Now that you have seen some ways to create data frames, let's discuss a number
of basic manipulations of data frames. I will show you some examples and then
you'll have the chance to put in practice the following operations:

- Selecting table elements:
    + select a given cell
    + select a set of cells
    + select a given row
    + select a set of rows
    + select a given column
    + select a set of columns
- Adding a new column
- Deleting a new column
- Renaming a column
- Moving a column
- Transforming a column


```{r echo = FALSE}
tbl <- data.frame(
  player = c('Thompson', 'Curry', 'Green', 'Durant', 'Pachulia'),
  position = c('SG', 'PG', 'PF', 'SF', 'C'),
  salary = c(16663575, 12112359, 15330435, 26540100, 2898000),
  points = c(1742, 1999, 776, 1555, 426),
  ppg = c(22.3, 25.3, 10.2, 25.1, 6.1),
  rookie = rep(FALSE, 5),
  stringsAsFactors = FALSE
)
```

Let's say you have a data frame `tbl` with the lineup of the Golden State Warriors:

```{r, echo = FALSE, comment = ""}
tbl
```


### Selecting elements

The data frame `tbl` is a 2-dimensional object: the 1st dimension corresponds
to the rows, while the 2nd dimension corresponds to the columns.
Because `tbl` has two dimensions, the bracket notation involves 
working with the data frame in this form: `tbl[ , ]`.
In other words, you have to specify values inside the 
brackets for the 1st index, and the 2nd index: `tbl[index1, index2]`.

```{r}
# select value in row 1 and column 1
tbl[1,1]

# select value in row 2 and column 5
tbl[2,5]

# select values in these cells
tbl[1:3,3:5]
```


If no value is specified for `index1` then all rows are included. Likewise,
if no value is specified for `index2` then all columns are included.

```{r}
# selecting first row
tbl[1, ]

# selecting third row
tbl[3, ]

# selecting second column
tbl[ ,2]

# selecting columns 3 to 5
tbl[ ,3:5]
```


### Adding a column

Perhaps the simplest way to add a column is with the dollar operator `$`.
You just need to give a name for the new column, and assign a vector (or factor):

```{r}
# adding a column
tbl$new_column <- c('a', 'e', 'i', 'o', 'u')
tbl
```

Another way to add a column is with the _column binding_ function `cbind()`:

```{r}
# vector of weights
weight <- c(215, 190, 230, 240, 270)

# adding weights to tbl
tbl <- cbind(tbl, weight)
tbl
```


### Deleting a column

The inverse operation of adding a column consists of __deleting__ a column.
This is possible with the `$` dollar operator. For instance, say you want to 
remove the column `new_column`. Use the `$` operator to select this column,
and assign it the value `NULL` (think of this as _NULLifying_ a column):

```{r}
# deleting a column
tbl$new_column <- NULL
tbl
```


### Renaming a column

What if you want to rename a column? There are various options to do this.
One way is by changing the column`names` attribute:

```{r}
# attributes
attributes(tbl)
```

which is more commonly accessed with the `names()` function:

```{r}
# column names
names(tbl)
```

Notice that `tbl` has a list of attributes. The element `names` is the vector
of column names.

You can directly modify the vector of `names`; for example let's change
`rookie` to `rooky`:

```{r}
# changing rookie to rooky
attributes(tbl)$names[6] <- "rooky"

# display column names
names(tbl)
```

By the way: this way of changing the name of a variable is very low level, and probably
unfamiliar to most useRs.


### Moving a column

A more challenging operation is when you want to move a column to a different 
position. What if you want to move `salary` to the last position (last column)?
One option is to create a vector of column names in the desired order, and then
use this vector (for the index of columns) to reassign the data frame like this:

```{r}
reordered_names <- c("player", "position", "points", "ppg", "rooky", "weight", "salary")

# moving salary at the end
tbl <- tbl[ ,reordered_names]
tbl
```


### Transforming a column

A more common operation than deleting or moving a column, is to transform the 
values in a column. This can be easily accomplished with the `$` operator.
For instance, let's say that we want to transform `salary` from dollars to
millions of dollars:

```{r}
# converting salary in millions of dollars
tbl$salary <- tbl$salary / 1000000
tbl
```

Likewise, instead of using the `$` operator, you can refer to the column using
bracket notation. Here's how to transform weight from pounds to kilograms
(1 pound = 0.453592 kilograms):

```{r}
# weight in kilograms
tbl[ ,"weight"] <- tbl[ ,"weight"] * 0.453592
tbl
```

There is also the `transform()` function which transform values _interactively_,
that is, temporarily:

```{r}
# transform weight to inches
transform(tbl, weight = weight / 0.453592)
```

`transform()` does its job of modifying the values of `weight` but only 
temporarily; if you inspect `tbl` you'll see what this means:

```{r}
# did weight really change?
tbl
```

To make the changes permanent with `transform()`, you need to reassign them
to the data frame:

```{r}
# transform weight to inches (permanently)
tbl <- transform(tbl, weight = weight / 0.453592)
tbl
```