scales-guides.qmd

# Scales and guides {#sec-scales-guides}

```{r}
#| echo: false
#| message: false
#| results: asis
source("common.R")
status("drafting")
```

```{=html}
<!-- DN: might be worth adding a section summarising how lims/labs/guides
span the same space as scale_*(name = , limits = , guide =), and highlighting
how scale_* has extra scale stuff, and the helpers have extra theme stuff ??? -->
```
```{r}
#| echo: false
toy <- data.frame(
  const = 1, 
  up = 1:4,
  txt = letters[1:4], 
  big = (1:4)*1000,
  log = c(2, 5, 10, 2000)
)
```

The scales toolbox in @sec-scale-position to @sec-scale-other provides extensive guidance for how to work with scales, focusing on solving common data visualisation problems.
The practical goals of the toolbox mean that topics are introduced when they are most relevant: for example, scale transformations are discussed in relation to continuous position scales (@sec-scale-transformation) because that is the most common situation in which you might want to transform a scale.
However, because ggplot2 aims to provide a grammar of graphics, there is nothing preventing you from transforming other kinds of scales (see @sec-scale-transformation-extras).
This chapter aims to illustrate these concepts: We'll discuss the theory underpinning scales and guides, and give examples showing how concepts that we've discussed specifically for position or colour scales also apply elsewhere.

## Theory of scales and guides {#sec-scales-guides-theory}

Formally, each scale is a function from a region in data space (the domain of the scale) to a region in aesthetic space (the range of the scale).
The axis or legend is the inverse function, known as the guide: it allows you to convert visual properties back to data.
You might find it surprising that axes and legends are the same type of thing, but while they look very different, they have the same purpose: to allow you to read observations from the plot and map them back to their original values.
The commonalities between the two are illustrated below:

| Argument name | Axis              | Legend    |
|:--------------|:------------------|:----------|
| `name`        | Label             | Title     |
| `breaks`      | Ticks & grid line | Key       |
| `labels`      | Tick label        | Key label |

```{r}
#| label: guides
#| echo: false
#| out.width: 100%
#| fig.cap: Common components of axes and legends
knitr::include_graphics("diagrams/scale-guides.png", dpi = 300, auto_pdf = TRUE)
```

However, legends are more complicated than axes, and consequently there are a number of topics that are specific to legends:

1.  A legend can display multiple aesthetics (e.g. colour and shape), from multiple layers (@sec-sub-layers-legends), and the symbol displayed in a legend varies based on the geom used in the layer (@sec-legend-glyph)

2.  Axes always appear in the same place.
    Legends can appear in different places, so you need some global way of positioning them.

    (1) 

3.  Legends have more details that can be tweaked: should they be displayed vertically or horizontally?
    How many columns?
    How big should the keys be?
    This is discussed in (@sec-scale-guide)

### Scale specification {#sec-scale-usage}

An important property of ggplot2 is the principle that every aesthetic in your plot is associated with exactly one scale.
For instance, when you write this

```{r}
#| label: default-scales
#| fig.show: hide
ggplot(mpg, aes(displ, hwy)) + 
  geom_point(aes(colour = class))
```

ggplot2 adds a default scale for each aesthetic used in the plot:

```{r}
#| fig.show: hide
ggplot(mpg, aes(displ, hwy)) + 
  geom_point(aes(colour = class)) +
  scale_x_continuous() + 
  scale_y_continuous() + 
  scale_colour_discrete()
```

The choice of default scale depends on the aesthetic and the variable type.
In this example `hwy` is a continuous variable mapped to the y aesthetic so the default scale is `scale_y_continuous()`; similarly `class` is discrete so when mapped to the colour aesthetic the default scale becomes `scale_colour_discrete()`.
Specifying these defaults would be tedious so ggplot2 does it for you.
But if you want to override the defaults, you'll need to add the scale yourself, like this: \index{Scales!defaults}

```{r}
#| fig.show: hide
ggplot(mpg, aes(displ, hwy)) + 
  geom_point(aes(colour = class)) + 
  scale_x_continuous(name = "A really awesome x axis label") +
  scale_y_continuous(name = "An amazingly great y axis label")
```

In practice you would typically use `labs()` for this, discussed in @sec-titles, but it is conceptually helpful to understand that axis labels and legend titles are both examples of scale names: see @sec-scale-names.

The use of `+` to "add" scales to a plot is a little misleading because if you supply two scales for the same aesthetic, the last scale takes precedence.
In other words, when you `+` a scale, you're not actually adding it to the plot, but overriding the existing scale.
This means that the following two specifications are equivalent: \indexc{+}

```{r}
#| label: multiple-scales
#| fig.show: hide
#| layout-ncol: 2
#| fig-width: 4
ggplot(mpg, aes(displ, hwy)) + 
  geom_point() + 
  scale_x_continuous(name = "Label 1") +
  scale_x_continuous(name = "Label 2")

ggplot(mpg, aes(displ, hwy)) + 
  geom_point() + 
  scale_x_continuous(name = "Label 2")
```

Note the message when you add multiple scales for the same aesthetic, which makes it harder to accidentally overwrite an existing scale.
If you see this in your own code, you should make sure that you're only adding one scale to each aesthetic.

If you're making small tweaks to the scales, you might continue to use the default scales, supplying a few extra arguments.
If you want to make more radical changes you will override the default scales with alternatives:

```{r}
#| fig.show: hide
ggplot(mpg, aes(displ, hwy)) + 
  geom_point(aes(colour = class)) +
  scale_x_sqrt() + 
  scale_colour_brewer()
```

Here `scale_x_sqrt()` changes the scale for the x axis scale, and `scale_colour_brewer()` does the same for the colour scale.

### Naming scheme

The scale functions intended for users all follow a common naming scheme.
You've probably already figured out the scheme, but to be concrete, it's made up of three pieces separated by "\_":

1.  `scale`
2.  The name of the primary aesthetic (e.g., `colour`, `shape` or `x`)
3.  The name of the scale (e.g., `continuous`, `discrete`, `brewer`).

The naming structure is often helpful, but can sometimes be ambiguous.
For example, it is immediately clear that `scale_x_*()` functions apply to the x aesthetic, but it takes a little more thought to recognise that they also govern the behaviour of other aesthetics that describe a horizontal position (e.g., the `xmin`, `xmax`, and `xend` aesthetics).
Similarly, while the name `scale_colour_continuous()` clearly refers to the colour scale associated with a continuous variables, it is less obvious that `scale_colour_distiller()` is simply a different method for creating colour scales for continuous variables.

### Fundamental scale types

It is useful to note that internally all scale functions in ggplot2 belong to one of three fundamental types; continuous scales, discrete scales, and binned scales.
Each fundamental type is handled by one of three scale constructor functions; `continuous_scale()`, `discrete_scale()` and `binned_scale()`.
Although you should never need to call these constructor functions, they provide the organising structure for scales and it is useful to know about them.

<!-- ### Exercises -->

<!-- 1.  Simplify the following plot specifications to make them easier to  -->

<!--     understand. -->

<!--     ```{r, eval = FALSE} -->

<!--     ggplot(mpg, aes(displ)) +  -->

<!--       scale_y_continuous("Highway mpg") +  -->

<!--       scale_x_continuous() + -->

<!--       geom_point(aes(y = hwy)) -->

<!--     ggplot(mpg, aes(y = displ, x = class)) +  -->

<!--       scale_y_continuous("Displacement (l)") +  -->

<!--       scale_x_discrete("Car type") + -->

<!--       scale_x_discrete("Type of car") +  -->

<!--       scale_colour_discrete() +  -->

<!--       geom_point(aes(colour = drv)) +  -->

<!--       scale_colour_discrete("Drive\ntrain") -->

<!--     ``` -->

<!-- 1.  What happens if you pair a discrete variable with a continuous scale? -->

<!--     What happens if you pair a continuous variable with a discrete scale? -->

## Scale names {#sec-scale-names}

Extend discussion of `labs()` in @sec-titles.

## Scale breaks {#sec-scale-breaks}

Discussion of what unifies the concept of `breaks` across continuous, discrete and binned scales: they are specific data values at which the guide needs to display something.
Include additional detail about break functions.

## Scale limits {#sec-oob}

@sec-scales-guides-theory introduced the concept that a scale defines a mapping from the data space to the aesthetic space.
Scale limits are an extension of this idea: they dictate the **region** of the data space over which the mapping is defined.
At a theoretical level this region is defined differently depending on the fundamental scale type.
For continuous and binned scales, the data space is inherently continuous and one-dimensional, so the limits can be specified by two end points.
For discrete scales, however, the data space is unstructured and consists only of a set of categories: as such the limits for a discrete scale can only be specified by enumerating the set of categories over which the mapping is defined.

The toolbox chapters outline the common practical goals for specifying the limits: for position scales the limits are used to set the end points of the axis, for example.
This leads naturally to the question of what ggplot2 should do if the data set contains "out of bounds" values that fall outside the limits.

```{=html}
<!-- DJN: this is taken from the positions "oob" section. I think it makes sense to 
document the scales::oob functions for continuous and binned scales as part of the 
theory chapter. The only reason we needed to talk about oob functions in position
chapter is because of the common error with using lims() to zoom, yes? If it weren't 
for that it would be an esoteric theory thing. -->
```
```{=html}
<!-- DJN: give examples with different aesthetics, and show for binned as well as 
continuous. try to cover most of the oob_* functions. might be valuable to highlight 
oob_keep() for position scales, as a way of making limits behave like zoom -->
```
The default behaviour in ggplot2 is to convert out of bounds values to `NA`, the logic for this being that if a data value is not part of the mapped region, it should be treated as missing.
This can occasionally lead to unexpected behaviour, as illustrated in @sec-zooming-in.
You can override this default by setting `oob` argument of the scale, a function that is applied to all observations outside the scale limits.
The default is `scales::oob_censor()` which replaces any value outside the limits with `NA`.
Another option is `scales::oob_squish()` which squishes all values into the range.
An example using a fill scale is shown below:

```{r}
#| layout-ncol: 3
#| fig-width: 3
#| fig-height: 4
df <- data.frame(x = 1:6, y = 8:13)
base <- ggplot(df, aes(x, y)) + 
  geom_col(aes(fill = x)) +                    # bar chart
  geom_vline(xintercept = 3.5, colour = "red") # for visual clarity only

base
base + scale_fill_gradient(limits = c(1, 3))
base + scale_fill_gradient(limits = c(1, 3), oob = scales::squish)
```

On the left the default fill colours are shown, ranging from dark blue to light blue.
In the middle panel the scale limits for the fill aesthetic are reduced so that the values for the three rightmost bars are replace with `NA` and are mapped to a grey shade.
In some cases, this is desired behaviour but often it is not: the right panel addresses this by modifying the `oob` function appropriately.

## Scale guides {#sec-scale-guide}

Scale guides are more complex than scale names: where the `name` argument (and `labs()` ) takes text as input, the `guide` argument (and `guides()`) require a guide object created by a **guide function** such as `guide_colourbar()` and `guide_legend()`.
These arguments to these functions offer additional fine control over the guide.

The table below summarises the default guide functions associated with different scale types:

| Scale type                                         | Default guide type |
|:---------------------------------------------------|:-------------------|
| continuous scales for colour/fill aesthetics       | colourbar          |
| binned scales for colour/fill aesthetics           | coloursteps        |
| position scales (continuous, binned and discrete)  | axis               |
| discrete scales (except position scales)           | legend             |
| binned scales (except position/colour/fill scales) | bins               |

Each of these guide types has appeared earlier in the toolbox:

-   `guide_colourbar()` is discussed in @sec-guide-colourbar
-   `guide_coloursteps()` is discussed in @sec-guide-coloursteps
-   `guide_axis()` is discussed in @sec-guide-axis
-   `guide_legend()` is discussed in @sec-guide-legend
-   `guide_bins()` is discussed in @sec-guide-bins

In addition to the functionality discussed in those sections, the guide functions have many arguments that are equivalent to theme settings like text colour, size, font etc, but only apply to a single guide.
For information about those settings, see @sec-polishing.

New stuff: show examples where something other than the default guide is used...

## Scale transformation {#sec-scale-transformation-extras}

The most common use for scale transformations is to adjust a continuous position scale, as discussed in @sec-scale-transformation.
However, they can sometimes be helpful to when applied to other aesthetics.
Often this is purely a matter of visual emphasis.
An example of this for the Old Faithful density plot is shown below.
The linearly mapped scale on the left makes it easy to see the peaks of the distribution, whereas the transformed representation on the right makes it easier to see the regions of non-negligible density around those peaks: \index{Transformation!scales}

```{r}
#| layout-ncol: 2
#| fig-width: 4
#| fig-height: 4
base <- ggplot(faithfuld, aes(waiting, eruptions)) + 
  geom_raster(aes(fill = density)) + 
  scale_x_continuous(NULL, NULL, expand = c(0, 0)) +
  scale_y_continuous(NULL, NULL, expand = c(0, 0))
  
base
base + scale_fill_continuous(trans = "sqrt")
```

Transforming size aesthetics is also possible:

```{r}
#| layout-ncol: 2
#| fig-width: 4
#| fig-height: 4
df <- data.frame(x = runif(20), y = runif(20), z = sample(20))
base <- ggplot(df, aes(x, y, size = z)) + geom_point()

base 
base + scale_size(trans = "reverse")
```

In the plot on the left, the `z` value is naturally interpreted as a "weight": if each dot corresponds to a group, the `z` value might be the size of the group.
In the plot on the right, the size scale is reversed, and `z` is more naturally interpreted as a "distance" measure: distant entities are scaled to appear smaller in the plot.

````{=html}
<!-- DN: This was the beginning of the binned transformation section. Tempted to omit this entirely?

### Binned scales

```{r}
base <- ggplot(mpg, aes(hwy)) + geom_bar()

p1 <- base + scale_x_binned(breaks = seq(-50,50,10), limits = c(-50, 50))
p2 <- base + scale_x_binned(breaks = seq(-50,50,10), limits = c(-50, 50), trans = "reverse")
```

Binned scales can be transformed, much like continuous scales, but some care is required because the bins are constructed in the transformed space. In some cases this can produce undesirable outcomes. In the code below, we take a uniformly distributed variable and use `scale_x_binned()` and `geom_bar()` to construct a histogram of the logarithmically transformed data. 

```{r}
df <- data.frame(val = runif(10000, 1, 20000))
ggplot(df, aes(log10(val))) + geom_bar() + scale_x_binned()
```

In this example the transformation takes place in the data: the x aesthetic is mapped to the value of `log10(val)`, and no scale transformation is applied. The bins are evenly spaced on this logarithmic scale. Alternatively, you can specify the transformation by setting `trans = "log10"` in the scale function:

```{r}
ggplot(df, aes(val)) + geom_bar() + scale_x_binned(trans="log10") 
```

The unevenly spaced bins occur due to an interaction of two things: (1) binned scales use breaks to construct the bins, and (2) the default breaks for a transformed scale are specified by the transformation and are designed to look nice, but may not be good for binning data. The solution to this is to override the default breaks: 

```{r}
ggplot(df, aes(val)) + geom_bar() + 
  scale_x_binned(trans="log10", breaks = 3^(0:9))  
```

-->
````

## Legend merging and splitting {#sec-legend-merge-split}

There is always a one-to-one correspondence between position scales and axes.
But the connection between non-position scales and legend is more complex: one legend may need to draw symbols from multiple layers ("merging"), or one aesthetic may need multiple legends ("splitting").

### Merging legends {#sec-sub-layers-legends}

\index{Legend}

Merging legends occurs quite frequently when using ggplot2.
For example, if you've mapped colour to both points and lines, the keys will show both points and lines.
If you've mapped fill colour, you get a rectangle.
Note the way the legend varies in the plots below:

```{r}
#| label: legend-geom
#| echo: false
#| layout-ncol: 3
#| fig-width: 3
#| fig-height: 3
base <- ggplot(toy, aes(const, up, colour = txt))
base + geom_point()
base + geom_point() + geom_path(aes(group = 1))
base + geom_raster(aes(fill = txt))
```

By default, a layer will only appear if the corresponding aesthetic is mapped to a variable with `aes()`.
You can override whether or not a layer appears in the legend with `show.legend`: `FALSE` to prevent a layer from ever appearing in the legend; `TRUE` forces it to appear when it otherwise wouldn't.
Using `TRUE` can be useful in conjunction with the following trick to make points stand out:

```{r}
#| layout-ncol: 2
#| fig-width: 4
#| fig-height: 4
ggplot(toy, aes(up, up)) + 
  geom_point(size = 4, colour = "grey20") +
  geom_point(aes(colour = txt), size = 2) 

ggplot(toy, aes(up, up)) + 
  geom_point(size = 4, colour = "grey20", show.legend = TRUE) +
  geom_point(aes(colour = txt), size = 2) 
```

ggplot2 tries to use the fewest number of legends to accurately convey the aesthetics used in the plot.
It does this by combining legends where the same variable is mapped to different aesthetics.
The figure below shows how this works for points: if both colour and shape are mapped to the same variable, then only a single legend is necessary.
\index{Legend!merging}

```{r}
#| label: legend-merge
#| layout-ncol: 3
#| fig-width: 3
#| fig-height: 4
base <- ggplot(toy, aes(const, up)) +
  scale_x_continuous(NULL, breaks = NULL)
base + geom_point(aes(colour = txt))
base + geom_point(aes(shape = txt))
base + geom_point(aes(shape = txt, colour = txt))
```

In order for legends to be merged, they must have the same `name`.
So if you change the name of one of the scales, you'll need to change it for all of them.
One way to do this is by using `labs()` helper function:

```{r}
#| layout-ncol: 3
#| fig-width: 3
#| fig-height: 4
base <- ggplot(toy, aes(const, up)) + 
  geom_point(aes(shape = txt, colour = txt)) + 
  scale_x_continuous(NULL, breaks = NULL)

base
base + labs(shape = "Split legend")
base + labs(shape = "Merged legend", colour = "Merged legend")
```

### Splitting legends

Splitting a legend is a much less common data visualisation task.
In general it is not advisable to map one aesthetic (e.g. colour) to multiple variables, and so by default ggplot2 does not allow you to "split" the colour aesthetic into multiple scales with separate legends.
Nevertheless, there are exceptions to this general rule, and it is possible to override this behaviour using the ggnewscale package [@ggnewscale].
The `ggnewscale::new_scale_colour()` command acts as an instruction to ggplot2 to initialise a new colour scale: scale and guide commands that appear above the `new_scale_colour()` command will be applied to the first colour scale, and commands that appear below are applied to the second colour scale.

To illustrate this the plot on the left uses `geom_point()` to display a large marker for each vehicle make in the `mpg` data, with a single colour scale that maps to the year.
On the right, a second `geom_point()` layer is overlaid on the plot using small markers: this layer is associated with a different colour scale, used to indicate whether the vehicle has a 4-cylinder engine.

```{r}
#| layout-ncol: 2
#| fig-width: 4
base <- ggplot(mpg, aes(displ, hwy)) + 
  geom_point(aes(colour = factor(year)), size = 5) + 
  scale_colour_brewer("year", type = "qual", palette = 5) 

base
base + 
  ggnewscale::new_scale_colour() + 
  geom_point(aes(colour = cyl == 4), size = 1, fill = NA) + 
  scale_colour_manual("4 cylinder", values = c("grey60", "black"))
```

Additional details, including functions that apply to other scale types, are available on the package website, <https://github.com/eliocamp/ggnewscale>.

## Legend key glyphs {#sec-legend-glyph}

In most cases the default glyphs shown in the legend key will be appropriate to the layer and the aesthetic.
Line plots of different colours will show up as lines of different colours in the legend, boxplots will appear as small boxplots in the legend, and so on.
Should you need to override this behaviour, the `key_glyph` argument can be used to associate a particular layer with a different kind of glyph.
For example:

```{r}
#| layout-ncol: 2
#| fig-width: 4
#| fig-height: 3
base <- ggplot(economics, aes(date, psavert, color = "savings"))

base + geom_line()
base + geom_line(key_glyph = "timeseries")
```

More precisely, each geom is associated with a function such as `draw_key_boxplot()` or `draw_key_path()` which is responsible for drawing the key when the legend is created.
You can pass the desired key drawing function directly: for example, `base + geom_line(key_glyph = draw_key_timeseries)` would also produce the plot shown above right.

<!-- ## Exercises -->

<!-- 1.  What are the three most important arguments that apply to both  -->

<!--     axes and legends? What do they do? Compare and contrast their -->

<!--     operation for axes vs. legends. -->