numeric value format for condition

[StefanMunnes]

Description

I have noticed that numeric values for mc or mc_multi variables are converted to strings and are also joined with a "," for multiple answers. At least for the last case I understand that this might be necessary for export to the database, but for internal reference it makes things more difficult:

SURVEY.R

sd_question(
  type = "mc_multi",
  id = "kind_transport",
  label = "What kind of transport you own?",
  option = c(
    "Bike" = 1,
    "Scooter" = 2,
    "Skateboard" = 3,
    "Car" == 4,
    "Boat" == 5
  )

sd_question(
  type = "mc",
  id = "buy_preference",
  label = "How interessted are you in buying a new transport?",
  option = c(
    "Very much" = 1,
    "Much" = 2,
    "Some" = 3,
    "Not much" = 4,
    "Not very much" = 5
  )
)

APP.R

sd_show_if(
  any(input$kind_transport) %in% c(1, 4) ~ "buy_preference",
  length(input$kind_transport) <= 2 ~ "buy_preference,
  input$buy_preference<= 3 ~ "what_to_buy",
)

These are some example where it could be handy to reference directly to the numeric values, provided as option values of the question.

And I would also mention, that it would be handy if the numeric values would be stored as numbers in the database. Specially for scales but also factor variables, we would transform them back anyway.

[jhelvy]

Great examples here. I'll separate these examples though as they are actually asking two different things.

1: Can we encode numbers as numbers and not strings?

We've encoded all question values as strings as it's just simply easier to work with on the backend for a whole lot of reasons. This is also the behavior for most shiny widgets (e.g. see this SO issue). I also feel like if we set this as a standard, then it's easier for users to know what to expect when writing their surveys. For example, in the example above, you can just wrap the input in as.numeric():

sd_show_if(
  as.numeric(input$buy_preference) <= 3 ~ "what_to_buy"
)

So we probably won't make this change and we'll keep everything as strings.

2: How should we use multi-option responses in show_if conditions?

This is a bigger challenge, as the way we've written things is to store the results inside a single concatenated string. For example, if a user chose "Car" and "Boat" in your kind_transport example question above, then this would be stored in the data under the kind_transport column as "4, 5". We do this primarily so the resulting data is easier to store.

Some things aren't too bad with this structure. Like in your example, we could write the condition like this:

input$kind_transport == "1, 4" ~ "buy_preference"

Or you could write a simple function to do the more complex work of converting the value back into a vector and then using the vector, something like this:

transport_less_than_two <- function(input) {
  x <- as.numeric(strsplit(input$kind_transport, ", ")[[1]])
  return(length(x) <= 2)
}

sd_show_if(
  transport_less_than_two(input) ~ "buy_preference"
)

This is certainly more work, but it's a pretty complex condition, and I don't know how else to specify it.

The only alternative I can think of for is to split the data into multiple columns for mc_multi type questions, similar to how we do with matrix type questions. In your example, instead of have just one kind_transport column, we would have 5 columns: kind_transport_1, kind_transport_2, kind_transport_3, kind_transport_4, kind_transport_5. I'm not a big fan of this design though. It also requires that we choose a value to store in these columns to indicate selection, like 1 and 0 or maybe TRUE and FALSE. With this design, the same condition would look something like this:

input$kind_transport_1 & input$kind_transport_4 ~ "buy_preference"

[StefanMunnes]

I know you are the developers and have a much better clue on what is going on under the hood. I just can give you my impression as a (future power) user.

1. That numeric values will be transformed to strings wasn't clear for me and should at least be written somewhere in the documentation and ROXYGEN help. Using the as.numeric() function is a way of dealing with the simple conditions.

2. General, I'm wondering if we can't separate between using the input values on the server and how to store the data? The problem with your first solution (input$kind_transport == "1, 4" ~ "buy_preference") is, that it doesn't have the flexibility like the %in% approach, which also is TRUE if it just matches one of the values. I think there is no way out, then have a little helper function that brings the collapsed values back to a (numeric) vector:

sd_split_multi <- function(input, numeric = TRUE) {
  str_vec <- strsplit(input, ",")[[1]]
  
  str_vec <- trimws(str_vec)
  
  if (all(grepl("^[0-9]+$", str_vec)) && numeric){
    return(as.numeric(str_vec))
  } else {
    return(str_vec)
  }  
}

3. I think storing the values in the way it is solved right now is feasible, but if you want to work with the data, you would usually split it up the way you said, a dummy variable (0/1) for every option. So maybe the comparison to the matrix question is correct and the "sub-questions" should also be more treated like row variables. But this would, again, make things more complicated when using in the condition functions.

One last point: From a user perspective, a lot of helper functions make things more confusing and the problems I came up with, result from my intuitive way to try to use and solve a survey logic.

[StefanMunnes]

Seeing this suggestion from @warnes, I want to add, that you can use this function also to just transform a single value to numeric and if you rename it to a more clear name, than it could be used more general (also instead of as.numeric())

And then I'm wondering if there is no way to implement this helper function in the backend of sd_show_if() function to re-transform all the given input (if possible).

[warnes]

2: How should we use multi-option responses in show_if conditions? This is a bigger challenge, as the way we've written things is to store the results inside a single concatenated string. For example, if a user chose "Car" and "Boat" in your kind_transport example question above, then this would be stored in the data under the kind_transport column as "4, 5". We do this primarily so the resulting data is easier to store. Some things aren't too bad with this structure. Like in your example, we could write the condition like this: input$kind_transport == "1, 4" ~ "buy_preference" Or you could write a simple function to do the more complex work of converting the value back into a vector

Perhaps it would be worthwhile for sd to provide utility functions to convert to/from the comma-delimited string representation. E.g., something along the lines of ```r str <- 'A, B, C' vals <- c('A','B','C') sd_str2vals <- function(str) { strsplit(str, ',', fixed = TRUE) |> unlist() } sd_vals2str <- function(vals) { paste(vals, collapse=', ') } sd_str2vals(str) sd_vals2str(vals) ```

…

and then using the vector, something like this: transport_less_than_two <- function(input) { x <- as.numeric(strsplit(input$kind_transport, ", ")[[1]]) return(length(x) <= 2) } sd_show_if( transport_less_than_two() ~ "buy_preference" ) This is certainly more work, but it's a pretty complex condition, and I don't know how else to specify it. The only alternative I can think of for is to split the data into multiple columns for mc_multi type questions, similar to how we do with matrix type questions. In your example, instead of have just one kind_transport column, we would have 5 columns: kind_transport_1, kind_transport_2, kind_transport_3, kind_transport_4, kind_transport_5. I'm not a big fan of this design though. It also requires that we choose a value to store in these columns to indicate selection, like 1 and 0 or maybe TRUE and FALSE. With this design, the same condition would look something like this: input$kind_transport_1 & input$kind_transport_4 ~ "buy_preference" — Reply to this email directly, view it on GitHub <#123 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABO4GX33PIPJV4UTKGOHC2DZ3PP2FAVCNFSM6AAAAABP445YVSVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTAOJTHA2DKMY>. You are receiving this because you are subscribed to this thread.

[jhelvy]

Ooo that's a good idea! Yeah going from a string to a vector of values is annoying code, very hard to remember strsplit(str, ',', fixed = TRUE) |> unlist()

The sd_str2vals() seems much more helpful though than the other way around. I can't think of a use case where that might be desired, and also it's not too hard to remember. So maybe we just need the one function?

What about sd_value(id) or sd_get_value(id)? This would take the question id and return the values as a vector. I'm suggesting design this because it may not be easy to remember that we need to convert from a string to a vector, so the sd_str2vals() naming might be harder to remember. With this design, the condition might look like this:

sd_show_if(
  length(sd_value("kind_transport") <= 2) ~ "buy_preference"
)

The other nice thing about this is that you don't have to use the input$ thing anymore. You can just hand it the id, so this could be useful for other questions too that have simpler values.

[warnes]

When creating transformation functions, I always try to do both to and from functions. Among other things, this allow the user to put the transformation on either side of a logic test.

And yes, I also think providing sd_get_values('kind_transport') and sd_get_string('kind_transport') would be helpful.

[jhelvy]

Okay, so pulling together some of the comments here, it seems like it would be helpful to have some sort of function like sd_value(id) that returns the value from a question. I like this as it could help users avoid needing to use input$id, and also because you can document the function and be more clear about the nature of the values that will be returned. For example, here is some expected behavior:

• Any multi-response question (e.g. mc_multi) will return a vector of the chosen options, making it much easier to work with in show_if conditions.
• The function should return the original data type used to define the question options. For example, if a user defines the options in a question as numbers, then sd_value(id) should return the values as a number too, avoiding the need to use as.numeric().

I don't think I would include something like sd_str2vec() (or the other way around) as this would require interacting directly with the input value, e.g. sd_str2vec(input$id). I would instead prefer that users just use sd_value(id) so that they can know how the value will be delivered (it's unclear how the data will be returned using input$id).

Finally, about the point @StefanMunnes made regarding working with the data post-survey for multi-response questions, I agree that one would probably want to convert the data into dummy-coded columns. This could be handled separately, e.g. inside sd_get_data() or perhaps as yet another helper function to convert the raw data into something easier to work with.

[warnes]

Finally, about the point @StefanMunnes made regarding working with the data post-survey for multi-response questions, I agree that one would probably want to convert the data into dummy-coded columns. This could be handled separately, e.g. inside sd_get_data() or perhaps as yet another helper function to convert the raw data into something easier to work with.

Perhaps sd_get_data() with an argument to specify what form to return. For example,

> sd_get_data("kind_transport", type='vector', label=FALSE) # c(1,3)
[1] 1 3 
> sd_get_data("kind_transport", type='vector', label=TRUE) # c("Bike","Skateboard")
[1] "Bike"       "Skateboard"
> sd_get_data("kind_transport", type='table', label=FALSE)  # c('1'=TRUE, '2'=FALSE, '3'=TRUE,'4'=FALSE,'5'=FALSE)
    1     2     3     4     5 
 TRUE FALSE  TRUE FALSE FALSE 
> sd_get_data("kind_transport", type='table', label=TRUE) # c("Bike" = TRUE, "Scooter" = FALSE,  "Skateboard" = TRUE, "Car" == FALSE, "Boat" == TRUE)
      Bike    Scooter Skateboard        Car       Boat 
      TRUE      FALSE       TRUE      FALSE       TRUE 

[jhelvy]

I assume you meant sd_value()? sd_get_data() is already a function, and it returns the entire dataset. If anything I was thinking we may update sd_get_data() to encode the data in a particular way when it is returned, something like this suggestion. But yes, could also add arguments like these to sd_value() to define how you want a particular question value to be returned.

[jhelvy]

Wouldn't it be the easiest way to at least have a copy from the original values in the object and refer to them in the server functions from interest?

This is a good point, but I don't know if it eliminates the need for something like sd_value(). This is because of how shiny works. When you use input$id, you get back the raw output from the shiny widget. For multi-response questions, I believe it gives you a list, which is difficult to work with. That's why I'm thinking some sort of sd_value() function might be helpful, because it could call a value stored in the all_data() list, which would be pre-formatted to whatever we want.

I don't remember why we chose to do the formatting at the early stage in the data handling process, but there was probably a good reason. It may have simply been that we wanted to get everything into a simple single value string so we didn't have to consider data structures later - it's just a list of string values, easy to use. But we could always make that conversion later when we're writing the data to the database, and otherwise we could keep some structure in the interim. So for like mc_multiple we could convert the result from a list into a vector and store that, then sd_value(id) would return a vector.

The other thing I was thinking is that we may actually be able to keep track of the data type used when the options argument was defined in sd_question(). I'm not sure if it's possible, but it would be pretty sweet if sd_value() returned the value encoded in the same way as the original format used to define the question. It'd take some careful thinking here to make this work, but I think we could actually embed this in the rendered html, which we could then parse later. That's actually already how we handle some things, so this is a promising idea.

[jhelvy]

Okay so I just tried this and we actually get back a vector for multi-select questions, not a list, so this may actually not be so bad as is. I tried this question:

sd_question(
  type = "mc_multiple",
  id = "kind_transport",
  label = "What kind of transport you own?",
  option = c(
    "Bike" = 1,
    "Scooter" = 2,
    "Skateboard" = 3,
    "Car" = 4,
    "Boat" = 5
  )
)

Then in my server I printed out the values like this:

  observe({
    val <- input$kind_transport
    print(val)
    print(length(val))
    print(as.numeric(val))
    print(all(as.numeric(val) %in% c(1, 4)))
  })

When I selected "Bike" and "Car", this printed the following:

[1] "1" "4"
[1] 2
[1] 1 4
[1] TRUE

So input$id does return a vector for multi-response questions. So we may actually not need a sd_value() function at all here - just directly accessing the shiny widget value with input$id gives you back a vector. The only thing you'd have to do is convert it to a number if you wanted to do something like val %in% c(1, 4), but that's easy enough to remember to do. Perhaps we've all been over-thinking this problem?

[StefanMunnes]

This is what I tried to ask for the whole time. ;) In the end, it just leaves out the paste(input$id, collapse = ", ") part until exporting the data, right? Then it would be easy to implement.
I was expecting, that the shiny::checkboxGroupInput() function will store the original numeric values, but it transforms them automatically to character. But if this is written in the documentation and help file, I think it's feasible to keep it this way. As you said, using as.numeric(input$kind_transport) will do it, or one could also do it with c("1,", "4").
And the best, a simple length(input$kind_transport) <= 2 should also do it then in the sd_show_if() function!

[pingfan-hu]

Hi @StefanMunnes ! Please check out our new demo survey, as well as its GitHub repo.

Now we have 4 scenarios.
The 3rd scenario is where we applied numerical condition:

as.numeric(input$car_number) > 1 ~ "ev_ownership"

The 4th scenario is where we applied multiple inputs (there are 2 conditions):

input$fav_fruits %in% c("apple", "banana") ~ "apple_or_banana",
length(input$fav_fruits) > 3 ~ "fruit_number"

These above conditions should comply your needs. I hope they are helpful.

[jhelvy]

@pingfan-hu this is such a good demo! So we can actually do basically everything that I think you wanted to do here the way it's currently designed, but it's not necessarily obvious how users can implement all of these conditions. Hopefully a rather comprehensive demo like this helps, and I might update the docs website to point to it as a good example. Here's what the full set of show_if conditions looks like btw:

  sd_show_if(

    # Simple Conditional Display
    input$penguins_simple == "other" ~ "penguins_simple_other",

    # Complex Conditional Display
    input$penguins_complex == "other" &
      input$show_other == "show" ~ "penguins_complex_other",

    # Conditional display based on a numeric value
    as.numeric(input$car_number) > 1 ~ "ev_ownership",

    # Conditional display based on multiple inputs
    input$fav_fruits %in% c("apple", "banana") ~ "apple_or_banana",
    length(input$fav_fruits) > 3 ~ "fruit_number"
  )

[StefanMunnes]

You all doing great work, thank you!
@jhelvy I would like to see all of these examples in the documentation page. One point, the last example length(input$fav_fruits) > 3 ~ "fruit_number" is not based on multiple inputs, it's more about "Conditional display based on range of possible inputs".

[jhelvy]

the last example length(input$fav_fruits) > 3 ~ "fruit_number" is not based on multiple inputs

Maybe I'm not understanding your point here, but this condition does indeed require multiple inputs to trigger. The user has to select four or more fruits for this condition to be TRUE.

And yes, I do plan to update the documentation to have conditions like this so it's more readily accessible for people to reference.

[StefanMunnes]

Ah, you're absolutely right, I was a bit sleepy. I overlooked the fact that the length() function in the last example also refers to the mc_multi question.
One further question: How does the formula deal with NA values? For the example as.numeric(input$car_number) > 1 ~ "ev_ownership",: Just in case there are values in the numeric "car_number" question that can't be transformed to a number, the as.numeric() function returns a NA. For example, if people define all values in the question from 1 to 9 and one "other" categorie. This will return for the first 9 values correct TRUE or FALSE values, but a NA for the "other" value. Will this throw an error?

[jhelvy]

It would throw an error depending on the context of how the vector is being used. If you're doing something like length(input$question_id) then that would work just fine, but in the case you just showed like as.numeric(input$question_id) > 1, then this would return NA if the chosen value was something like "other". In this case, the condition might error as it is always expecting a value of TRUE or FALSE to be in the condition. To get around that, you might have to write a small function that handles that case and returns FALSE, e.g.:

condition <- function(input) {
  if (is.na(input$question_id)) { return(FALSE) }
  return(as.numeric(input$question_id) > 1)
}

Then the condition would just be condition ~ "ev_ownership"

[jhelvy]

We updated the docs now to highlight some of the ways you can set up conditions in more detail. We now have a separate page on conditional control, and in it we have a section on common conditions. Hopefully this will help direct future users to ways to set up some of these more complex conditions.

[jhelvy]

Also btw we now have a sd_is_answered() function that returns TRUE if the question is answered at all, which simplifies the logic required for that specific condition.