Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

See if it's feasible to implement parallel processing for calc_palmsplus #9

Open
TheTS opened this issue Dec 31, 2017 · 1 comment
Open
Assignees
Labels

Comments

@TheTS
Copy link
Owner

TheTS commented Dec 31, 2017

Try and foreach this loop. If significant speed improvement, parameterise parallel_processing

  for (i in unique(data$identifier)) {
    x[[i]] <- data %>%
      filter(identifier == i) %>%
      mutate(!!! field_args) %>%
      mutate_if(is.logical, as.integer)

    if (verbose) {
      cat("[", j, "/", len, "] Computed palmsplus for: ", i, "\n", sep = "")
      j <- j + 1
    }
  }
@TheTS TheTS self-assigned this Dec 31, 2017
@TheTS
Copy link
Owner Author

TheTS commented Jan 1, 2018

Note: a quick test is ~300% faster on 7 vs. 1 core. Tricky to implement as palms dataset is copied to each worker (too much RAM). Also, to keep field tables dynamic, the global environment needs to be copied too.

Not a massive deal as < 6 min for 194 identifiers.

Might revisit this in future (and for palms_build_trajectories too).

@TheTS TheTS added the wontfix label Jan 9, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant