-
Notifications
You must be signed in to change notification settings - Fork 9
Integrating Rperform with Travis CI
Note: Check out the documentation for using Travis build system with R: Building an R project. In case you are new to Travis CI, go through their Getting Started page and other introductory docs.
Rperform can be used to obtain and visualize package performance metrics. It can also be integrated into a package's Travis-CI workflow in order to obtain (changes in) performance metrics with each Travis build. The results can be generated in the form of a static webpage and pushed to the package's gh-pages branch.
We can use Rperform with both regular as well as PR (pull request) builds on Travis. In case of regular builds, the function plot_webpage() can be used to generate performance plots or data. In case of PR builds, the function plot_PR_webpage() will be used.
You will need to generate an access token that will be used to allow you to push a webpage containing Rperform's results to your GitHub repo's gh-page branch. This will be some really long alphanumeric string. You can generate one by going to settings -> personal access tokens -> generate new token. If we are working with a public repo, selecting only the public_repo option under scope would do. Otherwise, the repo option must be selected.
You will also need to install the travis ruby framework.
gem install travis
After installing, navigate to your git repo for the project you want to enable automatic pushing of content for, and then login to travis-ci and secure the GitHub token.
travis login
travis encrypt GH_TOKEN="yourgithubtoken"
This will generate output that should be copied to your .travis.yml file in a subsequent step. We will refer to this output as yoursecurestring. Essentially we have created an environment variable GH_TOKEN with your actual GitHub token, that is encrypted on the travis-ci servers. So this way you don't expose your actual GitHub token to anyone who looks at your .travis.yml file.
Ignore the first step if you already have a gh-pages branch set up for your package repo.
- Create and checkout into an orphan gh-pages branch in the repo directory.
git checkout --orphan gh-pages
git rm -rf .
- Make an Rperform directory and create an empty index.html file inside it.
mkdir Rperform
touch Rperform/index.html
- Commit and push the changes. Then, checkout into the master branch.
git add Rperform/index.html
git commit -m "First commit"
git push origin gh-pages
git checkout master
This section assumes that you have got Travis-CI configured to work with your repo already.
-
For a sample .travis.yml file configured to work with Rperform, look here. We will use this sample file as our reference.
-
Add the package names given under r_packages and r_github_packages in the sample file to your .travis.yml file under the respective sections.
r_packages:
- microbenchmark
- roxygen2
- rmarkdown
r_github_packages:
- analyticalmonk/Rperform
- Add the command given under before_script in the sample file to your .travis.yml file.
before_script:
- travis_wait 30 source `Rscript -e "cat(find.package(\"Rperform\"))"`/push_gh_pages.sh
- Next, we need to configure our environment variables.
env:
global:
- secure: "ENTER_YOUR_ENCRYPTED_GITHUB_ACCESS_TOKEN_HERE"
- USER_EMAIL="INSERT_A_VALID_EMAIL_ID_HERE"
- USER_NAME="INSERT_USERNAME_HERE"
- PR_COMMAND="Rperform::plot_PR_webpage(test_path = './INSERT_PATH_TO_TEST_DIR/INSERT_TEST_NAME_HERE', metric = 'time')"
- RPERFORM_COMMAND="Rperform::plot_webpage(test_directory = './INSERT_PATH_TO_TEST_DIR/', metric = 'time')"
Your encrypted_github_access_token is nothing but the yoursecurestring generated at the beginning of this tutorial.
USER_EMAIL and USER_NAME aren't mandatory but put them in anyway if you want.
RPERFORM_COMMAND and PR_COMMAND are the Rperform commands run when a regular build and Pull Request build are triggered respectively.
When configured, these environment variables will look something like this:
env:
global:
- secure: "yoursecurestring"
- USER_EMAIL="[email protected]"
- USER_NAME="analyticalmonk"
- PR_COMMAND="Rperform::plot_PR_webpage(test_path = './test_directory/test_file', metric = 'time')"
- RPERFORM_COMMAND="Rperform::plot_webpage(test_directory = './test_directory', metric = 'time')"
- Add, commit and push the changes.
git add .
git commit -m "Configured Travis CI to run Rperform"
git push origin master
-
The plot_webpage() command generates performance plots for all test files contained in the provided directory for a given metric parameter. metric can be time, memory or testMetrics (both time and memory). It provides the metrics for last 5 commits by default.
-
The plot_PR_webpage() command generates performance plots comparing the commits from the incoming Pull Request (PR) with the head of the master branch of the repo. It works similarly to when comparing commits across two branches; it's comparing two directories instead. The time metric is supported for PR builds as of now. It provides the metrics for last 5 commits by default.
-
Once the Travis build gets completed, the required performance plots for files in the directory provided to RPERFORM_COMMAND should get generated at username.github.io/<project_name>/Rperform.
-
Old performance pages can be found inside the Rperform/ directory in the repo's gh-pages branch.