Contributing to eddy4R

Contents:

• Github Issue Tracker
• Github Forking Workflow
• eddy4R Branching Model
• Contributing code via pull request checklist
• Coding style
• Terms table
• Travis-CI

Github Issue Tracker

In order to facilitate the DevOps approach, the GitHub Issue Tracker is used as an interface for eddy4R users to report bug fixes and request additional features and functionality, while also providing developers a way to keep track of current and future work. The eddy4R development team works according to the Agile project management methodology, where discrete portions of work, or "user stories", are performed in two-week cycles called "sprints". New issues will be incorporated into the eddy4R development user story backlog, and subsequently scheduled for work in upcoming sprints according to priority, as determined by the eddy4R development team and the NEON surface atmosphere exchange technical working group. We encourage collaboration from the broader science community, and in the event that an eddy4R user would like to collaboratively work on development, GitHub issues will be used to communicate ideas for code contribution, keep track of work allocation, schedule, and progress, while issue comments can be used for group discussion (see Contributing code via pull request checklist for full contribution checklist).

Issues will be closed upon completion (or may be resolved without incorporation if so decided by the governance body) according to the release workflow that is outlined in our configuration management process. The archive of closed issues serves as a searchable and transparent record of previous development tasks.

• When submitting a GitHub issue to the eddy4R repository, please include the SHA-256 digest value for the image used.
• To determine the current image SHA-256 digest value use the following commands

  • pull latest Docker image from Dockerhub, also displays image digest sha256:

    docker pull REPO/IMAGE:TAG
    docker pull REPO/IMAGE@sha256:
    

  • to display image digest sha256: after download

    docker images --digests
    

Github Forking Workflow

The eddy4R repo is read access only; thus, to contribute to eddy4R you must use a "Forking Workflow" as described in the figure below.

eddy4R Branching Model

• The eddy4R branching model follows the principles outlined in ["A successful Git branching model"] (https://nvie.com/posts/a-successful-git-branching-model/) described by Vincent Driessen
• New feature release process
  • Open the NEONScience/eddy4R git project
  • Create new release branch from /deve (e.g. git checkout -b 0.3.0) then push to NEONScience/eddy4r repo (e.g. git push -u origin 0.3.0)
  • release from release branch (e.g., 0.3.0) with release tag (e.g., 0.3.0-rc1)
• bug fixes from release branch (e.g., 0.3.0)
  • check out branch "/fixDesc" (e.g., fixEcseNa) in personal fork (e.g. ddurden/eddy4R-ddurden) from target release branch (e.g., git checkout -b upstream/0.3.0 fixDesc)
  • Push bug fix branch "/fixDesc" to github fork (git push -u origin fixDesc)
  • resolve bug and test in branch "/fixDesc"
  • pull request from branch "/fixDesc" into the release branch (e.g. 0.3.0)
    • resolve conflicts (there shouldn't be any, unless another bug-fix was pulled in first)
    • perform Travis-CI (see Travis-CI section for details) test; further corrections if test fails
    • once Travis-CI successful, merge
    • make new release candidate (e.g., 0.3.0-rc2)
  • subsequent pull request from release branch 0.3.0 into branch deve
    • resolve conflicts
    • perform Travis-CI test; further corrections if test fails
    • once Travis-CI successful AND new base image build completed on Dockerhub, merge
  • delete branch "/fixDesc"
• Once the release is stable make the final release from master
  • pull request from release branch 0.3.0 to master
  • perform Travis-CI test; further corrections if test fails
  • once Travis-CI successful, merge
  • make official release (e.g., 0.3.0)
• bug fixes from tagged/final release (e.g., 0.3.0)
  • In personal fork (e.g. ddurden/eddy4R-ddurden)
  • check out branch fixDesc from target release (e.g., 0.3.0)

     git fetch --all --tags --prune
     git tag
     git checkout -b tags/0.3.0 fixDesc
     git push -u origin fixDesc
    

  • resolve bug and test in branch fixDesc
  • pull request from branch fixDesc into the NEONScience/eddy4R/master
    • resolve conflicts (there shouldn't be any, unless another bug-fix was pulled in first)
    • perform Travis-CI test; further corrections if test fails
    • once Travis-CI successful, merge
    • make new official release (e.g., 0.3.1)
  • subsequent pull request from branch master into branch deve
    • [merging from branch master instead of from branch fix: in this way branch deve always has the same commit history as branch master, incl. merge commits]
    • resolve conflicts
    • perform Travis-CI test; further corrections if test fails
    • once Travis-CI successful AND new base image build completed on Dockerhub, merge delete branch fixDesc

Contributing code via pull request checklist

• Have you entered a Github issue proposing the developments that you would like to have implemented in eddy4R?
• Have the proposed code contributions been accepted for absorption into the eddy4R repository maintainers?

Information to be added in pull request description

• What type of maintenance that needed?
• Who will responsible for the maintenance?
• Added functionality
• Dependencies (R, OS)
• Initial profiling for compute resources (execution time, CPU, max RAM, max disk space)
  • Rstudio built in profiling saved and exported

Preliminary checklist for pull request submission

• Do you agree to the eddy4R license conditions? Do they meet your requirements for intellectual property, disclosure, and license compatibility?
• Does the code adhere to the eddy4R Coding style?
  • Has proper Roxygen documentation been added to the header of any functions?
  • Have all the eddy4R terms been adhered to?
  • Has additional "#comment" documentation to help understand the flow of the function been added?
• Do you know where the code would fit inside the eddy4R family of R-packages?
• If pull requests contains functions inside a eddy4R package, has the package successful been built in the eddy4R docker? Has the gold test cases ran successfully, either locally or on Travis-CI (Note: Travis-CI runs automatically when a pull request is made)?

NEON Internal considerations

• Who will be responsible for the maintenance? (and what charge code?)
• Is this code redundant to current eddy4R code?
• Should this be added as a method to an existing function instead?

Coding style

File header

• standard header based on eddy4R.base/R/def.env.glob.R
• include Roxygen2 tags

Naming conventions

Based on (in descending order):

• https://journal.r-project.org/archive/2012-2/RJournal_2012-2_Baaaath.pdf
• http://blog.lmorchard.com/2013/01/23/naming-conventions/
• http://adv-r.had.co.nz/Style.html
• https://google.github.io/styleguide/Rguide.xml#identifiers

General concept:

• base-name (fundamental aspect) followed by modifier(s)
• each base-name or modifier min 2, max 4 alphanumeric characters long (incl. numbers 00, 01…)
• modifiers in descending hierarchical order: The majority of variations should be expected for the last modifier. Example for the variable temperature (temp) measured in a single cell (tempCell) at inlet (tempCellIn), center (tempCellMid), outlet (tempCellOut) positions.
• {no CAPITALIZATION exceptions for (i) chemical formulae (e.g., co2, h2o), (ii) abbreviations not exceeding 2 characters when not at the beginning of an object (e.g. qf)}

Convention for R-objects:

• variables all camelCase

  • baseModi
  • base-name is a noun carrying the unit of the variable
  • example: tempPot for potential temperature [K]
  • running indices idxModi (e.g. used in for-loops)
    • follows the hierarchical convention for variables
    • base-name is always 'idx', and Modi the object followed by variables or parameters it is executed over
    • example: idxMatVar for the index idx running through a matrix Mat by variables Var

• constants, parameters all PascalCase

  • BaseModi
  • base-name is a noun carrying the unit of the parameter
  • example: Temp00 for NIST standard temperature (293.5 K)
  • definition: constant (Cnst)
    • hard coded, e.g. in a definition-function, wrapper-function, internal data
    • single number
    • not changing through the course of a definition-function, wrapper-function or workflow evaluation
  • definition parameter (Para)
    • derived or supplied as an argument, e.g. to a definition-function or a wrapper-function
    • single number
    • not changing through the course of a definition-function, wrapper-function or workflow evaluation
  • definition: constant-set (SetCnst), parameter-set (SetPara)
    • multiple constants (parameters) are being supplied
    • constant-set (parameter-set) is a list with entries for individual constants (parameters)
    • constant-set (parameter-set) follows "constants, parameters all PascalCase" coding style

• environmental variables all UPPERCASE

  • BASEMODI
  • base-name is a noun carrying the unit of the environmental variable (if applicable)
  • example: DIRINP for the absolute path to the input data file
  • environmental variable definition
    • provided from the host operating system to the eddy4R R-session
    • user-specified environmental variables are imported to the eddy4R R-session using base::Sys.getenv()
    • upon import these user-specified environmental variables are converted to the parameter naming convention for further use (see above)

• functions and workflows - all lowercase with period separator: base.modi

  • no exceptions to the 2-4 character limit for each base or modifier name
  • base-name is a verb, one of def (define function), wrap (wrap several functions together), flow (workflow), docu
    • general structure
      • def.base.modi
      • wrap.base.modi
      • flow.base.modi

type of function	base	modi
general	dominating property generated in output (e.g., def.dspk)	methodological specifiers from least to most variation (e.g., def.dspk.filt.med)
quantity-specific	output quantity (e.g. def.dens)	output specifiers followed by (i) input specifiers and (ii) methodological specifiers from least to most variation (e.g., def.dens.pres.pois.R)

- example: def.env.glob() for a function defining parts of a global environment

• classes, structures all uppercase with period separator
  • BASE.MODI

Terms table

• eddy4R keeps a controlled set of terms that are used in naming variables, parameters, and function following the eddy4R Coding style.
• Current Terms table

Travis-CI

• Travis-CI documentation for building and testing an R project: https://docs.travis-ci.com/user/languages/r/

eddy4R implementation:

• Travis-CI tests the eddy4R docker build process, including installation of dependencies and R-package builds, and runs the gold file regression test (flow.test.gold.eddy.R) to ensure changes to the code base do not impact the resulting output (unless expected and verified, which would include an update to the reference gold file).

  • The eddy4R master repo is currently set to perform testing on all pull requests made to the repo, and the results of the test will be displayed on the pull request github page (see image below). Green checks indicate the pull request passed testing, and red x's indicate a failed test.

    

• Inside a new pull request, the following will be displayed if the automated Travis-CI testing passed.

  

Travis-CI users guide:

• First, signup for Travis-CI (https://travis-ci.org/) using your github ID.
• Next, you can activate repositories that you would like to perform builds and test on. However, not that the repositories available to you are only public repositories. Travis-CI has a separate webpage (https://travis-ci.com/) for private projects. So, you will need to sign up for a Travis Pro account, which comes with 100 free builds. Once you are signed up, you should be able to sync National Ecological Observatory Network private repos and activate them for building and testing.

• Once, accounts are activated you should see them in My Repositories tab. The NEONScience/eddy4R master repo should already be activated. If you want to test your updates prior to sending a pull request, you should activate your fork (e.g. ddurden/eddy4R-ddurden).

  

• Considering you have only 100 free builds, it's suggested that you make some changes to testing settings. These can be found by clicking "More options" ==> "Settings". The most important settings are under "General", I suggest requiring a .yml file be present to build, turn off build branch updates, and turn off build pull request updates (see below for my configuration).

• Then, if you want to test your code changes before submitting the pull request you can trigger a build. This can be found under "More options" ==> "Trigger build", which will give you the option of which branch you would like to build (see below). This will perform a custom build, and will not add any commits to your github code repo.

• Additional Notes: - Travis_CI for R is usually performed on a package basis, not a suite of packages - use devtools::use_travis() to automatically add a minimal .travis.yml to your package, add .travis.yml to .Rbuildignore, and add the code for the badge to your README.

  • Adding system requirements can be done by

• Adding docker to services in the travis.yml:

Future improvements:

• Travis-CI is integrated with testthat package - True unit testing functionality. Relatively simple to automate for packages, but would require some work up front. - Can be automated for directories or packages - R CMD 'check' will perform the package test

Travis-CI Resources:

• Hadley Wickham’s R Packages book has a section on Travis that is short but helpful; it is within the section on automated checking that also has relevant helpful information. The R docs at Travis itself are also quite helpful, although when I am having a real headscratcher of a problem I always wish there was more detail.