Recipes

Use Pools Of Orgs

If you have a bunch of continuous integration orgs, you may want to group them into pools of orgs. This allows you to split up the apex tests to run across multiple orgs, and allow multiple builds to run concurrently across pools.

deploylock can help by giving you a lock on a pool.

Suppose you have 20 ci orgs, that you want to split into four pools.

CI_ORGS=20
CI_ORG_POOLS=4
DEPLOYLOCK_SERVER=https://deploylock-74a2278a5a00.herokuapp.com/

acquire_lock() {
  lock_num=$1
  lock_info=$(deploylock-client acquire -n "ci$lock_num" "$DEPLOYLOCK_SERVER")
  echo "$lock_num $lock_info"
}
export -f acquire_lock

# Get the first available lock.  Set LOCK_NUM to the pool number, e.g. LOCK_NUM 1 corresponds with lock ci1.
lock_output=$(jot $CI_ORG_POOLS 0 | parallel --halt now,success=1 acquire_lock {} | head -1)
export LOCK_NUM=$(echo "$lock_output" | awk '{print $1}')
export LOCK=$(echo "$lock_output" | awk '{print $2}')
# Hang onto the lock until our build completes
deploylock-client renew -n "ci$LOCK_NUM" -l $LOCK $DEPLOYLOCK_SERVER &
# Use the 5 orgs from the pool
ORGS=$(($CI_ORGS / $CI_ORG_POOLS))
START=$(($ORGS * $LOCK_NUM))

# Start your build, splitting up the tests across the orgs in the pool
grep -ril IsTest build/classes | parallel echo --test {/.} | sort --random-sort | parallel --jobs $(jot $ORGS $START | wc -l) -m  echo | parallel --jobs 0 --halt now,fail=1 --link --linebuffer -a - 'eval force import {1} -c -d build -i -f junit -w --erroronfailure=false -a $SFUSER.ci{2} > $CIRCLE_TEST_REPORTS/{2}/report.xml' :::: <(jot $ORGS $START)