SPLaSh: Scaling Out Shell Scripts on Serverless Platforms

A system for deploying POSIX shell scripts to serverless.

Running SPLaSh

To parallelize, say, ./evaluation/intro/hello-world.sh with a parallelization degree of 2× run:

./pa.sh --serverless_exec -w 2 ./evaluation/intro/hello-world.sh

Run ./pa.sh --help to get more information about the available commands.

Installation

On Ubuntu, Fedora, and Debian, run the following to set up SPLaSh.

wget https://raw.githubusercontent.com/binpash/pash/main/scripts/up.sh
sh up.sh
export PASH_TOP="$PWD/pash/"
## Run PaSh with echo hi
"$PASH_TOP/pa.sh" -c "echo hi"

For more details, manual installation, or other platforms see installation instructions.

Repo Structure

This repo is a frozen version of splash. For the most up-to-date version, visit https://github.com/binpash/pash. The structure is as follows:

• compiler: Shell-dataflow translations, associated parallelization transformations, and serverless transformations/primitives.
• docs: Design documents, tutorials, installation instructions, etc.
• evaluation: Shell pipelines and example scripts used for the evaluation.
• runtime: Runtime components for the shell—e.g., eager, split—and serverless execution—e.g., invoke-lambda,send-object.
• scripts: Scripts related to continuous integration, deployment, and testing.

Community & More

Chat:

• Discord Server (Invite)

Mailing Lists:

• pash-devs: Join this mailing list for discussing all things pash
• pash-commits: Join this mailing list for commit notifications

Development/contributions:

• Contribution guide: docs/contributing
• Continuous Integration Server: ci.binpa.sh