Skip to content

Commit

Permalink
added simplest example
Browse files Browse the repository at this point in the history
  • Loading branch information
wxtim committed Jun 28, 2024
1 parent b4c42f8 commit 70accc0
Show file tree
Hide file tree
Showing 4 changed files with 92 additions and 0 deletions.
20 changes: 20 additions & 0 deletions simplest-useful/bin/get_data.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
#!/bin/bash

HEAD="[MY SUPERCOMPUTER] "

echo "${HEAD}Waiting for resources"
sleep 7
echo "${HEAD}"

if [[ -f output ]]; then
rm output
fi

if [[ $((RANDOM % 4)) == 0 ]]; then
echo "Data Retrieval Succeeded"
touch output
exit 0
else
echo "Data Retrieval Failed"
exit 1
fi
8 changes: 8 additions & 0 deletions simplest-useful/bin/process_data.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
#!/bin/bash

cylc message -- "what is the shortest possible timespan for a pretend hpc job?"
if [[ ! -f output ]]; then
echo 'ERROR - NO INPUT DATA!'
exit 1
fi
sleep 14
56 changes: 56 additions & 0 deletions simplest-useful/flow.cylc
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
[meta]
title = Pretty much the simplest use case.
description = """
## What is the smallest workflow worth converting to Cylc?
Consider the case where you want to get some data and
do some number crunching:
```bash
#!bin/bash
#@supercomputer --time 300
#@supercomputer --memory LOTS
#@supercomputer --CPU MANY
get_data.sh
process_data.sh
```
Where `get_data.sh` is a data retrieval programme:
* High IO & Slow
* Low Memory and CPU requirements
* Might be flaky?
And `process_data.sh`
* High memory and CPU
## Why use Cylc
#. Don't ask for resources until pre-reqs are done
#. Retry flaky steps
#. Using `execution time limit` means that Cylc knows that
a task has taken too long even if communications are
disrupted.
"""

[scheduling]
[[graph]]
R1 = get_data => process_data

[runtime]
[[get_data]]
script = get_data.sh
platform = any_old_server
execution retry delays = 4*PT15M, PT1D

[[process_data]]
script = process_data.sh
platform = supercomputer
# Cylc converts to #@supercomputer --time 300
execution time limit = PT5M
[[[directives]]]
--memory LOTS
--CPU MANY
8 changes: 8 additions & 0 deletions simplest-useful/simple_script.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
#!/bin/bash
#@supercomputer --time 300
#@supercomputer --memory LOTS
#@supercomputer --CPU MANY

./bin/get_data.sh

./bin/process_data.sh

0 comments on commit 70accc0

Please sign in to comment.