-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
4 changed files
with
92 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,20 @@ | ||
#!/bin/bash | ||
|
||
HEAD="[MY SUPERCOMPUTER] " | ||
|
||
echo "${HEAD}Waiting for resources" | ||
sleep 7 | ||
echo "${HEAD}" | ||
|
||
if [[ -f output ]]; then | ||
rm output | ||
fi | ||
|
||
if [[ $((RANDOM % 4)) == 0 ]]; then | ||
echo "Data Retrieval Succeeded" | ||
touch output | ||
exit 0 | ||
else | ||
echo "Data Retrieval Failed" | ||
exit 1 | ||
fi |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,8 @@ | ||
#!/bin/bash | ||
|
||
cylc message -- "what is the shortest possible timespan for a pretend hpc job?" | ||
if [[ ! -f output ]]; then | ||
echo 'ERROR - NO INPUT DATA!' | ||
exit 1 | ||
fi | ||
sleep 14 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,56 @@ | ||
[meta] | ||
title = Pretty much the simplest use case. | ||
description = """ | ||
## What is the smallest workflow worth converting to Cylc? | ||
Consider the case where you want to get some data and | ||
do some number crunching: | ||
```bash | ||
#!bin/bash | ||
#@supercomputer --time 300 | ||
#@supercomputer --memory LOTS | ||
#@supercomputer --CPU MANY | ||
get_data.sh | ||
process_data.sh | ||
``` | ||
Where `get_data.sh` is a data retrieval programme: | ||
* High IO & Slow | ||
* Low Memory and CPU requirements | ||
* Might be flaky? | ||
And `process_data.sh` | ||
* High memory and CPU | ||
## Why use Cylc | ||
#. Don't ask for resources until pre-reqs are done | ||
#. Retry flaky steps | ||
#. Using `execution time limit` means that Cylc knows that | ||
a task has taken too long even if communications are | ||
disrupted. | ||
""" | ||
|
||
[scheduling] | ||
[[graph]] | ||
R1 = get_data => process_data | ||
|
||
[runtime] | ||
[[get_data]] | ||
script = get_data.sh | ||
platform = any_old_server | ||
execution retry delays = 4*PT15M, PT1D | ||
|
||
[[process_data]] | ||
script = process_data.sh | ||
platform = supercomputer | ||
# Cylc converts to #@supercomputer --time 300 | ||
execution time limit = PT5M | ||
[[[directives]]] | ||
--memory LOTS | ||
--CPU MANY |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,8 @@ | ||
#!/bin/bash | ||
#@supercomputer --time 300 | ||
#@supercomputer --memory LOTS | ||
#@supercomputer --CPU MANY | ||
|
||
./bin/get_data.sh | ||
|
||
./bin/process_data.sh |