Docs/ingestion process detailed (#108)

neicnordic · Feb 19, 2024 · 32e1c67 · 32e1c67
2 parents 39f9a80 + 6e0b239
commit 32e1c67
Show file tree

Hide file tree

Showing 2 changed files with 23 additions and 21 deletions.
diff --git a/docs/guides/deploy-k8s.md b/docs/guides/deploy-k8s.md
@@ -3,6 +3,7 @@
 ## Guide summary
 
 This guide explains how to deploy the Sensitive Data Archive (SDA) in kubernetes.
+
 - What it intends to cover
 - What to expect, scope, explain level of details
 - How self-contained the guide is
@@ -22,8 +23,6 @@ The services could be divided into two trust boundaries
 
 The innermost trust zone contains the database and the archive, which be can accessed only from internal cluster.
 
-
-
 ## Charts overview
 
 ### sda-db - Database component for Sensitive Data Archive (SDA) installation
@@ -227,11 +226,6 @@ Below is a minimal list of variables that need to be configured in the [values.y
 - `global.tls.clusterIssuer`: The cluster issuer for TLS
 - `global.tls.secretName`: The name by which the kubernetes secret for TLS is referenced in the Helm charts
 
-## Security issues
-
-- Enabling TLS example
-- Secret handling example
-
 ## Network policies
 
 - DNS names and ingress for services

diff --git a/docs/submission.md b/docs/submission.md
@@ -41,51 +41,59 @@ Structure of the message and its contents are described in
       deactivate Inbox
       Central EGA RabbitMQ-->>SDA RabbitMQ: federated msg: [from_cega][ingest type]
       SDA RabbitMQ-->>Intercept: Intercept reads message
+      Intercept -->> SDA RabbitMQ: Forwards ingest message <br/> to queue
+      alt Ingest is successful 
       Intercept->>Ingest: msg: [sda][ingest] begin ingestion
-      activate Ingest
+      activate Ingest 
       Ingest->>SDA Database: mark ingested
-      opt
+      Note over Ingest: store file in Archive
+      Ingest->>SDA Database: mark archived
+      Ingest-->>SDA RabbitMQ: msg [sda][archived]
+      else Error occurred in ingestion process
       Ingest-->>SDA RabbitMQ: msg: error
       SDA RabbitMQ-->>Central EGA RabbitMQ: shovel msg:[to_cega][files.error]
       end
-      Ingest->>SDA Database: mark archived
-      Ingest-->>SDA RabbitMQ: msg [sda][archived]
       deactivate Ingest
+      alt Verify is successful 
       activate Verify
       SDA RabbitMQ-->>Verify: msg [sda][archived] triggers verify
-      opt
+      Verify->>SDA Database: mark verified
+      Verify-->>SDA RabbitMQ: msg: [sda][verified]
+      else Error occurred in verify process
       Verify-->>SDA RabbitMQ: msg: error
       SDA RabbitMQ-->>Central EGA RabbitMQ: shovel msg:[to_cega][files.error]
       end
-      Verify->>SDA Database: mark verified
-      Verify-->>SDA RabbitMQ: msg: [sda][verified]
       deactivate Verify
       SDA RabbitMQ-->>Central EGA RabbitMQ: shovel msg:[to_cega][files.verified]
       Central EGA RabbitMQ-->>SDA RabbitMQ: federated msg: [from_cega][accession type]
       SDA RabbitMQ-->>Intercept: Intercept reads message
+      Intercept -->> SDA RabbitMQ: Forwards accession ID message <br/> to queue
       Intercept->>Finalize: msg: [sda][accession] map file to accession ID
+      alt Finalize is successful 
       activate Finalize
       note right of Finalize: Finalize makes the file backup
-      opt
+      Finalize->>SDA Database: mark completed
+      Finalize-->>SDA RabbitMQ: msg: [sda][completed]
+      else Error occurred in finalize process
       Finalize-->>SDA RabbitMQ: msg: error
       SDA RabbitMQ-->>Central EGA RabbitMQ: shovel msg:[to_cega][files.error]
       end
-      Finalize->>SDA Database: mark completed
-      Finalize-->>SDA RabbitMQ: msg: [sda][completed]
       deactivate Finalize
       SDA RabbitMQ-->>Central EGA RabbitMQ: shovel msg:[to_cega][files.completed]
       Central EGA RabbitMQ-->>SDA RabbitMQ: federated msg: [from_cega][mappings type]
       SDA RabbitMQ-->>Intercept: Intercept reads message
+      Intercept -->> SDA RabbitMQ: Forwards mapper message <br/> to queue
       Intercept->>Mapper: msg: [sda][mappings] begin ingestion
+      alt Mapper is successful 
       activate Mapper
-      opt
+      Mapper->>SDA Database: map file to dataset accession ID
+      Mapper->>Inbox: remove file from inbox
+      else Error occurred in mapper process
       Mapper-->>SDA RabbitMQ: msg: error
       SDA RabbitMQ-->>Central EGA RabbitMQ: shovel msg:[to_cega][files.error]
       end
-      Mapper->>SDA Database: map file to dataset accession ID
-      Mapper->>Inbox: remove file from inbox
       deactivate Mapper
-    
+
 ```
 
 > NOTE: