Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sunset intake module 🫡 #494

Merged
merged 1 commit into from
Nov 18, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 0 additions & 2 deletions .github/workflows/build-push-image-provider.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,6 @@ on:
- 'docker/images/yoda_irods_icat/**'
- 'roles/irods_completion/files/irods_completion.sh'
- 'roles/pam_python/files/pam_python.so'
- 'roles/yoda_rulesets/files/run-intake-movetovault.sh'

jobs:
push-image:
Expand Down Expand Up @@ -55,4 +54,3 @@ jobs:
tags: ghcr.io/utrechtuniversity/yoda-provider:${{ steps.extract_branch.outputs.branch }}
build-args: |
TAG=${{ steps.extract_branch.outputs.branch }}

3 changes: 0 additions & 3 deletions docker/images/yoda_irods_icat/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -177,10 +177,7 @@ RUN for script in scheduled-copytovault.sh admin-remove-orphan-vault-if-empty.sh
do install -m 0755 /etc/irods/yoda-ruleset/tools/${script} /var/lib/irods/msiExecCmd_bin/${script} ; \
done && \
/etc/irods/yoda-ruleset/tools/setup_tokens.sh /etc/irods/yoda-ruleset/accesstokens.db test
COPY stage/run-intake-movetovault.sh /tmp/run-intake-movetovault.sh
COPY job_movetovault.r /var/lib/irods/.irods/job_movetovault.r
COPY job_clearintakelocks.r /var/lib/irods/.irods/job_clearintakelocks.r
RUN install -o irods -g irods -m 0755 /tmp/run-intake-movetovault.sh /var/lib/irods/.irods/run-intake-movetovault.sh
COPY cronjob-revision-cleanup.sh /tmp/cronjob-revision-cleanup.sh
RUN install -o irods -g irods -m 0755 /tmp/cronjob-revision-cleanup.sh /var/lib/irods/.irods/cronjob-revision-cleanup.sh
COPY service_account.config /tmp/service_account.config
Expand Down
29 changes: 0 additions & 29 deletions docker/images/yoda_irods_icat/job_clearintakelocks.r

This file was deleted.

67 changes: 0 additions & 67 deletions docker/images/yoda_irods_icat/job_movetovault.r

This file was deleted.

1 change: 0 additions & 1 deletion docker/images/yoda_irods_icat/rules_uu.cfg
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,6 @@ eus_api_tls_verify = 'false'

enable_deposit = 'true'
enable_open_search = 'false'
enable_intake = 'true'
enable_datarequest = 'true'
yoda_portal_fqdn = 'portal.yoda'

Expand Down
1 change: 0 additions & 1 deletion docker/images/yoda_irods_icat/stage-uploads.sh
Original file line number Diff line number Diff line change
Expand Up @@ -16,4 +16,3 @@ fi

cp ../../../roles/pam_python/files/pam_python.so stage
cp ../../../roles/irods_completion/files/irods_completion.sh stage
cp ../../../roles/yoda_rulesets/files/run-intake-movetovault.sh stage
2 changes: 0 additions & 2 deletions docker/images/yoda_portal/yoda_portal_init.sh
Original file line number Diff line number Diff line change
Expand Up @@ -92,8 +92,6 @@ YODA_COMMIT = '$YODA_COMMIT'
RESEARCH_ENABLED = True
OPEN_SEARCH_ENABLED = False
DEPOSIT_ENABLED = True
INTAKE_ENABLED = True
INTAKE_EXT_TIMEOUT = 1800
DATAREQUEST_ENABLED = True
TOKENS_ENABLED = True
TOKEN_LIFETIME = 72
Expand Down
4 changes: 0 additions & 4 deletions docker/run-cronjob.sh
Original file line number Diff line number Diff line change
Expand Up @@ -25,10 +25,6 @@ case "$1" in
docker exec "$EXEC_OPTIONS" provider.yoda sudo -iu irods /bin/bash /etc/irods/yoda-ruleset/tools/notification/notification-data-access-token-expiry.sh
;;

intakevault)
docker exec "$EXEC_OPTIONS" provider.yoda sudo -iu irods /var/lib/irods/.irods/run-intake-movetovault.sh
;;

publication)
docker exec "$EXEC_OPTIONS" provider.yoda sudo -iu irods /bin/irule -r irods_rule_engine_plugin-irods_rule_language-instance -F /etc/irods/yoda-ruleset/tools/process-publication.r
;;
Expand Down
8 changes: 0 additions & 8 deletions docs/administration/configuring-yoda.md
Original file line number Diff line number Diff line change
Expand Up @@ -282,14 +282,6 @@ Variable | Description
-----------------------------|---------------------
enable_deposit | Enable deposit module

### Intake module configuration

Variable | Description
-----------------------------|---------------------
enable_intake | Enable intake module
intake_groups | List of intake groups (without the "grp-intake-" prefix)
intake_extended_timeout | If the intake module is enabled, portal iRODS session timeouts and portal VHost timeouts will be changed to this value (in seconds), in order to be able to handle long synchronous operations, such as intake scans (default: 1800)

### Datarequest module configuration

Variable | Description
Expand Down
1 change: 0 additions & 1 deletion docs/design/api/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,5 +11,4 @@ has_toc: false
- [Core API documentation](https://petstore.swagger.io/?url=https://utrechtuniversity.github.io/yoda-ruleset/api_core.json)

### Module API
- [Intake API documentation](https://petstore.swagger.io/?url=https://utrechtuniversity.github.io/yoda-ruleset/api_intake.json)
- [Datarequest API documentation](https://petstore.swagger.io/?url=https://utrechtuniversity.github.io/yoda-ruleset/api_datarequest.json)
5 changes: 0 additions & 5 deletions docs/design/overview/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,11 +39,6 @@ The user module provides settings and notifications.

## Additional modules

### Intake
This module processes uploaded YOUth programme experiment data, and facilitate its archival in the vault by a data manager.

[Technical design](intake.md){: .btn }

### Datarequest
Module adding support for the submission, review and approval of requests to obtain research data stored in Yoda.

Expand Down
135 changes: 0 additions & 135 deletions docs/design/overview/intake.md

This file was deleted.

14 changes: 1 addition & 13 deletions docs/design/processes/asynchronous-processes.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,6 @@ This page contains an overview of asynchronous processes in Yoda.
* [Statistics job](#statistics-job)
* [Archiving](#archiving)
* [Archiving - retry-copy-to-vault job](#archiving-retry-copy-job)
* [Archiving - intake to vault job](#archiving-intake-to-vault-job)
* [Archiving - publication job](#archiving-publication-job)

<a name="metadata"/>
Expand Down Expand Up @@ -141,7 +140,7 @@ and communities.

## Archiving and publication

Asynchronous jobs are also used to copy data packages from a research or intake folder to the vault,
Asynchronous jobs are also used to copy data packages from a research folder to the vault,
as well as to process publications.

<a name="archiving-retry-copy-job"/>
Expand All @@ -158,17 +157,6 @@ as well as to process publications.
By default, groups that are to be copied to the vault are marked with a metadata attribute named
`cronjob_copy_to_vault`.

<a name="archiving-intake-to-vault-job"/>

### Intake copy to vault job

| | |
|---|---|
| Script | /var/lib/irods/.irods/job_movetovault.r |
| Purpose | copy data packages from intake groups to the vault |
| Lock file | uses lock attribute on the vault folder |
| Typically started by | cronjob, runs every five minutes |

<a name="archiving-publication-job"/>

### Process publication job
Expand Down
3 changes: 0 additions & 3 deletions environments/development/allinone/group_vars/allinone.yml
Original file line number Diff line number Diff line change
Expand Up @@ -35,9 +35,6 @@ opensearch_server: combined.yoda.test
# Deposit module configuration
enable_deposit: true # Enable deposit module

# Intake module configuration
enable_intake: true # Enable intake module

# Datarequest module configuration
enable_datarequest: true # Enable datarequest module
datarequest_help_contact_name: PLACEHOLDER # Help contact name
Expand Down
3 changes: 0 additions & 3 deletions environments/development/el8-database/group_vars/el8.yml
Original file line number Diff line number Diff line change
Expand Up @@ -30,9 +30,6 @@ opensearch_server: combined.yoda.test
# Deposit module configuration
enable_deposit: true # Enable deposit module

# Intake module configuration
enable_intake: true # Enable intake module

# Datarequest module configuration
enable_datarequest: true # Enable datarequest module
datarequest_help_contact_name: PLACEHOLDER # Help contact name
Expand Down
3 changes: 0 additions & 3 deletions environments/development/full/group_vars/full.yml
Original file line number Diff line number Diff line change
Expand Up @@ -32,9 +32,6 @@ opensearch_server: portal.yoda.test
# Deposit module configuration
enable_deposit: true # Enable deposit module

# Intake module configuration
enable_intake: true # Enable intake module

# Datarequest module configuration
enable_datarequest: true # Enable datarequest module
datarequest_help_contact_name: PLACEHOLDER # Help contact name
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -37,9 +37,6 @@ opensearch_server: combined.yoda.test
# Deposit module configuration
enable_deposit: true # Enable deposit module

# Intake module configuration
enable_intake: true # Enable intake module

# Datarequest module configuration
enable_datarequest: true # Enable datarequest module
datarequest_help_contact_name: PLACEHOLDER # Help contact name
Expand Down
Loading