tests

LocalEGA testsuite

Unit Tests are run with pytest, coverage and tox.

The other tests use BATS. They also require expect, ssh, ssh-agent and crypt4gh.

Install the required packages with pip install -r requirements.txt.

Integration Tests

These tests treat the system as a black box, only checking the expected output for a given input.

  $ bats tests/integration

• Ingesting a 10MB file
  Expected outcome: Message in the CentralEGA completed queue

• Ingesting a "big" file
  Expected outcome: Message in the CentralEGA completed queue

• Ingesting a directory with multiple files and/or subdirectories
  Expected outcome: Messages in the CentralEGA completed queue

• Upload 2 files encrypted with same session key
  Expected outcome: Message in the CentralEGA user-error queue for the second file

• (skipped) Use 2 stable IDs for the same ingested file
  Expected outcome: Error captured (where?)

• Ingest a file with a user that does not exist in CentralEGA
  Expected outcome: Authentication "fails", and the ingestion does not start

• Ingest a file with a user in CentralEGA, using the wrong password
  Expected outcome: Authentication "fails", and the ingestion does not start

• Ingest a file with a user in CentralEGA, using the wrong sshkey
  Expected outcome: Fallback to password (previous scenario)

• Ingest a file for a given LocalEGA using the key of another one
  Expected outcome: Message in the CentralEGA error queue, with the relevant content.

• Ingestion with wrong file format
  Expected outcome: Message in the CentralEGA error queue, with the relevant content.

• Outgesting a cleartext file
  Expected outcome: Matching the original input file Note: Insecure, we should not support that

• Outgesting a cleartext file, with a range
  Expected outcome: Matching the relevant part of the original input file

• Outgesting a C4GH-formatted file
  Expected outcome: Decrypt and match the original input file

Robustness Tests

These tests will not treat the system as a black box.
They require some knowledge on how the components are interconnected.

  $ bats tests/robustness

• Check Archive+DB consistency
  Expected outcome: Re-checksums the files after several ingestions

• DB restarted after n seconds
  Expected outcome: Combining an ingestion before and one after, the latest one should still "work"

• DB restarted in the middle of an ingestion
  Expected outcome: File ingested as usual

• MQ restarted, test delivery mode
  Expected outcome: queued tasks completed

• Central MQ restarted while LocalEGA shoveling the completion message
  Expected outcome: queued tasks completed

• Retry message 3 times if rejected before error or timeout
  Expected outcome: queued tasks completed

• Restart all components in between 2 ingestions
  Expected outcome: Business as usual

Stress

These tests treat the system as a black box and "measure" performance

  $ bats tests/stress

• Multiple ingestions by the same user
  Expected outcome:

• Ingestions by multiple users
  Expected outcome:

• (Auto?)-Scaling
  Expected outcome:

Security

These tests will not treat the system as a black box.
They require some knowledge on how the components are interconnected.

  $ bats tests/security

• Network access forbidden from some selected components
  Expected outcome:

• Inbox user isolation: A user cannot access the files of another user
  Expected outcome: File not found or access denied

• Database not reachable from the vault (only if S3-backed)
  Expected outcome: ping "db" fails from archive

• Database not reachable from the inbox
  Expected outcome: ping "db" fails from inbox

• Vault not reachable from the db (only if S3-backed)
  Expected outcome: ping "archive" fails from db

• Vault not reachable from the inbox (only if S3-backed)
  Expected outcome: ping "archive" fails from inbox