Skip to content

Commit

Permalink
Simplified tests with two log messages for four test environments
Browse files Browse the repository at this point in the history
The PATH values vary based on a user's local system and could have additional values which could fail the test in a local environment run without docker containers.

Why do we even need the PATH, PYTHONPATH values in the assertion log messages.
This is because we are printing the config in get_database.py which contains environment variables or config file values.

Now, the question is why to print the entire set of values if we are only using DB_HOST, DB_RESULT_LIMIT?

I am now printing only these values in the get_database.py. This has two benefits:
- Allows to eliminate variable environment variables like PATH that might vary in local testing environments for every user.
- Since PATH is no longer printed, helps avoid issue of 'apple security' values.
- Helps reduce the number of log messages the 4 testing environments: local without docker, local with docker, CI/CD without docker, CI/CD with docker. The two docker environments have the same log message; similarly for the two non-docker environments. Thus we only need two log messages in the assertion tests.
  • Loading branch information
Mahadik, Mukul Chandrakant authored and Mahadik, Mukul Chandrakant committed Aug 26, 2024
1 parent 1a13a49 commit 0efbdc8
Show file tree
Hide file tree
Showing 2 changed files with 9 additions and 17 deletions.
2 changes: 1 addition & 1 deletion emission/core/get_database.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,9 +15,9 @@
config = ecbc.get_config('conf/storage/db.conf',
{"DB_HOST": "timeseries.url", "DB_RESULT_LIMIT": "timeseries.result_limit"})

print("Retrieved config %s" % config)
url = config.get("DB_HOST", "localhost")
result_limit = config.get("DB_RESULT_LIMIT", 250000)
print("Retrieved config: {'DB_HOST': '%s', 'DB_RESULT_LIMIT': '%s'}" % (url, result_limit))

try:
parsed=pymongo.uri_parser.parse_uri(url)
Expand Down
24 changes: 8 additions & 16 deletions emission/tests/storageTests/TestTokenQueries.py
Original file line number Diff line number Diff line change
Expand Up @@ -168,28 +168,20 @@ def test_run_script_show(self):
esdt.insert({'token':'y'})
esdt.insert({'token':'z'})
sp = subprocess.run(["python3", "bin/auth/insert_tokens.py", "--show"], capture_output=True)
# The first message is displayed when we run tests locally
# The second is displayed when we run in the CI/CD, but with the local install
# The third is displayed when we run in the docker CI since the `DB_HOST` is set to `db`
# The fourth message is displayed when we locally run a docker container and run tests with the emission environment; we are now setting DB_HOST to `db` by default
# The first message is displayed when we run tests locally or run in the CI/CD, but with the local install
# The second is displayed when we run tests locally in a docker container or run in the docker CI; the `DB_HOST` is set to `db` by default
self.assertIn(sp.stdout,
[b'Config file not found, returning a copy of the environment variables instead...\nRetrieved config {\'DB_HOST\': \'localhost\', \'DB_RESULT_LIMIT\': 250000}\nURL not formatted, defaulting to "Stage_database"\nConnecting to database URL localhost\nx\ny\nz\n',
b'Config file not found, returning a copy of the environment variables instead...\nRetrieved config {\'PYTHONPATH\': \'.\', \'PATH\': \'/home/runner/miniconda-23.5.2/envs/emissiontest/bin:/home/runner/miniconda-23.5.2/condabin:/snap/bin:/home/runner/.local/bin:/opt/pipx_bin:/home/runner/.cargo/bin:/home/runner/.config/composer/vendor/bin:/usr/local/.ghcup/bin:/home/runner/.dotnet/tools:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/home/runner/.dotnet/tools\', \'LC_CTYPE\': \'C.UTF-8\'}\nURL not formatted, defaulting to "Stage_database"\nConnecting to database URL localhost\nx\ny\nz\n',
b'Config file not found, returning a copy of the environment variables instead...\nRetrieved config {\'PYTHONPATH\': \'.\', \'DB_HOST\': \'db\', \'PATH\': \'/root/miniconda-23.5.2/envs/emissiontest/bin:/root/miniconda-23.5.2/condabin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin\', \'LC_CTYPE\': \'C.UTF-8\'}\nURL not formatted, defaulting to "Stage_database"\nConnecting to database URL db\nx\ny\nz\n',
b'Config file not found, returning a copy of the environment variables instead...\nRetrieved config {\'PYTHONPATH\': \'.\', \'DB_HOST\': \'db\', \'PATH\': \'/root/miniconda-23.5.2/envs/emission/bin:/root/miniconda-23.5.2/condabin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin\', \'LC_CTYPE\': \'C.UTF-8\'}\nURL not formatted, defaulting to "Stage_database"\nConnecting to database URL db\nx\ny\nz\n'
[b'Config file not found, returning a copy of the environment variables instead...\nRetrieved config: {\'DB_HOST\': \'localhost\', \'DB_RESULT_LIMIT\': \'250000\'}\nURL not formatted, defaulting to "Stage_database"\nConnecting to database URL localhost\nx\ny\nz\n',
b'Config file not found, returning a copy of the environment variables instead...\nRetrieved config: {\'DB_HOST\': \'db\', \'DB_RESULT_LIMIT\': \'250000\'}\nURL not formatted, defaulting to "Stage_database"\nConnecting to database URL db\nx\ny\nz\n'
])

def test_run_script_empty(self):
sp = subprocess.run(["python3", "bin/auth/insert_tokens.py"], capture_output=True)
# The first message is displayed when we run tests locally
# The second is displayed when we run in the CI/CD, but with the local install
# The third is displayed when we run in the docker CI since the `DB_HOST` is set to `db`
# The fourth message is displayed when we locally run a docker container and run tests with the emission environment; we are now setting DB_HOST to `db` by default
# The first message is displayed when we run tests locally or run in the CI/CD, but with the local install
# The second is displayed when we run tests locally in a docker container or run in the docker CI; the `DB_HOST` is set to `db` by default
self.assertIn(sp.stdout,
[b'Config file not found, returning a copy of the environment variables instead...\nRetrieved config {\'DB_HOST\': \'localhost\', \'DB_RESULT_LIMIT\': 250000}\nURL not formatted, defaulting to "Stage_database"\nConnecting to database URL localhost\nPlease provide the script with an argument. Use the "--help" option for more details\n',
b'Config file not found, returning a copy of the environment variables instead...\nRetrieved config {\'PYTHONPATH\': \'.\', \'PATH\': \'/home/runner/miniconda-23.5.2/envs/emissiontest/bin:/home/runner/miniconda-23.5.2/condabin:/snap/bin:/home/runner/.local/bin:/opt/pipx_bin:/home/runner/.cargo/bin:/home/runner/.config/composer/vendor/bin:/usr/local/.ghcup/bin:/home/runner/.dotnet/tools:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/home/runner/.dotnet/tools\', \'LC_CTYPE\': \'C.UTF-8\'}\nURL not formatted, defaulting to "Stage_database"\nConnecting to database URL localhost\nPlease provide the script with an argument. Use the "--help" option for more details\n',
b'Config file not found, returning a copy of the environment variables instead...\nRetrieved config {\'PYTHONPATH\': \'.\', \'DB_HOST\': \'db\', \'PATH\': \'/root/miniconda-23.5.2/envs/emissiontest/bin:/root/miniconda-23.5.2/condabin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin\', \'LC_CTYPE\': \'C.UTF-8\'}\nURL not formatted, defaulting to "Stage_database"\nConnecting to database URL db\nPlease provide the script with an argument. Use the "--help" option for more details\n',
b'Config file not found, returning a copy of the environment variables instead...\nRetrieved config {\'PYTHONPATH\': \'.\', \'DB_HOST\': \'db\', \'PATH\': \'/root/miniconda-23.5.2/envs/emission/bin:/root/miniconda-23.5.2/condabin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin\', \'LC_CTYPE\': \'C.UTF-8\'}\nURL not formatted, defaulting to "Stage_database"\nConnecting to database URL db\nPlease provide the script with an argument. Use the "--help" option for more details\n'
[b'Config file not found, returning a copy of the environment variables instead...\nRetrieved config: {\'DB_HOST\': \'localhost\', \'DB_RESULT_LIMIT\': \'250000\'}\nURL not formatted, defaulting to "Stage_database"\nConnecting to database URL localhost\nPlease provide the script with an argument. Use the "--help" option for more details\n',
b'Config file not found, returning a copy of the environment variables instead...\nRetrieved config: {\'DB_HOST\': \'db\', \'DB_RESULT_LIMIT\': \'250000\'}\nURL not formatted, defaulting to "Stage_database"\nConnecting to database URL db\nPlease provide the script with an argument. Use the "--help" option for more details\n'
])

#test that no two options can be used together
Expand Down

0 comments on commit 0efbdc8

Please sign in to comment.