Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Support for Dynamic Settings #1837

Merged
merged 8 commits into from
Nov 12, 2023

Conversation

rochacbruno
Copy link
Member

@rochacbruno rochacbruno commented Aug 9, 2023

Issue: AAH-2009

Feature is deactivated by default,
to activate Requires Dynaconf 3.2.2

What this PR is doing?

  1. This PR adds a new table to the models config.Setting, this table will store key:value pairs to be dynamically added to Dynaconf data and be available through django.conf.settings
    IMPORTANT key/value are stores in the database as str, upon access value can be read raw or parsed by dynaconf/toml, model also stores a timestamp

  2. Along with the model, the PR also adds a schema on dynamic_settings.py:DYNACMIC_SETTINGS_SCHEMA which defines which keys can be stored on the database, as we cannot allow all settings to be dynamic.

  3. A management command django-admin galaxy-settings is also added with commands to set,get,delete,list,inspect the settings on the database.

  4. Every time a Setting is mutated on database it will also update a cache on Redis, if Redis is not available then caching will be skipped (and logged)

  5. Finally, the PR adds a Dynaconf method hook to the dynaconf:Settings:get method, making access to any variables listed on the schema to trigger an immediate lookup on cache -> db IMPORTANT this last feature requires Dynaconf 3.2.2 (to be released)

NOTE The PR can be merged before Dynaconf 3.2.2 is released, there will be no breaking, only the feature n 5 will not be enabled.

  1. This PR adds a new variable to settings.py GALAXY_DYNAMIC_SETTINGS=False this is the main toggle for the feature n 5, this is by default set to False, which means, django.conf.settings will lookup for keys in the cache/db only if this is set to True.

How to add settings to the database?

django-admin galaxy-settings set --key=GALAXY_FEATURE_FLAGS__power_mode --value=true
django-admin galaxy-settings get --key GALAXY_FEATURE_FLAGS 

More details on django-admin galaxy-settings --help

Or via script/shell

from galaxy_ng.app.models.config import Setting
Setting.set_value_in_db("GALAXY_FEATURE_FLAGS", "@merge power_mode=true")
#or equivalent 
Setting.set_value_in_db("GALAXY_FEATURE_FLAGS__power_mode", "true")

Currently there is no API (WIP) so command and script is the only way to manage the settings.

Due to the validation, it is not recommended to write directly to the database.

How to read settings from database with current version of dynaconf?

from galaxy_ng.app.models.config import Setting
result = Setting.get("GALAXY_FEATURE_FLAGS")
# Will lookup DB and then merge with current django/dynaconf settings
# before returning the result

How to read settings from db/cache after the upgrade to Dynaconf 3.2.2

1st enable GALAXY_DYNAMIC_SETTINGS=True via settings.py or PULP_ prefixed envvar.

then access settings normally using django.

from django.conf import settings
result = settings.GALAXY_FEATURE_FLAGS  
# upon access to key listed on DYNAMIC_SETTINGS_SCHEMA
# dynaconf will magically lookup on cache/db to merge data
# before returning the result.

What about the API/UI?

  • API will be added in another PR
  • UI will be created based on DYNAMIC_SETTINGS_SCHEMA and will be added after the API

@github-actions github-actions bot added backport-4.2 This PR should be backported to stable-4.2 (1.2) backport-4.4 This PR should be backported to stable-4.4 (2.1) backport-4.5 This PR should be backported to stable-4.5 (2.2) backport-4.6 This PR should be backported to stable-4.6 (2.3) backport-4.7 This PR should be backported to stable-4.7 (2.4) labels Aug 9, 2023
@rochacbruno rochacbruno removed backport-4.2 This PR should be backported to stable-4.2 (1.2) backport-4.4 This PR should be backported to stable-4.4 (2.1) backport-4.5 This PR should be backported to stable-4.5 (2.2) backport-4.6 This PR should be backported to stable-4.6 (2.3) backport-4.7 This PR should be backported to stable-4.7 (2.4) labels Aug 9, 2023
rochacbruno added a commit to dynaconf/dynaconf that referenced this pull request Aug 9, 2023
This PR allows changing the wrapped class with
a class that allows hooks on specific methods.

This will enable ansible/galaxy_ng#1837

This PR also fixes: #969
rochacbruno added a commit to dynaconf/dynaconf that referenced this pull request Aug 9, 2023
This PR allows changing the wrapped class with
a class that allows hooks on specific methods.

This will enable ansible/galaxy_ng#1837

This PR also fixes: #969
rochacbruno added a commit to dynaconf/dynaconf that referenced this pull request Aug 9, 2023
This PR allows changing the wrapped class with
a class that allows hooks on specific methods.

This will enable ansible/galaxy_ng#1837

This PR also fixes: #969
@rochacbruno rochacbruno force-pushed the new_dynamic_settings branch 4 times, most recently from fdea3ad to 441f981 Compare August 10, 2023 21:31
rochacbruno added a commit to dynaconf/dynaconf that referenced this pull request Aug 11, 2023
This PR allows changing the wrapped class with
a class that allows hooks on specific methods.

This will enable ansible/galaxy_ng#1837

This PR also fixes: #969
@rochacbruno rochacbruno marked this pull request as ready for review August 18, 2023 18:33
@rochacbruno rochacbruno force-pushed the new_dynamic_settings branch 3 times, most recently from 3dc80db to 47cc64b Compare September 20, 2023 16:00
@rochacbruno
Copy link
Member Author

/retest

Copy link
Contributor

@slemrmartin slemrmartin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 Awesome work

@rochacbruno rochacbruno merged commit 9eaf00a into ansible:master Nov 12, 2023
22 checks passed
@rochacbruno rochacbruno deleted the new_dynamic_settings branch November 12, 2023 14:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants