[Resolve #1212] Conditional Stacks via "ignore" and "obsolete"

docs/_source/docs/faq.rst

@@ -92,3 +92,22 @@ Sceptre project, see the `sceptre-sam-handler`_ page on PyPI.
 
 
 .. _sceptre-sam-handler: https://pypi.org/project/sceptre-sam-handler/
+
+My CI/CD process uses ``sceptre launch``. How do I delete stacks that aren't needed any more?
+---------------------------------------------------------------------------------------------
+
+Running the ``launch`` command is a very useful "1-stop-shop" to apply changes from Stack Configs,
+creating stacks that don't exist and updating stacks that do exist. This makes it a very useful
+command to configure your CI/CD system to invoke. However, sometimes you need to delete a stack that
+isn't needed any more and you want this automatically applied by the same process.
+
+This "clean up" is complicated by the fact that Sceptre doesn't know about anything that isn't
+in its Stack and StackGroup Configs; If you delete a Stack Config, Sceptre won't know to clean it up.
+
+Therefore, the way to accomplish this "clean up" operation is to perform the change in 2 stages:
+
+1. First, add ``launch_action: "exclude"`` to the Stack Config(s) you want to clean up. This will
+   cause that stack to be deleted (if it exists) when you run ``sceptre launch``. For more information
+   on ``launch_action``, see the :ref:`Stack Config entry on it<launch_action>`.
+2. Once you've launched the Stack(s) to have them deleted this way, only **then** should you
+   delete the Stack Configs.

docs/_source/docs/stack_config.rst

@@ -16,6 +16,7 @@ particular Stack. The available keys are listed below.
 -  `template_path`_ or `template`_ *(required)*
 -  `dependencies`_ *(optional)*
 -  `hooks`_ *(optional)*
+-  `launch_action`_ *(optional)*
 -  `notifications`_ *(optional)*
 -  `on_failure`_ *(optional)*
 -  `parameters`_ *(optional)*
@@ -100,6 +101,48 @@ hooks
 A list of arbitrary shell or Python commands or scripts to run. Find out more
 in the :doc:`hooks` section.
 
+
+.. _launch_action:
+
+launch_action
+~~~~~~~~~~~~~
+* Resolvable: No
+* Can be inherited from StackGroup: Yes
+* Inheritance strategy: Overrides parent if set
+
+This setting determines whether or not the stack should be included or excluded when running
+``sceptre launch``. This must be one of two values: "include" or "exclude". The default value for
+this (which doesn't need to be specified) is "include". This **only** applies to the ``launch``
+command.
+
+If the ``launch_action`` is set to ``"exclude"``, it means that:
+ * If the stack does NOT exist, the Stack will not be created and will be skipped over
+ * If the stack *does* currently exist, it will be deleted.
+
+This setting is especially useful in two situations:
+
+1. You can use it to make conditional stacks that may or may not be included in a Stack Group based
+   upon some Jinja2 logic.
+2. If your CI/CD deployment process runs ``sceptre launch``, you can use this to have stacks deleted
+   when it runs. Once the stack(s) have been deleted, you can then delete the StackConfig file.
+
+For Example:
+
+.. code-block:: yaml
+
+   template:
+       path: "my/test/resources.yaml"
+
+   {% if not var.use_test_resources %}
+   launch_action: "exclude"
+   {% endif %}
+
+.. note::
+
+   The ``launch_action`` configuration only applies to the **launch** command. You can still run
+   ``create``, ``update``, or ``delete`` commands on a stack marked with ``launch_action: "exclude"``.
+   If you do, the command will operate just like it normally does.
+
 notifications
 ~~~~~~~~~~~~~
 * Resolvable: Yes
@@ -245,7 +288,7 @@ For more information on this configuration, its implications, and its uses, see
 :ref:`Sceptre and IAM: iam_role <iam_role_permissions>`.
 
 iam_role_session_duration
-~~~~~~~~
+~~~~~~~~~~~~~~~~~~~~~~~~~
 * Resolvable: No
 * Can be inherited from StackGroup: Yes
 * Inheritance strategy: Overrides parent if set

integration-tests/features/launch-stack-group.feature

@@ -22,19 +22,19 @@ Feature: Launch stack_group
 
   Scenario: launch a stack_group with updates that partially exists
     Given stack "2/A" exists in "CREATE_COMPLETE" state
-    and stack "2/B" does not exist
-    and stack "2/C" does not exist
-    and the template for stack "2/A" is "updated_template.json"
+    And stack "2/B" does not exist
+    And stack "2/C" does not exist
+    And the template for stack "2/A" is "updated_template.json"
     When the user launches stack_group "2"
     Then stack "2/A" exists in "UPDATE_COMPLETE" state
-    and stack "2/B" exists in "CREATE_COMPLETE" state
-    and stack "2/C" exists in "CREATE_COMPLETE" state
+    And stack "2/B" exists in "CREATE_COMPLETE" state
+    And stack "2/C" exists in "CREATE_COMPLETE" state
 
   Scenario: launch a stack_group with updates that already exists
     Given all the stacks in stack_group "2" are in "CREATE_COMPLETE"
-    and the template for stack "2/A" is "updated_template.json"
-    and the template for stack "2/B" is "updated_template.json"
-    and the template for stack "2/C" is "updated_template.json"
+    And the template for stack "2/A" is "updated_template.json"
+    And the template for stack "2/B" is "updated_template.json"
+    And the template for stack "2/C" is "updated_template.json"
     When the user launches stack_group "2"
     Then all the stacks in stack_group "2" are in "UPDATE_COMPLETE"
 
@@ -47,3 +47,16 @@ Feature: Launch stack_group
     Given stack_group "2" does not exist
     When the user launches stack_group "2" with ignore dependencies
     Then all the stacks in stack_group "2" are in "CREATE_COMPLETE"
+
+  Scenario: launch a StackGroup with stacks with launch_type = excluded that has not been launched
+    Given stack_group "launch-actions" does not exist
+    When the user launches stack_group "launch-actions"
+    Then stack "launch-actions/excluded" does not exist
+    And stack "launch-actions/included" exists in "CREATE_COMPLETE" state
+
+  Scenario: launch a StackGroup with stacks with launch_type = excluded that currently exist
+    Given  stack "launch-actions/excluded" exists using "valid_template.json"
+    And stack "launch-actions/included" exists using "valid_template.json"
+    When the user launches stack_group "launch-actions"
+    Then stack "launch-actions/excluded" does not exist
+    And stack "launch-actions/included" exists in "CREATE_COMPLETE" state

integration-tests/features/launch-stack.feature

@@ -2,24 +2,34 @@ Feature: Launch stack
 
   Scenario: launch a new stack
     Given stack "1/A" does not exist
-    and the template for stack "1/A" is "valid_template.json"
+    And the template for stack "1/A" is "valid_template.json"
     When the user launches stack "1/A"
     Then stack "1/A" exists in "CREATE_COMPLETE" state
 
   Scenario: launch a stack that was newly created
     Given stack "1/A" exists in "CREATE_COMPLETE" state
-    and the template for stack "1/A" is "updated_template.json"
+    And the template for stack "1/A" is "updated_template.json"
     When the user launches stack "1/A"
     Then stack "1/A" exists in "UPDATE_COMPLETE" state
 
   Scenario: launch a stack that has been previously updated
     Given stack "1/A" exists in "UPDATE_COMPLETE" state
-    and the template for stack "1/A" is "valid_template.json"
+    And the template for stack "1/A" is "valid_template.json"
     When the user launches stack "1/A"
     Then stack "1/A" exists in "UPDATE_COMPLETE" state
 
   Scenario: launch a new stack with ignore dependencies
     Given stack "1/A" does not exist
-    and the template for stack "1/A" is "valid_template.json"
+    And the template for stack "1/A" is "valid_template.json"
     When the user launches stack "1/A" with ignore dependencies
     Then stack "1/A" exists in "CREATE_COMPLETE" state
+
+  Scenario: launch a stack with launch_type = exclude that doesn't exist
+    Given stack "launch-actions/excluded" does not exist
+    When the user launches stack "launch-actions/excluded"
+    Then stack "launch-actions/excluded" does not exist
+
+  Scenario: launch a stack with launch_type = exclude that does exist
+    Given stack "launch-actions/excluded" exists using "valid_template.json"
+    When the user launches stack "launch-actions/excluded"
+    Then stack "launch-actions/excluded" does not exist

integration-tests/sceptre-project/config/launch-actions/excluded.yaml

@@ -0,0 +1,4 @@
+# This stack shouldn't ever be launched
+launch_action: exclude
+
+# Since the launch type is exclude, we don't need any other attributes

integration-tests/sceptre-project/config/launch-actions/included.yaml

@@ -0,0 +1,5 @@
+# This is the default so it isn't required, but for testing let's explicitly set it.
+launch_action: include
+
+template:
+  path: valid_template.yaml

integration-tests/steps/templates.py

@@ -29,6 +29,7 @@ def set_template_path(context, stack_name, template_name):
     if "template_path" in stack_config:
         stack_config["template_path"] = template_path
     if "template" in stack_config:
+        stack_config["template"]["type"] = stack_config["template"].get("type", "file")
         template_handler_type = stack_config["template"]["type"]
         if template_handler_type.lower() == 's3':
             segments = stack_config["template"]["path"].split('/')

sceptre/cli/launch.py

@@ -1,10 +1,13 @@
 import click
+from click import Context
+from colorama import Fore, Style
 
 from sceptre.context import SceptreContext
 from sceptre.cli.helpers import catch_exceptions
 from sceptre.cli.helpers import confirmation
 from sceptre.cli.helpers import stack_status_exit_code
 from sceptre.plan.plan import SceptrePlan
+from sceptre.stack import LaunchAction
 
 
 @click.command(name="launch", short_help="Launch a Stack or StackGroup.")
@@ -14,15 +17,21 @@
 )
 @click.pass_context
 @catch_exceptions
-def launch_command(ctx, path, yes):
+def launch_command(ctx: Context, path: str, yes: bool):
     """
-    Launch a Stack or StackGroup for a given config PATH.
+    Launch a Stack or StackGroup for a given config PATH. This command is intended as a catch-all
+    command that will apply any changes from Stack Configs indicated via the path.
+
+    \b
+    * Any Stacks that do not exist will be created
+    * Any stacks that already exist will be updated (if there are any changes)
+    * If any stacks are marked with launch_type: exclude, they will NOT be created and, if they exist,
+    will be deleted.
+
     \f
 
     :param path: The path to launch. Can be a Stack or StackGroup.
-    :type path: str
     :param yes: A flag to answer 'yes' to all CLI questions.
-    :type yes: bool
     """
     context = SceptreContext(
         command_path=path,
@@ -33,6 +42,20 @@ def launch_command(ctx, path, yes):
     )
 
     plan = SceptrePlan(context)
+    plan.resolve(plan.launch.__name__)
+
+    stacks_to_exclude = []
+    for stacks in plan.launch_order:
+        for stack in stacks:
+            if stack.launch_action == LaunchAction.exclude:
+                stacks_to_exclude.append(stack)
+
+    if stacks_to_exclude:
+        delete_message = "The following stacks are excluded from the launch. They will be deleted, if they exist:\n"
+        for stack in stacks_to_exclude:
+            delete_message += f"{Fore.YELLOW}{stack.name}{Style.RESET_ALL}\n"
+
+        print(delete_message)
 
     confirmation(plan.launch.__name__, yes, command_path=path)
     responses = plan.launch()

sceptre/config/reader.py

@@ -32,7 +32,7 @@
 from sceptre.exceptions import VersionIncompatibleError
 from sceptre.exceptions import ConfigFileNotFoundError
 from sceptre.helpers import sceptreise_path
-from sceptre.stack import Stack
+from sceptre.stack import Stack, LaunchAction
 from sceptre.config import strategies
 
 ConfigAttributes = collections.namedtuple("Attributes", "required optional")
@@ -58,7 +58,8 @@
     "template_bucket_name": strategies.child_wins,
     "template_key_value": strategies.child_wins,
     "template_path": strategies.child_wins,
-    "template": strategies.child_wins
+    "template": strategies.child_wins,
+    "launch_action": strategies.child_wins
 }
 
 STACK_GROUP_CONFIG_ATTRIBUTES = ConfigAttributes(
@@ -566,6 +567,7 @@ def _construct_stack(self, rel_path, stack_group_config=None):
             notifications=config.get("notifications"),
             on_failure=config.get("on_failure"),
             stack_timeout=config.get("stack_timeout", 0),
+            launch_action=LaunchAction[config.get('launch_action', 'include')],
             stack_group_config=parsed_stack_group_config
         )
 

sceptre/plan/actions.py

@@ -25,6 +25,7 @@
 from sceptre.exceptions import UnknownStackChangeSetStatusError
 from sceptre.exceptions import UnknownStackStatusError
 from sceptre.hooks import add_stack_hooks
+from sceptre.stack import LaunchAction
 from sceptre.stack_status import StackChangeSetStatus
 from sceptre.stack_status import StackStatus
 
@@ -175,7 +176,7 @@ def cancel_stack_update(self):
         return self._wait_for_completion()
 
     @add_stack_hooks
-    def launch(self):
+    def launch(self) -> StackStatus:
         """
         Launches the Stack.
 
@@ -185,10 +186,15 @@ def launch(self):
         performed, launch exits gracefully.
 
         :returns: The Stack's status.
-        :rtype: sceptre.stack_status.StackStatus
         """
         self._protect_execution()
-        self.logger.info("%s - Launching Stack", self.stack.name)
+
+        if self.stack.launch_action == LaunchAction.exclude:
+            self.logger.info(f'{self.stack.name} is excluded - Deleting Stack (if it exists)')
+            return self.delete()
+
+        self.logger.info("f{self.stack.name} - Launching Stack")
+
         try:
             existing_status = self._get_status()
         except StackDoesNotExistError:
@@ -1008,10 +1014,10 @@ def drift_detect(self) -> Dict[str, str]:
         Show stack drift for a running stack.
 
         :returns: The stack drift detection status.
-        If the stack does not exist, we return a detection and
-        stack drift status of STACK_DOES_NOT_EXIST.
-        If drift detection times out after 5 minutes, we return
-        TIMED_OUT.
+            If the stack does not exist, we return a detection and
+            stack drift status of STACK_DOES_NOT_EXIST.
+            If drift detection times out after 5 minutes, we return
+            TIMED_OUT.
         """
         try:
             self._get_status()

sceptre/plan/executor.py

@@ -38,10 +38,10 @@ def __init__(self, command, launch_order):
 
     def execute(self, *args):
         """
-        Execute is responsible executing the sets of Stacks in `launch_order`
+        Execute is responsible executing the sets of Stacks in ``launch_order``
         concurrently, in the correct order.
 
-        :param *args: Any arguments that should be passed through to the\
+        :param args: Any arguments that should be passed through to the
                 StackAction being called.
         """
         responses = {}