SM dev - SM datamodel and CRUD skeleton

[bowenlan-amzn]

Signed-off-by: bowenlan-amzn [email protected]

Issue #, if available:
#280

Description of changes:

Data model

snapshot management policy

{
    "name": "daily-snapshot",
    "id": "daily-snapshot-sm",
    "description": "Daily snapshot",
    "creation": {
        "schedule": {
            "cron": {
                "expression": "* * * * *",
                "timezone": "America/Los_Angeles"
            }
        },
        "timeout": "1h" // if snapshot creation time is longer than this, skip to next schedule
    },
    "deletion": {
        "schedule": {
            "cron": {
                "expression": "*/2 * * * *",
                "timezone": "America/Los_Angeles"
            }
        },
        "timeout": "1h", // if snapshot deletion time is longer than this, skip to next schedule
        "condition": {
            "max_count": 50, // max number of snapshots kept under this policy
            "max_age": "7d", // max age of snapshots kept under this policy
            "min_count": 5 // min number of snapshots kept under this policy, regardless of max_age condition
        }
    },
    "snapshot_config": {
        "date_expression": "{now/d}", // customize the date format of date shown in snapshot name
        "indices": "*", // other fields in snapshot_config are params for native create snapshot API
        "repository": "repo",
        "ignore_unavailable": "",
        "include_global_state": "",
        "partial": "",
        "metadata": {}
    },
    "last_updated_time": 1648694952,
    "enabled": true, // whether the policy is running or not
    "enabled_time": 1648694952,
    "schedule": { // the schedule to run this policy
      "interval": {
        "period": 1,
        "unit": "Minutes",
        "start_time": 1602100553
      }
    },
    "seq_no": 2,
    "primary_term": 1
}

snapshot management metadata

{
    "policy_seq_no": 2, // policy seqNo and primaryTerm used to find out if policy has been changed
    "policy_primary_term": 1,
    "current_state": "create_condition_met", // the current state of state machine
    "info": { // user facing information
        "pre_condition_failed": "cluster is red, will not create snapshot now.",
        "exception": "[2022-03-25]: Caught exception while...",
        "last_success": "",
        "last_failure": ""
    },
    "creation": {
        "trigger": { // recurring actions trigger
            "next_exeution_time": 1602100553
        },
        "started": { // contains the newest snapshot info that has been started to create
          "name": "daily-snapshot-002",
          "start_time": 1602100553 // used to decide whether timeouts
        },
        "finished": { // record the info of last created snapshot
          "name": "daily-snapshot-001",
          "start_time": 1602100553,
          "end_time": 1602100553
        }
    },
    "deletion": {
        "trigger": {
            "next_exeution_time": 1602100553
        },
        "started": [ // list of snapshots info that has been started to delete
          {
            "name": "daily-snapshot-001",
            "start_time": 1602100553,
            "end_time": 1602100553
          }
        ], 
        "started_time": 1602100553 // the time when last deletion started, used to decide whether timeouts
    },
    "id": "daily-snapshot-sm-metadata",
    "seq_no": 2,
    "primary_term": 1
}

APIs

Pour in draft for several APIs, not the finalized version, just the basic structure.

• Create policy
• Update policy
• Get policy
• Delete policy

CheckList:

• Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[downsrob]

Do you want to leave a todo to remove the unnecessary logging like this so we don't forget later? Maybe we could use some standard format like

// TODO SLM <todo>

for things like this so we can easily search later and we don't miss any of the pre-release todos?

[bowenlan-amzn]

How about we use log.info("sm dev: .....") for all the logs that probably need to be removed.
And for all the TODO we add in the project, we use // TODO SM

[bowenlan-amzn]

This commit #345 on main branch are left out, because DCO problem. But when we merge sm-dev to main, this should be covered.

[codecov-commenter]

Codecov Report

Merging #355 (fe9c202) into sm-dev (534bb10) will decrease coverage by 3.41%.
The diff coverage is 9.93%.

@@             Coverage Diff              @@
##             sm-dev     #355      +/-   ##
============================================
- Coverage     75.25%   71.84%   -3.42%     
- Complexity     2144     2151       +7     
============================================
  Files           262      277      +15     
  Lines         12444    13032     +588     
  Branches       1966     2026      +60     
============================================
- Hits           9365     9363       -2     
- Misses         2012     2594     +582     
- Partials       1067     1075       +8     

Impacted Files	Coverage Δ
...t/indexstatemanagement/model/ManagedIndexConfig.kt	81.13% <ø> (ø)
...dexmanagement/indexstatemanagement/model/Policy.kt	84.25% <ø> (ø)
...transport/action/indexpolicy/IndexPolicyRequest.kt	92.59% <0.00%> (ø)
...exmanagement/opensearchapi/OpenSearchExtensions.kt	75.45% <0.00%> (-10.26%)	⬇️
...ch/indexmanagement/rollup/RollupMetadataService.kt	74.28% <ø> (ø)
.../opensearch/indexmanagement/rollup/model/Rollup.kt	85.58% <ø> (-0.47%)	⬇️
...rch/indexmanagement/rollup/model/RollupMetadata.kt	86.04% <ø> (ø)
...ment/api/transport/delete/DeleteSMPolicyRequest.kt	0.00% <0.00%> (ø)
...ent/api/transport/delete/DeleteSMPolicyResponse.kt	0.00% <0.00%> (ø)
...management/api/transport/get/GetSMPolicyRequest.kt	0.00% <0.00%> (ø)
... and 31 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 534bb10...fe9c202. Read the comment docs.