[Epic] Automatic schema updates in a deployment

[fridex]

Is your feature request related to a problem? Please describe.

If there are schema changes, new components are updated and they all run code that expects new schema. Once the release is done, one needs to manually trigger the schema update procedure in the deployment so that the database schema matches code run in components.

The manual schema update is prone to errors and we can easily forgot to run the update.

Describe the solution you'd like

Create a mechanism that runs schema update on each release, as necessary. This can be a simple job that is run on each new release and makes sure the database schema is updated.

Describe alternatives you've considered

Keep this as a manual procedure, but that is prone to errors.

Additional context

As of now, we have schema updates in workflow-helpers.

It would be great if the update procedure would report any issues so a rollback can be done (also on components level so the system keeps operational).

Acceptance criteria

• [spike][2pt] Research on methods to trigger automatic schema update. #2582
• Implement the method for the trigger. #2583
  • (alternative) use argocd notification job to trigger management-api schema update endpoint.
• Use bots to trigger kebechet updates and make releases. #2584

Goal

• Automate the schema update flow.

[fridex]

/triage accepted
/priority backlog

CC @codificat if this needs better labeling.

[codificat]

if this needs better labeling

it is perfect, thanks!

Having said that, let me add:
/sig devops
while I review some updates to the SIGs...

[sesheta]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

/lifecycle stale

[sesheta]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

/lifecycle rotten

[codificat]

/remove-lifecycle rotten

[sesheta]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

/lifecycle stale

[sesheta]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

/lifecycle rotten

[harshad16]

/remove-lifecycle rotten
/lifecycle frozen