add data source.md

Creating a Kestra workflow to accomplish the specified tasks involves a series of steps. Here's how you can set up the workflow:

1. Take input from the user for the database connection hostname, port number, and service name.
2. Update the Trino egress network policy file and push it to Bitbucket.
3. Update the config map and mount it in the Trino deployment and stateful set.
4. Kickoff the Jenkins pipeline to deploy these changes to the Kubernetes cluster.

Here's a detailed Kestra workflow definition:

id: trino-update-deployment
namespace: devops
description: |
  A workflow to update the Trino egress network policy file, push it to Bitbucket,
  update the config map, mount it in the Trino deployment and stateful set, 
  and kickoff the Jenkins pipeline to deploy the changes to a Kubernetes cluster.

inputs:
  - name: db_hostname
    type: STRING
    required: true
  - name: db_port
    type: STRING
    required: true
  - name: db_service_name
    type: STRING
    required: true

tasks:
  - id: clone-repo
    type: io.kestra.plugin.scripts.shell.Bash
    description: Clone the Bitbucket repository
    commands:
      - git clone https://bitbucket.org/your-repo/trino-configs.git
      - cd trino-configs
    errorBehavior: STOP

  - id: update-network-policy
    type: io.kestra.plugin.scripts.shell.Bash
    description: Update the Trino egress network policy file with the new database connection details
    commands:
      - cd trino-configs
      - |
        sed -i 's/old-db-hostname/${{ inputs.db_hostname }}/g' trino-egress-policy.yaml
        sed -i 's/old-db-port/${{ inputs.db_port }}/g' trino-egress-policy.yaml
        sed -i 's/old-db-service-name/${{ inputs.db_service_name }}/g' trino-egress-policy.yaml
      - git config user.name "kestra-bot"
      - git config user.email "[email protected]"
      - git add trino-egress-policy.yaml
      - git commit -m "Update egress network policy with new DB connection details: ${{ inputs.db_hostname }}, ${{ inputs.db_port }}, ${{ inputs.db_service_name }}"
      - git push origin main
    errorBehavior: STOP

  - id: update-config-map
    type: io.kestra.plugin.scripts.shell.Bash
    description: Update the config map with the new database connection details
    commands:
      - cd trino-configs
      - |
        sed -i 's/old-db-hostname/${{ inputs.db_hostname }}/g' config-map.yaml
        sed -i 's/old-db-port/${{ inputs.db_port }}/g' config-map.yaml
        sed -i 's/old-db-service-name/${{ inputs.db_service_name }}/g' config-map.yaml
      - git config user.name "kestra-bot"
      - git config user.email "[email protected]"
      - git add config-map.yaml
      - git commit -m "Update config map with new DB connection details: ${{ inputs.db_hostname }}, ${{ inputs.db_port }}, ${{ inputs.db_service_name }}"
      - git push origin main
    errorBehavior: STOP

  - id: update-deployment
    type: io.kestra.plugin.kubernetes.jobs.JobExec
    description: Update the Trino deployment with the new config map
    namespace: trino-namespace
    jobSpec:
      metadata:
        name: update-trino-deployment
      spec:
        template:
          spec:
            containers:
              - name: kubectl
                image: bitnami/kubectl:latest
                command:
                  - /bin/sh
                  - -c
                  - |
                    kubectl apply -f trino-configs/config-map.yaml
                    kubectl rollout restart deployment trino-deployment
                    kubectl rollout restart statefulset trino-statefulset
    errorBehavior: STOP

  - id: trigger-jenkins-pipeline
    type: io.kestra.plugin.http.Request
    description: Trigger the Jenkins pipeline to deploy the changes
    uri: https://jenkins.yourdomain.com/job/your-pipeline/buildWithParameters
    method: POST
    headers:
      - key: Authorization
        value: Basic your_jenkins_token
    body: |
      {
        "parameter": [
          {
            "name": "DB_HOSTNAME",
            "value": "${{ inputs.db_hostname }}"
          },
          {
            "name": "DB_PORT",
            "value": "${{ inputs.db_port }}"
          },
          {
            "name": "DB_SERVICE_NAME",
            "value": "${{ inputs.db_service_name }}"
          }
        ]
      }
    errorBehavior: STOP

  - id: wait-for-jenkins
    type: io.kestra.plugin.jenkins.JobStatus
    description: Wait for the Jenkins pipeline to complete
    jenkins:
      url: https://jenkins.yourdomain.com
      username: your_jenkins_username
      apiToken: your_jenkins_token
      job: your-pipeline
    pollingInterval: PT30S
    timeout: PT30M

  - id: notify-success
    type: io.kestra.plugin.slack.SendMessage
    description: Send a Slack notification on successful deployment
    token: your_slack_token
    channel: your-slack-channel
    text: "The Trino egress network policy and config map have been successfully updated and deployed for DB connection: ${{ inputs.db_hostname }}, ${{ inputs.db_port }}, ${{ inputs.db_service_name }}"

  - id: notify-failure
    type: io.kestra.plugin.slack.SendMessage
    description: Send a Slack notification on deployment failure
    token: your_slack_token
    channel: your-slack-channel
    text: "Failed to update and deploy the Trino egress network policy and config map for DB connection: ${{ inputs.db_hostname }}, ${{ inputs.db_port }}, ${{ inputs.db_service_name }}"

    when:
      condition: "{{ task('wait-for-jenkins').status == 'FAILED' }}"

  - id: clean-up
    type: io.kestra.plugin.scripts.shell.Bash
    description: Clean up the local repository
    commands:
      - rm -rf trino-configs

    when:
      condition: "{{ task('wait-for-jenkins').status == 'COMPLETED' }} || {{ task('wait-for-jenkins').status == 'FAILED' }}"

Explanation

1. User Input: The inputs section collects the database connection details from the user.
2. Clone Repository: The clone-repo task clones the Bitbucket repository containing the Trino configuration files.
3. Update Network Policy: The update-network-policy task updates the Trino egress network policy file with the new database connection details and pushes the changes to Bitbucket.
4. Update Config Map: The update-config-map task updates the config map with the new database connection details and pushes the changes to Bitbucket.
5. Update Deployment: The update-deployment task applies the updated config map to the Kubernetes cluster and restarts the Trino deployment and stateful set.
6. Trigger Jenkins Pipeline: The trigger-jenkins-pipeline task triggers the Jenkins pipeline to deploy the changes.
7. Wait for Jenkins Completion: The wait-for-jenkins task polls Jenkins to check the status of the pipeline.
8. Notification: The notify-success and notify-failure tasks send Slack notifications based on the result of the Jenkins pipeline.
9. Clean Up: The clean-up task removes the cloned repository to clean up the local environment.

Ensure you replace placeholders like your-repo, yourdomain.com, your-pipeline, your_jenkins_token, your_jenkins_username, and your_slack_token with your actual values. This workflow automates the entire process from user input to deployment, providing notifications and cleaning up afterward.