[FEATURE] Integrate multitenancy into workflow provisioning

[dbwiddis]

Is your feature request related to a problem?

PR #980 implements multi-tenancy for the Flow Framework system indices (templates, workflow state, and config). The tenant ID has not yet ben integrated into (de-)(re-)provisioning, however.

What solution would you like?

Pass the tenantId available (once #980 is merged) in the (de-)(re-)provisioning transport actions to the workflow steps that execute the provisioning, and eventually to the respective client calls.

Do you have any additional context?

Start point

The tenantId has already been passed to the necessary TransportActions far enough to validate it but has not yet been passed to the workflow execution steps. This should probably start with the executeWorkflowAsync() calls in the Provision and Reprovision actions, and executeDeprovisionSequence() call in the Deprovision action.

End point

The calls to the MLClient here need to be changed to pass the tenantId parameter expected here

• Note this code is actively under development. Flow Framework implementation is on main and the linked MLClient changes are on the ML Commons plugin feature/multi-tenancy branch (tracking 2.x). @dhrubo-os is migrating that code to ML Commons main in Primary setup for Multi-tenancy ml-commons#3307 and follow-on PRs. Initial development should probably be done on 2.x using that feature branch as a local publication; and then cherry-picked to a branch off main prior to being submitted as a Pull Request.
• Note that while some APIs add a tenantId parameter (e.g., Delete Connector) others which will include the tenant ID in one of the existing arguments do not add that parameter (e.g., Create Connector where it's included in the connector input parameter). So each client/API call needs to be evaluated to see exactly how the tenant ID needs to be passed.

In between

Exactly how to get from start to end is an exercise left to the reader. :)

• One possible solution is to pass additional argument(s) on the ProcessNode method(s) or constructor. Note that backwards compatibility must be retained, so new methods (or constructors) with a tenant ID should be called from the existing methods (or constructors) as an overload, passing null for the id.
• Other solutions may exist leveraging existing ways to get input arguments into workflow steps.

[dbwiddis]

@siddharthabingi You don't need to wait for #980 to get merged to get started on this. This commit: 99d273b shows how I just added a new overload method signature, temporarily with deprecated annotation. Once you get the tenant ID into the workflow steps I can add it to the method call and remove the deprecated one.

Here's the old method signature, delegating to the new one (with deprecated annotation added so we can easily find it later and make sure we get it removed, and a link to the issue tracking why it's there):

    /**
     * Removes a resource from the state index, including common exception handling
     * @param workflowId The workflow document id in the state index
     * @param resourceToDelete The resource to delete
     * @param listener the ActionListener for this step to handle completing the future after update
     * @deprecated here temporarily until tenantID passed https://github.com/opensearch-project/flow-framework/issues/987
     */
    @Deprecated
    public void deleteResourceFromStateIndex(String workflowId, ResourceCreated resourceToDelete, ActionListener<WorkflowData> listener) {
        deleteResourceFromStateIndex(workflowId, null, resourceToDelete, listener);
    }

And the new method with the new argument:

    /**
     * Removes a resource from the state index, including common exception handling
     * @param workflowId The workflow document id in the state index
     * @param tenantId The tenant id
     * @param resourceToDelete The resource to delete
     * @param listener the ActionListener for this step to handle completing the future after update
     */
    public void deleteResourceFromStateIndex(
        String workflowId,
        String tenantId,
        ResourceCreated resourceToDelete,
        ActionListener<WorkflowData> listener
    ) {
        // existing code from original method here
    }