Integrate Azure Functions deployment with In-Vitro #569

cavinkavi · 2024-12-19T14:11:52Z

Summary

A small summary of the requirements (in one/two sentences).

To integrate Azure Functions deployment with In-Vitro. This feature enables deployment of serverless functions on the Azure Functions platform.

Implementation Notes ⚒️

Briefly outline the overall technical solution. If necessary, identify talking points where the reviewer's attention should be drawn to.

This PR adds a workflow to deploy multiple functions to Azure Functions via ZIP deployment.

Documentation on the steps to deploy the functions to Azure are included in loader.md.

azurefunctions_setup: this folder contains the relevant config and workload files to be used for deployment.
azure_functions.go: initializes Azure resources required for deployment to Azure Functions, such as Resource Group, Storage Account, and Function App. In order to deploy multiple functions via ZIP, each function must have its own specific folder containing required dependencies, before zipping and deploying all the function folders as a single package. After successful deployment, local temporary folders holding the functions and the zip package are cleaned up.
(Note: All cleanup, including local temporary folders, will be done after entire experiment is completed.)

In order to run test script (azure_functions_test.go), follow steps 1 to 5 in loader.md and run go test -v ./pkg/driver/deployment from root.

External Dependencies 🍀

gopkg.in/yaml.v3 for reading Azure configuration files.
Azure CLI for managing Azure resources and deployment.

Breaking API Changes ⚠️

N/A

Simply specify none (N/A) if not applicable.

azurefunctions_setup/shared_azure_workload/azureworkload.py

pkg/driver/deployment/azure_functions.go

wanghanchengchn · 2025-01-15T12:41:41Z

Looks like there is an error in the checks.

Missing commit signatures: It seems the commits are missing Signed-off-by signatures. Please use git commit -s -m "Your commit message" when committing.
Commit history: Consider squashing your commits into a single commit to maintain a clean history.

You can find more details about these requirements in our contributing guide:
https://github.com/vhive-serverless/vHive/blob/main/docs/contributing_to_vhive.md

wanghanchengchn · 2025-01-16T05:10:11Z

Hi @leokondrashov, I've completed my initial review of this PR. @cavinkavi has made the corresponding changes, and the current version looks good to me. Do you have any additional feedback or concerns?

leokondrashov

First, there are no tests for the infrastructure, please add. Better even add them into CI tests.
Second, does Azure support Go deployments? We don't use Python that much and I'm not sure that our main Python implementation is well-behaved. It would be preferable to use Go instead.

leokondrashov · 2025-01-17T06:29:54Z

azurefunctions_setup/shared_azure_workload/azureworkload.py

+# Simulate the busySpin function
+def busy_spin(duration_ms: int) -> None:
+    end_time = perf_counter() + duration_ms / 1000  # Convert ms to seconds
+    while perf_counter() < end_time:
+        continue
+
+# Convert TraceFunctionExecution
+def trace_function_execution(start: float, time_left_milliseconds: int) -> str:
+    time_consumed_milliseconds = int((time.time() - start) * 1000)
+    if time_consumed_milliseconds < time_left_milliseconds:
+        time_left_milliseconds -= time_consumed_milliseconds
+        if time_left_milliseconds > 0:
+            busy_spin(time_left_milliseconds)
+
+    return f"OK - {hostname}"


Please reuse this parts from existing Python implementation (server/trace-func-py/trace_func.py). Don't reimplement because this creates duplicates in the code and incorrect performance comparisons.

leokondrashov · 2025-01-17T06:38:42Z

No need to throw out the Python deployment, though. It could be an option to deploy one of them during runtime. But the default should be Go to align with other deployments.

Signed-off-by: Kavithran <[email protected]>

leokondrashov

Also, please change the message of your commit to something meaningful. Addressing my comments doesn't make any sense when you are looking at the history of commits.

leokondrashov · 2025-01-20T01:59:15Z

pkg/driver/deployment/azure_functions.go

+}
+
+// Helper function to copy files
+func copyFile(src, dst string) error {


Put this function in utils

leokondrashov · 2025-01-24T03:28:58Z

pkg/driver/deployment/azure_functions_test.go

Here you have the end-to-end test. Please add a test that would check the potential zip file for being operational. This wouldn't require the Azure credentials, so we can run it without any concerns about spending. This test would show when we have problems with our file copies in the archive.

Please also add them to the CI, so we can see the results right in the PRs.

At the same time, we can have an end-to-end test by just running the loader.go with the correct config file and checking for the output. There is no reason to add the script that duplicates the experiment pipeline. Better to repurpose it to test specific parts of the pipeline (deploying/cleaning functions, checking for the zip health).

wanghanchengchn self-requested a review January 9, 2025 08:36