Skip to content

Commit

Permalink
[UPDATE] instructions in README -> MANAGED TO DEPLOY ON GCLOUD
Browse files Browse the repository at this point in the history
  • Loading branch information
BenCretois committed Oct 3, 2023
1 parent 5d72c86 commit 545959b
Show file tree
Hide file tree
Showing 5 changed files with 103 additions and 14 deletions.
65 changes: 55 additions & 10 deletions cloud_analysis/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,26 +4,64 @@

- If lost, there is a button `console` on the upper right corner that brings us back to the projects

## Create the Docker image for Cloud analysis

The Docker image used for the cloud analysis is slightly different:

- The entrypoint uses `cloud_analysis/main.py`
- Some libraries are added to the base `pyproject.toml` (i.e. `Flask`, `distlib`)
- `snowmobile_cloud` also has a `.env` file containing the information for `Gmail` loggin

Make sure you have the following `gmail_logs.env` in the folder `cloud_analysis`:

```
[email protected]
GMAIL_PASS=XXX
[email protected]
```

You may need to **Use an App Password** that you can set from your [Google Account](https://myaccount.google.com/) -> Security -> 2-step validation -> App Password (bottom of the page).

Then run:

```bash
docker build -t snowmobile_cloud -f cloud_Dockerfile .
```

It is possible to test the image by sending a file to the endpoint:

```
docker run --rm -p 8080:8080 -v $PWD:/app snowmobile_cloud
```

And in another terminal:

```bash
cd cloud_analysis
./test.sh
```

The sender should receive an email.

## Login stuff

1- Generate the Google Cloud credentials:

```
```bash
gcloud auth application-default login
```

The credentials are stored in `~/.config/gcloud/`

2- Login to your Google Cloud account:

```
```bash
gcloud auth login
```

3- Log in to the project

```
```bash
gcloud config set project snokuter-akustikk
```

Expand All @@ -33,7 +71,7 @@ gcloud config set project snokuter-akustikk

Enable the Cloud Functions, Cloud Run, and Cloud Build APIs:

```
```bash
gcloud services enable cloudfunctions.googleapis.com
gcloud services enable run.googleapis.com
gcloud services enable cloudbuild.googleapis.com
Expand All @@ -43,22 +81,29 @@ gcloud services enable cloudbuild.googleapis.com

The credentials are stored in `~/.docker/config.json`

```
```bash
gcloud auth configure-docker
```

3- Change the name of the image to push it on GCloud

The Docker image will be saved in Google's container registry, which is directly supported by Cloud Run

```
docker tag ghcr.io/ninanor/snowmobile_analyzer:main gcr.io/snoskuter-akustikk/snowmobile_analyzer:main
docker push gcr.io/snoskuter-akustikk/snowmobile_analyzer:main
```bash
docker tag snowmobile_cloud gcr.io/snoskuter-akustikk/snowmobile_cloud:main
docker push gcr.io/snoskuter-akustikk/snowmobile_cloud:main

```

4- Deploy the Docker image to Google Cloud run

```bash
gcloud run deploy \
model \
--image gcr.io/snoskuter-akustikk/snowmobile_cloud:main \
--memory 1Gi \
--region europe-north1 \
--platform managed
```
gcloud run deploy model --image gcr.io/snoskuter-akustikk/snowmobile_analyzer:main
```

Be sure to allocate enough memory for the service to be able to process the files and run the model.
6 changes: 2 additions & 4 deletions cloud_analysis/main.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@

#!/usr/env/bin python3

from flask import Flask, request, jsonify
Expand Down Expand Up @@ -56,7 +55,6 @@ def send_email(subject, body):
logging.error(f"Error sending email: {e}")



def analyseAudioFile(
audio_file_path, batch_size=1, num_workers=4, min_hr = 0.1, min_conf = 0.99
):
Expand Down Expand Up @@ -138,8 +136,8 @@ def process_audio_endpoint():

detection_count = on_process_audio(audio_id, audio_rec, audio_file_path)

#if detection_count > 0:
send_email("Snowmobile Detection Alert", f"{detection_count} snowmobile detections were made in the audio file!")
if detection_count > 0:
send_email("Snowmobile Detection Alert", f"{detection_count} snowmobile detections were made in the audio file!")

return jsonify({"message": "Audio processing completed!"})

Expand Down
10 changes: 10 additions & 0 deletions cloud_analysis/test_cloud.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
ID_TOKEN=$(gcloud auth print-identity-token)

curl -X POST \
-H "Content-Type: application/json" \
-H "Authorization: Bearer ${ID_TOKEN}" \
-d '{"audio_file_path": "/app/example/example_audio.mp3", "audio_id": "test-id", "audio_rec": {"location": {"latitude": 0, "longitude": 0}}}' \
https://model-4uhtnq5xla-lz.a.run.app/process-audio


# /home/benjamin.cretois/data/snowmobile/example_audio.mp3
File renamed without changes.
36 changes: 36 additions & 0 deletions cloud_analysis/trigger_audio_analysis.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
import requests
import os
from google.cloud import storage

def trigger_audio_analysis(data, context):
"""Background Cloud Function to be triggered by Cloud Storage.
This function is triggered when a file is uploaded to the GCS bucket.
"""
file_name = data['name']
bucket_name = data['bucket']

# URL of your Cloud Run service
cloud_run_url = "https://model-4uhtnq5xla-lz.a.run.app/process-audio-gcs"

# ID Token for authentication
id_token = os.environ.get("ID_TOKEN")

# Payload to send to Cloud Run
payload = {
"bucket_name": bucket_name,
"blob_name": file_name,
"audio_id": "example-audio-id",
"audio_rec": {"location": {"latitude": 0, "longitude": 0}}
}

# Headers for the request
headers = {
"Content-Type": "application/json",
"Authorization": f"Bearer {id_token}"
}

# Send a POST request to the Cloud Run service
response = requests.post(cloud_run_url, json=payload, headers=headers)

# Log the response
print(f"Response from Cloud Run: {response.text}")

0 comments on commit 545959b

Please sign in to comment.