Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

added the aws and gcp connectors #187

Open
wants to merge 5 commits into
base: main
Choose a base branch
from
Open

added the aws and gcp connectors #187

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
4f3ec60
added the aws and gcp connectors
Ris-code Aug 22, 2024
9b6211f
added the GCP and AWS env variables in language services of docker-co…
Ris-code Aug 26, 2024
1209f33
added more indian languages for AWS and GCP, updated the poetry.lock,…
Ris-code Aug 28, 2024
8bc8984
shifted the aws credentials to key section in .env.template
Ris-code Sep 5, 2024
9e15e64
made the cosmetic changes and mounted the gcp credentials during runtime
Ris-code Sep 14, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions .env-dev.template
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,14 @@ AZURE_SPEECH_REGION=
AZURE_TRANSLATION_KEY=
AZURE_TRANSLATION_RESOURCE_LOCATION=

# AWS Speech/Translation Keys
AWS_ACCESS_KEY_ID=
AWS_SECRET_ACCESS_KEY=
AWS_DEFAULT_REGION=

# Set S3 Bucket name when using AWS speech services
S3_BUCKET_NAME=

# Storage
# -------
# Storage Configuation (Default is local)
Expand All @@ -38,6 +46,9 @@ PUBLIC_URL_PREFIX= # Set Tunnel URL if using local storage
# AZURE_STORAGE_ACCOUNT_KEY=
# AZURE_STORAGE_CONTAINER=

# Set GCP credentials (json)
GOOGLE_APPLICATION_CREDENTIALS=

# Encryption key for storing credentials
ENCRYPTION_KEY=

Expand Down
11 changes: 11 additions & 0 deletions .env.template
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,14 @@ AZURE_SPEECH_REGION=
AZURE_TRANSLATION_KEY=
AZURE_TRANSLATION_RESOURCE_LOCATION=

# AWS Speech/Translation Keys
AWS_ACCESS_KEY_ID=
AWS_SECRET_ACCESS_KEY=
AWS_DEFAULT_REGION=

# Set S3 Bucket name when using AWS speech services
S3_BUCKET_NAME=

# Storage
# -------
# Set Azure storage keys if using Azure Blob Storage
Expand All @@ -39,6 +47,9 @@ AZURE_STORAGE_ACCOUNT_URL=
AZURE_STORAGE_ACCOUNT_KEY=
AZURE_STORAGE_CONTAINER=

# Set GCP credentials (json)
GOOGLE_APPLICATION_CREDENTIALS=

# Encryption key for storing credentials
ENCRYPTION_KEY=

Expand Down
6 changes: 6 additions & 0 deletions docker-compose.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -73,16 +73,22 @@ services:
- AZURE_SPEECH_REGION=${AZURE_SPEECH_REGION}
- AZURE_TRANSLATION_KEY=${AZURE_TRANSLATION_KEY}
- AZURE_TRANSLATION_RESOURCE_LOCATION=${AZURE_TRANSLATION_RESOURCE_LOCATION}
- AWS_ACCESS_KEY_ID=${AWS_ACCESS_KEY_ID}
- AWS_SECRET_ACCESS_KEY=${AWS_SECRET_ACCESS_KEY}
- AWS_DEFAULT_REGION=${AWS_DEFAULT_REGION}
- S3_BUCKET_NAME=${S3_BUCKET_NAME}
- STORAGE_TYPE=${STORAGE_TYPE}
- AZURE_STORAGE_ACCOUNT_URL=${AZURE_STORAGE_ACCOUNT_URL}
- AZURE_STORAGE_ACCOUNT_KEY=${AZURE_STORAGE_ACCOUNT_KEY}
- AZURE_STORAGE_CONTAINER=${AZURE_STORAGE_CONTAINER}
- PUBLIC_URL_PREFIX=${PUBLIC_URL_PREFIX}
- GOOGLE_APPLICATION_CREDENTIALS=/app/credentials/google-credentials.json
depends_on:
- kafka
- postgres
volumes:
- ./media:/mnt/jb_files
- ${GOOGLE_APPLICATION_CREDENTIALS}:/app/credentials/google-credentials.json:ro
flow:
environment:
- POSTGRES_DATABASE_NAME=${POSTGRES_DATABASE_NAME}
Expand Down
849 changes: 637 additions & 212 deletions language/poetry.lock
View file
Open in desktop

Large diffs are not rendered by default.

4 changes: 4 additions & 0 deletions language/pyproject.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,10 @@ azure-cognitiveservices-speech = "^1.38.0"
httpx = "^0.27.0"
aiohttp = "^3.10.0"
pydub = "^0.25.1"
boto3 = "^1.35.3"
google-cloud-speech = "^2.27.0"
google-cloud-texttospeech = "^2.17.1"
google-cloud-translate = "^3.16.0"
KaranrajM marked this conversation as resolved.
Show resolved Hide resolved

[tool.poetry.group.dev.dependencies]
lib = {path = "../jb-lib", develop = true}
Expand Down
8 changes: 5 additions & 3 deletions language/src/extension.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,16 +5,18 @@
AzureSpeechProcessor,
CompositeSpeechProcessor,
DhruvaSpeechProcessor,
AWSSpeechProcessor,
GCPSpeechProcessor,
)
from .translator import AzureTranslator, CompositeTranslator, DhruvaTranslator
from .translator import AzureTranslator, CompositeTranslator, DhruvaTranslator, AWSTranslator, GCPTranslator

# ---- Speech Processor ----
speech_processor = CompositeSpeechProcessor(
DhruvaSpeechProcessor(), AzureSpeechProcessor()
DhruvaSpeechProcessor(), AzureSpeechProcessor(), AWSSpeechProcessor(), GCPSpeechProcessor()
)

# ---- Translator ----
translator = CompositeTranslator(DhruvaTranslator(), AzureTranslator())
translator = CompositeTranslator(DhruvaTranslator(), AzureTranslator(), AWSTranslator(), GCPTranslator())

# ---- Storage ----
storage = StorageHandler.get_async_instance()
Loading