Skip to content
This repository has been archived by the owner on Jan 2, 2025. It is now read-only.

Commit

Permalink
add autoscaling
Browse files Browse the repository at this point in the history
  • Loading branch information
Louis-Jellyfish committed Nov 20, 2023
1 parent b8d4760 commit 7e7162c
Show file tree
Hide file tree
Showing 2 changed files with 4 additions and 0 deletions.
2 changes: 2 additions & 0 deletions deployment/deploy_cloud.sh
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,8 @@ python3 -m main \
--template_location "gs://$2/templates/megalista" \
--num_workers 1 \
--autoscaling_algorithm=NONE \
--autoscaling_algorithm=THROUGHPUT_BASED \
--number_of_worker_harness_threads=16 \
--service_account_email "$4" \
--collect_usage_stats "$5"
echo "Copy megalista_medata to bucket $2"
Expand Down
2 changes: 2 additions & 0 deletions megalista_dataflow/steps/processing_steps.py
Original file line number Diff line number Diff line change
Expand Up @@ -523,6 +523,8 @@ def expand(self, executions):
20,
TransactionalType.UUID,
)
| "Prevent Fusion"
>> beam.Reshuffle()
| "Upload - GA 4 measurement protocol"
>> beam.ParDo(
GoogleAnalytics4MeasurementProtocolUploaderDoFn(
Expand Down

0 comments on commit 7e7162c

Please sign in to comment.