Skip to content

Commit

Permalink
chore: dynamic options
Browse files Browse the repository at this point in the history
  • Loading branch information
abhinavkumarsph committed Feb 13, 2024
1 parent 8576a46 commit 7e2842c
Show file tree
Hide file tree
Showing 3 changed files with 25 additions and 17 deletions.
20 changes: 10 additions & 10 deletions modules/ingestion/pipeline/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,17 +35,17 @@

| Name | Description | Type | Default | Required |
|------|-------------|------|---------|:--------:|
| <a name="input_configuration_body"></a> [configuration\_body](#input\_configuration\_body) | The Data Prepper pipeline configuration in YAML format | `string` | n/a | yes |
| <a name="input_enable_logging"></a> [enable\_logging](#input\_enable\_logging) | If true, will create a cloudwatch log group to monitor the pipeline | `bool` | `true` | no |
| <a name="input_iam_role_name"></a> [iam\_role\_name](#input\_iam\_role\_name) | Name of the pipeline IAM role | `string` | n/a | yes |
| <a name="input_log_group_retention_days"></a> [log\_group\_retention\_days](#input\_log\_group\_retention\_days) | Duration in days for cloudwatch log group retention | `number` | `30` | no |
| <a name="input_max_units"></a> [max\_units](#input\_max\_units) | The maximum pipeline capacity, in Ingestion Compute Units | `number` | n/a | yes |
| <a name="input_min_units"></a> [min\_units](#input\_min\_units) | The minimum pipeline capacity, in Ingestion Compute Units | `number` | n/a | yes |
| <a name="input_name"></a> [name](#input\_name) | Name of the ingestion pipeline | `string` | n/a | yes |
| <a name="input_configuration_body"></a> [configuration\_body](#input\_configuration\_body) | The Data Prepper pipeline configuration in YAML format | `string` | n/a | yes |
| <a name="input_enable_logging"></a> [enable\_logging](#input\_enable\_logging) | If true, will create a cloudwatch log group to monitor the pipeline | `bool` | `true` | no |
| <a name="input_iam_role_name"></a> [iam\_role\_name](#input\_iam\_role\_name) | Name of the pipeline IAM role | `string` | n/a | yes |
| <a name="input_log_group_retention_days"></a> [log\_group\_retention\_days](#input\_log\_group\_retention\_days) | Duration in days for cloudwatch log group retention | `number` | `30` | no |
| <a name="input_max_units"></a> [max\_units](#input\_max\_units) | The maximum pipeline capacity, in Ingestion Compute Units | `number` | n/a | yes |
| <a name="input_min_units"></a> [min\_units](#input\_min\_units) | The minimum pipeline capacity, in Ingestion Compute Units | `number` | n/a | yes |
| <a name="input_name"></a> [name](#input\_name) | Name of the ingestion pipeline | `string` | n/a | yes |
| <a name="input_persistent_buffer_enabled"></a> [persistent\_buffer\_enabled](#input\_persistent\_buffer\_enabled) | Whether persistent buffering should be enabled | `bool` | `false` | no |
| <a name="input_security_group_ids"></a> [security\_group\_ids](#input\_security\_group\_ids) | Security group IDs to attach to the pipeline | `list(string)` | `null` | no |
| <a name="input_subnet_ids"></a> [subnet\_ids](#input\_subnet\_ids) | Subnet IDs to deploy pipeline in. Only needed if pipeline is to be deployed in VPC mode | `list(string)` | `null` | no |
| <a name="input_tags"></a> [tags](#input\_tags) | A map of tags to add to all resources | `map(string)` | `{}` | no |
| <a name="input_security_group_ids"></a> [security\_group\_ids](#input\_security\_group\_ids) | Security group IDs to attach to the pipeline | `list(string)` | `[]` | no |
| <a name="input_subnet_ids"></a> [subnet\_ids](#input\_subnet\_ids) | Subnet IDs to deploy pipeline in. Only needed if pipeline is to be deployed in VPC mode | `list(string)` | `[]` | no |
| <a name="input_tags"></a> [tags](#input\_tags) | A map of tags to add to all resources | `map(string)` | `{}` | no |

## Outputs

Expand Down
18 changes: 13 additions & 5 deletions modules/ingestion/pipeline/pipeline.tf
Original file line number Diff line number Diff line change
Expand Up @@ -5,16 +5,24 @@ resource "aws_osis_pipeline" "this" {
min_units = var.min_units
max_units = var.max_units

vpc_options {
subnet_ids = var.subnet_ids
security_group_ids = var.security_group_ids
dynamic "vpc_options" {
for_each = length(var.subnet_ids) > 0 ? [1] : []

content {
subnet_ids = var.subnet_ids
security_group_ids = var.security_group_ids
}
}

log_publishing_options {
is_logging_enabled = var.enable_logging

cloudwatch_log_destination {
log_group = local.pipeline_log_group
dynamic "cloudwatch_log_destination" {
for_each = var.enable_logging ? [1] : []

content {
log_group = local.pipeline_log_group
}
}
}

Expand Down
4 changes: 2 additions & 2 deletions modules/ingestion/pipeline/variables.tf
Original file line number Diff line number Diff line change
Expand Up @@ -32,13 +32,13 @@ variable "enable_logging" {
variable "subnet_ids" {
description = "Subnet IDs to deploy pipeline in. Only needed if pipeline is to be deployed in VPC mode"
type = list(string)
default = null
default = []
}

variable "security_group_ids" {
description = "Security group IDs to attach to the pipeline"
type = list(string)
default = null
default = []
}

variable "persistent_buffer_enabled" {
Expand Down

0 comments on commit 7e2842c

Please sign in to comment.