Skip to content

Commit

Permalink
Data Server Updates (#185)
Browse files Browse the repository at this point in the history
  • Loading branch information
thpang authored Aug 18, 2021
1 parent 949135f commit f414906
Show file tree
Hide file tree
Showing 12 changed files with 175 additions and 154 deletions.
2 changes: 1 addition & 1 deletion Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ RUN apk --update --no-cache add git openssh \
&& chmod 755 ./kubectl /viya4-iac-azure/docker-entrypoint.sh \
&& mv ./kubectl /usr/local/bin/kubectl \
&& chmod g=u -R /etc/passwd /etc/group /viya4-iac-azure \
&& chdir /viya4-iac-azure ; terraform init
&& terraform init

ENV TF_VAR_iac_tooling=docker
ENTRYPOINT ["/viya4-iac-azure/docker-entrypoint.sh"]
Expand Down
64 changes: 49 additions & 15 deletions docs/CONFIG-VARS.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ Supported configuration variables are listed in the tables below. All variables
- [NFS Server VM (only when `storage_type=standard`)](#nfs-server-vm-only-when-storage_typestandard)
- [Azure NetApp Files (only when `storage_type=ha`)](#azure-netapp-files-only-when-storage_typeha)
- [Azure Container Registry (ACR)](#azure-container-registry-acr)
- [PostgreSQL](#postgresql)
- [Postgres Servers](#postgres-servers)

Terraform input variables can be set in the following ways:

Expand Down Expand Up @@ -298,20 +298,54 @@ When `storage_type=ha` (high availability), [Microsoft Azure NetApp Files](https
| container_registry_admin_enabled | Enables the admin user | bool | false | |
| container_registry_geo_replica_locs | List of Azure locations where the container registry should be geo-replicated. | list of strings | null | This is only supported when `container_registry_sku` is set to `"Premium"`. |

## PostgreSQL
## Postgres Servers

When setting up ***external database servers***, you must provide information about those servers in the `postgres_servers` variable block. Each entry in the variable block represents a ***single database server***.

This code only configures database servers. No databases are created during the infrastructure setup.

The variable has the following format:

```terraform
postgres_servers = {
default = {},
...
}
```

**NOTE**: The `default = {}` elements is always required when creating external databases. This is the systems default database server.

Each server element, like `foo = {}`, can contain none, some, or all of the parameters listed below:

| Name | Description | Type | Default | Notes |
| :--- | ---: | ---: | ---: | ---: |
| create_postgres | Create an Azure Database for PostgreSQL server instance | bool | false | |
| postgres_sku_name| The SKU Name for the PostgreSQL Server | string | "GP_Gen5_32" | The name pattern is the SKU, followed by the tier + family + cores (e.g. B_Gen4_1, GP_Gen5_4).|
| postgres_storage_mb | Max storage allowed for the PostgreSQL server. | number | 51200 | Possible values are between 5120 MB(5GB) and 1048576 MB(1TB) for the Basic SKU and between 5120 MB(5GB) and 4194304 MB(4TB) for General Purpose/Memory Optimized SKUs |
| postgres_backup_retention_days | Backup retention days for the PostgreSQL server. | number | 7 | Supported values are between 7 and 35 days. |
| postgres_geo_redundant_backup_enabled | Whether to enable Geo-redundant for server backup. | bool | false | Not supported for the basic tier. |
| postgres_administrator_login | The Administrator Login for the PostgreSQL server. Changing this forces a new resource to be created. | string | "pgadmin" | The admin login name cannot be azure_superuser, azure_pg_admin, admin, administrator, root, guest, or public. It cannot start with pg_. See: [Microsoft Quickstart Server Database](https://docs.microsoft.com/en-us/azure/postgresql/quickstart-create-server-database-portal) |
| postgres_administrator_password | The password associated with the postgres_administrator_login for the PostgreSQL Server. | string | | The password must contain between 8 and 128 characters and must contain characters from three of the following categories: English uppercase letters, English lowercase letters, numbers (0 through 9), and non-alphanumeric characters (!, $, #, %, etc.). |
| postgres_server_version | The version of the Azure Database for PostgreSQL server instance. Valid values are "9.5", "9.6", "10.0", and "11". Changing this forces a new resource to be created.| string | "11" | |
| postgres_ssl_enforcement_enabled | Enforce SSL on connection to the Azure Database for PostgreSQL server instance | bool | true | |
| postgres_db_names | List of names for databases to create for the Azure Database for PostgreSQL server instance. Each name needs to be a valid PostgreSQL identified. Changes this forces a new resource to be created. | list of strings | [] | |
| postgres_db_charset | The Charset for the PostgreSQL Database. Must be a valid PostgreSQL Charset. Changing this forces a new resource to be created. | string | "UTF8" | |
| postgres_db_collation | The Collation for the PostgreSQL Database. Needs to be a valid PostgreSQL Collation. Changing this forces a new resource to be created. |string| "English_United States.1252" | |
| postgres_configurations | Configurations to enable on the PostgreSQL Database server instance. | map | {} | |
| sku_name| The SKU Name for the PostgreSQL Server | string | "GP_Gen5_32" | The name pattern is the SKU, followed by the tier + family + cores (e.g. B_Gen4_1, GP_Gen5_4).|
| storage_mb | Max storage allowed for the PostgreSQL server | number | 51200 | Possible values are between 5120 MB(5GB) and 1048576 MB(1TB) for the Basic SKU and between 5120 MB(5GB) and 4194304 MB(4TB) for General Purpose/Memory Optimized SKUs |
| backup_retention_days | Backup retention days for the PostgreSQL server | number | 7 | Supported values are between 7 and 35 days. |
| geo_redundant_backup_enabled | Enable Geo-redundant or not for server backup | bool | false | Not supported for the basic tier. |
| administrator_login | The Administrator Login for the PostgreSQL Server. Changing this forces a new resource to be created. | string | "pgadmin" | The admin login name cannot be azure_superuser, azure_pg_admin, admin, administrator, root, guest, or public. It cannot start with pg_. See: [Microsoft Quickstart Server Database](https://docs.microsoft.com/en-us/azure/postgresql/quickstart-create-server-database-portal) |
| administrator_password | The Password associated with the administrator_login for the PostgreSQL Server | string | "my$up3rS3cretPassw0rd" | The password must contain between 8 and 128 characters and must contain characters from three of the following categories: English uppercase letters, English lowercase letters, numbers (0 through 9), and non-alphanumeric characters (!, $, #, %, etc.). |
| server_version | The version of the Azure Database for PostgreSQL server instance. Changing this forces a new resource to be created.| string | "11" | |
| ssl_enforcement_enabled | Enforce SSL on connection to the Azure Database for PostgreSQL server instance | bool | true | |
| postgresql_configurations | Configurations to enable on the PostgreSQL Database server instance | map(string) | {} | More details can be found [here](https://docs.microsoft.com/en-us/azure/postgresql/howto-configure-server-parameters-using-cli) |

Here is a sample of the `postgres_servers` variable with the `default` entry only overriding the `administrator_password` parameter and the `cps` entry overriding all of the parameters:

```terraform
postgres_servers = {
default = {
administrator_password = "D0ntL00kTh1sWay"
},
another_server = {
sku_name = "GP_Gen5_32"
storage_mb = 51200
backup_retention_days = 7
geo_redundant_backup_enabled = false
administrator_login = "pgadmin"
administrator_password = "1tsAB3aut1fulDay"
server_version = "11"
ssl_enforcement_enabled = true
postgresql_configurations = { foo = "true", bar = "false" }
}
}
```
10 changes: 6 additions & 4 deletions examples/sample-input-byo.tfvars
Original file line number Diff line number Diff line change
Expand Up @@ -29,10 +29,12 @@ default_public_access_cidrs = [] # e.g., ["123.45.6.89/32"]
# Tags for all taggable items in your cluster.
tags = {} # e.g., { "key1" = "value1", "key2" = "value2" }

# Azure Postgres config
create_postgres = true # set this to "false" when using internal Crunchy Postgres
postgres_ssl_enforcement_enabled = false
postgres_administrator_password = "mySup3rS3cretPassw0rd"
# Postgres config - By having this entry a database server is created. If you do not
# need an external database server remove the 'postgres_servers'
# block below.
postgres_servers = {
default = {},
}

# Azure Container Registry config
create_container_registry = false
Expand Down
10 changes: 6 additions & 4 deletions examples/sample-input-ha.tfvars
Original file line number Diff line number Diff line change
Expand Up @@ -19,10 +19,12 @@ default_public_access_cidrs = [] # e.g., ["123.45.6.89/32"]
# Tags can be specified matching your tagging strategy.
tags = {} # for example: { "owner|email" = "<you>@<domain>.<com>", "key1" = "value1", "key2" = "value2" }

# Azure Postgres config
create_postgres = true # set this to "false" when using internal Crunchy Postgres
postgres_ssl_enforcement_enabled = false
postgres_administrator_password = "mySup3rS3cretPassw0rd"
# Postgres config - By having this entry a database server is created. If you do not
# need an external database server remove the 'postgres_servers'
# block below.
postgres_servers = {
default = {},
}

# Azure Container Registry config
create_container_registry = false
Expand Down
10 changes: 6 additions & 4 deletions examples/sample-input-minimal.tfvars
Original file line number Diff line number Diff line change
Expand Up @@ -19,10 +19,12 @@ default_public_access_cidrs = [] # e.g., ["123.45.6.89/32"]
# Tags can be specified matching your tagging strategy.
tags = {} # for example: { "owner|email" = "<you>@<domain>.<com>", "key1" = "value1", "key2" = "value2" }

# Azure Postgres config
create_postgres = false # set this to "false" when using internal Crunchy Postgres
postgres_ssl_enforcement_enabled = false
postgres_administrator_password = "mySup3rS3cretPassw0rd"
# Postgres config - By having this entry a database server is created. If you do not
# need an external database server remove the 'postgres_servers'
# block below.
# postgres_servers = {
# default = {},
# }

# Azure Container Registry config
create_container_registry = false
Expand Down
10 changes: 6 additions & 4 deletions examples/sample-input-ppg.tfvars
Original file line number Diff line number Diff line change
Expand Up @@ -19,10 +19,12 @@ default_public_access_cidrs = [] # e.g., ["123.45.6.89/32"]
# Tags can be specified matching your tagging strategy.
tags = {} # for example: { "owner|email" = "<you>@<domain>.<com>", "key1" = "value1", "key2" = "value2" }

# Azure Postgres config
create_postgres = true # set this to "false" when using internal Crunchy Postgres
postgres_ssl_enforcement_enabled = false
postgres_administrator_password = "mySup3rS3cretPassw0rd"
# Postgres config - By having this entry a database server is created. If you do not
# need an external database server remove the 'postgres_servers'
# block below.
postgres_servers = {
default = {},
}

# Azure Container Registry config
create_container_registry = false
Expand Down
10 changes: 6 additions & 4 deletions examples/sample-input.tfvars
Original file line number Diff line number Diff line change
Expand Up @@ -21,10 +21,12 @@ default_public_access_cidrs = [] # e.g., ["123.45.6.89/32"]
# Tags can be specified matching your tagging strategy.
tags = {} # for example: { "owner|email" = "<you>@<domain>.<com>", "key1" = "value1", "key2" = "value2" }

# Azure Postgres config
create_postgres = true # set this to "false" when using internal Crunchy Postgres
postgres_ssl_enforcement_enabled = false
postgres_administrator_password = "mySup3rS3cretPassw0rd"
# Postgres config - By having this entry a database server is created. If you do not
# need an external database server remove the 'postgres_servers'
# block below.
postgres_servers = {
default = {},
}

# Azure Container Registry config
create_container_registry = false
Expand Down
35 changes: 27 additions & 8 deletions locals.tf
Original file line number Diff line number Diff line change
@@ -1,21 +1,40 @@
locals {
is_private = var.infra_mode == "private" ? true : false
is_standard = var.infra_mode == "standard" ? true : false

# Useful flags
is_private = var.infra_mode == "private" ? true : false
is_standard = var.infra_mode == "standard" ? true : false
create_jump_public_ip = var.create_jump_public_ip == null ? local.is_standard : var.create_jump_public_ip
create_nfs_public_ip = var.create_nfs_public_ip == null ? local.is_standard : var.create_nfs_public_ip

# CIDR/Network
default_public_access_cidrs = local.is_private ? [] : (var.default_public_access_cidrs == null ? [] : var.default_public_access_cidrs)
vm_public_access_cidrs = local.is_private ? [] : (var.vm_public_access_cidrs == null ? local.default_public_access_cidrs : var.vm_public_access_cidrs)
acr_public_access_cidrs = local.is_private ? [] : (var.acr_public_access_cidrs == null ? local.default_public_access_cidrs : var.acr_public_access_cidrs)
cluster_endpoint_public_access_cidrs = local.is_private ? [] : (var.cluster_endpoint_public_access_cidrs == null ? local.default_public_access_cidrs : var.cluster_endpoint_public_access_cidrs)
postgres_public_access_cidrs = local.is_private ? [] : (var.postgres_public_access_cidrs == null ? local.default_public_access_cidrs : var.postgres_public_access_cidrs)
subnets = { for k, v in var.subnets : k => v if ! ( k == "netapp" && var.storage_type == "standard")}

create_jump_public_ip = var.create_jump_public_ip == null ? local.is_standard : var.create_jump_public_ip
create_nfs_public_ip = var.create_nfs_public_ip == null ? local.is_standard : var.create_nfs_public_ip
# Kubernetes
kubeconfig_filename = "${var.prefix}-aks-kubeconfig.conf"
kubeconfig_path = var.iac_tooling == "docker" ? "/workspace/${local.kubeconfig_filename}" : local.kubeconfig_filename

postgres_firewall_rules = [for addr in local.postgres_public_access_cidrs : { "name" : replace(replace(addr, "/", "_"), ".", "_"), "start_ip" : cidrhost(addr, 0), "end_ip" : cidrhost(addr, abs(pow(2, 32 - split("/", addr)[1]) - 1)) }]
# PostgreSQL
postgres_servers = var.postgres_servers == null ? {} : { for k, v in var.postgres_servers : k => merge( var.postgres_server_defaults, v, )}
postgres_firewall_rules = [for addr in local.postgres_public_access_cidrs : { "name" : replace(replace(addr, "/", "_"), ".", "_"), "start_ip" : cidrhost(addr, 0), "end_ip" : cidrhost(addr, abs(pow(2, 32 - split("/", addr)[1]) - 1)) }]

subnets = { for k, v in var.subnets : k => v if ! ( k == "netapp" && var.storage_type == "standard")}
postgres_outputs = length(module.postgresql) != 0 ? { for k,v in module.postgresql :
k => {
"server_name" : module.postgresql[k].server_name,
"fqdn" : module.postgresql[k].server_fqdn,
"admin" : "${module.postgresql[k].administrator_login}@${module.postgresql[k].server_name}",
"password" : module.postgresql[k].administrator_password,
"server_port" : "5432", # TODO - Create a var when supported
"ssl_enforcement_enabled" : local.postgres_servers[k].ssl_enforcement_enabled,
"internal" : false
}
} : {}

# Container Registry
container_registry_sku = title(var.container_registry_sku)

kubeconfig_filename = "${var.prefix}-aks-kubeconfig.conf"
kubeconfig_path = var.iac_tooling == "docker" ? "/workspace/${local.kubeconfig_filename}" : local.kubeconfig_filename
}
35 changes: 19 additions & 16 deletions main.tf
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
## Azure-AKS
#
# Terraform Registry : https://registry.terraform.io/namespaces/Azure
# GitHub Repository : https://github.com/terraform-azurerm-modules
#
provider "azurerm" {

subscription_id = var.subscription_id
Expand Down Expand Up @@ -291,25 +296,23 @@ module "postgresql" {
source = "Azure/postgresql/azurerm"
version = "2.1.0"

count = var.create_postgres ? 1 : 0
for_each = local.postgres_servers != null ? length(local.postgres_servers) != 0 ? local.postgres_servers : {} : {}

resource_group_name = module.resource_group.name
location = var.location
server_name = lower("${var.prefix}-pgsql")
sku_name = var.postgres_sku_name
storage_mb = var.postgres_storage_mb
backup_retention_days = var.postgres_backup_retention_days
geo_redundant_backup_enabled = var.postgres_geo_redundant_backup_enabled
administrator_login = var.postgres_administrator_login
administrator_password = var.postgres_administrator_password
server_version = var.postgres_server_version
ssl_enforcement_enabled = var.postgres_ssl_enforcement_enabled
db_names = var.postgres_db_names
db_charset = var.postgres_db_charset
db_collation = var.postgres_db_collation
firewall_rule_prefix = "${var.prefix}-postgres-firewall-"
server_name = lower("${var.prefix}-${each.key}-pgsql")
sku_name = each.value.sku_name
storage_mb = each.value.storage_mb
backup_retention_days = each.value.backup_retention_days
geo_redundant_backup_enabled = each.value.geo_redundant_backup_enabled
administrator_login = each.value.administrator_login
administrator_password = each.value.administrator_password
server_version = each.value.server_version
ssl_enforcement_enabled = each.value.ssl_enforcement_enabled
firewall_rule_prefix = "${var.prefix}-${each.key}-postgres-firewall-"
firewall_rules = local.postgres_firewall_rules
vnet_rule_name_prefix = "${var.prefix}-postgresql-vnet-rule-"
postgresql_configurations = var.postgres_configurations
vnet_rule_name_prefix = "${var.prefix}-${each.key}-postgresql-vnet-rule-"
postgresql_configurations = each.value.postgresql_configurations
tags = module.resource_group.tags

## TODO : requires specific permissions
Expand Down
24 changes: 3 additions & 21 deletions outputs.tf
Original file line number Diff line number Diff line change
Expand Up @@ -22,28 +22,10 @@ output "aks_cluster_password" {
}

#postgres
output "postgres_server_name" {
value = var.create_postgres ? element(coalescelist(module.postgresql.*.server_name, [" "]), 0) : null
}
output "postgres_fqdn" {
value = var.create_postgres ? element(coalescelist(module.postgresql.*.server_fqdn, [" "]), 0) : null
}
output "postgres_admin" {
value = var.create_postgres ? "${element(coalescelist(module.postgresql.*.administrator_login, [" "]), 0)}@${element(coalescelist(module.postgresql.*.server_name, [" "]), 0)}" : null
}
output "postgres_password" {
value = var.create_postgres ? element(coalescelist(module.postgresql.*.administrator_password, [" "]), 0) : null
sensitive = true
}
output "postgres_server_id" {
value = var.create_postgres ? element(coalescelist(module.postgresql.*.server_id, [" "]), 0) : null
}
output "postgres_server_port" {
value = var.create_postgres ? "5432" : null
}

output "postgres_ssl_enforcement_enabled" {
value = var.create_postgres ? var.postgres_ssl_enforcement_enabled : null
output "postgres_servers" {
value = length(module.postgresql) != 0 ? local.postgres_outputs : null
sensitive = true
}

# jump server
Expand Down
Loading

0 comments on commit f414906

Please sign in to comment.