This Terraform module deploys a Management Virtual Network Hub Overlay using the Microsoft recommended Hub-Spoke network topology. Usually, only one hub in each region with multiple spokes and each of the spokes can also be in separate subscriptions.
Since this module is built for both public and us government clouds. The environment variable defaults to public for Azure Cloud. When using this module with the Azure Government Cloud, you must set the environment variable to usgovernment.
You will also need to set the azurerm provider environment variable to the proper cloud as well. This will ensure that the correct Azure Government Cloud endpoints are used. You will also need to set the location variable to a valid Azure Government Cloud location.
Example Usage for Azure Government Cloud:
provider "azurerm" {
environment = "usgovernment"
}
module "overlays-management-hub" {
source = "azurenoops/overlays-management-hub/azurerm"
version = "2.0.0"
location = "usgovvirginia"
environment = "usgovernment"
...
}
Terraform requires the following resource providers to be available:
- Microsoft.Network
- Microsoft.Storage
- Microsoft.Compute
- Microsoft.KeyVault
- Microsoft.Authorization
- Microsoft.Resources
- Microsoft.OperationalInsights
- Microsoft.GuestConfiguration
- Microsoft.Insights
- Microsoft.Advisor
- Microsoft.Security
- Microsoft.OperationsManagement
- Microsoft.AAD
- Microsoft.AlertsManagement
- Microsoft.Authorization
- Microsoft.AnalysisServices
- Microsoft.Automation
- Microsoft.Subscription
- Microsoft.Support
- Microsoft.PolicyInsights
- Microsoft.SecurityInsights
- Microsoft.Security
- Microsoft.Monitor
- Microsoft.Management
- Microsoft.ManagedServices
- Microsoft.ManagedIdentity
- Microsoft.Billing
- Microsoft.Consumption
Please note that some of the resource providers may not be available in Azure Government Cloud. Please check the Azure Government Cloud documentation for more information.
This module is be SCCA compliant and can be used in a SCCA compliant Network. Enable SCCA compliant network rules to make it SCCA compliant.
For more information, please read the SCCA documentation.
If you want to contribute to this repository, feel free to to contribute to our Terraform module.
More details are available in the CONTRIBUTING.md file.
The following reference architecture shows how to implement a SCCA compliant hub-spoke topology in Azure. The Management Hub Overlay is a virtual network in Azure that acts as a central point of connectivity to an optional on-premises network.
The spokes are virtual networks that peer with the Management Hub Overlay and can be used to isolate workloads. Traffic flows between the on-premises datacenter and the hub can be achieved through an ExpressRoute or VPN gateway connection.
AzureFirewallSubnet and GatewaySubnet will not contain any UDR (User Defined Route) or NSG/Rules (Network Security Group). Management and DMZ subnets will route all outgoing traffic through firewall instance.
Source: Microsoft Azure Hub-Spoke Topology Documentation
- Virtual Network
- Subnets
- Subnet Service Delegation
- Virtual Network service endpoints
- AzureNetwork DDoS Protection Plan
- Network Security Groups
- Azure Firewall
- Azure Firewall Application Rule Collection
- Azure Firewall Network Rule Collection
- Azure Firewall NAT Rule Collection
- Route Table
- Role Assignment for Peering
- Storage Account for Log Archive
- Network Watcher
- Network Watcher Workflow Logs
- Private DNS Zone
# Azurerm provider configuration
provider "azurerm" {
features {}
}
module "mod_vnet_hub" {
source = "azurenoops/overlays-management-hub/azurerm"
version = "x.x.x"
# By default, this module will create a resource group, provide the name here
# To use an existing resource group, specify the existing resource group name,
# and set the argument to `create_hub_resource_group = false`. Location will be same as existing RG.
create_hub_resource_group = true
location = "eastus"
deploy_environment = "dev"
org_name = "anoa"
environment = "public"
workload_name = "hub"
# Provide valid VNet Address space and specify valid domain name for Private DNS Zone.
virtual_network_address_space = ["10.0.0.0/16"] # (Required) Hub Virtual Network Parameters
firewall_subnet_address_prefix = ["10.0.100.0/26"] # (Required) Hub Firewall Subnet Parameters
firewall_management_snet_address_prefix = ["10.0.100.128/26"] # (Optional) Hub Firewall Management Subnet Parameters
gateway_subnet_address_prefix = ["10.0.100.192/27"] # (Optional) Hub Gateway Subnet Parameters
# (Required) Hub Subnets
# Default Subnets, Service Endpoints
# This is the default subnet with required configuration, check README.md for more details
# First address ranges from VNet Address space reserved for Firewall Subnets.
# ex.: For 10.0.100.128/27 address space, usable address range start from 10.0.100.0/24 for all subnets.
# default subnet name will be set as per Azure NoOps naming convention by default.
# Multiple Subnets, Service delegation, Service Endpoints, Network security groups
# These are default subnets with required configuration, check README.md for more details
# NSG association to be added automatically for all subnets listed here.
# First two address ranges from VNet Address space reserved for Gateway And Firewall Subnets.
# ex.: For 10.1.0.0/16 address space, usable address range start from 10.1.2.0/24 for all subnets.
# subnet name will be set as per Azure naming convention by default. expected value here is: <App or project name>
hub_subnets = {
default = {
name = "hub"
address_prefixes = ["10.0.100.64/26"]
service_endpoints = ["Microsoft.Storage"]
private_endpoint_network_policies_enabled = false
private_endpoint_service_endpoints_enabled = true
}
dmz = {
name = "appgateway"
address_prefixes = ["10.0.100.224/27"]
service_endpoints = ["Microsoft.Storage"]
private_endpoint_network_policies_enabled = false
private_endpoint_service_endpoints_enabled = true
nsg_subnet_rules = [
{
name = "allow-443",
description = "Allow access to port 443",
priority = 100,
direction = "Inbound",
access = "Allow",
protocol = "*",
source_port_range = "*",
destination_port_range = "443",
source_address_prefix = "*",
destination_address_prefix = "*"
}
{
name = "ntp_out",
description = "Allow NTP out on 123",
priority = 400,
direction = "Outbound",
access = "Allow",
protocol = "Udp",
source_port_range = "123",
destination_port_range = "",
source_address_prefix = "0.0.0.0/0",
destination_address_prefix = "*"
}
]
}
}
# Firewall Settings
# By default, Azure NoOps will create Azure Firewall in Hub VNet.
# If you do not want to create Azure Firewall,
# set enable_firewall to false. This will allow different firewall products to be used (Example: F5).
enable_firewall = true
# By default, forced tunneling is enabled for Azure Firewall.
# If you do not want to enable forced tunneling,
# set enable_forced_tunneling to false.
enable_forced_tunneling = true
# (Optional) To enable the availability zones for firewall.
# Availability Zones can only be configured during deployment
# You can't modify an existing firewall to include Availability Zones
firewall_zones = [1, 2, 3]
# # (Optional) specify the Network rules for Azure Firewall l
# This is default values, do not need this if keeping default values
firewall_network_rules_collection = [
{
name = "AllowAzureCloud"
priority = "100"
action = "Allow"
rules = [
{
name = "AzureCloud"
protocols = ["Any"]
source_addresses = ["*"]
destination_addresses = ["AzureCloud"]
destination_ports = ["*"]
}
]
},
{
name = "AllowTrafficBetweenSpokes"
priority = "200"
action = "Allow"
rules = [
{
name = "AllSpokeTraffic"
protocols = ["Any"]
source_addresses = ["10.96.0.0/19"]
destination_addresses = ["*"]
destination_ports = ["*"]
}
]
}
]
# (Optional) specify the application rules for Azure Firewall
# This is default values, do not need this if keeping default values
firewall_application_rule_collection = [
{
name = "AzureAuth"
priority = "110"
action = "Allow"
rules = [
{
name = "msftauth"
source_addresses = ["*"]
destination_fqdns = ["aadcdn.msftauth.net", "aadcdn.msauth.net"]
protocols = {
type = "Https"
port = 443
}
}
]
}
]
# Private DNS Zone Settings
# By default, Azure NoOps will create Private DNS Zones in Hub VNet.
# If you do want to create additional Private DNS Zones,
# add in the list of private_dns_zones to be created.
# else, remove the private_dns_zones argument.
private_dns_zones = ["privatelink.file.core.windows.net"]
# By default, this module will create a bastion host,
# and set the argument to `enable_bastion_host = false`, to disable the bastion host.
enable_bastion_host = true
azure_bastion_host_sku = "Standard"
azure_bastion_subnet_address_prefix = ["10.0.200.0/27"]
# By default, this will apply resource locks to all resources created by this module.
# To disable resource locks, set the argument to `enable_resource_locks = false`.
enable_resource_locks = false
# Tags
add_tags = {
Example = "Management Hub Overlay"
} # Tags to be applied to all resources
}
Hub Networking is set up in a Management Hub Overlay design based on the SCCA Hub/Spoke architecture. The Management Hub Overlay is a central point of connectivity to many different networks.
The following parameters affect Management Virtual Network Hub Overlay.
| Parameter name | Location | Default Value | Description |
|---|---|---|---|
virtual_network_address_space |
variables.vnet.tf |
'10.0.100.0/24' | The CIDR Virtual Network Address Prefix for the Hub Virtual Network. |
subnet_address_prefix |
variables.snet.tf |
'10.0.100.128/27' | The CIDR Subnet Address Prefix for the default Hub subnet. It must be in the Hub Virtual Network space. |
firewall_client_snet_address_prefix |
variables.fw.tf |
'10.0.100.0/26' | The CIDR Subnet Address Prefix for the Azure Firewall Subnet. It must be in the Hub Virtual Network space. It must be /26. |
firewall_management_snet_address_prefix |
variables.fw.tf |
'10.0.100.64/26' | The CIDR Subnet Address Prefix for the Azure Firewall Management Subnet. It must be in the Hub Virtual Network space. It must be /26. |
This module handles the creation and a list of address spaces for subnets. This module uses for_each to create subnets and corresponding service endpoints, service delegation, and network security groups. This module associates the subnets to network security groups as well with additional user-defined NSG rules.
This module creates 4 subnets by default: Gateway Subnet, AzureFirewallSubnet, AzureFirewallManagementSubnet and AzureBastionSubnet.
| Name | Description |
|---|---|
| GatewaySubnet | Contain VPN Gateway, Express route Gateway |
| AzureFirewallSubnet | If added the Firewall module, it Deploys an Azure Firewall that will monitor all incoming and outgoing traffic |
| AzureFirewallManagementSubnet | An additional dedicated subnet named AzureFirewallManagementSubnet (minimum subnet size /26) is required with its own associated public IP address. This public IP address is for management traffic. It is used exclusively by the Azure platform and can't be used for any other purpose. |
| AzureBastionSubnet | Management subnet for Bastion host, accessible from gateway |
| PrivateEndpointSubnet | Hosts the private endpoints used by the Azure Monitor. |
Both Gateway Subnet and AzureFirewallSubnet allow traffic out and can have public IPs. Management subnets route traffic through the firewall and does not support public IPs due to asymmetric routing.
Service Endpoints allows connecting certain platform services into virtual networks. With this option, Azure virtual machines can interact with Azure SQL and Azure Storage accounts, as if they’re part of the same virtual network, rather than Azure virtual machines accessing them over the public endpoint.
This module supports enabling the service endpoint of your choosing under the virtual network and with the specified subnet. The list of Service endpoints to associate with the subnet values includes:
Microsoft.AzureActiveDirectory, Microsoft.AzureCosmosDB, Microsoft.ContainerRegistry, Microsoft.EventHub, Microsoft.KeyVault, Microsoft.ServiceBus, Microsoft.Sql, Microsoft.Storage and Microsoft.Web.
module "vnet-hub" {
source = "azurenoops/overlays-management-hub/azurerm"
version = "x.x.x"
# .... omitted
# Multiple Subnets, Service delegation, Service Endpoints
subnets = {
mgmt_subnet = {
subnet_name = "management"
subnet_address_prefix = "10.1.2.0/24"
service_endpoints = ["Microsoft.Storage"]
}
}
# ....omitted
}Subnet delegation enables you to designate a specific subnet for an Azure PaaS service of your choice that needs to be injected into your virtual network. The Subnet delegation provides full control to manage the integration of Azure services into virtual networks.
This module supports enabling the service delegation of your choosing under the virtual network and with the specified subnet. For more information, check the terraform resource documentation.
module "vnet-hub" {
source = "azurenoops/overlays-management-hub/azurerm"
version = "x.x.x"
# .... omitted
# Multiple Subnets, Service delegation, Service Endpoints
subnets = {
mgmt_subnet = {
subnet_name = "management"
subnet_address_prefix = "10.1.2.0/24"
delegation = {
name = "demodelegationcg"
service_delegation = {
name = "Microsoft.ContainerInstance/containerGroups"
actions = ["Microsoft.Network/virtualNetworks/subnets/join/action", "Microsoft.Network/virtualNetworks/subnets/prepareNetworkPolicies/action"]
}
}
}
}
# ....omitted
}Network policies, like network security groups (NSG), are not supported for Private Link Endpoints. In order to deploy a Private Link Endpoint on a given subnet, you must set the private_endpoint_network_policies_enabled attribute to true.
This setting is only applicable for the Private Link Endpoint, for all other resources in the subnet access is controlled based via the Network Security Group which can be configured using the azurerm_subnet_network_security_group_association resource.
This module Enable or Disable network policies for the private link endpoint on the subnet. The default value is false. If you are enabling the Private Link Endpoints on the subnet you shouldn't use Private Link Services as it's conflicts.
module "vnet-hub" {
source = "azurenoops/overlays-management-hub/azurerm"
version = "x.x.x"
# .... omitted
# Multiple Subnets, Service delegation, Service Endpoints
subnets = {
mgmt_subnet = {
subnet_name = "management"
subnet_address_prefix = "10.1.2.0/24"
private_endpoint_network_policies_enabled = true
}
}
}
}
# ....omitted
}In order to deploy a Private Link Service on a given subnet, you must set the private_link_service_network_policies_enabled attribute to true. This setting is only applicable for the Private Link Service, for all other resources in the subnet access is controlled based on the Network Security Group which can be configured using the azurerm_subnet_network_security_group_association resource.
This module Enable or Disable network policies for the private link service on the subnet. The default value is false. If you are enabling the Private Link service on the subnet then, you shouldn't use Private Link endpoints as it's conflicts.
module "vnet-hub" {
source = "azurenoops/overlays-management-hub/azurerm"
version = "x.x.x"
# .... omitted
# Multiple Subnets, Service delegation, Service Endpoints
subnets = {
mgmt_subnet = {
subnet_name = "management"
subnet_address_prefix = "10.1.2.0/24"
private_link_service_network_policies_enabled = true
}
}
}
}
# ....omitted
}Azure Firewall is a managed, cloud-based network security service that protects your Azure Virtual Network resources. It's a fully stateful firewall as a service with built-in high availability and unrestricted cloud scalability.
Source: Azure Firewall Documentation
You can centrally create, enforce, and log application and network connectivity policies across subscriptions and virtual networks. Azure Firewall uses a static public IP address for your virtual network resources allowing outside firewalls to identify traffic originating from your virtual network. The service is fully integrated with Azure Monitor for logging and analytics.
This is designed to support hub and spoke architecture in the azure and further security hardening would be recommend to add appropriate firewall application/network/NAT rules to use this for any production workloads.
All network traffic is directed through the firewall residing in the Management Virtual Network Hub Overlay resource group. The firewall is configured as the default route for all other spokes including T0 (Identity and Authorization), T1 (Operations), T2 (Shared Services) Management Spokes and T3 (workload/team environments) workload spoke as follows:
| Name | Address prefix | Next hop type | Next hop IP address |
|---|---|---|---|
| default_route | 0.0.0.0/0 | Virtual Appliance | 10.0.100.4* |
*-example IP for firewall
The default firewall configured for Management Virtual Network Hub Overlay is Azure Firewall Premium. Firewall Premium is the only version that meets SCCA compliance as it can break-and-inspect traffic whereas Firewall Standard can not.
Azure Firewall can be configured during deployment to span multiple Availability Zones for increased availability. With Availability Zones, your availability increases to 99.99% uptime.
To specifies the availability zones in which the Azure Firewall should be created, set the argument firewall_zones = [1, 2, 3]. This is by default is not enabled and set to none. There's no additional cost for a firewall deployed in an Availability Zone. However, there are additional costs for inbound and outbound data transfers associated with Availability Zones.
Note: Availability Zones can only be configured during deployment. You can't modify an existing firewall to include Availability Zones
This module centrally create allow or deny network filtering rules by source and destination IP address, port, and protocol. Azure Firewall is fully stateful, so it can distinguish legitimate packets for different types of connections. Rules are enforced and logged across multiple subscriptions and virtual networks.
To define the firewall rules, use the input variables firewall_application_rules, firewall_network_rules and firewall_nat_rules.
Presently, there are two firewall rules configured to ensure access to the Azure Portal and to facilitate interactive logon via PowerShell and Azure CLI, all other traffic is restricted by default. Below are the collection of rules configured for Azure Commercial and Azure Government clouds:
| Rule Collection Priority | Rule Collection Name | Rule name | Source | Port | Protocol |
|---|---|---|---|---|---|
| 100 | AllowAzureCloud | AzureCloud | * | * | Any |
| 110 | AzureAuth | msftauth | * | Https:443 | aadcdn.msftauth.net, aadcdn.msauth.net |
module "vnet-hub" {
source = "azurenoops/overlays-management-hub/azurerm"
version = "x.x.x"
# ....omitted
# (Optional) specify the application rules for Azure Firewall
firewall_application_rules = [
{
name = "microsoft"
action = "Allow"
source_addresses = ["10.0.0.0/8"]
target_fqdns = ["*.microsoft.com"]
protocol = {
type = "Http"
port = "80"
}
},
]
# (Optional) specify the Network rules for Azure Firewall
firewall_network_rules = [
{
name = "ntp"
action = "Allow"
source_addresses = ["10.0.0.0/8"]
destination_ports = ["123"]
destination_addresses = ["*"]
protocols = ["UDP"]
},
]
# (Optional) specify the NAT rules for Azure Firewall
# Destination address must be Firewall public IP
# `fw-public` is a variable value and automatically pick the firewall public IP from module.
firewall_nat_rules = [
{
name = "testrule"
action = "Dnat"
source_addresses = ["10.0.0.0/8"]
destination_ports = ["53", ]
destination_addresses = ["fw-public"]
translated_port = 53
translated_address = "8.8.8.8"
protocols = ["TCP", "UDP", ]
},
]
# ....omitted
}By default, the network security groups connected to subnets will only allow necessary traffic and block everything else (deny-all rule). Use nsg_subnet_rules in this Terraform module to create a Network Security Group (NSG) for each subnet and allow it to add additional rules for inbound flows.
In the Source and Destination columns, VirtualNetwork, AzureLoadBalancer, and Internet are service tags, rather than IP addresses. In the protocol column, Any encompasses TCP, UDP, and ICMP. When creating a rule, you can specify TCP, UDP, ICMP or *. 0.0.0.0/0 in the Source and Destination columns represents all addresses.
You cannot remove the default rules, but you can override them by creating rules with higher priorities.
This module supports enabling the NSG Rules of your choosing under the virtual network and with the specified subnet. For more information, check the terraform resource documentation.
module "vnet-hub" {
source = "azurenoops/overlays-management-hub/azurerm"
version = "x.x.x"
# .... omitted
# Multiple Subnets, Service delegation, Service Endpoints
subnets = {
mgmt_subnet = {
subnet_name = "management"
subnet_address_prefix = "10.1.2.0/24"
nsg_subnet_rules = [
{
name = "allow-443",
description = "Allow access to port 443",
priority = 100,
direction = "Inbound",
access = "Allow",
protocol = "*",
source_port_range = "*",
destination_port_range = "443",
source_address_prefix = "*",
destination_address_prefix = "*"
}
]
}
}
}
# ....omitted
}To peer spoke networks to the hub networks requires the service principal that performs the peering has Network Contributor role on hub network. Linking the Spoke to Hub DNS zones, the service principal also needs the Private DNS Zone Contributor role on hub network.
Management Hub Overlay has optional features that can be enabled by setting parameters on the deployment.
By default, this module will create a resource group and the name of the resource group to be given in an argument resource_group_name located in variables.naming.tf. If you want to use an existing resource group, specify the existing resource group name, and set the argument to create_hub_resource_group = false.
Note: If you are using an existing resource group, then this module uses the same resource group location to create all resources in this module.
By default, this module will not create a DDoS Protection Plan. You can enable/disable it by appending an argument create_ddos_plan located in variables.vnet.tf. If you want to enable a DDoS plan using this module, set argument create_ddos_plan = true
This module handles the lookup of Network Watcher resource. It will use network watcher, flow logs and traffic analytics for all the subnets in the Virtual Network.
Note: Log Analytics workspace is required for NSG Flow Logs and Traffic Analytics. If you want to enable NSG Flow Logs and Traffic Analytics, you must create a Log Analytics workspace and provide the workspace name set argument
log_analytics_workspace_nameand rg set argumentlog_analytics_workspace_resource_group_name
By default, this module will not create a force tunnel on the firewall. You can enable/disable it by appending an argument enable_force_tunneling. Enabling this feature will ensure that the firewall is the default route for all the T0 through T3 Network routes.
"The Encrypted Transport Add-on Encrypted Transport Add-On requires modifications to the Firewall in the form of a new Route Table on the AzureFirewallSubnet as well as a new route. If you decide to use the Encrypted Transport Add-on then set the enable_encrypted_transport argument to true."
Addition to enabling Encrypted Transport, you will need to add encrypted_transport_address_prefix, encrypted_transport_next_hop_in_ip_address and encrypted_transport_next_hop_type arguments.
module "vnet-hub" {
source = "azurenoops/overlays-management-hub/azurerm"
version = "x.x.x"
# .... omitted
# Enable Encrypted Transport
enable_encrypted_transport = true
encrypted_transport_address_prefix = "10.0.100.0/27"
encrypted_transport_next_hop_in_ip_address = "10.0.100.0/27"
encrypted_transport_next_hop_type = "VirtualAppliance"
# ....omitted
}This module facilitates the private DNS zone for the virtual network. To create a zone, set the domain name for the private DNS zone with variable private_dns_zone_name. This will additionally link the virtual network hub to the private DNS zone. It will assign all principals that have peering access as contributors so, spokes can remain linked to the same zone.
If you want to remotely access the network and the resources you've deployed you can use Azure Bastion to remotely access virtual machines within the network without exposing them via Public IP Addresses.
By default, this module will not create a Azure Bastion Host. You can enable/disable it by appending an argument enable_bastion_host located in variables.bastion.tf If you want to enable a Azure Bastion Host using this module, set argument enable_bastion_host = true.
If you would like to create a jumpbox VM in the network, you can use the Azure Bastion Jumpbox Virtual Machine module.
By default, Management Hub Overlay deploys Azure Firewall Premium. Not all regions support Azure Firewall Premium.
Check here to see if the region you're deploying to supports Azure Firewall Premium. If necessary you can set a different firewall SKU or location.
You can manually specify which SKU of Azure Firewall to use for your deployment by specifying the firewallSkuTier parameter. This parameter only accepts values of Standard or Premium.
| Parameter name | Default Value | Description |
|---|---|---|
firewallSkuTier |
'Premium' | [Standard/Premium] The SKU for Azure Firewall. It defaults to "Premium". |
If you'd like to specify a different region to deploy your resources into, change the location of the deployment. For example, when using the AZ CLI set the deployment command's --location argument.
By default, Management Hub Overlay uses the shorten location name for naming resources. For example, eus2 is used instead of eastus2. You can enable/disable it by appending an argument use_location_short_name located in variables.tf If you want to disable shorten location name using this module, set argument use_location_short_name = false.
Using tags to properly organize your Azure resources, resource groups, and subscriptions into a taxonomy. Each tag is made up of a name and a value pair. For example, you can apply the term Environment and the value Production to all production resources.
See Resource name and tagging choice guide for advice on how to apply a tagging strategy.
Important : For operations, tag names are case-insensitive. A tag with a tag name is updated or retrieved, independent of casing. The resource provider, on the other hand, may preserve the casing you supply for the tag name. Cost reports will show that casing. The case of tag values is important.
An effective naming convention creates resource names by incorporating vital resource information into the name. A public IP resource for a production SharePoint workload, for example, is named pip-sharepoint-prod-westus-001 using these recommended naming conventions.