This repository contains a module to create several resources needed by AWS WAF WebACL to:
- Enable logging of traffic information[1].
- Store logs in Parquet format[2] for more optimized query using Athena[3].
- Provision query-ready Athena Database and Table which based on AWS Glue Data Catalog [4].
Based on the diagram above, the resources going to be created are:
- S3 Bucket to store all traffic logs.
- Kinesis Data Firehose[5] to deliver traffic logs from WAF WebACL to the S3.
- Cloudwatch Log Group and Stream to store the Firehose delivery error information.
- AWS Glue Catalog Database and Table which store metadata/schema of the log data.
- One function of those resources is to make it possible the conversion from JSON to Parquet.
- The other function is to provision Amazon Athena Database and Table which is ready to use to perform queries.
- IAM Role and Permissions for the Firehose to do all those actions above.
Please Note: This module WILL NOT CREATE AWS WAF Rules and WebACL.
If you are wondering what kind of rules that you should create to protect your App, you might want to take a look at Related Modules section below.
Also, to get a full picture on how to make use of this module together with AWS WAF WebACL and Rules, check Examples section.
References
- [1] : https://docs.aws.amazon.com/waf/latest/developerguide/logging.html
- [2] : https://cwiki.apache.org/confluence/display/Hive/Parquet
- [3] : https://docs.aws.amazon.com/athena/latest/ug/columnar-storage.html
- [4] : https://docs.aws.amazon.com/athena/latest/ug/glue-athena.html
- [5] : https://docs.aws.amazon.com/firehose/latest/dev/what-is-this-service.html
AWS WAF is divided into two, Global and Regional resources. Read this to understand better. These examples provided will tell you more on how they are implemented:
Apache 2 Licensed. See LICENSE for full details.