Merge pull request #745 from Security-Onion-Solutions/dev

1.3.0
Security-Onion-Solutions · May 20, 2020 · 9eb4d6c · 9eb4d6c
2 parents cb59d21 + 35646b7
commit 9eb4d6c
Show file tree

Hide file tree

Showing 166 changed files with 12,973 additions and 4,466 deletions.
diff --git a/.gitignore b/.gitignore
@@ -1,2 +1,59 @@
+
+# Created by https://www.gitignore.io/api/macos,windows
+# Edit at https://www.gitignore.io/?templates=macos,windows
+
+### macOS ###
+# General
 .DS_Store
-.idea
+.AppleDouble
+.LSOverride
+
+# Icon must end with two \r
+Icon
+
+# Thumbnails
+._*
+
+# Files that might appear in the root of a volume
+.DocumentRevisions-V100
+.fseventsd
+.Spotlight-V100
+.TemporaryItems
+.Trashes
+.VolumeIcon.icns
+.com.apple.timemachine.donotpresent
+
+# Directories potentially created on remote AFP share
+.AppleDB
+.AppleDesktop
+Network Trash Folder
+Temporary Items
+.apdisk
+
+### Windows ###
+# Windows thumbnail cache files
+Thumbs.db
+Thumbs.db:encryptable
+ehthumbs.db
+ehthumbs_vista.db
+
+# Dump file
+*.stackdump
+
+# Folder config file
+[Dd]esktop.ini
+
+# Recycle Bin used on file shares
+$RECYCLE.BIN/
+
+# Windows Installer files
+*.cab
+*.msi
+*.msix
+*.msm
+*.msp
+
+# Windows shortcuts
+*.lnk
+
+# End of https://www.gitignore.io/api/macos,windows
diff --git a/README.md b/README.md
@@ -1,42 +1,34 @@
-## Hybrid Hunter Beta 1.2.2 - Beta 1
-
-### Changes:  
-
-- Updated Saltstack to 2019.2.4 to address [CVE-2020-11651](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-11651)
-- Updated Suricata to 4.1.8 to address some possible security issues. Details [here](https://suricata-ids.org/2020/04/28/suricata-4-1-8-released/).
-- Fixed an issue that was preventing Strelka to function properly.
-- ISO installs should now use the built in docker containers instead of re-downloading them.
-
-
-## Hybrid Hunter Beta 1.2.1 - Beta 1
+## Hybrid Hunter Beta 1.3.0 - Beta 2
 
 ### Changes:
 
-- Full support for Ubuntu 18.04. 16.04 is no longer supported for Hybrid Hunter.
-- Introduction of the Security Onion Console. Once logged in you are directly taken to the SOC.
-- New authentication using Kratos.
-- During install you must specify how you would like to access the SOC ui. This is for strict cookie security.
-- Ability to list and delete web users from the SOC ui.
-- The soremote account is now used to add nodes to the grid vs using socore. 
-- Community ID support for Zeek, osquery, and Suricata. You can now tie host events to connection logs!
-- Elastic 7.6.1 with ECS support.
-- New set of Kibana dashboards that align with ECS.
-- Eval mode no longer uses Logstash for parsing (Filebeat -> ES Ingest)
-- Ingest node parsing for osquery-shipped logs (osquery, WEL, Sysmon).
-- Fleet standalone mode with improved Web UI & API access control.
-- Improved Fleet integration support.
-- Playbook now has full Windows Sigma community ruleset builtin.
-- Automatic Sigma community rule updates.
-- Playbook stability enhancements.
-- Zeek health check. Zeek will now auto restart if a worker crashes.
-- zeekctl is now managed by salt.
-- Grafana dashboard improvements and cleanup.
-- Moved logstash configs to pillars.
-- Salt logs moved to /opt/so/log/salt.
-- Strelka integrated for file-oriented detection/analysis at scale
-
-### Known issues:
-
+- New Feature: Codename: "Onion Hunt". Select Hunt from the menu and start hunting down your adversaries! 
+- Improved ECS support.
+- Complete refactor of the setup to make it easier to follow.
+- Improved setup script logging to better assist on any issues.
+- Setup now checks for minimal requirements during install.
+- Updated Cyberchef to version 9.20.3.
+- Updated Elastalert to version 0.2.4 and switched to alpine to reduce container size.
+- Updated Redis to 5.0.9 and switched to alpine to reduce container size.
+- Updated Salt to 2019.2.5
+- Updated Grafana to 6.7.3.
+- Zeek 3.0.6
+- Suricata 4.1.8
+- Fixes so-status to now display correct containers and status.
+- local.zeek is now controlled by a pillar instead of modifying the file directly.
+- Renamed so-core to so-nginx and switched to alpine to reduce container size.
+- Playbook now uses MySQL instead of SQLite.
+- Sigma rules have all been updated.
+- Kibana dashboard improvements for ECS.
+- Fixed an issue where geoip was not properly parsed.
+- ATT&CK Navigator is now it's own state.
+- Standlone mode is now supported.
+- Mastersearch previously used the same Grafana dashboard as a Search node. It now has its own dashboard that incorporates panels from the Master node and Search node dashboards.
+
+### Known Issues:
+
+- The Hunt feature is currently considered "Preview" and although very useful in its current state, not everything works. We wanted to get this out as soon as possible to get the feedback from you! Let us know what you want to see! Let us know what you think we should call it!
+- You cannot pivot to PCAP from Suricata alerts in Kibana or Hunt.
 - Updating users via the SOC ui is known to fail. To change a user, delete the user and re-add them. 
 - Due to the move to ECS, the current Playbook plays may not alert correctly at this time.
 - The osquery MacOS package does not install correctly.

diff --git a/VERSION b/VERSION
@@ -1 +1 @@
-1.2.2
+1.3.0
diff --git a/pillar/data/mastersearchtab.sls b/pillar/data/mastersearchtab.sls
@@ -0,0 +1 @@
+mastersearchtab:
diff --git a/pillar/docker/config.sls b/pillar/docker/config.sls
@@ -10,7 +10,7 @@
 
 eval:
   containers:
-    - so-core
+    - so-nginx
     - so-telegraf
     {% if  GRAFANA == '1' %}
     - so-influxdb
@@ -54,7 +54,7 @@ eval:
     {% endif %}
 heavy_node:
   containers:
-    - so-core
+    - so-nginx
     - so-telegraf
     - so-redis
     - so-logstash
@@ -69,7 +69,7 @@ heavy_node:
     {% endif %}
 helix:
   containers:
-    - so-core
+    - so-nginx
     - so-telegraf
     - so-idstools
     - so-steno
@@ -79,14 +79,14 @@ helix:
     - so-filebeat
 hot_node:
   containers:
-    - so-core
+    - so-nginx
     - so-telegraf
     - so-logstash
     - so-elasticsearch
     - so-curator
 master_search:
   containers:
-    - so-core
+    - so-nginx
     - so-telegraf
     - so-soc
     - so-kratos
@@ -127,7 +127,7 @@ master_search:
 master:
   containers:
     - so-dockerregistry
-    - so-core
+    - so-nginx
     - so-telegraf
     {% if  GRAFANA == '1' %}
     - so-influxdb
@@ -169,12 +169,12 @@ master:
     {% endif %}
 parser_node:
   containers:
-    - so-core
+    - so-nginx
     - so-telegraf
     - so-logstash
 search_node:
   containers:
-    - so-core
+    - so-nginx
     - so-telegraf
     - so-logstash
     - so-elasticsearch
@@ -185,7 +185,7 @@ search_node:
     {% endif %}
 sensor:
   containers:
-    - so-core
+    - so-nginx
     - so-telegraf
     - so-steno
     - so-suricata
@@ -196,7 +196,7 @@ sensor:
     - so-filebeat
 warm_node:
   containers:
-    - so-core
+    - so-nginx
     - so-telegraf
     - so-elasticsearch
 fleet:
@@ -206,6 +206,6 @@ fleet:
     - so-fleet
     - so-redis
     - so-filebeat
-    - so-core
+    - so-nginx
     - so-telegraf
     {% endif %}
diff --git a/pillar/firewall/analyst.sls b/pillar/firewall/analyst.sls
diff --git a/pillar/firewall/beats_endpoint.sls b/pillar/firewall/beats_endpoint.sls
diff --git a/pillar/firewall/forward_nodes.sls b/pillar/firewall/forward_nodes.sls
diff --git a/pillar/firewall/masterfw.sls b/pillar/firewall/masterfw.sls
diff --git a/pillar/firewall/minions.sls b/pillar/firewall/minions.sls
diff --git a/pillar/firewall/osquery_endpoint.sls b/pillar/firewall/osquery_endpoint.sls
diff --git a/pillar/firewall/search_nodes.sls b/pillar/firewall/search_nodes.sls
diff --git a/pillar/firewall/wazuh_endpoint.sls b/pillar/firewall/wazuh_endpoint.sls
diff --git a/pillar/healthcheck/standalone.sls b/pillar/healthcheck/standalone.sls
@@ -0,0 +1,5 @@
+healthcheck:
+  enabled: False
+  schedule: 300
+  checks:
+    - zeek
diff --git a/pillar/top.sls b/pillar/top.sls
@@ -1,7 +1,10 @@
 base:
   '*':
     - patch.needs_restarting
-    - docker.config
+
+  '*_eval or *_helix or *_heavynode or *_sensor or *_standalone':
+    - match: compound
+    - zeek
 
   '*_mastersearch or *_heavynode':
     - match: compound
@@ -37,6 +40,18 @@ base:
     - healthcheck.eval
     - minions.{{ grains.id }}
 
+  '*_standalone':
+    - logstash
+    - logstash.master
+    - logstash.search
+    - firewall.*
+    - data.*
+    - brologs
+    - secrets
+    - healthcheck.standalone
+    - static
+    - minions.{{ grains.id }}
+
   '*_node':
     - static
     - firewall.*

diff --git a/pillar/zeek/init.sls b/pillar/zeek/init.sls
@@ -0,0 +1,55 @@
+zeek:
+  zeekctl:
+    MailTo: root@localhost
+    MailConnectionSummary: 1
+    MinDiskSpace: 5
+    MailHostUpDown: 1
+    LogRotationInterval: 3600
+    LogExpireInterval: 0
+    StatsLogEnable: 1
+    StatsLogExpireInterval: 0
+    StatusCmdShowAll: 0
+    CrashExpireInterval: 0
+    SitePolicyScripts: local.zeek
+    LogDir: /nsm/zeek/logs
+    SpoolDir: /nsm/zeek/spool
+    CfgDir: /opt/zeek/etc
+    CompressLogs: 1
+  local:
+    '@load':
+      - misc/loaded-scripts
+      - tuning/defaults
+      - misc/capture-loss
+      - misc/stats
+      - frameworks/software/vulnerable
+      - frameworks/software/version-changes
+      - protocols/ftp/software
+      - protocols/smtp/software
+      - protocols/ssh/software
+      - protocols/http/software
+      - protocols/dns/detect-external-names
+      - protocols/ftp/detect
+      - protocols/conn/known-hosts
+      - protocols/conn/known-services
+      - protocols/ssl/known-certs
+      - protocols/ssl/validate-certs
+      - protocols/ssl/log-hostcerts-only
+      - protocols/ssh/geo-data
+      - protocols/ssh/detect-bruteforcing
+      - protocols/ssh/interesting-hostnames
+      - protocols/http/detect-sqli
+      - frameworks/files/hash-all-files
+      - frameworks/files/detect-MHR
+      - policy/frameworks/notice/extend-email/hostnames
+      - ja3
+      - hassh
+      - intel
+      - cve-2020-0601
+      - securityonion/bpfconf
+      - securityonion/communityid
+      - securityonion/file-extraction
+    '@load-sigs':
+      - frameworks/signatures/detect-windows-shells
+    redef:
+      - LogAscii::use_json = T;
+      - LogAscii::json_timestamps = JSON::TS_ISO8601;