Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

False Positives when checking all rules #144

Open
bryaend opened this issue Mar 14, 2023 · 2 comments · May be fixed by #145
Open

False Positives when checking all rules #144

bryaend opened this issue Mar 14, 2023 · 2 comments · May be fixed by #145

Comments

@bryaend
Copy link

bryaend commented Mar 14, 2023

cis-dil-benchmark/controls/3_5_firewall_configuration.rb

Lines 52 to 78 in e1dc70a

rules = ip6tables.retrieve_rules
describe.one do
rules.each do |rule|
describe rule do
it { should match(/(?=.*-A INPUT)(?=.*-i lo)(?=.*-j ACCEPT)/) }
end
end
end
describe.one do
rules.each do |rule|
describe rule do
it { should match(/(?=.*-A OUTPUT)(?=.*-o lo)(?=.*-j ACCEPT)/) }
end
end
end
describe.one do
rules.each do |rule|
describe rule do
it { should match(/(?=.*-A INPUT)(?=.*-s ::1)(?=.*-j DROP)/) }
end
end
end
only_if { ipv6.zero? }

When this section is run, I am getting hundreds of failures, one for every rule that doesn't match the filter:

-P INPUT DROP
is expected to match /(?=.*-A INPUT)(?=.*-i lo)(?=.*-j ACCEPT)/0.00076s
  Got 2 failures:

  1) Failure/Error: �[0m�[1;34;4mDEFAULT_FAILURE_NOTIFIER�[0m = lambda { |failure, _opts| raise failure }

       expected "-P INPUT DROP" to match /(?=.*-A INPUT)(?=.*-i lo)(?=.*-j ACCEPT)/
       Diff:
       @@ -1 +1 @@
       -/(?=.*-A INPUT)(?=.*-i lo)(?=.*-j ACCEPT)/
       +"-P INPUT DROP"
     # cis-dil-benchmark/controls/3_5_firewall_configuration.rb:158:in `block (5 levels) in load_with_context'
     # /var/lib/gems/2.7.0/gems/inspec-core-5.21.29/lib/inspec/runner.rb:356:in `each'
     # /var/lib/gems/2.7.0/gems/inspec-core-5.21.29/lib/inspec/runner.rb:356:in `find_all'
     # /var/lib/gems/2.7.0/gems/inspec-core-5.21.29/lib/inspec/runner.rb:356:in `add_resource'
     # /var/lib/gems/2.7.0/gems/inspec-core-5.21.29/lib/inspec/runner.rb:311:in `get_check_example'
     # /var/lib/gems/2.7.0/gems/inspec-core-5.21.29/lib/inspec/runner.rb:319:in `block in register_rule'
     # /var/lib/gems/2.7.0/gems/inspec-core-5.21.29/lib/inspec/runner.rb:318:in `each'
     # /var/lib/gems/2.7.0/gems/inspec-core-5.21.29/lib/inspec/runner.rb:318:in `flat_map'
     # /var/lib/gems/2.7.0/gems/inspec-core-5.21.29/lib/inspec/runner.rb:318:in `register_rule'
     # /var/lib/gems/2.7.0/gems/inspec-core-5.21.29/lib/inspec/runner.rb:137:in `block in load'
     # /var/lib/gems/2.7.0/gems/inspec-core-5.21.29/lib/inspec/runner.rb:135:in `each'
     # /var/lib/gems/2.7.0/gems/inspec-core-5.21.29/lib/inspec/runner.rb:135:in `load'
     # /var/lib/gems/2.7.0/gems/inspec-core-5.21.29/lib/inspec/runner.rb:163:in `run'
     # /var/lib/gems/2.7.0/gems/inspec-core-5.21.29/lib/inspec/cli.rb:366:in `exec'
     # /var/lib/gems/2.7.0/gems/thor-1.2.1/lib/thor/command.rb:27:in `run'
     # /var/lib/gems/2.7.0/gems/thor-1.2.1/lib/thor/invocation.rb:127:in `invoke_command'
     # /var/lib/gems/2.7.0/gems/thor-1.2.1/lib/thor.rb:392:in `dispatch'
     # /var/lib/gems/2.7.0/gems/thor-1.2.1/lib/thor/base.rb:485:in `start'
     # /var/lib/gems/2.7.0/gems/inspec-core-5.21.29/lib/inspec/base_cli.rb:35:in `start'

  2) Failure/Error: �[0m�[1;34;4mDEFAULT_FAILURE_NOTIFIER�[0m = lambda { |failure, _opts| raise failure }

       expected "-P INPUT DROP" to match /(?=.*-A INPUT)(?=.*-i lo)(?=.*-j ACCEPT)/
       Diff:
       @@ -1 +1 @@
       -/(?=.*-A INPUT)(?=.*-i lo)(?=.*-j ACCEPT)/
       +"-P INPUT DROP"
     # cis-dil-benchmark/controls/3_5_firewall_configuration.rb:158:in `block (5 levels) in load_with_context'
     # /var/lib/gems/2.7.0/gems/inspec-core-5.21.29/lib/inspec/runner_rspec.rb:97:in `run'
     # /var/lib/gems/2.7.0/gems/inspec-core-5.21.29/lib/inspec/runner.rb:193:in `run_tests'
     # /var/lib/gems/2.7.0/gems/inspec-core-5.21.29/lib/inspec/runner.rb:164:in `run'
     # /var/lib/gems/2.7.0/gems/inspec-core-5.21.29/lib/inspec/cli.rb:366:in `exec'
     # /var/lib/gems/2.7.0/gems/thor-1.2.1/lib/thor/command.rb:27:in `run'
     # /var/lib/gems/2.7.0/gems/thor-1.2.1/lib/thor/invocation.rb:127:in `invoke_command'
     # /var/lib/gems/2.7.0/gems/thor-1.2.1/lib/thor.rb:392:in `dispatch'
     # /var/lib/gems/2.7.0/gems/thor-1.2.1/lib/thor/base.rb:485:in `start'
     # /var/lib/gems/2.7.0/gems/inspec-core-5.21.29/lib/inspec/base_cli.rb:35:in `start'

This should be checking to see if the rule exists in the list, not checking every rule in the list against the matcher.
@bryaend
Copy link
Author

bryaend commented Mar 14, 2023

Upon checking the documentation, this is how describe.one is expected to work in the case that no check meets the criteria to pass the check, however, it creates a lot of clutter for what is essentially only one failure.

@bryaend
Copy link
Author

bryaend commented Mar 14, 2023

Also, as a final note, the failures are still present despite the check passing:

-A INPUT -i lo -j ACCEPT
is expected to match /(?=.*-A INPUT)(?=.*-i lo)(?=.*-j ACCEPT)/

@bryaend bryaend linked a pull request Mar 15, 2023 that will close this issue
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Don't test each rule, match on whole array. bryaend/cis-dil-benchmark
1 participant
@bryaend