summarizing multiple similar findings into problems

[LeoOMaia]

Description

• Feature:
  We created a "Problems" tab that disambiguates similar findings based on the script_id that detected them. This allows us to consolidate most findings into a single problem, enabling the vulnerability analyst to more accurately identify all types of issues without duplication.
• Bug fix implemented by this PR:
  A bug in the OpenVAS XML parser has been fixed, where it was not correctly identifying the script_id and was always returning None.

Test results

We tested the creation and association of findings to a specific problem according to the JSON file we provided, which already identifies similar findings by grouping them based on their script_id. We also tested that after creating the findings and problems, deleting a finding would update the problem by reducing the number of associated findings, and if all findings related to a problem were deleted, the problem would be automatically removed. Additionally, we verified the logic where if all findings become inactive, the problem status changes from open to closed, and if at least one finding remains active, the problem stays open.

[dryrunsecurity]

DryRun Security Summary

The GitHub Pull Request introduces a new "Problem" concept in the Dojo application to enhance security management by mapping security findings to specific vulnerabilities, improving tracking, reporting, and prioritization of security issues.

Expand for full summary

Summary:

The changes in this GitHub Pull Request are focused on enhancing the security management and tracking capabilities of the Dojo application. The key changes include:

1. Introduction of a "Problem" Concept: The application is integrating a new "Problem" model and functionality to associate security findings with specific security issues or vulnerabilities. This provides better context, prioritization, and tracking of security problems across the application.

2. Mapping Findings to Problems: The code changes in various parsers (e.g., Nmap, OpenVAS, Nuclei) now include functionality to associate the identified findings with the corresponding "problems" in the application. This helps to better organize and manage security issues.

3. Caching and Updating of Mappings: The application now includes a mechanism to download, cache, and periodically update a JSON file that contains mappings between script IDs and problem IDs. This helps to improve the performance and reliability of the problem-finding association process.

4. Improved Reporting and Visualization: The changes include updates to the templates and views that display the list of problems and their associated findings. This provides better filtering, sorting, and export capabilities, which can enhance the security team's ability to prioritize and address the most critical issues.

5. Secure Coding Practices: The code changes generally follow secure coding practices, such as input validation, error handling, and the use of Django's built-in security features. However, it's important to continuously review the entire codebase and dependencies to ensure that no new security vulnerabilities are introduced.

Files Changed:

• .dryrunsecurity.yaml: Adds a new sensitive code path to the list of monitored areas.
• dojo/db_migrations/0219_problem_finding_problem.py: Introduces a new Problem model and associates it with the existing Finding model.
• dojo/models.py: Defines the Problem model with fields for name, ID, creation/update timestamps, and severity.
• dojo/problem/update_mappings.py: Adds a new asynchronous task to update the cache of problem-to-script ID mappings.
• dojo/problem/urls.py: Adds new URL patterns for listing and retrieving problems and their associated findings.
• dojo/problem/views.py: Implements the views for listing problems, open problems, closed problems, and the findings associated with a specific problem.
• dojo/problem/helper.py: Handles the downloading, caching, and mapping of problem-to-script ID data.
• dojo/settings/settings.dist.py: Adds a new setting for the URL of the problem-to-script ID mapping JSON file.
• dojo/templates/dojo/*: Updates various templates to display the problem-related functionality.
• dojo/tools/*: Updates the parsers for Nmap, OpenVAS, and Nuclei to associate findings with problems.
• dojo/urls.py: Adds a new URL pattern for the problem-related functionality.

Overall, the changes in this Pull Request appear to be a significant step forward in enhancing the security management and tracking capabilities of the Dojo application. The introduction of the "Problem" concept and the associated functionality provide a more structured and comprehensive approach to managing security issues, which can lead to improved prioritization, remediation, and overall security posture of the application.

Code Analysis

We ran 9 analyzers against 21 files and 2 analyzers had findings. 7 analyzers had no findings.

Analyzer	Findings
Configured Codepaths Analyzer	13 findings
Sensitive Files Analyzer	4 findings

Overall Riskiness

🔴 Risk threshold exceeded.

We've notified @mtesauro, @grendel513.

View PR in the DryRun Dashboard.

[cunha]

The most important comments are changing some of the algorithms to have better asymptotic performance.