Native library detection plugin

[wangmot]

Summary

If merged this pull request will detect native libraries from files. It will find matches either through file name or file content using regex patterns. Statically linked libraries will also be detected.

First, run get_emba_db.py to generate the EMBA database.

Then, just run surfactant and in the output file there will be a nativeLibraries: [] for each file that states all the libraries that it was able to detect. The output will have either isLibrary: [] meaning the file is the library, or containsLibrary: [] which means the libraries were statically linked within the file.

[nightlark]

Looks pretty good, mostly a few changes related to where the database of patterns is stored.

Let's add this helper function to the ConfigManager class (in surfactant/configmanager.py):

        """Determines the path to the data directory, for storing things such as databases.

        Returns:
            Path: The path to the data directory.
        """
        if platform.system() == "Windows":
            data_dir = Path(os.getenv("LOCALAPPDATA", os.path.expanduser("~\\AppData\\Local")))
        else:
            data_dir = Path(os.getenv("XDG_DATA_HOME", os.path.expanduser("~/.local/share")))
        data_dir = data_dir / self.app_name
        return data_dir

(There are also ~3 things flagged by the pre-commit CI check to change).

[nightlark]

Interesting, I came across two patterns that don't compile and cause an error:

• C++\ RTMP\ Server\ .*\ version\ v[0-9](\.[0-9])+?\ r\.[0-9]+ due to the ++ at the start being intended to match literally; emba may be taking advantage of some non-standard behavior in the tool they use for regex parsing, whereas Python's re module (and every other regexp parser on https://regex101.com/) either doesn't like that construct and complains about the pattern, or treats it as matching multiple C's.
• User-Agent: Siemens Canada Limited - ROX2 - [0-9](\.[0-9]+)?+$ is disliked by Python's re module due to the ?+ towards the end. Other regexp parsers give an error, or treat it as basically the same as just having a ?... I'm not sure if it intends to match against a literal + or not.. or maybe they meant to just have + at the end so it matches e.g. 4.2.4.4 with an arbitrary number of dots, instead of just a maximum of one dot in the version number.

Will need to test whatever they are using for matching with the regular expressions, but it is possible the first doesn't have the behavior they intended (and the 2nd is just ambiguous).

[nightlark]

I think they are using grep -o -a -E "<pattern>" for matching, and for both of those patterns grep says repetition-operator operand invalid. So I think these are actually just broken patterns in the EMBA database.

[nightlark]

Suggested change

	if name == content:
	if name.lower() == content.lower():

For the filename, let's make it a case-insensitive comparison.