Move from yara.exe to library version of Yara

[JohnLaTwC]

Today our installer ships yara64.exe to support our Yara rules. I think eventually we want to move to incorporating yara as a library so it's linked into raccine.exe. Some pros/cons to think through:

Pros for status quo:

1. Today getting a new drop of yara is as simple as taking the release binaries from the yara project and including them.

2. No need to build yara library ourselves or deal with keeping yara source in sync with its master

Cons (arguments for moving to a linked yaralib):

1. Every interception calls CreateProcess for each .yar rule--imagine when a user copies their favorite rule repo of 100 .yar rules in our rules directory. This is an expensive API. It's also a common interception point by local anti-virus and security programs. The more work you do in the CP code path, the more potential for conflicts and side effects down the road in deployment. Having a yara.lib would avoid the need for additional CreateProcess calls because all the yara checks would happen in raccine.exe. As raccine interception points grow, some of them may be invoked quite frequently--we already hook powershell. So we need to be thoughtful about this.

2. We can have better control over our use of Yara. Examples:

   1. We may want tighter runtime limits. When we add support for scanning memory YARA Scan of Images and Process Memory in the Process Tree #58, we may want bounds / timeouts on this (i.e. no more than 5 seconds). We would have more control when incorporating yara into our sources.
   2. We have limits today imposed by yara. We pass additional context for rules (e.g. process command line) and we do this through external definitions (passed to yara64.exe via -d params). Today yara limits us on the number we can pass (#define MAX_ARGS_EXT_VAR 32). We can either alter this limit or (better) write a yara module that exposes the additional properties (so we can do import context just like the pe module and add additional context in a more natural way).
      https://github.com/VirusTotal/yara/blob/7517bbdf8778c37fa494966b39623dc6c2ccfce9/cli/yara.c#L122

References:
Sources: https://github.com/VirusTotal/yara
Yara C API: https://yara.readthedocs.io/en/stable/capi.html
Visual Studio files: https://github.com/VirusTotal/yara/tree/master/windows

[N3mes1s]

adding my2cent in this issue, could be interesting even to integrate in the pipeline the use of https://github.com/microsoft/vcpkg to handle the yara lib.

The support to yara 4.0.2 was merged not long ago: https://github.com/microsoft/vcpkg/tree/master/ports/yara

[JohnLaTwC]

Another reason this is important is to remove a possible infinite loop. Imagine someone sets raccine to intercept cmd.exe and it gets invoked. Well, raccine intercepts it via IFEO keys and spawns runyara.bat to apply yara rules. Running a batch file creates another...cmd.exe. Which invokes raccine, which runs runyara.bat, which... you get the picture. So probably need to look at this sooner rather than later.