Problems to generate API tracer Database (sqlite)

[LordGarithm]

Hi,

I am trying to use your project to speed up malware unpacking. So, I am trying to set up malware monitor.

Following the instructions I have compile the DbGenerator from the project Deviare2 after patching it as you saw (first just this tool and after that the full project). The issue is that to use Database/DbBuilder/ run build_db32 you need a prepossess headers(in this case preprocessW32.h) or just a .h file with the headers you want to build the database for. The preprocessW32.h headers are missing, at least in my case. I have seem many headers file under the directory HeaderBuilder. What would be the proper header file to be used with Malware monitor in Pyrebox?

Cheers,

Vicente

[xabiugarte]

Hi Vicente,

If you only need the 32 bit version of the database, you can grab it from the repository: https://github.com/Cisco-Talos/pyrebox/blob/master/mw_monitor/third_party/deviare2_db/deviare32_populated.sqlite

64 bit is still not fully supported, but it will be in the near future. Once support is added, I will probably add a 64bit database as well.

Regards,

[LordGarithm]

Hi, Thank you for your fast support. In any case I would appreciate if you explain me a little bit which is the role of the headers file missing required by build_db32 to create the db. Finally what do you mean by 64 is still not fully supported? You mean by malware monitor or by Pyrebox. Cheers,

…

On Thu, May 24, 2018 at 11:58 AM, xabiugarte ***@***.***> wrote: Hi Vicente, If you only need the 32 bit version of the database, you can grab it from the repository: https://github.com/Cisco-Talos/pyrebox/blob/master/mw_ monitor/third_party/deviare2_db/deviare32_populated.sqlite 64 bit is still not fully supported, but it will be in the near future. Once support is added, I will probably add a 64bit database as well. Regards, — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#50 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/Alu8muS8VdEJzgyhZQ2LuF9rLQtKrmp7ks5t1oRHgaJpZM4UK5VJ> .

[xabiugarte]

Hi Vicente,

I'll reproduce the steps with the latest version of Deviare2 and will update the documentation. I will update this bug once this is ready.

The lack of 64 bit support refers to malware monitor. While this set of scripts is certainly designed to support 64 bit, it has not been tested with 64 bit as much as with 32 bit. For instance, there are some known issues with Wow64 bit processes.

Refining this piece of software is one of my first-priority goals right now.

Thanks,

[LordGarithm]

Clear now thanks.

…

On Thu, May 24, 2018 at 3:50 PM, xabiugarte ***@***.***> wrote: Hi Vicente, I'll reproduce the steps with the latest version of Deviare2 and will update the documentation. I will update this bug once this is ready. The lack of 64 bit support refers to malware monitor. While this set of scripts is certainly designed to support 64 bit, it has not been tested with 64 bit as much as with 32 bit. For instance, there are some known issues with Wow64 bit processes. Refining this piece of software is one of my first-priority goals right now. Thanks, — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#50 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/Alu8mhMJrFSPqfOu8cZ0QAdrDyl2ByKbks5t1rqkgaJpZM4UK5VJ> .

[xorhex]

After much tinkering and banging my head against the wall, I managed to create the preprocessed64W.h with this command (after compiling everything in VS first):

Deviare2\Database\HeaderBuilder> buildall.bat Full

Also had to comment out multiple lines in the "Deviare2\Database\HeaderBuilder\Full\headers.h" file to ignore errors recorded in the log file until it gave me a clean "build". However after generating that file, build_db64.bat failed me due multiple syntax errors in preprocessed64W.h. :-(

Hope this helps some in trouble shooting the creation of the API database.

I'll wait until the 64 bit processes are better supported (as noted above). Thanks xabiugarte for all of the support!

[xabiugarte]

I have just uploaded an untested 64bit database to the dev branch of this repository. In order to generate it, I just needed to first generate the header file (preprocessed64W.h) by using the build script under Database\HeaderBuilder. I had to comment multiple lines on headers.h in order to make it work.

The build_db64.bat also fails, as xorhex pointed out. But I fixed all the syntax errors in the header file manually.

Support for 64 bit is still undergoing development and testing, and will be added to the refactored version of malware monitor (mw_monitor2), currently being developed in the 'dev' branch of this repository. I am leaving this issue open until this functionality is fully tested and merged into master.