One Generation Step For All Files

[LittleCoinCoin]

[Warning] this pull request is bqsed on the modifications under consideration in #7. Hence, commits f52d6ce, 11982dc, 4b9506c, and 996fe0f do not include modifications directly related to this pull request.

Motivation

Kodgen is currently powerful to generate (at least) one new file for each parsed file, but is impractical to generate (at least) one new file for all parsed files. The need for such generation feature might arise when one wishes to automatically generate a classes that will tally logic related to many other classes defined in different files.
The use case that motivated this pull request is to be able to have a class containing vectors of objects to store and manipulate them.

Goal

Modify (improve?) Kodgen's API to be able to switch between the current (unique) generation scheme that is One-Generate-For-Each-File and a new generation strategy that would be One-Generate-For-All-Files.

[Note] One-Generate should not be understood as "Output One File", but rather as the generation step in its entirety that might output several files depending on the user's code.

Suggested solution

The solution in this pull request relies on the existing API, but slightly modifies the internal execution flow. The proposed changes are based on the observation that preGenerateCode and postGenerateCode names may suggest a different behavior from what is actually implemented. Indeed, both methods appear in the CodeGenUnit::generateCode in the following order:

generateCode(...)
{
  ...
  preGenerateCode(...)
  ...
  initialGenerateCodeInternal(...) --> // different function body
    for each CodeGenerator:
     initialGenerateCode(...)
    <-- // going back to generateCode body
  ...
  foreachCodeGenEntityPair(...)
  ...
 finalGenerateCodeInternal(...) --> // different function body
    for each CodeGenerator:
      finalGenerateCode(...)
    <-- // going back to generateCode body
  ...
  postGenerateCode(...)
}

Note that in the original API documentation of preGenerateCode and postGenerateCode, it says that they are run right before and after foreachCodeGenEntityPair when this description actually matches initialGenerateCode and finalGenerateCode.

In any case, both functions run every time generateCode is called, that is for each parsed file (see CodeGenManager::processFile). Which means preGenerateCode and postGenerateCode are not strictly executed outside a generation step associated to a parsed file, hence are not truly pre and post steps.

Trying to use the functions in a way closer to what their names suggest is helpful to easily access to both One-Generate-For-Each-File and One-Generate-For-All-Files. In fact, the solution moves preGenerateCode and postGenerateCode outside of generateCode, associated with a new enum called EGenerationStrategies to support both the legacy generation scheme One-Generate-For-Each-File, and the new generation scheme One-Generate-For-All-Files.

Major changes to Code Generation Process

Modifications to CodeGenUnit

• BREAKING CHANGE: createCodeGenEnv, preGenerateCode, and postGenerateCode are public and not protected anymore. See 54dfac8
• BREAKING CHANGE: preGenerateCode is not taking FileParsingResult anymore. See 3c1c709
  • Because in the case of One-Generate-For-All-Files, the question arises to know which FileParsingResult we could be using in the preGenerateCode function.
  • Instead it FileParsingResult is set in the CodeGenEnv at the top of generateCode.
  • Thus, a way to fix this change for One-Generate-For-Each-File is simply to move the code using FileParsinResult from preGenerateCode to initialGenerateCode.
• Corresponding declarations in the derived class MacroCodeGenUnit were updated to reflect these changes.

New Generation Strategies

• Introduced EGenerationStrategies enum to define various generation strategies, including ForceReparseAll, ForceRegenerateAll, OneGenerateForEachFile, and OneGenerateForAllFiles. (Kodgen/Include/Kodgen/CodeGen/EGenerationStrategies.h). See 5ea0e22
  • ForceReparseAll, ForceRegenerateAll are notably used to assume in finer detail the parsing and generation behavior when going over all detected files in CodeGenManager::identifyFilesToProcess. This was previously controlled by a unique boolean value forceRegenerateAll. See 5ea0e22

Modifications to CodeGenManager

See 3c52973 for all bellow:

• processFile was renamed oneGenerateForEachParsedFile
• Added oneGenerateForAllParsedFiles to handle the new strategy. It is based upon oneGenerateForEachParsedFile.
  • The distribution of the parsing is also multi-threaded
  • The generation process (reading the parsed code and making std::string before assembling everything in a file), is NOT multi-threaded anymore. Given that we have only one generation step centralized by a unique CodeGenUnit, multi-threading would lead to data race where the std::string in the code generation unique might get appended at the same time by different generation tasks. A solution to resolve the data race was not investigated at all.
• Updated the run method to pass a EGenerationStrategies argument instead of the original boolean forRegenerateAll that could only influence the identification of source files on which the generation process should be applied.

Demonstration

You can check commit 84caea5

The example project in kodgen was updated to include a demonstration of the usage of the Generation Strategies. In particular, a new set of objects deriving from CodeGenUnit, CodeGenUnitSettings, CodeGenEnvironment, CodeGenModule, and PropertyCodeGen was implemented. These are respectively called DataStateCGU, DataStateCGUS, DataStateCGE, DataStateCGM, and DataPCG. They also serve as a demonstration of how it is possible to build one's own code generation unit and its dependences in addition to the existing MacroCodeGenUnit & Co.

DataStateCGU & Co are used to generate a class DataState (which can be renamed in the settings) that stores a variable number of data. In fact, every class (or struct) marked with kodgen's corresponding macro and that is further marked with the property "Data" will be added to the data state.

"Data" was added to the already existing example classes SomeClass and SomeOtherClass. As a result, the generated data state contains two std::vectors to store instances of these classes (that can be added with a generated DataState::EmplaceBackData), and that can, of course, be accessed with a getter.

However, the content of the data state is note as simple as it could be, simply to exemplify how to complex we can get with kodgen. In particular, the getter is a function pointer in an array one can access thanks to an enum. In addition, the getter and emplace back functions have constexpr if statements to help the compiler.