Enhance description of the output files from pre-processing and description of input format that Marius requires.

[thodrek]

What is the documentation lacking? Please describe.
Please add a clear description of the output of pre-processing; specifically, describe all files, their format, schema, and encoding requirements that are output by pre-processing.

Describe the improvement you'd like
Add this description in the comments of the general_parser function

Additional context
The above enhancement will enable writing custom (scalable) pre-processors that can emit Marius input files and won't require one starting from a raw CSV file.