Implement a *code* parser for improved auto-documentation

[rtopfer]

Parsing functions' and methods' arguments blocks to enable auto-documenting of input constraints (size, type, validation functions) and default assignments so that these details don't need to be manually documented.

(Note: for defaults, it might be more sensible generally to copy the assignment portion of the code itself (e.g. = zeros(100,100)) rather than the value itself!)

Note: nargin and nargout are readily obtained for functions and methods, but getting the remaining details will require something more involved...

Suggested approaches:

• This simple code search could be a useful starting point.

• An even simpler option is probably just to use MATLAB's textscan and/or type( ) functions and parse the return. (Probably not that difficult when arguments blocks are used as they need to fall between the leading documentation comments and the start of the main code.)

Related issue: #16 —adding the option to include implementation details in the documentation

Related note/rant re: class introspection in MATLAB:
meta.class objects provide a lot of useful material to document classes and their members.
E.g.:

Mc = meta.class.fromName( 'Documentor' ) ; 

Mc.MethodList then contains info on Documentor's methods -- including the input names (counting them then gives the number of arguments).

Curiously, however, it seems that if you add an arguments block to a class method---even if you don't specify default values, which normally renders arguments optional---suddenly the input names disappear and are replaced by an uninformative varargin??

Generally it seems that the whole meta.class library isn't fully/coherently implemented. (e.g. many properties that sound as though they would be very useful (e.g. Description, DetailedDescription) are not currently used (according to the official documentation).

Furthermore, if you convert a meta.class object to struct (e.g. struct( Mc.MethodsList ))
hidden properties are then displayed (such as InputTypes) that clearly haven't been implemented: E.g. InputTypes always seems to be {'any'} even when an arguments block explicitly states what the permitted input types are...

[gaspardcereza]

@rtopfer I just discussed with @jcohenadad the need for a parser that would allow us to convert the Matlab doc into the .md used for shimming-toolbox.org and I opened an issue here just before finding about that one.
Are you currently working on that parsing stuff ? I don't want to duplicate the work and as you have been working on the subject for a longer time you certainly have a better overview of how we could do it without "butchering" what has already been done. However I would be glad to discuss the different options we have with you if you need some feedback !

[gaspardcereza]

Also, now that we decided to go with packages instead of classes, I think we will have to find a solution that doesn't involve meta.class right ? (I know that meta.package exist so maybe it could be of interest here).

[jcohenadad]

Also, now that we decided to go with packages instead of classes

do we actually get rid of all classes? a bit of OO might still be relevant...

[rtopfer]

no I'm not working on it (I would've self-assigned the issue).
To be sure, the issue described above is more about parsing the code to glean (meta) info about it...

If you just want to parse/reformat the .md text prior to printing, it should be more straightforward: Nothing necessarily needs be added to the repo for this, it should just be a matter of calling regexprep with the appropriate inputs.

I.e. the documentation process is as follows:

1. Choose the files or directory you want documented, e.g. src could be the main toolbox folder
2. Create the documentor instance Dr = Documentor( src ) — this automatically starts drafting the text content and stores it under the property Dr.docContent.
3. Now, you can print to file with Dr.printdoc — but if you want to parse/replace text, just call replace (for known strings) or regexprep (for patterns) to modify docContent prior to calling Dr.printdoc

Note, if there are multiple .m files in src (unless they're all from the same class) Dr is returned as an object array: ie after step 2, each Dr(i) element i corresponds to a potential .md file (paths listed under Dr.docFile), so if you want to alter the output for a specific file, look at the names in there, note the index of the file of interest (say it's the 4th file), then change docContent using the index: Dr(4).docContent = ...

(p.s. packages aren't an alternative to classes. re: existing toolbox classes, the idea is to reorganize, simplify, and maybe get rid of a couple of them, not drop them altogether)

[rtopfer]

p.s. Can you please change the issue name back to what it was before? These are 2 different issues being discussed.

[jcohenadad]

ok, so after taking the time to properly read #17 (comment), i realize that the main purpose of this issue is to analyze input/output arguments. This would open the door to, e.g. doc checker (consistency between header's doc and arguments) and/or auto-doc of arguments.

so yes, i agree we are talking about two different issues, and the title should be changed to something different. However, i have to admit that "Elaborate auto-documentation feature" did not resonate with me (which led to this confusion of issue purposes). I would instead suggest something like: "Examine input/output arguments to functions/classes for verifications and auto-documentation". Just a suggestion... I let you modify the title as you see fit.

Now, regarding the "other" issue, i.e., parsing the header to change the output format for writing the .md file. Following on #17 (comment), i think it will take more than regexprep to address the issue. For example, one thing that needs to be done:
from there:

% INPUTS
%
%  bla (str)
%   blablabla
%
%  bla2 (float)
%    blablablablabla

to there:

| name | type | description
| bla | str | blablabla
…

it would be nice to implement that feature in HelpDocMd.

Also, we should probably re-open #19 and move that discussion there. @gaspardcereza pls

[rtopfer]

I thought it might just be a question of replacing things like INPUTS: with Inputs:, but the above example is indeed more complicated. I think regexprep would probably still be relevant, but I'm not sure too how to go about it...