add handling of double underscores in generated html

[benjub]

Almost certainly buggy since I don't know the codebase nor the language. Also, probably inefficient since I copied code from a more complex case than just replacing __ with _. This is more like pseudocode than anything else, actually, but I wanted to get the ball rolling since the need for this feature was emphasized in metamath/set.mm#3389 (comment) by @avekens (i.e., we want to display command names like MINIMIZE_WITH and TRACE_BACK without triggering italics or subscripts in the generated html).

Help would be appreciated (it may be simpler to start another PR). Maybe from @digama0 or @wlammen or @tirix ?

[tirix]

I've added some comments, but that's a lot of different changes.
I could not propose changes directly in the repository (maybe I don't have the rights for that).
What I came up with is:

      /* Double-underscore handling: double-underscores are "escaped
      underscores", so replace a double-underscore with a single
      underscore and do not modify italic or subscript. */
      if (cmt[pos1] == '_') {
            if (g_htmlFlag) {  /* HTML */
              let(&cmt, cat(left(cmt, pos1), /* Skip (delete) "_" */
                  right(cmt, pos1 + 2), NULL));
              let(&cmtMasked, cat(left(cmtMasked, pos1), /* Skip (delete) "_" */
                  right(cmtMasked, pos1 + 2), NULL));
              pos1 ++; /* Adjust for 1 extra char '_' */
            } else {  /* LaTeX */
              let(&cmt, cat(left(cmt, pos1 - 1),  /* Skip (delete) "_" */
                  "\\texttt{\\_}",
                  right(cmt, pos1 + 2), NULL));
              let(&cmtMasked, cat(left(cmtMasked, pos1 - 1),  /* Skip (delete) "_" */
                  "\\texttt{\\_}",
                  right(cmtMasked, pos1 + 2), NULL));
              pos1 = pos1 + 11; /* Adjust for 11 extra chars "\texttt{\_}" */
            }
        continue;
      } /* End of double-underscore handling */

I've tested it successfully with HTML mode, but it does not work for TEX as the whole thing seems to be skipped at line 1941 with a test on the g_htmlFlag global variable.

[digama0]

needs a test

[benjub]

Added Thierry's fix and tests.

[benjub]

As for the tex: the if (g_htmlFlag) at Line 1970 of mmwtex.c is inside the if (g_htmlFlag != 0 && at Line 1941, and there semms to be no side effects affecting g_htmlFlag in between, so the else branch beginning Line 1976 is probably useless. (Same thing for Lines 2013/2024, and 2049/2057, which are not part of this PR.)

[tirix]

My compiler did not like this: \t is a tab, \_ is an unknown escape.
You have to escape the backslashes, as in "\\texttt{\\_}"

[tirix]

The cat is a concatenation.
Here you concatenate:

• a left part,
• an empty string (?)
• additional characters which include the second underscore and the character right after it,
• another empty string (?)
• the right part

That seems to be too much. We only need:

• a left part, including one underscore,
• the right part, after the second underscore.

[tirix]

It looks like you have to use cmt[pos1] here, without the +1.

[tirix]

As for the tex: the if (g_htmlFlag) at Line 1970 of mmwtex.c is inside the if (g_htmlFlag != 0 && at Line 1941, and there semms to be no side effects affecting g_htmlFlag in between, so the else branch beginning Line 1976 is probably useless. (Same thing for Lines 2013/2024, and 2049/2057, which are not part of this PR.)

That one is probably best left aside for now. This is not directly related with this PR.
One would have to understand if annotations are handled somewhere else for LaTeX (and those cases don't need to be handled here) or not (and special care has to be taken for LaTeX in that case too).

[benjub]

Reverted the last commit, as @tirix suggested. This looks ready to me. Better reviewed as a whole, since there was a commit revert. @digama0 ?

[tirix]

LGTM!

[benjub]

@digama0 does it look good to you ?

By the way: if we keep tests/.gitignore, we should add *.produced to it and remove it from .gitignore, or else we only keep a single ./gitignore at the root and put everything there.

[benjub]

@digama0 does it look good to you ?

[digama0]

LGTM, I didn't check the replacement logic in detail but I'm happy to follow up with bugfixes if we find any issues later. This should be replicated in metamath-knife if it hasn't been done already.

[benjub]

This should be replicated in metamath-knife if it hasn't been done already.

Not to my knowledge... and I'm not that fluent in Rust. Maybe @tirix knows/can add it ?

[tirix]

Maybe @tirix knows/can add it ?

The first part is in metamath/metamath-knife#127.