You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I've split out the lexer phase to avoid dealing with whitespaces and comments at the grammar level, so now there is a Pidgin parser for String->[Token] and a Pidgin parser for [Token]->AST.
How do I wire the original token locations correctly?
So far I found the SourcePosDelta/SourcePos types with CurrentSourcePosDelta/CurrentPos parsers, and the WithPosCalculator ext method. My hope was that I could plug SourcePosDelta in it using WithPosCalculator(token => token.Position), but the implementation treats it as the delta between the tokens (something like a token length, but not quite), not the delta from the beginning of the stream/enumerable/whatever-the-source-is.
Is my only option to compute token "lengths"?
If I make a PR that adds WithPosProvider ext method and
if(_sourcePosProvideris not null){varcurrent=(int)(location-_bufferStartLocation);return_sourcePosProvider(_span[current]);}
may I hope that it's accepted?
The text was updated successfully, but these errors were encountered:
I've split out the lexer phase to avoid dealing with whitespaces and comments at the grammar level, so now there is a Pidgin parser for String->[Token] and a Pidgin parser for [Token]->AST.
How do I wire the original token locations correctly?
So far I found the
SourcePosDelta
/SourcePos
types withCurrentSourcePosDelta
/CurrentPos
parsers, and theWithPosCalculator
ext method. My hope was that I could plug SourcePosDelta in it usingWithPosCalculator(token => token.Position)
, but the implementation treats it as the delta between the tokens (something like a token length, but not quite), not the delta from the beginning of the stream/enumerable/whatever-the-source-is.Is my only option to compute token "lengths"?
If I make a PR that adds
WithPosProvider
ext method andmay I hope that it's accepted?
The text was updated successfully, but these errors were encountered: