Can we reuse the results from the scanner/lexer for the full parse?

[f-markus]

For performance reasons, I'm starting with just the lexer (so the app is really responsive to user inputs):

var parser = new Irony.Parsing.Parser.Parser(_langData);
parser.Scanner.VsSetSource(source, 0, 0);
do { 
  var token = parser.Scanner.VsReadToken(ref state); 
} while (...)

Afterwards, for the full parse, it would be nice to reuse the already calculated tokens. Something like so

var tree = parser.Parse(tokenlist);

I already played around a little bit, but I was not successful. I managed to supply a list of tokens, but it seems that during parsing, some additional state is stored with the tokens. This state seems not to be there when just calling VsReadToken().

Am I missing something?

[rivantsov]

You are right, there's no such direct facility, unfortunately. You can try workaround.
Add token list to ParseContext.BufferedTokens; it is internal but you can reach it using Reflection. And then provide empty source text to Parser. The parser would consume all bufferred tokens first and then empty actual source.

[rivantsov]

or just add another consturctor to Parser class, and add tokens to BufferTokens