Pull based mempool

[dnkolegov]

In most Mine functions we use polling in the default clause of the select statement. This leads to the following million lines in the logs and probably big inefficiency.

2022-08-27T16:09:33.424Z	INFO	messagepool	messagepool/selection.go:472	message selection done	{"took": 0.000081971}
2022-08-27T16:09:33.424Z	INFO	messagepool	messagepool/selection.go:472	message selection done	{"took": 0.000091091}
2022-08-27T16:09:33.424Z	INFO	messagepool	messagepool/selection.go:472	message selection done	{"took": 0.000077593}
2022-08-27T16:09:33.424Z	INFO	messagepool	messagepool/selection.go:472	message selection done	{"took": 0.000106447}
2022-08-27T16:09:33.425Z	INFO	messagepool	messagepool/selection.go:472	message selection done	{"took": 0.0001174}
2022-08-27T16:09:33.425Z	INFO	messagepool	messagepool/selection.go:472	message selection done	{"took": 0.000089388}
2022-08-27T16:09:33.426Z	INFO	messagepool	messagepool/selection.go:472	message selection done	{"took": 0.000085548}
2022-08-27T16:09:33.426Z	INFO	messagepool	messagepool/selection.go:472	message selection done	{"took": 0.000086867}
2022-08-27T16:09:33.427Z	INFO	messagepool	messagepool/selection.go:472	message selection done	{"took": 0.000084708}
2022-08-27T16:09:33.427Z	INFO	messagepool	messagepool/selection.go:472	message selection done	{"took": 0.000087421}
2022-08-27T16:09:33.427Z	INFO	messagepool	messagepool/selection.go:472	message selection done	{"took": 0.000087834}
2022-08-27T16:09:33.427Z	INFO	messagepool	messagepool/selection.go:472	message selection done	{"took": 0.000084959}
2022-08-27T16:09:33.428Z	INFO	messagepool	messagepool/selection.go:472	message selection done	{"took": 0.000081306}
2022-08-27T16:09:33.428Z	INFO	messagepool	messagepool/selection.go:472	message selection done	{"took": 0.000129739}
2022-08-27T16:09:33.429Z	INFO	messagepool	messagepool/selection.go:472	message selection done	{"took": 0.000084564}
2022-08-27T16:09:33.429Z	INFO	messagepool	messagepool/selection.go:472	message selection done	{"took": 0.000087551}
2022-08-27T16:09:33.429Z	INFO	messagepool	messagepool/selection.go:472	message selection done	{"took": 0.00008116}
2022-08-27T16:09:33.429Z	INFO	messagepool	messagepool/selection.go:472	message selection done	{"took": 0.000089143}
2022-08-27T16:09:33.430Z	INFO	messagepool	messagepool/selection.go:472	message selection done	{"took": 0.00007906}
2022-08-27T16:09:33.430Z	INFO	messagepool	messagepool/selection.go:472	message selection done	{"took": 0.00007827}
2022-08-27T16:09:33.431Z	INFO	messagepool	messagepool/selection.go:472	message selection done	{"took": 0.000081545}
2022-08-27T16:09:33.431Z	INFO	messagepool	messagepool/selection.go:472	message selection done	{"took": 0.000085226}
2022-08-27T16:09:33.431Z	INFO	messagepool	messagepool/selection.go:472	message selection done	{"took": 0.000080777}
2022-08-27T16:09:33.431Z	INFO	messagepool	messagepool/selection.go:472	message selection done	{"took": 0.000088139}
2022-08-27T16:09:33.432Z	INFO	messagepool	messagepool/selection.go:472	message selection done	{"took": 0.000077117}
2022-08-27T16:09:33.432Z	INFO	messagepool	messagepool/selection.go:472	message selection done	{"took": 0.0000788}

Could we improve it at least from a usability perspective:

• do not log these message selections with INFO level
• replace select-default to ticker-based approach <-receiveFromMempool.C , but in this case, it is not obvious how often we should read from mempool
• add notifications to mempool and use pulling in the Mine

The first suggestion is the most trivial one.

@adlrocha What do you think?

[adlrocha]

This is a great point, and I agree we should get it fixed.

The first option is the friction-less one (and the most obvious).

I think the the other two may have their trade-offs:

• A ticker-based approach may not be a bad idea, but there's a risk that we may not be able to choose the right ticker time. I think this should be consensus-specific and configurable.
• A push-based mempool is the most efficient alternative, but in this case we may still need some kind of ticker for consensus algorithms that support empty blocks (we may still need to propose new blocks even if there are no messages in the mempool).

I'd suggest to start with 1, document 2 and 3 in an issue, and decide after we re-architect the code. WDYT?

[dnkolegov]

The first option added - 6fb898b