FastHeadersSyncFeed explicitly flush before setting metadata

[Scooletz]

Closes #8029

This PR makes the flush of the underlying databases explicit for HeadersSyncFeed inserting headers. It's a change similar to a fix done in #6227 which explicitly calls the flush after a specific boundary is breached. This PR flushes the headers after each batch. If a performance penalty occurs, it should be done in an amortized way, every N batches set.

In addition BodiesSyncFeed is modified, so that it Flushes the underlying database in the same way HeadersSyncFeed does it. It calls the .Flush with a specific reason.

Changes

• explicit call out to .Flush of the underlying database

Types of changes

What types of changes does your code introduce?

• Bugfix (a non-breaking change that fixes an issue)
• New feature (a non-breaking change that adds functionality)
• Breaking change (a change that causes existing functionality not to work as expected)
• Optimization
• Refactoring
• Documentation update
• Build-related changes
• Other: Description

Testing

Requires testing

• Yes
• No

If yes, did you write tests?

• Yes
• No

Notes on testing

Optional. Remove if not applicable.

Documentation

Requires documentation update

• Yes
• No

If yes, link the PR to the docs update or the issue with the details labeled docs. Remove if not applicable.

Requires explanation in Release Notes

• Yes
• No

If yes, fill in the details here. Remove if not applicable.

Remarks

Optional. Remove if not applicable.

[Scooletz]

@asdacap You authored #6227 and I think you'd be most suited to provide the initial review. If you could, it'd be great.

[asdacap]

Do you have some performance figure? Also, where is the NoWAL flag in fast header? If that is not use, there is not much use for an explicit flush.

The thing about blocks and receipts is that, once the compaction writes was reduced enough, the portion of the writes for the WAL file was large enough that it is worth optimizing away.

The thing about headers is that, it is not parallelized when inserting because of the total difficulty calculation, so it straight up can't saturate the IO bandwidth. Latency, sure, but not bandwith. Having a way insert batch of headers instead of one by one would help it a lot.

[Scooletz]

Do you have some performance figure? Also, where is the NoWAL flag in fast header? If that is not use, there is not much use for an explicit flush.

I'll clear it out.

The thing about blocks and receipts is that, once the compaction writes was reduced enough, the portion of the writes for the WAL file was large enough that it is worth optimizing away.

I see.

The thing about headers is that, it is not parallelized when inserting because of the total difficulty calculation, so it straight up can't saturate the IO bandwidth. Latency, sure, but not bandwith. Having a way insert batch of headers instead of one by one would help it a lot.

This was not intended to be addressed here, but surely I'll take a look.

[Scooletz]

@asdacap I've reviewed the answers and I think I need more imperative input or a review that points to the missing ones. Specifically:

1. NoWAL there's no change in FastHeadersSyncFeed that changes the flush behavior. The only thing is that we flush before the the metadata are assigned. What do you mean by If that is not use, there is not much use for an explicit flush.
2. writes for the WAL I spent some time and I think I'm not following. This PR tries not to touch anything peformance/throughput related. It tries to ensure that we update metadata ONLY IF the underlying headers that we'll point to will be ok
3. total difficulty I think that if we want to tackle it, a separate PR should be provided. This aims only in providing a proper capture of the last inserted header.

[asdacap]

Discussed offline, so basically, there is no need for flush unless NoWAL flag is used, in which case, you MUST flush. Especially in the case that kamil gave as the system does not crash, only the app.