The file name rule for daily rotate log?

[sinall]

I posted a thread on stackoverflow few months ago, no one answered it. It's not very urgent since it is only log file name so I just left it.

Recently I noticed that the changelog of 1.66.0 said it has been fixed.

For me I still have some issues about the file name. I can see some log file names after I upgrade boost to 1.66.0 just like it behaves in 1.65.1.

Here's the code:

logging::wsettings settings;
settings["Core"]["Filter"] = L"%Severity% >= debug";
settings["Core"]["DisableLogging"] = false;
settings["Sinks.File"]["Destination"] = L"TextFile";
settings["Sinks.File"]["FileName"] = L"App_%Y-%m-%d.%N.log";
settings["Sinks.File"]["Format"] = L"%TimeStamp% [%ThreadID%] %Severity(format=\"%-7s\")% - %Message%";
settings["Sinks.File"]["AutoFlush"] = true;
settings["Sinks.File"]["RotationTimePoint"] = L"00:00:00";
settings["Sinks.File"]["RotationSize"] = 10 * 1024 * 1024;
settings["Sinks.File"]["Target"] = filePath_;
settings["Sinks.File"]["ScanForFiles"] = L"All";
settings["Sinks.File"]["Append"] = true;

Here's some log file examples:

App_2018-01-09.7.log
App_2018-01-10.8.log
App_2018-01-10.0.log
App_2018-01-11.1.log
App_2018-01-11.0.log
(Sorted by creation time ASC)

My understanding is that the counter and date-time part are unrelated, the counter increases whenever a new file is being created. Is my understanding correct?

But this makes the file names confusing. For example:

1. Start app on 2018-01-11, then the file name is App_2018-01-11.0.log
2. A new filename will be created as App_2018-01-12.1.log tomorrow
3. If I restart app tomorrow, then a new filename created as App_2018-01-12.0.log

Is this the current behavior?
If so, my suggestion is to reset the file counter for a new day like this:

1. Start app on 2018-01-11, then the file name is App_2018-01-11.0.log
2. A new filename created as App_2018-01-12.0.log tomorrow
3. If I restart app tomorrow, then a new filename will be created as App_2018-01-12.0.log
4. If the file becomes too large, then the file counter will be increased when creating new file name.

Does this make sense?

[Lastique]

My understanding is that the counter and date-time part are unrelated, the counter increases whenever a new file is being created. Is my understanding correct?

Yes, mostly. The file counter is incremented whenever a new file name is generated by the sink backend. This happens whenever the log file is rotated. In certain cases though the counter is not incremented - mostly when the backend is supposed to append to an existing file. For example, as a recovery measure against the filesystem was being full and now having free space. In this case the file stream would be marked as bad (because the previous write failed) and the sink reopens the same file to attempt to continue writing to it.

The counter can be updated to a starting value by calling scan_for_files on the backend. The call will attempt to discover the last used counter value and start counting new files from the next value. The fix in 1.66 was related to this call. This method may not be exactly what you want, but it may be better than always starting from 0.

If so, my suggestion is to reset the file counter for a new day like this:

This requires the sink backend to be coupled with the file name generator. In particular, the sink would have to know that the generated filename without the counter has changed (i.e. from 2018-01-11 to 2018-01-12 or from 2018-01-11_23_59_59 to 2018-01-12_00_00_00 or whatever the generator produces). I don't like that kind of coupling.

I suppose, I could extend the interface of the backend to allow users to set their own file name generators instead of relying just on the filename pattern. That would allow you to implement any behavior you want without the added coupling. But on the other hand, it would break scan_for_files, because it needs the filename pattern. I'm not sure what the solution is here.

[sinall]

I haven't go through the code, it is too complex for me. In my experience, a 'counter' generating strategy could be introduced:

1. global - increases no matter the filename with/without the counter has changed
2. filename based - only increases when for same filename without counter or reset for new filename

The global strategy could be used as default.

Generally speaking the filename generating and counter management could be handled in filename generator with counter strategy (as well as other strategies).

However, in practice, the intention of 'counter' is for distinguish filenames; but if the major part of filename has already been significantly changed, the counter part is not very helpful therefore should be reset. So the introducing a strategy idea is might be over-engineering.

---

I guess most users want to use logging library out-of-the-box with a few configurations. It could handle most of common cases, only need to be extended for some particular cases (for example a different persistence layer).

[hoangtocdo90]

App_2018-01-09.7.log
App_2018-01-10.8.log
App_2018-01-10.0.log
App_2018-01-11.1.log
App_2018-01-11.0.log

I think have some problem with log file number auto increment with no limit.
Example App_2018-01-11.999999999999999999999.log

[IevgenVovk]

I was wondering if there any plans to resolve this in some way.

My team is presently working on the control software for the new telescope for Cherekov Telescope Array, we're trying to use Boost (already our dependency) for logging. However we are required to use the "filename based" as opposed to "global" (using the terms from the post above) counter in the logs, so we've also faced this issue.

[Lastique]

No specific plans. The counter and timestamp are designed to work independently at present and there is no easy way to change it. There is no analysis of a would-be filename without the counter and frankly I don't fully understand how that would work.

If the current counter behavior does not suit you, my suggestion is to just not use it and instead use a more precise timestamp in the filename.

[IevgenVovk]

I imagine a way to realize this would be to have an additional, joint date/counter name generator, triggered by the presence of the specific formatter (say "%n" instead of "%N") in the log name specification. The generator may keep the presently used date and reset the counter when the date changes. Unable to say how difficult would it be to implement this given the present name generation logic, I rely here on your good judgement.

Regarding alternating the log name format, I'm afraid this is not in my team's hands (event if personally I'd be also in favor of finer time stamps).

Nevertheless, thank you for considering this. Will look for another option then.

[Lastique]

You can always write a custom sink backend that implements the behavior you require. Although that will involve more than just filename generation.