anansi-project/comicinfo

[SenorSmartyPants]

https://github.com/anansi-project/comicinfo

In case this project wasn't aware of the anansi project. They have been trying to document/advance the comicinfo.xml format. I'm not part of that project or sure of the buy in of other comic related projects but I wanted CRCE to be aware of it.

[maforget]

I've posted yesterday an updated schema based on the code. I've looked a little bit at that page trying to figure out the original schema. Like I said in that other post, maybe the dev had a schema, but ComicRack just exports it's class to an XML. The only thing you need to do to export a schema is just use this command xsd.exe ComicRack.Engine.dll /t:ComicInfo to export it from the source binaries. It comes with the .NET framework.

I've already added Tags to the ComicInfo earlier, other than that not really certain what is changed. The schema shows an enum for ComicPageType, while the existing one is just a string. I've looked at the code and it just outputs as a string, but still uses an enum. Doesn't change anything really, but to show in the exported schema I had to modify the code slightly so it shows in the export.

Other than that they also created an enum for Age Ratings instead of just a string, I don't know about this one, not all rating are the same in all the world. But the have all of the Default ones.

All in all I don't see a lot of changes being made, the resulting files will be mostly the same. Only addition I see is that they added a Translator field, that could be a good idea to add to the program. There is also a GTIN, which I don't know what it is. Anything else?

• Add Translator Field

ComicInfo.xsd

[maforget]

I can always add an XmlAnyAttribute to the ComicPageInfo. Although I don't know if any unknown tags exists. We already have the Bookmark that does basically the same as a chapter in ComicRack.

[flaser]

I'm not sure as according its desc XmlAnyAttribute only captures whole XML Elements without mapping, not XML attributes.
(The name is a bit misleading, as I think the Attribute here refers to the C# concept instead the XML one >_<)

What I'm talking about is an unknown XML attribute (e.g. ImageWidth) in a parsed XML Element (e.g. Page).
If I'm correct, any unknown XML attribute of parsed XML elements should be quietly dropped at the moment.

UPDATE 1:

• Did a quick test, unknown XML attributes in parsed XML elements are stripped out.

UPDATE 2:

• I've seen parsing breaking and comicinfo reset if I add unknown XML elements. I'll do more investigation.
• This was caused by malformed XML, see UPDATE 5 below.

UPDATE 3:

• Recognized source of some of the weirdness I'm seeing.
• The 1st time a file is loaded is when it's stored in memory.
• Unless I force CR to reload from file (e.g. restart app), it will keep & serialize this state.

UPDATE 4:

• I can confirm multiple unknown XML Elements are stored & reserialized.

Type	Attributes	Example
regular	no	<MyElement1>LoremIpsum</MyElement1>
regular	yes	<MyElement2 Attr="x">LoremIpsum</MyElement1>
self-closing	no	<MyElement3 />
self-closing	yes	<MyElement4 Height="1" Weight="2"/>
nested	no	<MyElement5><Nest1>Lorem Ipsum</Nest1></MyElement5>
nested	yes	<MyElement5><Nest2 Attr="x">Lorem Ipsum</Nest2></MyElement5>

• I couldn't reproduce breaking of parsing by adding extra XML attributes to existing XML elements.
• I've rechecked and can confirm that these extra XML attributes get stripped out.

UPDATE 5:

• A malformed XML will break parsing --> This was causing my earlier issue
• BUG! CR will always update a malformed ComicInfo.xml
  (Even if the user choosed not update files outside the DB)

[maforget]

I think you are confused, I added a XmlAnyElementAttribute to the ComicInfo class. The XmlAnyAttributeAttribute is the same but for attributes, since it hasn't been added then it's normal that they are ignored. You just don't write the last Attribute when you use it.

No need to use an Event and then pick apart the Unknown, the idea is to just have someplace to store them. I was talking adding it to the ComicPageInfo because it's the only place that actually uses attributes, the ComicInfo only uses Elements.

The idea is to make sure CR doesn't remove some of the other program tags, not have some Frankenstein of possible XML format. The XmlElement that stores the Unparsed can contain attribute, that is why you are seeing the Attributes being save for Unknown Tags, but not the rest. Normally Attribute would be Properties of a subclass, you wouldn't have say a Title Element have an attribute, because it's supposed to be a string. That is why in the schema only The ComicPageInfo uses attributes.

Of course in a normal XML you could have a string have an attribute, but this isn't an XML => OBJECT it's an OBJECT => XML. It's a representation of the object in memory and has such, some of these theoretical wouldn't happen. To do what you want would require use to actually inspect each XmlElement & XmlNode, figure out if the element has a corresponding property. Then what if it has other unexpected elements after them, figure out a way to store them so they can be restored in the correct order. This is a mess and it isn't really required because the XmlSerializer does it for us and we shouldn't encounter these anyway. I don't know how the other software do it, but it's probably something similar to serializing.

Now I will have to check on that bug, because if you don't update the file it shouldn't touch the file regardless of it being malformed. But then again why would it be malformed, if it created by the serializer and is just a representation of an object?

[flaser]

You're correct, I was mixing things up with XmlAnyElementAtttribute.
Indeed, XmlAnyAttributeAttribute should do exactly what is required.
(The example even demonstartes this very use - d'oh, better look more carefully next time!)

You're also correct there's no need to support any arbitrary XML==>Object schema.
I was indeed thinking of ways how to preserve data added by other programs.
As for how it can get malformed? Simplest is user error, or a bug in an external scraper.

[maforget]

The change also as the effect of storing the unparsed elements in the database.

[therobbiedavis]

The point of the Anansi Project is to try and move the comicinfo schema forward as a standard for the community at large. The issue is it’s been multiple years without much or any progress forward and the current schema version is sorely lacking in some areas (hence Kavita moving forward with supporting non-standard fields).

I don’t think the Anansi Project as it stands right now is the way forward, but I think the idea behind it is. The culmination of all the key players working towards a better standardized schema for all.

[maforget]

A User on Reddit suggested adding the reading progress to the xml. There is the Current Page/Last Page Read properties that are related. I don't have any opinion on that either way. There was a reason it wasn't added, especially if sharing files. Good Idea? Bad Idea?

[maforget]

I feel like for something like this it goes into the re-write direction that we commented on in another thread. If CR's own sqlite database held that info then it would be store data like that. It would then also be easier for users to move in or out of different systems, without worrying if the reader they are using supports the very specific tag. Scripts could be made to query the db directly so that progress from other tracking apps like Kavita / Komga / Stump / Codex can be either inserted or exported.

This is irrelevant to either the form of data storage. The info is already stored in the db, that it is in sqlite or not is irrelevant. The question is do we make that info also saved in the file? That file is only used when restoring or sharing a comic. It is never used by ComicRack directly beside storing them.

Plugins just refer to the data that the ComicBook object contains. When CR loads a file the source of that data is usually the database, if it's not in the database, it will check if a XML exists inside, the NTFS alternate stream or either a side XML beside the file. The source of the data is irrelevant and should be transparent to the program. Script can already access all the info in the ComicBook, regardless of the source. You have ComicBook that inherits ComicInfo, the info that are in ComicInfo are saved in ComicInfo.xml (hence the name). While the data in ComicBook is just saved in the DB (which includes ComicInfo). Having the data saved in the XML is only a question of transferring the property from one class to the other.

My question is not whether it is compatible with other software, with all the different version I don't believe it is a problem. Normal XML already ignore unknown tags and to be frank it is these program problem. We can play fair with other and try not to erase tags we don't know like I did already, Maybe because I don't use these software, but my focus is just ComicRack. If I can do anything to fix a compatibility problem and it isn't a big deal, I will. But I ain't gonna bother what might happen if I implement what is best for ComicRack.

As for what another file would do, it might help prevent any compatibility problem with the OG ComicRack or other software, if there is any reason for concern. It might be easy to decouple the reading status from the ComicBook object, but since we would need to rewire the program to access this field anyway, so it won't change nothing.

[azuravian]

What is the purpose of writing the reading progress to the file itself instead of just keeping it in the db?

[maforget]

What is the purpose of writing the reading progress to the file itself instead of just keeping it in the db?

That user reason was that he frequently reinstall his windows (and doesn't seem to backup the db) and would like reading progress to be saved.

Although not the best reason I could see the usage, but like others have said I don't know if it's place is in the file itself.

[SenorSmartyPants]

Reading progress is about the user, not the book. It makes no sense to write it to the book if you consider multiple users.

[Nadiar]

Would make more sense to have a separate file in the directory with the cbz not inside of it, and probably as a plugin. You could just do a <filename-username.json> (format doesn't matter, just for illustration I'm assuming Jason) document to store "local" information.

[TheRealC0unt]

It might be useful to get in touch with the author of Komga server to coordinate about comicinfo tags.
At least he as well participates in the Anansi Project, if I'm not mistaken, and Komga is pretty widespread these days.
I am used to tag all my Comics and mangas with CR prior to out them on my Komga server, for example.

[maforget]

All we have done is add Tags to comicinfo which the Anansi project also does and support for any unknown tags that could be there. I don't know what else there is to add or change to it. Not planning on doing any adding any new tags to it. CR => Komga will work correctly like it does. Any other Software => CR should keep these data also.

[killo3967]

Here is a discussion about GTIN in the Anasi Project.

anansi-project/comicinfo#50

I think it's an important filed. Basically it's the id's of each web that contains metadata that you scrape.

[maforget]

Right now the 2.1 draft shows that GTIN is a single string entry and the proposal would be to have multiples identifiers and add an attribute called type where we can set which type of identifier it is?

Has anyone actually did changes in that regard or to even support the GTIN single entry?

No point to implement a new field when there isn't anything certain yet.

[killo3967]

Your right, so in add into the anasi-project DTIN an entry to proposing that GTIN has multiple tipes of entries as Calibre do with the ID's of the books.

[maforget]

I've added support for reading the MetronInfo.xml from the MetronInfo project. Doing so I've made change for support for multiple xml file. So it would be possible to have multiple types of files with various schema be supported.

The internal of the program are still the ComicInfo, so for now this only maps the corresponding fields from the MetronInfo.xml to the ComicInfo.xml. There is an order specified in the code, so it will prefer the ComicInfo.xml if both are present and really only read MetronInfo.xml when it is the only file present. Also a note that exporting will delete any MetronInfo.xml file but keep the metadata inside the ComicInfo.xml.