Support SPDX 2.3

[licquia]

SPDX 2.3 is about to be released; we should support it ASAP.

[armintaenzertng]

Fields currently missing in tools-python:

• files not belonging to a package

in packages:

• primary package purpose
• release date
• built date
• valid until date
• Add missing package properties for SPDX v2.3 #269

in snippets:

• byte range (mandatory)
• line range
• snippet class misses range field #262
• annotations (probably a parsing issue with (at least) xml) [jsonyamlxml] parse annotations that are nested within packages/snippets/files #275

---

Fields that are mandatory in tools-python, but should be optional:

• packages Documents do not have to contain packages or files any longer #239
• files in packages Documents do not have to contain packages or files any longer #239
• Make properties optional that are not mandatory anymore in 2.3 #272
  • package concluded license
  • package declared license
  • package copyright text
  • license information from files
  • package verification code
  • file concluded license
  • license information in file
  • file copyright text
  • snippet concluded license
  • snippet copyright text
• annotation ids (when the annotation belongs to an SPDX element, the Id of that element is implied)

These lists are probably not exhaustive. During my experimentation I found lots of issues with the xml parser/writer not detecting fields. These will probably be addressed during the major refactoring.

[armintaenzertng]

Issues with the xml parser:

The following fields are now using different tags/values:

in snippets:

• Rename snippet properties #287
  • fileId is now snippetFromFile
  • licenseInfoFromSnippet is now licenseInfoInSnippets

in files:

• file type values are now of the format SOURCE, not fileType_source
• Support multiple file types and fix the casing #309

---

During conversion from xml to xml the following fields get lost (whether they are not parsed or not written remains to be determined):

• other checksums than SHA1
• Add support for multiple file types and fix file type casing #310
• Recover missing properties during parsing/writing #278:
  • in packages:
    • comment
    • originator
  • in files:
    • attribution texts
    • in fileContributors: file contributor is converted to ['file contributor']

[armintaenzertng]

known shortcomings that we might want to fix later:

• rdf parser, builder and writer, see rdf support for 2.3 #295
• rdf parser, externalDocumentRefs: if the first child node of externalDocumentRef is not the externalDocumentId the parser fails with an IndexError, see "don't work for SPDX v2.2" #196 (comment))
• unpackaged files (depending on this: the tv-parser checks if packages for files exists and stops parsing with an OrderError if not, see "don't work for SPDX v2.2" #196 (comment))
• attributionText should be a list
• In json/yaml/xml, annotations cannot reference elements from external documents. This is an issue with the json schema, see Annotations object doesn't have SPDXID property in json spec2.2 spdx-spec#517 (comment)
• license expressions (not sure where we stand there, but Add support for parsing complex license expressions #111 made an attempt at implementation that to my knowledge has not been picked up since)
  • WITH is not parsed correctly

[armintaenzertng]

The new refactored version now found on main features complete v2.3 support. Please report any problems you may still find with it!