Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[issue-184] validate against json-spec #254

Merged
merged 10 commits into from
Nov 3, 2022
Merged

[issue-184] validate against json-spec #254

merged 10 commits into from
Nov 3, 2022

Conversation

meretp
Copy link
Collaborator

@meretp meretp commented Oct 27, 2022

This PR is taken over from the original PR #197 by @dholth.

Besides writing the json according to the spec this PR also handles parsing json/yaml/xml files.

This PR doesn't cover files that do not belong to any package since this would need more work on the internal data model. Also it is not possible to parse checksums other than sha1, there is another PR #200 dealing with this, which can hopefully be rebased on this and add this functionality.

fixes #184, #234,#240, #242.

Copy link
Collaborator

@armintaenzertng armintaenzertng left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Phew, that was quite the amount of work :O
Thanks for addressing these non-trivial matters! :)

The remarks regarding the json files also apply to the xml and yaml versions as far as I checked.

data/SPDXJsonExample.json Outdated Show resolved Hide resolved
data/SPDXJsonExample.json Show resolved Hide resolved
data/SPDXJsonExample.json Show resolved Hide resolved
spdx/file.py Outdated
@@ -63,7 +63,7 @@ def __init__(self, name, spdx_id=None, chk_sum=None):
self.spdx_id = spdx_id
self.comment = None
self.type = None
self.chk_sum = chk_sum
self.checksums = [None]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
self.checksums = [None]
self.checksums = [chk_sum]

and maybe rename chk_sum to checksum

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the intention here was to distinguish the package and file checksum by using different names chk_sum for file and check_sum for package. I agree that it is better to use chksum and checksum but I would stick to the different names for the objects since they are slightly different: checksums for packages are optional whereas checksums for files are required.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To give my opinion here (already discussed with @meretp ):

  • I find the distinction between the two variable names rather arbitrary and obscure. I would question whether it's necessary at all, and if it is, I would prefer something like file_checksum and package_checksum. That would make it clear to a reader which one is which, and also provide a hint at why the naming distinction is made in the first place
  • Considering that this is probably an issue in quite a lot of places, and we are planning a major redesign in the near future, I want to be pragmatic about this. If renaming the variables everywhere is a lot of effort, let's just stick with the status quo until the redesign.

spdx/parsers/jsonyamlxml.py Outdated Show resolved Hide resolved
tests/data/formats/SPDXJsonExample.json Show resolved Hide resolved
tests/data/formats/SPDXJsonExample2.2.json Outdated Show resolved Hide resolved
tests/data/formats/SPDXJsonExample2.2.json Show resolved Hide resolved
tests/utils_test.py Outdated Show resolved Hide resolved
tests/utils_test.py Outdated Show resolved Hide resolved
@meretp meretp mentioned this pull request Oct 28, 2022
@nicoweidner nicoweidner dismissed armintaenzertng’s stale review November 2, 2022 12:36

Currently sick and unable to do another round

nicoweidner
nicoweidner previously approved these changes Nov 2, 2022
Copy link
Collaborator

@nicoweidner nicoweidner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just checked the outstanding comments, since @armintaenzertng already did an extensive review. I think this is good to go now (as soon as appveyor starts working again or we disable it somehow...)

nicoweidner
nicoweidner previously approved these changes Nov 3, 2022
Copy link
Collaborator

@nicoweidner nicoweidner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-approving after rebase

Copy link
Collaborator

@nicoweidner nicoweidner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Once more

dholth and others added 10 commits November 3, 2022 17:34
leads to failing tests caused by jsonyamlxml writer since json and yaml expect surrounding document

Signed-off-by: Meret Behrens <[email protected]>
This leads to the failure of tests caused by the jsonyamlxml writer.
Signed-off-by: Meret Behrens <[email protected]>
 - correct checksum in json example
 - add JSONExample2.2 from spec but exclude in tests since 2.2 is not yet completely supported
 - add files only once if they appear in multiple packages
 - parse only spdxid in documentDescribes, delete commented out code
 - delete unused XMLWriter and JsonYamlWriter class, updated xml test results
 - rework create_document_describes method
 - delete surrounding document in json/yaml test
 - rename licenseinfoinfiles method according to variable
 - rename chk_sum/ check_sum to chksum/checksum
 - delete duplicated relationships from json/yaml/xml

Signed-off-by: Meret Behrens <[email protected]>
@meretp meretp merged commit a3679d3 into spdx:main Nov 3, 2022
@meretp meretp deleted the checksums branch November 7, 2022 07:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

JSON is not copatible with v2.2 spec - "packages" and "files" lists should be outside "documentDescribes"
4 participants