HUGE warning for deprecated licences

[vargenau]

The following SPDX file has:
one line with PackageLicenseDeclared: LGPL-2.1
two lines with PackageLicenseDeclared: LGPL-2.1+

airflow-tern2.12.1.spdx.txt

So we expect warnings about deprecated licenses.

The issue is that the warning message is HUGE and mentions relationships.

Extract:

The following warning(s) were raised by airflow-tern2.12.1.spdx: [Relationship warning: Relationship warning: Relationship warning: Relationship warning: Relationship warning: Relationship warning: Relationship warning: Relationship warning: Relationship warning: Relationship warning: Relationship warning: Relationship warning: LGPL-2.1 is deprecated. in gcc-9-base in 7ccd46673b6aaf6a36223e9dfc465558ec677e554b3d9235f061d40d3f1c977d in 7ccd46673b6aaf6a36223e9dfc465558ec677e554b3d9235f061d40d3f1c977d in 36254b46d1f9a0328bade29f39118458bccce9bfea8d103bfded0d2a6fae417b in 36254b46d1f9a0328bade29f39118458bccce9bfea8d103bfded0d2a6fae417b in bc867b663af10df4812a0da86e7848606228f19e2d9400faf5893e7d13fc9f7e in bc867b663af10df4812a0da86e7848606228f19e2d9400faf5893e7d13fc9f7e in 6e11a4f8cea53e565c3f90626b2e56a2aff21956bd048c2b801ac8609fe7c58d in 6e11a4f8cea53e565c3f90626b2e56a2aff21956bd048c2b801ac8609fe7c58d in 0d539a55d59a940033e43dc6512c5438706923f6d89dd3dbe70463721d64445f in 0d539a55d59a940033e43dc6512c5438706923f6d89dd3dbe70463721d64445f in 84876d8dac764bbdb7d3c632608d524735a396302c3b7ae9172deb5750ce6eee in 84876d8dac764bbdb7d3c632608d524735a396302c3b7ae9172deb5750ce6eee in c1056de4735af96d75efc981d0ff63d4dbba58b266d571ed8eb2529f8a083fb4 in c1056de4735af96d75efc981d0ff63d4dbba58b266d571ed8eb2529f8a083fb4 in c2ea7373b660aed37248b647f7e00469efe8294d6c0007194ad10e9d809a0fbb in c2ea7373b660aed37248b647f7e00469efe8294d6c0007194ad10e9d809a0fbb in 35549c1d7df23d9951d851c69b6b65ce72bda95fa1ffb7f30d50ee61953d4e04 in 35549c1d7df23d9951d851c69b6b65ce72bda95fa1ffb7f30d50ee61953d4e04 in b0319bc418f0a6419cc4849e5078f6e1912a55be42925f7f5f36e5e816fb5705 in b0319bc418f0a6419cc4849e5078f6e1912a55be42925f7f5f36e5e816fb5705 in apache/airflow in apache/airflow in Tern report for apache/airflow, Relationship warning: Relationship warning: Relationship warning: Relationship warning: Relationship warning: Relationship warning: Relationship warning: Relationship warning: Relationship warning: Relationship warning: Relationship warning: Relationship warning: Relationship warning: Relationship warning: LGPL-2.1 is deprecated. in libnpth0 in 58248e3207652b9482aac6217ea284ee889b27edffcdfe78a697af34c68166e8 in 58248e3207652b9482aac6217ea284ee889b27edffcdfe78a697af34c68166e8 in 50c1f9516b61082d67aed87693f0c984c7d67ed79ee2a226a3bf9db6a80a1bc4 in 50c1f9516b61082d67aed87693f0c984c7d67ed79ee2a226a3bf9db6a80a1bc4 in 7ccd46673b6aaf6a36223e9dfc465558ec677e554b3d9235f061d40d3f1c977d in 7ccd46673b6aaf6a36223e9dfc465558ec677e554b3d9235f061d40d3f1c977d in 36254b46d1f9a0328bade29f39118458bccce9bfea8d103bfded0d2a6fae417b in 36254b46d1f9a0328bade29f39118458bccce9bfea8d103bfded0d2a6fae417b in bc867b663af10df4812a0da86e7848606228f19e2d9400faf5893e7d13fc9f7e in bc867b663af10df4812a0da86e7848606228f19e2d9400faf5893e7d13fc9f7e in 6e11a4f8cea53e565c3f90626b2e56a2aff21956bd048c2b801ac8609fe7c58d in 6e11a4f8cea53e565c3f90626b2e56a2aff21956bd048c2b801ac8609fe7c58d in 0d539a55d59a940033e43dc6512c5438706923f6d89dd3dbe70463721d64445f in 0d539a55d59a940033e43dc6512c5438706923f6d89dd3dbe70463721d64445f in 84876d8dac764bbdb7d3c632608d524735a396302c3b7ae9172deb5750ce6eee in 

[vargenau]

It seems to be a "heisenbug".

I had the huge message above the first time; running it a second time gave the correct warning message:

The following warning(s) were raised: [Package at line 35850 invalid: LGPL-2.1 is deprecated. in libseccomp2, Package at line 9216 invalid: LGPL-2.1 is deprecated. in gcc-9-base, Package at line 72638 invalid: LGPL-2.1 is deprecated. in libnpth0]

It occurred in "validate" and "convert".

[goneall]

Thanks @vargenau for reporting the issue. I'm able to reproduce it using the command line tool-java.

Based on the results from the command line tool, the long string should be returned on every validate - it looks like on some executions it is using the older version of the Java library validator. This is likely a server configuration issue resulting from the last hardware upgrade.

There are two separate issues involving two separate libraries. Both of these are introduced with the fix for verifying related elements:

• In the SPDX Java Library, Verify will now report any Relationship as invalid if it references an invalid Element, this will create one line for the package, and one line each for the the relationships that reference the package. What makes the worse is that it is recursive - so if a package has a relationship which references a different package with a different relationship that references the package with the invalid license, you'll get a very long String for the invalid relationship with all the intermediate relationship ID's. I've opened Issue 189 to track.
• In the SPDX tag/value parser, each relationship is verified. With the fix for verifying related elements this verification now verifies any related package and related relationships recursively. This could generate a large number of duplicate verification errors. I created Issue 50 to track.

[goneall]

One possible solution is to parse the results in the tools-java library. Added spdx/tools-java#134 with the proposal.

[vargenau]

@goneall
Thank you for the detailed explanations.

As a minor remark, it would be good to add some CR/LF to the output in order not to have the huge message on a single line.

[goneall]

@vargenau I noticed that too - turns out the Java library is adding CR/LF, but the online tools are treating them as whitespace when rendering the HTML - so some kind of conversion is needed - yet one issue

[vargenau]

Hi @goneall

I have not tested, but something like that might do the trick:

In file src/app/core.py:

-     ajaxdict["data"] = "The following warning(s) were raised:\n" + str(retval)
+     warnings = str(retval)
+     ajaxdict["data"] = "The following warning(s) were raised:\n" + warnings.replace('\n', '<br />')

(you have to do it 3 times in this file)

[goneall]

I have not tested, but something like that might do the trick:

Thanks @vargenau - @BassCoder2808 - Is this something you can do?

[BassCoder2808]

Hi @goneall, sure I will look into this and will let you know if I am able to add the following

[goneall]

Hi @goneall, sure I will look into this and will let you know if I am able to add the following

Thanks @BassCoder2808

[BassCoder2808]

Hi @goneall I have created the PR #495, let me know if anything else needs to be done in that

[goneall]

Since @BassCoder2808 solved the formatting issue related to this repo and there are other issues added to address the main issue raised here, I'll close this issue.

If I missed something, please feel free to open a new issue.