Skip to content

Commit

Permalink
🚑 Improve handling of exceptions when alternatives license sources exist
Browse files Browse the repository at this point in the history
  • Loading branch information
@pmonks
pmonks committed Oct 31, 2023
1 parent 63c372c commit 8e3b771
Show file tree
Hide file tree
Showing 4 changed files with 44 additions and 10 deletions.
27 changes: 22 additions & 5 deletions src/lice_comb/deps.clj
View file
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@
"Functionality related to combing tools.deps dependency maps and lib maps for
license information."
(:require [clojure.string :as s]
[clojure.tools.logging :as log]
[dom-top.core :as dom]
[lice-comb.maven :as lcmvn]
[lice-comb.files :as lcf]
Expand Down Expand Up @@ -61,6 +62,26 @@
version (:mvn/version info)]
(lcihttp/gav->pom-uri group-id artifact-id version))))

(defn- expressions-from-dep
"Find license expressions in the given dep, ignoring exceptions."
[dep]
(when dep
(let [info (second dep)
pom-uri (dep->pom-uri dep)]
(if-let [pom-expressions (try
(lcmvn/pom->expressions-info pom-uri)
(catch javax.xml.stream.XMLStreamException xse
(log/warn (str "Failed to parse " pom-uri " - ignoring") xse)
nil))]
pom-expressions
; If we didn't find any licenses in the dep's POM, check the dep's JAR(s)
(into {} (filter identity (dom/real-pmap #(try
(lcf/zip->expressions-info %)
(catch java.util.zip.ZipException ze
(log/warn (str "Failed to unzip " % " - ignoring") ze)
nil))
(:paths info))))))))

(defmulti dep->expressions-info
"Returns an expressions-info map for the given tools.dep dep (a MapEntry or
two-element vector of `['groupId/artifactId dep-info]`), or nil if no
Expand All @@ -71,11 +92,7 @@
(defmethod dep->expressions-info :mvn
[dep]
(when dep
(let [info (second dep)
pom-uri (dep->pom-uri dep)
expressions (if-let [expressions (lcmvn/pom->expressions-info pom-uri)]
expressions
(into {} (dom/real-pmap lcf/zip->expressions-info (:paths info))))] ; If we didn't find any licenses in the dep's POM, check the dep's JAR(s)
(when-let [expressions (expressions-from-dep dep)]
(lciei/prepend-source (dep->string dep) expressions))))

(defmethod dep->expressions-info :deps
Expand Down
19 changes: 16 additions & 3 deletions src/lice_comb/files.clj
View file
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@
for license information."
(:require [clojure.string :as s]
[clojure.java.io :as io]
[clojure.tools.logging :as log]
[lice-comb.matching :as lcmtch]
[lice-comb.maven :as lcmvn]
[lice-comb.impl.expressions-info :as lciei]
Expand Down Expand Up @@ -132,14 +133,26 @@
The optional `opts` map has these keys:
* `include-zips?` (boolean, default false) - controls whether zip compressed
files found in the directory are recursively included in the scan or not"
files found in the directory are recursively included in the scan or not
Note: logs and ignores errors (XML parsing errors, ZIP file errors)"
([dir] (dir->expressions-info dir nil))
([dir {:keys [include-zips?] :or {include-zips? false}}]
(when (lciu/readable-dir? dir)
(lciei/prepend-source (lciu/filepath dir)
(let [file-expressions (into {} (map file->expressions-info (probable-license-files dir)))]
(let [file-expressions (into {} (filter identity (map #(try
(file->expressions-info %)
(catch Exception e
(log/warn (str "Unexpected exception while processing " % " - ignoring") e)
nil))
(probable-license-files dir))))]
(if include-zips?
(let [zip-expressions (into {} (map #(try (zip->expressions-info %) (catch Exception _ nil)) (zip-compressed-files dir)))]
(let [zip-expressions (into {} (filter identity (map #(try
(zip->expressions-info %)
(catch Exception e
(log/warn (str "Unexpected exception while processing " % " - ignoring") e)
nil))
(zip-compressed-files dir))))]
(merge file-expressions zip-expressions))
file-expressions))))))

Expand Down
4 changes: 3 additions & 1 deletion src/lice_comb/maven.clj
View file
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,9 @@
If an InputStream is provided, it is the caller's responsibility to open and
close it, and a filepath associated with the InputStream *must* be provided as
the second parameter (it is optional for other types of input)."
the second parameter (it is optional for other types of input).
Note: throws on XML parsing error"
{:arglists '([pom] [pom filepath])}
(fn [& args] (type (first args))))

Expand Down
4 changes: 3 additions & 1 deletion test/lice_comb/files_test.clj
View file
Original file line number Diff line number Diff line change
Expand Up @@ -138,6 +138,8 @@
#{"GPL-2.0-only WITH Classpath-exception-2.0" "BSD-3-Clause" "Apache-2.0" "Unlicense AND CC0-1.0" "MIT" "MPL-2.0"}
(dir->expressions "."))))
(testing "Valid directory - include ZIP compressed files"
(println "\nℹ️ This next test emits logging output that includes exception information - this is expected/correct behaviour.\n")
(is (valid= ;#{"GPL-2.0-only WITH Classpath-exception-2.0" "BSD-3-Clause" "Apache-2.0" "Unlicense AND CC0-1.0" "MIT" "MPL-2.0" "CC-BY-4.0" "AGPL-3.0-or-later"} ; CC-BY-4.0 failing due to https://github.com/spdx/license-list-XML/issues/1960
#{"GPL-2.0-only WITH Classpath-exception-2.0" "BSD-3-Clause" "Apache-2.0" "Unlicense AND CC0-1.0" "MIT" "MPL-2.0" "AGPL-3.0-or-later"}
(dir->expressions "." {:include-zips? true})))))
(dir->expressions "." {:include-zips? true})))
(println "\nℹ️ The previous test emitted logging output that includes exception information - this is expected/correct behaviour.\n")))

0 comments on commit 8e3b771

Please sign in to comment.