Export advisories in OSV format #599

jaylinski · 2021-11-17T23:26:54Z

Fixes #576

This commit adds an automatic OSV export to the osv branch while keeping the current repository as is.

Inspired by rustsec: https://github.com/rustsec/advisory-db/blob/main/.github/workflows/export-osv.yml

Preview

https://github.com/jaylinski/security-advisories/tree/osv

Possible improvements

Validate generated JSON against spec (https://github.com/ossf/osv-schema/blob/main/validation/schema.json)
Validate uniqueness of IDs

Before merging

Create an empty osv branch with a readme similar to this one: https://github.com/rustsec/advisory-db/blob/osv/README.md

jaylinski · 2021-11-18T00:53:16Z

export-osv.php

+            array_push($events, ['introduced' => $branch[0]]);
+            array_push($events, ['fixed' => $branch[1]]);
+        } else {
+            array_push($events, ['fixed' => $branch[0]]);


⚠️ This is not correct, because this DB only specifies ranges that are affected, but not the fixed version.

Maybe we can just convert versions like <8.22.1 to 8.22.1.

How useful is OSV if we cannot list fixed versions at all?

@naderman I worked around this by retrieving all package tags from the Packagist API and comparing the ranges with the tags by calling Semver::satisfies($version, implode(' ', $constraints)).

Example: https://github.com/jaylinski/security-advisories/blob/osv/packagist/CVE-2021-43608.json

What do you think about it?

Would it make more sense to add this to packagist.org's security advisories API as a new output format (which pulls data from this repo here)? As you'd have access to live package data there?

Yes, definitely. I wasn't aware of the packagist.org security advisories API.

Should I open a PR at the packagist.org repo?

Sure, go ahead! :-)

It's pretty simple right now, just a way to list advisories for a set of packages, see https://packagist.org/apidoc#list-security-advisories

stof · 2021-11-18T10:45:49Z

export-osv.php

+    foreach (array_column($branches, 'versions') as $branch) {
+        if (count($branch) === 2) {
+            array_push($events, ['introduced' => $branch[0]]);
+            array_push($events, ['fixed' => $branch[1]]);


be careful. We have cases like >= 1.3.0, <2.0 where things are not fixed in 2.0 (if another branch has >=2.0 in its affected constraints)

That's true. I reworked it, see #599 (comment).

stof · 2021-11-18T10:46:26Z

export-osv.php

+
+    foreach (array_column($branches, 'versions') as $branch) {
+        if (count($branch) === 2) {
+            array_push($events, ['introduced' => $branch[0]]);


$branch[0] is not a version but a constraint

That's true. I reworked it, see #599 (comment).

stof · 2021-11-18T10:47:10Z

export-osv.php

+            ] : null,
+            [
+                'type' => 'PACKAGE',
+                'url' => 'https://packagist.org/packages/' . $package,


this looks broken for packages using a custom repository

I'm now skipping packages that have a custom or no composer-repository.

stof · 2021-11-18T10:47:27Z

export-osv.php

+            array_key_exists('link', $advisory) ? [
+                'type' => 'ADVISORY',
+                'url' => $advisory['link'],
+            ] : null,


putting null in the array looks wrong to me

True. Since the validator enforces links anyway, I just skipped that check.

Fixes FriendsOfPHP#576

icanhazstring · 2022-02-20T13:32:16Z

Any new update here to get the fixed version into the list as well?

jaylinski · 2022-02-20T19:18:06Z

@icanhazstring The PHP and OSV vulnerability schemes don't have a fixed-field. Only the affected versions are listed.

(Or maybe I'm misunderstanding your question?)

icanhazstring · 2022-02-20T19:54:22Z

@jaylinski was referring to the comment from @naderman about the Packagist advisory api about the fixed version.

Or is it somewhat possible to get security issue listed with affected version and the next which fixes it?

ohader · 2022-06-14T20:16:00Z

Just wanted to leave that here (probably you knew it already):
e.g. https://github.com/github/advisory-database/blob/b07a1c25e2ec4fe59bf3dae2c6b7db3b02f4ae75/advisories/github-reviewed/2022/04/GHSA-x7cr-6qr6-2hh6/GHSA-x7cr-6qr6-2hh6.json

gives an example of fixed and introduced nested in affected
OSV items do exist already in GitHub's advisory DB - can we some "synchronize" multiple sources (did not think about the details & consequences, yet)

naderman · 2022-06-14T20:34:53Z

@ohader The packagist.org API already synchronizes and merges github's db and friendsofphp, e.g. see here: https://packagist.org/packages/guzzlehttp/guzzle/advisories?version=6278149

It would really just need someone to build an OSV compatible output for the data we collect there to have an OSV database for PHP.

jaylinski force-pushed the export-osv branch 14 times, most recently from e0825fa to e41fbe9 Compare November 18, 2021 00:41

jaylinski commented Nov 18, 2021

View reviewed changes

stof reviewed Nov 18, 2021

View reviewed changes

jaylinski force-pushed the export-osv branch 3 times, most recently from 1b49ea4 to 5d2ade5 Compare November 18, 2021 19:36

Export advisories in OSV format

1827365

Fixes FriendsOfPHP#576

jaylinski force-pushed the export-osv branch from 5d2ade5 to 1827365 Compare November 18, 2021 19:42

jaylinski mentioned this pull request Apr 12, 2022

Import advisories from the Github security vulnerability database automatically #626

Closed

Raen111 approved these changes Dec 14, 2023

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Export advisories in OSV format #599

Export advisories in OSV format #599

jaylinski commented Nov 17, 2021 •

edited

Loading

jaylinski Nov 18, 2021 •

edited

Loading

naderman Nov 18, 2021

jaylinski Nov 18, 2021

naderman Nov 25, 2021

jaylinski Nov 26, 2021 •

edited

Loading

naderman Nov 26, 2021

stof Nov 18, 2021

jaylinski Nov 18, 2021

stof Nov 18, 2021

jaylinski Nov 18, 2021

stof Nov 18, 2021

jaylinski Nov 18, 2021

stof Nov 18, 2021

jaylinski Nov 18, 2021

icanhazstring commented Feb 20, 2022

jaylinski commented Feb 20, 2022

icanhazstring commented Feb 20, 2022

ohader commented Jun 14, 2022 •

edited

Loading

naderman commented Jun 14, 2022

Export advisories in OSV format #599

Are you sure you want to change the base?

Export advisories in OSV format #599

Conversation

jaylinski commented Nov 17, 2021 • edited Loading

Preview

Possible improvements

Before merging

jaylinski Nov 18, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jaylinski Nov 26, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

icanhazstring commented Feb 20, 2022

jaylinski commented Feb 20, 2022

icanhazstring commented Feb 20, 2022

ohader commented Jun 14, 2022 • edited Loading

naderman commented Jun 14, 2022

jaylinski commented Nov 17, 2021 •

edited

Loading

jaylinski Nov 18, 2021 •

edited

Loading

jaylinski Nov 26, 2021 •

edited

Loading

ohader commented Jun 14, 2022 •

edited

Loading