From 774df1da4bc1f8e9c292a188fd27354c7ca966d0 Mon Sep 17 00:00:00 2001 From: Hayley Denbraver Date: Thu, 16 Nov 2023 23:08:47 +0000 Subject: [PATCH] Adding C/C++ info to the docs (#648) I am open to feedback! Updated pages preview: - [Supported languages and lockfiles](https://hayleycd.github.io/osv-scanner/supported-languages-and-lockfiles/) - [Usage](https://hayleycd.github.io/osv-scanner/usage/) --------- Signed-off-by: Hayley Denbraver Co-authored-by: Andrew Pollock Co-authored-by: Gareth Jones Co-authored-by: Oliver Chang --- docs/configuration.md | 2 +- docs/contribute.md | 2 +- docs/experimental.md | 2 +- docs/github-action.md | 2 +- docs/index.md | 2 +- docs/installation.md | 2 +- docs/output.md | 2 +- docs/supported_languages_and_lockfiles.md | 105 ++++++++++++++++++++++ docs/usage.md | 89 +++++------------- 9 files changed, 134 insertions(+), 74 deletions(-) create mode 100644 docs/supported_languages_and_lockfiles.md diff --git a/docs/configuration.md b/docs/configuration.md index b4b0cc68ecf..aece78fd2c8 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -2,7 +2,7 @@ layout: page title: Configuration permalink: /configuration/ -nav_order: 4 +nav_order: 5 --- # Configure OSV-Scanner diff --git a/docs/contribute.md b/docs/contribute.md index a12cb39b458..7ab78231c91 100644 --- a/docs/contribute.md +++ b/docs/contribute.md @@ -2,7 +2,7 @@ layout: page title: Contribute permalink: /contribute/ -nav_order: 8 +nav_order: 9 --- # Contribute diff --git a/docs/experimental.md b/docs/experimental.md index 0d1d80147c1..bb2b77e868e 100644 --- a/docs/experimental.md +++ b/docs/experimental.md @@ -2,7 +2,7 @@ layout: page title: Experimental Features permalink: /experimental/ -nav_order: 7 +nav_order: 8 --- # Experimental Features diff --git a/docs/github-action.md b/docs/github-action.md index 7dee78e8851..601b71fcad7 100644 --- a/docs/github-action.md +++ b/docs/github-action.md @@ -2,7 +2,7 @@ layout: page title: GitHub Action permalink: /github-action/ -nav_order: 6 +nav_order: 7 --- # GitHub Action diff --git a/docs/index.md b/docs/index.md index 5be847751f2..c6be03d8bb6 100644 --- a/docs/index.md +++ b/docs/index.md @@ -22,4 +22,4 @@ OSV-Scanner provides an officially supported frontend to the [OSV database](http The above all results in fewer, more actionable vulnerability notifications, which reduces the time needed to resolve them. Check out our [announcement blog post] for more details! -[announcement blog post]: https://security.googleblog.com/2022/12/announcing-osv-scanner-vulnerability.html +[announcement blog post]: https://security.googleblog.com/2022/12/announcing-osv-scanner-vulnerability.html \ No newline at end of file diff --git a/docs/installation.md b/docs/installation.md index e7489b15f05..2f3775264c3 100644 --- a/docs/installation.md +++ b/docs/installation.md @@ -2,7 +2,7 @@ layout: page title: Installation permalink: /installation/ -nav_order: 2 +nav_order: 3 --- # Installation diff --git a/docs/output.md b/docs/output.md index 9c569555002..d1c8deb2693 100644 --- a/docs/output.md +++ b/docs/output.md @@ -2,7 +2,7 @@ layout: page title: Output permalink: /output/ -nav_order: 5 +nav_order: 6 --- # Output {: .no_toc } diff --git a/docs/supported_languages_and_lockfiles.md b/docs/supported_languages_and_lockfiles.md new file mode 100644 index 00000000000..cdc67346ab3 --- /dev/null +++ b/docs/supported_languages_and_lockfiles.md @@ -0,0 +1,105 @@ +--- +layout: page +title: Supported Languages and Lockfiles +permalink: /supported-languages-and-lockfiles/ +nav_order: 2 +--- + +# Supported Languages and Lockfiles + +{: .no_toc } + +
+ + Table of contents + + {: .text-delta } +- TOC +{:toc} +
+ +## Supported lockfiles + +A wide range of lockfiles are supported by utilizing this [lockfile package](https://github.com/google/osv-scanner/tree/main/pkg/lockfile). + +| Language | Compatible Lockfile(s) | +|:-----------|:------------------| +| C/C++ | `conan.lock`
[C/C++ commit scanning](#cc-scanning)| +| Dart | `pubspec.lock`| +| Elixir | `mix.lock` | +| Go | `go.mod` | +| Java | `buildscript-gradle.lockfile`
`gradle.lockfile`
`pom.xml`[\*](https://github.com/google/osv-scanner/issues/35) | +| Javascript | `package-lock.json`
`pnpm-lock.yaml`
`yarn.lock`| +| PHP | `composer.lock`| +| Python | `Pipfile.lock`
`poetry.lock`
`requirements.txt`[\*](https://github.com/google/osv-scanner/issues/34) | +| Ruby | `Gemfile.lock`| +| Rust | `Cargo.lock`| + +## Alpine Package Keeper and Debian Package Keeper + +The scanner also supports: +- `installed` files used by the Alpine Package Keeper (apk) that typically live at `/lib/apk/db/installed` +- `status` files used by the Debian Package manager (dpkg) that typically live at `/var/lib/dpkg/status` + +however you must [specify](./usage.md/#specify-lockfiles) them explicitly using the `--lockfile` flag: + +```bash +osv-scanner --lockfile 'apk-installed:/lib/apk/db/installed' +osv-scanner --lockfile 'dpkg-status:/var/lib/dpkg/status' +``` + +## C/C++ scanning + +With the addition of [vulnerable commit ranges](https://osv.dev/blog/posts/introducing-broad-c-c++-support/) to the OSV.dev database, OSV-Scanner now supports vendored and submoduled C/C++ dependencies + +Because the C/C++ ecosystem does not have a centralized package manager, C/C++ dependencies tend to be bundled with the project. Dependencies are either [submoduled](#submoduled-dependencies) or [vendored](#vendored-dependencies). In either case, OSV-Scanner is able to find known vulnerabilities in your project dependencies. + +OSV-Scanner's C/C++ support is based on commit-level data. OSV's commit-level data covers the majority of C/C++ vulnerabilities within the OSV database, but users should be aware that there may be vulnerabilities in their dependencies that could be excluded from OSV-Scanner results. Adding more commit-level data to the database is an ongoing project. + +### Submoduled dependencies + +Submoduled dependencies are included in the project folder retain their Git histories. To scan a C/C++ project with submoduled dependencies: + +1. Navigate to the root folder of your project. +2. Ensure that your submodules are up to date using `git submodule update`. +3. Run scanner using `osv-scanner -r .`. + +### Vendored dependencies + +Vendored dependencies have been directly copied into the project folder, but do not retain their Git histories. OSV-Scanner uses OSV's [determineversion API](https://google.github.io/osv.dev/post-v1-determineversion/) to estimate each dependency's version (and associated Git Commit). Vulnerabilities for the estimated version are returned. This process requires no additional work from the user. Run OSV-Scanner as you normally would. + +## Custom Lockfiles + +If you have a custom lockfile that we do not support or prefer to do your own custom parsing, you can extract the custom lockfile information and create a custom intermediate file containing dependency information so that osv-scanner can still check for vulnerabilities. + +Once you extracted your own dependency information, place it in a `osv-scanner.json` file, with the same format as the JSON output of osv-scanner, e.g.: + +``` +{ + "results": [ + { + "packages": [ + { + "package": { + "name": "github.com/repo/url", + "commit": "9a6bd55c9d0722cb101fe85a3b22d89e4ff4fe52" + } + }, + { + "package": { + "name": "react", + "version": "1.2.3", + "ecosystem": "npm" + } + }, + // ... + ] + } + ] +} +``` + +Then pass this to `osv-scanner` with this: +``` +osv-scanner --lockfile osv-scanner:/path/to/osv-scanner.json +``` \ No newline at end of file diff --git a/docs/usage.md b/docs/usage.md index e907cc2a1ef..db8619e0764 100644 --- a/docs/usage.md +++ b/docs/usage.md @@ -2,7 +2,7 @@ layout: page title: Usage permalink: /usage/ -nav_order: 3 +nav_order: 4 --- # Usage @@ -69,36 +69,7 @@ It is possible to specify more than one lockfile at a time; you can also specify osv-scanner --lockfile 'requirements.txt:/path/to/your/extra-requirements.txt' ``` -A wide range of lockfiles are supported by utilizing this [lockfile package](https://github.com/google/osv-scanner/tree/main/pkg/lockfile). This is the current list of supported lockfiles: - -- `buildscript-gradle.lockfile` -- `Cargo.lock` -- `composer.lock` -- `conan.lock` -- `Gemfile.lock` -- `go.mod` -- `gradle.lockfile` -- `mix.lock` -- `package-lock.json` -- `packages.lock.json` -- `Pipfile.lock` -- `pnpm-lock.yaml` -- `poetry.lock` -- `pom.xml`[\*](https://github.com/google/osv-scanner/issues/35) -- `pubspec.lock` -- `requirements.txt`[\*](https://github.com/google/osv-scanner/issues/34) -- `yarn.lock` - -The scanner also supports: -- `installed` files used by the Alpine Package Keeper (apk) that typically live at `/lib/apk/db/installed` -- `status` files used by the Debian Package manager (dpkg) that typically live at `/var/lib/dpkg/status` - -however you must specify them explicitly using the `--lockfile` flag: - -```bash -osv-scanner --lockfile 'apk-installed:/lib/apk/db/installed' -osv-scanner --lockfile 'dpkg-status:/var/lib/dpkg/status' -``` +The list of supported lockfile formats can be found [here](/osv-scanner/supported-languages-and-lockfiles/). If the file you are scanning is located in a directory that has a colon in its name, you can prefix the path to just a colon to explicitly signal to the scanner that @@ -108,42 +79,6 @@ it should infer the parser based on the filename: osv-scanner --lockfile ':/path/to/my:projects/package-lock.json' ``` -### Custom Lockfiles - -If you have a custom lockfile that we do not support or prefer to do your own custom parsing, you can extract the custom lockfile information and create a custom intermediate file containing dependency information so that osv-scanner can still check for vulnerabilities. - -Once you extracted your own dependency information, place it in a `osv-scanner.json` file, with the same format as the JSON output of osv-scanner, e.g.: - -``` -{ - "results": [ - { - "packages": [ - { - "package": { - "name": "github.com/repo/url", - "commit": "9a6bd55c9d0722cb101fe85a3b22d89e4ff4fe52" - } - }, - { - "package": { - "name": "react", - "version": "1.2.3", - "ecosystem": "npm" - } - }, - // ... - ] - } - ] -} -``` - -Then pass this to `osv-scanner` with this: -``` -osv-scanner --lockfile osv-scanner:/path/to/osv-scanner.json -``` - ## Scanning a Debian based docker image packages Preview {: .label } @@ -190,3 +125,23 @@ The `--output` flag can be used to save the scan results to a file instead of be ```bash osv-scanner -L package-lock.json --output scan-results.txt ``` + +## C/C++ scanning + +OSV-Scanner supports C/C++ projects. + +Because the C/C++ ecosystem does not have a centralized package manager, C/C++ dependencies tend to be bundled with the project's source code. Dependencies are either [submoduled](#submoduled-dependencies) or [vendored](#vendored-dependencies). In either case, OSV-Scanner is able to find known vulnerabilities in your project dependencies. + +OSV-Scanner's C/C++ support is based on commit-level data. OSV's commit-level data covers the majority of C/C++ vulnerabilities within the OSV database, but users should be aware that there may be vulnerabilities in their dependencies that may not be in the OSV database and therefore not included in OSV-Scanner results. Adding more commit-level data to the database is an ongoing project, follow [#783](https://github.com/google/osv.dev/issues/783) for more details. + +### Submoduled dependencies + +[Submoduled](https://git-scm.com/book/en/v2/Git-Tools-Submodules) dependencies are included in the project's source code and retain their Git histories. To scan a C/C++ project with submoduled dependencies: + +1. Navigate to the root folder of your project. +2. Ensure that your submodules are up to date using `git submodule update`. +3. Run scanner using `osv-scanner -r .`. + +### Vendored dependencies + +Vendored dependencies have been directly copied into the project folder, but do not retain their Git histories. OSV-Scanner uses OSV's [determineversion API](https://google.github.io/osv.dev/post-v1-determineversion/) to estimate each dependency's version (and associated Git commit). Vulnerabilities for the estimated version are returned. This process requires no additional work from the user. Run OSV-Scanner as you normally would.