Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: Update v2 to main #1374

Merged
merged 22 commits into from
Nov 5, 2024
Merged

Conversation

another-rex
Copy link
Collaborator

Using a separate branch to resolve the merge conflicts, but other than that this is a straightforward update of the v2 branch up to the latest main branch.

cuixq and others added 20 commits October 24, 2024 13:36
google#1286 adds support for Maven
registry during resolution.

As a follow up, this PR updates the documentation for transitive
scanning about specifying data source during resolution as well as
specifying Maven registry.

This PR also corrects the deps.dev API version we are using.

We also need to update the documentation in
google#1181.
…l-no-resolve` flags (google#1342)

Closes google#1339 and closes google#1121 
Adds flags to use offline mode for vulnerabilities
(`--experimental-offline-vulnerabilities`) and transitive resolution
separately (`--experimental-no-resolve`)

The original `--experimental-offline` flag retains the same behaviour by
functionally setting both of these flags.
Currently we can't reliably merge into v2 because checks don't
automatically trigger
Another day, another snapshot change 😄
I'm not sure exactly where this file came from but we're not using it so
it can go
…er group (google#1349)

Bumps the bundler group in /docs with 1 update:
[rexml](https://github.com/ruby/rexml).

Updates `rexml` from 3.3.8 to 3.3.9
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/ruby/rexml/releases">rexml's
releases</a>.</em></p>
<blockquote>
<h2>REXML 3.3.9 - 2024-10-24</h2>
<h3>Improvements</h3>
<ul>
<li>Improved performance.
<ul>
<li><a
href="https://redirect.github.com/ruby/rexml/issues/210">GH-210</a></li>
<li>Patch by NAITOH Jun.</li>
</ul>
</li>
</ul>
<h3>Fixes</h3>
<ul>
<li>
<p>Fixed a parse bug for text only invalid XML.</p>
<ul>
<li><a
href="https://redirect.github.com/ruby/rexml/issues/215">GH-215</a></li>
<li>Patch by NAITOH Jun.</li>
</ul>
</li>
<li>
<p>Fixed a parse bug that <code>&amp;#0x...;</code> is accepted as a
character
reference.</p>
</li>
</ul>
<h3>Thanks</h3>
<ul>
<li>NAITOH Jun</li>
</ul>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/ruby/rexml/blob/master/NEWS.md">rexml's
changelog</a>.</em></p>
<blockquote>
<h2>3.3.9 - 2024-10-24 {#version-3-3-9}</h2>
<h3>Improvements</h3>
<ul>
<li>Improved performance.
<ul>
<li><a
href="https://redirect.github.com/ruby/rexml/issues/210">GH-210</a></li>
<li>Patch by NAITOH Jun.</li>
</ul>
</li>
</ul>
<h3>Fixes</h3>
<ul>
<li>
<p>Fixed a parse bug for text only invalid XML.</p>
<ul>
<li><a
href="https://redirect.github.com/ruby/rexml/issues/215">GH-215</a></li>
<li>Patch by NAITOH Jun.</li>
</ul>
</li>
<li>
<p>Fixed a parse bug that <code>&amp;#0x...;</code> is accepted as a
character
reference.</p>
</li>
</ul>
<h3>Thanks</h3>
<ul>
<li>NAITOH Jun</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/ruby/rexml/commit/38eaa86ac7abe0d31cf49d8df57ad239fdeb80e9"><code>38eaa86</code></a>
Add 3.3.9 entry</li>
<li><a
href="https://github.com/ruby/rexml/commit/ce59f2eb1aeb371fe1643414f06618dbe031979f"><code>ce59f2e</code></a>
parser: fix a bug that &amp;#0x...; is accepted as a character
reference</li>
<li><a
href="https://github.com/ruby/rexml/commit/a09646d395a07399cbf9bc3bc8d6d8bb1d13ecea"><code>a09646d</code></a>
test: fix indent</li>
<li><a
href="https://github.com/ruby/rexml/commit/cf0fb9c9ca3dc0d725c8e4644aa0e728025f42ce"><code>cf0fb9c</code></a>
Fix <code>IOSource#readline</code> for <code>@pending_buffer</code> (<a
href="https://redirect.github.com/ruby/rexml/issues/215">#215</a>)</li>
<li><a
href="https://github.com/ruby/rexml/commit/1d0c362526f6e25e2abcd13e2fcefcc718c20e78"><code>1d0c362</code></a>
Optimize <code>IOSource#read_until</code> method (<a
href="https://redirect.github.com/ruby/rexml/issues/210">#210</a>)</li>
<li><a
href="https://github.com/ruby/rexml/commit/622011f25ac1519fd553d6c56da52d7eba14a787"><code>622011f</code></a>
Bump version</li>
<li>See full diff in <a
href="https://github.com/ruby/rexml/compare/v3.3.8...v3.3.9">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=rexml&package-manager=bundler&previous-version=3.3.8&new-version=3.3.9)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore <dependency name> major version` will close this
group update PR and stop Dependabot creating any more for the specific
dependency's major version (unless you unignore this specific
dependency's major version or upgrade to it yourself)
- `@dependabot ignore <dependency name> minor version` will close this
group update PR and stop Dependabot creating any more for the specific
dependency's minor version (unless you unignore this specific
dependency's minor version or upgrade to it yourself)
- `@dependabot ignore <dependency name>` will close this group update PR
and stop Dependabot creating any more for the specific dependency
(unless you unignore this specific dependency or upgrade to it yourself)
- `@dependabot unignore <dependency name>` will remove all of the ignore
conditions of the specified dependency
- `@dependabot unignore <dependency name> <ignore condition>` will
remove the ignore condition of the specified dependency and ignore
conditions
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/google/osv-scanner/network/alerts).

</details>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Xueqin Cui <[email protected]>
# PR Summary
Small PR - adjusts the sources to use the correct `usage.md` file.

Signed-off-by: Emmanuel Ferdman <[email protected]>
Co-authored-by: Rex P <[email protected]>
Scalibr is crashing with a SIGSEGV while trying to parse this directory:


https://github.com/semgrep/semgrep/tree/develop/cli/tests/default/e2e/targets/dependency_aware

This is due to the lack of an array length check after the
dependencyPath split.

Added failing lockfile as a test.

## Steps to reproduce it

```
$ git clone https://github.com/semgrep/semgrep
$ osv-scanner semgrep/cli/tests/default/e2e/targets/dependency_aware/pnpm-error-key/pnpm-lock.yaml

panic: runtime error: index out of range [0] with length 0

goroutine 1 [running]:
github.com/google/osv-scanner/pkg/lockfile.extractPnpmPackageNameAndVersion({0xc000328fd0?, 0xb?}, 0x401599999999999a?)
	/tmp/osv-scanner/pkg/lockfile/parse-pnpm-lock.go:95 +0x34f
github.com/google/osv-scanner/pkg/lockfile.parsePnpmLock({0xc000814230?, 0xc0004a6fc0?})
	/tmp/osv-scanner/pkg/lockfile/parse-pnpm-lock.go:141 +0x145
github.com/google/osv-scanner/pkg/lockfile.PnpmLockExtractor.Extract({}, {0x7f545125f268, 0xc00015de40})
	/tmp/osv-scanner/pkg/lockfile/parse-pnpm-lock.go:208 +0x25b
github.com/google/osv-scanner/pkg/lockfile.ExtractDeps({0x7f545125f268, 0xc00015de40}, {0x0, 0x0})
	/tmp/osv-scanner/pkg/lockfile/extract.go:61 +0x24a
github.com/google/osv-scanner/pkg/osvscanner.scanLockfile({0x19374a8, 0xc000359e00}, {0xc00037e540, 0x69}, {0x0, 0x0}, 0x0)
	/tmp/osv-scanner/pkg/osvscanner/osvscanner.go:371 +0x8aa
github.com/google/osv-scanner/pkg/osvscanner.scanDir.func1({0x7ffd822dfd35?, 0x100?}, {0x1932558, 0xc0004a4b90}, {0x0?, 0x0?})
	/tmp/osv-scanner/pkg/osvscanner/osvscanner.go:168 +0x745
path/filepath.walkDir({0x7ffd822dfd35, 0x54}, {0x1932558, 0xc0004a4b90}, 0xc000814970)
	/usr/lib/golang/src/path/filepath/path.go:443 +0x50
path/filepath.WalkDir({0x7ffd822dfd35, 0x54}, 0xc000814970)
	/usr/lib/golang/src/path/filepath/path.go:533 +0x7b
github.com/google/osv-scanner/pkg/osvscanner.scanDir({0x19374a8?, 0xc000359e00?}, {0x7ffd822dfd35?, 0x54?}, 0x0?, 0x0?, 0x1?, 0x0?)
	/tmp/osv-scanner/pkg/osvscanner/osvscanner.go:126 +0x212
github.com/google/osv-scanner/pkg/osvscanner.DoScan({{0x0, 0x0, 0x0}, {0x0, 0x0, 0x0}, {0xc0004a4a80, 0x1, 0x1}, {0x0, ...}, ...}, ...)
	/tmp/osv-scanner/pkg/osvscanner/osvscanner.go:906 +0xf73
github.com/google/osv-scanner/cmd/osv-scanner/scan.action(0xc000359500, {0x19255e0, 0xc0000f6028}, {0x19255e0, 0xc0000f6030})
	/tmp/osv-scanner/cmd/osv-scanner/scan/main.go:208 +0xca8
github.com/google/osv-scanner/cmd/osv-scanner/scan.Command.func2(0xc000815ab8?)
	/tmp/osv-scanner/cmd/osv-scanner/scan/main.go:145 +0x2c
github.com/urfave/cli/v2.(*Command).Run(0xc0005d2160, 0xc000359500, {0xc000688080, 0x2, 0x2})
	GODIR/pkg/mod/github.com/urfave/cli/[email protected]/command.go:276 +0x97d
github.com/urfave/cli/v2.(*Command).Run(0xc0005d2840, 0xc0003593c0, {0xc0004a6060, 0x3, 0x3})
	GODIR/pkg/mod/github.com/urfave/cli/[email protected]/command.go:269 +0xbb7
github.com/urfave/cli/v2.(*App).RunContext(0xc00024c600, {0x1932328, 0x2397500}, {0xc0004a6060, 0x3, 0x3})
	GODIR/pkg/mod/github.com/urfave/cli/[email protected]/app.go:333 +0x5a5
github.com/urfave/cli/v2.(*App).Run(...)
	GODIR/pkg/mod/github.com/urfave/cli/[email protected]/app.go:307
main.run({0xc0000400a0, 0x2, 0x2}, {0x19255e0, 0xc0000f6028}, {0x19255e0, 0xc0000f6030})
	/tmp/osv-scanner/cmd/osv-scanner/main.go:52 +0x678
main.main()
	/tmp/osv-scanner/cmd/osv-scanner/main.go:126 +0x45
```

---------

Co-authored-by: Gareth Jones <[email protected]>
Co-authored-by: Xueqin Cui <[email protected]>
Having ecosystems sorted by their name makes it easier to review this
section of code
…oogle#1365)

While I'm pretty these are technically invalid, they're easy to support
and without this Alpine is the only comparator that panics when parsing
an empty string which I think is a little sad.
…ions (google#1360)

While I'm a little reluctant to do this as I'm sure I included these for
a reason, they're apparently not covered by any tests and with
`semantic` expected to go public soon, I think it's a bit nicer to have
them gone until someone can prove they're needed as some of these could
arguably be a breaking change to remove.

I'm pretty sure the bulk of this was present as part of having
`semantic` rebuild the parsed version for debugging when I was writing
the implementation, but that's not actually a feature so we don't
explicitly need to be doing it - this won't stop us from reintroducing
the logic in future if we decide we want it
…oogle#1361)

I realised that I have overly complicated the act of returning the
result of comparing components in this function, and that the last two
conditions will never be true as we loop over the largest number of
components 😅
…tor (google#1362)

Currently this return is not covered because we're explicitly doing all
three possible comparisons, so we might as well move the equality check
to the end as the default
…ons (google#1366)

These letters naturally compare in the right order, so we can just
compare them directly which saves us a loop and an untestable `panic`.

(this will conflict with google#1362)
There's no reason for us to retain these in the internal packages, so
they can just go

Co-authored-by: Rex P <[email protected]>
Since this is an internal package it's not a breaking change to rename
this

Co-authored-by: Rex P <[email protected]>
…e#1364)

Notably this adds coverage over the Ubuntu entry, since that reuses
Debian so we don't have any explicit tests for that in the comparison
tests.

We also no longer need to be manually adding `CRAN` since `lockfile`
supports it
@another-rex another-rex requested a review from hogo6002 November 4, 2024 04:28
@another-rex another-rex changed the title Update v2 to main chore: Update v2 to main Nov 4, 2024
@another-rex another-rex enabled auto-merge (squash) November 5, 2024 02:23
@another-rex another-rex merged commit c9a0635 into google:v2 Nov 5, 2024
13 checks passed
@codecov-commenter
Copy link

codecov-commenter commented Nov 5, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 68.75%. Comparing base (b15b566) to head (0cc32c7).
Report is 1 commits behind head on v2.

Additional details and impacted files
@@            Coverage Diff             @@
##               v2    #1374      +/-   ##
==========================================
+ Coverage   68.66%   68.75%   +0.09%     
==========================================
  Files         188      188              
  Lines       18075    18029      -46     
==========================================
- Hits        12411    12396      -15     
+ Misses       4979     4954      -25     
+ Partials      685      679       -6     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

another-rex added a commit that referenced this pull request Nov 29, 2024
Using a separate branch to resolve the merge conflicts, but other than
that this is a straightforward update of the v2 branch up to the latest
main branch.

---------

Signed-off-by: dependabot[bot] <[email protected]>
Signed-off-by: Emmanuel Ferdman <[email protected]>
Co-authored-by: Xueqin Cui <[email protected]>
Co-authored-by: Michael Kedar <[email protected]>
Co-authored-by: Gareth Jones <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Emmanuel Ferdman <[email protected]>
Co-authored-by: Ignacio Vazquez <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants