chore(deps): update go to 1.22 #2760

baizon · 2024-06-10T09:09:45Z

As requested bumped go 1.22.0.

Notes:

I had to use 1.22.0, because of: cmd/go: download go1.22 for darwin/arm64: toolchain not available golang/go#65568, also see: https://go.dev/doc/toolchain#version
scripts/docker_install.sh changed tohttps://dl.google.com/go/go1.22.4.linux-amd64.tar.gz (the latest from https://go.dev/dl/)
AppVeyor Go 1.22 uses a differen PATH Go 1.22.1 x64 (C:\go - default in PATH), see: https://www.appveyor.com/docs/windows-images-software/

benma

Thanks ❤️ Haven't finished checking all yet, just left a quick comment for now.

vendor/github.com/ethereum/go-ethereum/.travis.yml

benma · 2024-06-10T13:05:13Z

TIL that Go simply downloads a new toolchain (1.22) if not available. I don't really like that, the Go dep, like the vendored deps, should be fixed and local. Seem like GOTOOLCHAIN=local would avoid this. We should add this to the CI and to the make ... targets for all platforms. Do you want to try or should I open a separate PR for that?

The CI here will then fail, but I'll upload a new image with the updated Go version as discussed.

baizon · 2024-06-10T13:16:42Z

Yeah, I'll integrate it, you can then review it.

benma · 2024-06-10T13:31:02Z

Thanks.

I had to use 1.22.0, [...]

This will also not be needed then anymore. The AppVeyor comment would be nice to have in the commit message body.

baizon · 2024-06-10T15:16:43Z

OK, all error msg are now go: ..\..\..\go.mod requires go >= 1.22 (running go 1.21.8; GOTOOLCHAIN=local). Now you need to update the Image.

benma

Please split the GOTOOLCHAIN=local changes into a separate commit (or even PR), as it is independent of the Go 1.22 change.

appveyor.yml

benma · 2024-06-11T08:17:08Z

Makefile

@@ -33,7 +33,7 @@ envinit:
 # Initializiation on MacOS


It seems we have a problem.

Above on line 31 there is:

# The gomobile/gobind libs are also needed when using `gomobile bind`, but it's not compatible with vendoring: # https://github.com/golang/go/issues/50994#issuecomment-1032754206 GO111MODULE=off go get -u golang.org/x/mobile/cmd/gomobile

This fails in Go 1.22 because go get is now disabled outside of modules. I locally built an image simply removing this line, but then cd backend/mobileserver; make fails with

unable to import bind: no Go package in golang.org/x/mobile/bind "golang.org/x/mobile/bind" is not found; run go get golang.org/x/mobile/bind: no Go package in golang.org/x/mobile/bind

Running go get golang.org/x/mobile/bind did not actually fix it for me.

Any idea how to resolve this?

I also tried adding a dummy import _ "golang.org/x/mobile/bind" to force it to be vendored, but go mod vendor only vendors files in Go folders, but bind has files like these that are then missing:

gomobile: /opt/go/bin/gobind -lang=go,java -outdir=/tmp/gomobile-work-2993124781 github.com/BitBoxSwiss/bitbox-wallet-app/backend/mobileserver failed: exit status 1 failed to open Java support file: open /opt/go/src/github.com/BitBoxSwiss/bitbox-wallet-app/vendor/golang.org/x/mobile/bind/java/Seq.java: no such file or directory

EDIT: according to golang/go#43736 (comment) this should have been fixed 🤔

EDIT2: With the _ "golang.org/x/mobile/cmd/gobind" import path the Java files are vendored, but the generated code is not modules aware it seems:

PWD=$WORK/src-android-386 GOMODCACHE=$GOPATH/pkg/mod GOOS=android GOARCH=386 CC=$ANDROID_HOME/ndk/21.2.6472646/toolchains/llvm/prebuilt/linux-x86_64/bin/i686-linux-android16-clang CXX=$ANDROID_HOME/ndk/21.2.6472646/toolchains/llvm/prebuilt/linux-x86_64/bin/i686-linux-android16-clang++ CGO_ENABLED=1 GOPATH=$WORK:$GOPATH go mod tidy rm -r -f "$WORK" gomobile: go mod tidy failed: exit status 1 go: error reading go.mod: missing module declaration. To specify the module path: go mod edit -module=example.com/mod

I filed an issue here: golang/go#67927

A workaround that seems to work is to mv vendor vendor.disabled before make android and move it back after 🙀

baizon · 2024-07-20T14:11:37Z

Could You provide a shiftcrypto/bitbox-wallet-app:24 Docker Image with go 1.22.5. I'll take a deeper look into this issue. Thanks in advance.

benma · 2024-07-20T16:39:09Z

Could You provide a shiftcrypto/bitbox-wallet-app:24 Docker Image with go 1.22.5. I'll take a deeper look into this issue. Thanks in advance.

Thanks!

I wouldn't push an image until we know if/how it will work, as we may need to push other image updates until then. You can simply build it locally though using make dockerinit, would that work? If not, I can push an image anyway.

baizon · 2024-07-21T08:46:09Z

I've solved golang/go#67927
Then tried to replace the gomobile with my fix (https://github.com/baizon/bitbox-wallet-app/blob/040a1781ae3fc7042dffde62542b764139bce106/go.mod#L31), but no success for now. Any recommendations?
Else we have to wait, until my PR is merged.

benma · 2024-07-25T09:48:28Z

This is amazing work, thanks @baizon.

What exactly didn't work? gomobile is also installed here:

bitbox-wallet-app/Makefile

Lines 28 to 32 in c221fca

    
           go install golang.org/x/mobile/cmd/gomobile@latest 
        
           # The gomobile/gobind libs are also needed when using `gomobile bind`, but it's not compatible with vendoring: 
        
           # https://github.com/golang/go/issues/50994#issuecomment-1032754206 
        
           GO111MODULE=off go get -u golang.org/x/mobile/cmd/gomobile 
        
           gomobile init

So maybe you'd need to change that path instead?

You may also need to do this to vendor the related packages? golang/go#67927 (comment)

baizon · 2024-07-26T18:08:11Z

Sorry, wasn't very specific. My steps:

Set replace within the go.mod, like this:
replace golang.org/x/mobile => github.com/baizon/mobile v0.0.0-20240721071641-1d3b1e5c533f
Changed within the Makefile you mentioned
go install github.com/baizon/mobile/cmd/gomobile@latest
Run make dockerinit, the result is:

go install github.com/baizon/mobile/cmd/gomobile@latest
go: downloading github.com/baizon/mobile v0.0.0-20240721071641-1d3b1e5c533f
go: github.com/baizon/mobile/cmd/gomobile@latest: version constraints conflict:
        github.com/baizon/[email protected]: parsing go.mod:
        module declares its path as: golang.org/x/mobile
                but was required as: github.com/baizon/mobile
make: Leaving directory '/tmp'
make: *** [Makefile:28: envinit] Error 1

benma · 2024-07-27T09:25:53Z

@baizon try also replacing the module name in your fork at https://github.com/baizon/mobile/blob/1d3b1e5c533f3e2b92f8cdde1921d96b618eb361/go.mod#L1

baizon · 2024-07-27T18:37:18Z

That was a nice challenge. Thanks for the support.

BUILD SUCCESSFUL in 3s
75 actionable tasks: 10 executed, 65 up-to-date
make[1]: Leaving directory '/opt/go/src/github.com/BitBoxSwiss/bitbox-wallet-app/frontends/android'

Next the 23 (go1.22) docker image?

benma

@baizon very nice, a local test building the android app seemed to work.

What's the GOMODCACHE env var needed for?

If you don't mind, later I'll fork your fork to the BitBoxSwiss org, so we can use that instead.

benma · 2024-07-27T19:25:41Z

android-env.mk.inc

@@ -1,2 +1,3 @@
 export ANDROID_SDK_ROOT := /opt/android-sdk
 export ANDROID_NDK_HOME := /opt/android-sdk/ndk/21.2.6472646
+export GOMODCACHE_ROOT := /tmp/gomodcache/pkg/mod


Why this? Is it required? If so, please add a comment why.

Added a comment.

backend/mobileserver/Makefile

backend/mobileserver/implicit.go

baizon · 2024-07-27T19:49:49Z

export GOMODCACHE_ROOT := /tmp/gomodcache/pkg/mod is set, because gomobile runs a go mod tidy and tries to create cache within /opt/go/pkg/mod/cache/download/. This fails, because there is no write access.
https://github.com/golang/mobile/blob/master/cmd/gomobile/build.go#L336

go: writing go.mod cache: mkdir /opt/go/pkg/mod/cache/download/github.com/!bit!box!swiss: permission denied
go: writing go.mod cache: mkdir /opt/go/pkg/mod/cache/download/github.com/!azure: permission denied
go: writing go.mod cache: mkdir /opt/go/pkg/mod/cache/download/github.com/!bit!box!swiss: permission denied
...

I've picked the /tmp folder, but it can be changed, or a different solution can be implemented.

Makefile

benma · 2024-07-28T21:26:26Z

@baizon I pushed the :24 image.

I noticed that while the android build works, it does not seem to be using the vendor folder - I can change deps in it or remove the whole folder, and the android build still works.

Granted, before we also didn't (couldn't) use the vendor lib, but since you went into the gomobile internals, maybe you have a clue why or how to fix it? Ideally it behaves like the normal build with -mod=vendor.

baizon · 2024-08-02T20:20:07Z

Image :24 is reporting an error:

+ docker pull ghcr.io/bitboxswiss/bitbox-wallet-app-ci:24
Error response from daemon: manifest unknown

The reason why go list fails:

PWD=$WORK/src-android-arm GOMODCACHE=/tmp/gomodcache/pkg/mod GOOS=android GOARCH=arm CC=$ANDROID_HOME/ndk/21.2.6472646/toolchains/llvm/prebuilt/linux-x86_64/bin/armv7a-linux-androideabi16-clang CXX=$ANDROID_HOME/ndk/21.2.6472646/toolchains/llvm/prebuilt/linux-x86_64/bin/armv7a-linux-androideabi16-clang++ CGO_ENABLED=1 GOARM=7 GOPATH=$WORK:$GOPATH go mod tidy -v
go: error reading go.mod: missing module declaration. To specify the module path:
        go mod edit -module=example.com/mod
go: error reading go.mod: missing module declaration. To specify the module path:
        go mod edit -module=example.com/mod
go: error reading go.mod: missing module declaration. To specify the module path:
        go mod edit -module=example.com/mod
go: error reading go.mod: missing module declaration. To specify the module path:
        go mod edit -module=example.com/mod

A check is performed which isn't working with the vendor option. With -mod=readonly or -mod=mod it can be performed:

go: found github.com/BitBoxSwiss/bitbox-wallet-app/backend/mobileserver in github.com/BitBoxSwiss/bitbox-wallet-app v0.0.0-00010101000000-000000000000
go: found github.com/BitBoxSwiss/bitbox-wallet-app/backend/mobileserver in github.com/BitBoxSwiss/bitbox-wallet-app v0.0.0-00010101000000-000000000000
go: found github.com/BitBoxSwiss/bitbox-wallet-app/backend/mobileserver in github.com/BitBoxSwiss/bitbox-wallet-app v0.0.0-00010101000000-000000000000
go: found github.com/BitBoxSwiss/bitbox-wallet-app/backend/mobileserver in github.com/BitBoxSwiss/bitbox-wallet-app v0.0.0-00010101000000-000000000000

go list seems to be the issue. Same problem and solution here: microsoft/component-detection#189

benma · 2024-08-03T08:26:23Z

@baizon

Image :24 is reporting an error:

I had a typo in the image name, should be fixed now.

The reason why go list fails:

Sorry I am confused, what does this relate to? The original issue to build the Android app with a vendor folder present works with -mod=mod with your patch, without errors, but my question was about why the Android Go build does not use the vendor folder at all now. It's possible however I am misreading your message 😇

Edit: now only a bunch of 1.22 lint issues remain about the new semantics for loop vars. Maybe we could disable the warning for now, or fix it in a separate commit.

benma

Somehow appveyor reports

go: ..\..\..\go.mod requires go >= 1.22 (running go 1.21.8; GOTOOLCHAIN=local)

Not sure why, the appveyor file seems ok.

appveyor.yml

baizon · 2024-08-03T09:45:56Z

So, while building the android app gomobile is running go list (see: https://github.com/golang/mobile/blob/master/cmd/gomobile/bind.go#L229) and go mod tidy. While running go list the issue the vendor issue occurs. See: https://github.com/golang/go/blob/master/src/cmd/go/internal/list/list.go#L545

Indeed, I'll add the lint exceptions.

Edit:

Go 1.22.1 x64 (C:\go - default in PATH)

I could try to remove GOROOT, because it seems to included in PATH.
Edit2:
No, that didn't work.

benma · 2024-08-03T10:59:51Z

So, while building the android app gomobile is running go list (see: https://github.com/golang/mobile/blob/master/cmd/gomobile/bind.go#L229) and go mod tidy. While running go list the issue the vendor issue occurs. See: https://github.com/golang/go/blob/master/src/cmd/go/internal/list/list.go#L545

This is the issue you fixed in your fork of gomobile, right?

But even with your fork, the vendor module does not seem to be used when building the android app. I checked by modifying a dep in the vendor module (added some syntax error) but the build finished fine anyway, suggesting that the vendor folder was ignored when building the android app. Any idea how one would fix that? As I mentioned before, it is not a high prio as even before 1.22, the vendor module was not used for the android app, but it would be nice to solve this too if you know how.

When building the Windows/macOS/linux apps, the vendor folder is used as expected.

benma

@baizon could you please rebase and squash? Once we figure out the appveyor problem we can merge. Maybe simply re-running it will do 🤷

baizon · 2024-08-14T05:30:04Z

Done.
To fix the mobile vendor issue needs a bigger fix. Rewriting the go list code. When I have more time I will look into it.

benma · 2024-08-26T07:35:49Z

@baizon I made a PR to your branch to fix the appveyor CI at baizon#3 - please apply the commit 😄

https://www.appveyor.com/docs/windows-images-software/#golang lists > Go 1.22.1 x64 (C:\go - default in PATH) But the image or docs are buggy, C:\go is a symlink of C:\go121 and Go 1.22 does not seem to exist. We switch to choco to install the desired Go version. The benefit is that choco receives new updates faster too and has more granular version control.

baizon · 2024-08-26T11:39:48Z

Sorry, much to do at work at the moment. Thank You for the fix. I did merge your PR and did a rebase.

benma

Thanks a lot for your patience and your effort @baizon!

benma requested changes Jun 10, 2024

View reviewed changes

vendor/github.com/ethereum/go-ethereum/.travis.yml Outdated Show resolved Hide resolved

baizon force-pushed the feature/go_122 branch from 39145fa to 4c212df Compare June 10, 2024 15:18

baizon changed the title ~~chore(deps): update go to 1.22.0~~ chore(deps): update go to 1.22 Jun 10, 2024

benma requested changes Jun 11, 2024

View reviewed changes

baizon force-pushed the feature/go_122 branch 3 times, most recently from bb4737b to 604bc9b Compare June 11, 2024 11:52

baizon force-pushed the feature/go_122 branch from 0f7fca0 to 6b52cf6 Compare July 20, 2024 14:27

baizon force-pushed the feature/go_122 branch from 6b52cf6 to 5bfa2f2 Compare July 21, 2024 08:25

baizon force-pushed the feature/go_122 branch from d4e8ae6 to 040a178 Compare July 21, 2024 11:48

baizon force-pushed the feature/go_122 branch from 040a178 to bbaed8e Compare July 27, 2024 08:38

baizon force-pushed the feature/go_122 branch from c5713cf to 06ace7a Compare July 27, 2024 18:29

baizon force-pushed the feature/go_122 branch from d39e098 to 48dba09 Compare July 27, 2024 18:46

benma reviewed Jul 27, 2024

View reviewed changes

baizon force-pushed the feature/go_122 branch from 199dc3e to 86800e1 Compare July 27, 2024 19:50

benma requested changes Jul 27, 2024

View reviewed changes

Makefile Outdated Show resolved Hide resolved

baizon force-pushed the feature/go_122 branch from 6516968 to 8fb0806 Compare July 28, 2024 08:02

baizon force-pushed the feature/go_122 branch from 76b3581 to bc9701f Compare August 2, 2024 22:40

benma reviewed Aug 3, 2024

View reviewed changes

appveyor.yml Show resolved Hide resolved

baizon force-pushed the feature/go_122 branch from 06f6490 to 80a83dc Compare August 3, 2024 08:43

baizon force-pushed the feature/go_122 branch 2 times, most recently from b728b6d to dffc31f Compare August 3, 2024 10:24

benma requested changes Aug 13, 2024

View reviewed changes

baizon force-pushed the feature/go_122 branch from dffc31f to ddd1d1c Compare August 14, 2024 05:28

baizon and others added 4 commits August 26, 2024 13:38

chore(deps): update go to 1.22, appveyor changed from go121 to go

82477f6

feat: added GOTOOLCHAIN

3ddb71d

fix: resolved gomobile issue

684e62e

baizon force-pushed the feature/go_122 branch from de87e3a to 505177a Compare August 26, 2024 11:38

fix: micromatch trivy fix

eac3a04

benma approved these changes Aug 26, 2024

View reviewed changes

benma merged commit bdf270c into BitBoxSwiss:master Aug 26, 2024
6 checks passed

baizon deleted the feature/go_122 branch October 18, 2024 17:50

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

chore(deps): update go to 1.22 #2760

chore(deps): update go to 1.22 #2760

baizon commented Jun 10, 2024 •

edited

Loading

benma left a comment

benma commented Jun 10, 2024

baizon commented Jun 10, 2024

benma commented Jun 10, 2024

baizon commented Jun 10, 2024 •

edited

Loading

benma left a comment

benma Jun 11, 2024

benma Jun 11, 2024 •

edited

Loading

benma Jun 11, 2024

baizon commented Jul 20, 2024 •

edited

Loading

benma commented Jul 20, 2024 •

edited

Loading

baizon commented Jul 21, 2024 •

edited

Loading

benma commented Jul 25, 2024

baizon commented Jul 26, 2024

benma commented Jul 27, 2024

baizon commented Jul 27, 2024 •

edited

Loading

benma left a comment

benma Jul 27, 2024

baizon Jul 27, 2024

baizon commented Jul 27, 2024 •

edited

Loading

benma commented Jul 28, 2024

baizon commented Aug 2, 2024 •

edited

Loading

benma commented Aug 3, 2024 •

edited

Loading

benma left a comment

baizon commented Aug 3, 2024 •

edited

Loading

benma commented Aug 3, 2024

benma left a comment

baizon commented Aug 14, 2024

benma commented Aug 26, 2024

baizon commented Aug 26, 2024

benma left a comment

chore(deps): update go to 1.22 #2760

chore(deps): update go to 1.22 #2760

Conversation

baizon commented Jun 10, 2024 • edited Loading

benma left a comment

Choose a reason for hiding this comment

benma commented Jun 10, 2024

baizon commented Jun 10, 2024

benma commented Jun 10, 2024

baizon commented Jun 10, 2024 • edited Loading

benma left a comment

Choose a reason for hiding this comment

benma Jun 11, 2024

Choose a reason for hiding this comment

benma Jun 11, 2024 • edited Loading

Choose a reason for hiding this comment

benma Jun 11, 2024

Choose a reason for hiding this comment

baizon commented Jul 20, 2024 • edited Loading

benma commented Jul 20, 2024 • edited Loading

baizon commented Jul 21, 2024 • edited Loading

benma commented Jul 25, 2024

baizon commented Jul 26, 2024

benma commented Jul 27, 2024

baizon commented Jul 27, 2024 • edited Loading

benma left a comment

Choose a reason for hiding this comment

benma Jul 27, 2024

Choose a reason for hiding this comment

baizon Jul 27, 2024

Choose a reason for hiding this comment

baizon commented Jul 27, 2024 • edited Loading

benma commented Jul 28, 2024

baizon commented Aug 2, 2024 • edited Loading

benma commented Aug 3, 2024 • edited Loading

benma left a comment

Choose a reason for hiding this comment

baizon commented Aug 3, 2024 • edited Loading

benma commented Aug 3, 2024

benma left a comment

Choose a reason for hiding this comment

baizon commented Aug 14, 2024

benma commented Aug 26, 2024

baizon commented Aug 26, 2024

benma left a comment

Choose a reason for hiding this comment

baizon commented Jun 10, 2024 •

edited

Loading

baizon commented Jun 10, 2024 •

edited

Loading

benma Jun 11, 2024 •

edited

Loading

baizon commented Jul 20, 2024 •

edited

Loading

benma commented Jul 20, 2024 •

edited

Loading

baizon commented Jul 21, 2024 •

edited

Loading

baizon commented Jul 27, 2024 •

edited

Loading

baizon commented Jul 27, 2024 •

edited

Loading

baizon commented Aug 2, 2024 •

edited

Loading

benma commented Aug 3, 2024 •

edited

Loading

baizon commented Aug 3, 2024 •

edited

Loading