Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Setup CI #10

Open
buffrr opened this issue Apr 27, 2022 · 5 comments
Open

Setup CI #10

buffrr opened this issue Apr 27, 2022 · 5 comments

Comments

@buffrr
Copy link
Member

buffrr commented Apr 27, 2022

We're gonna need some CI infra to do builds for Beacon this should make things a lot easier. Chromium uses LUCI it's a bit too googly i'm not sure if it's documented well enough to be usable for non-google projects

@mdaniel
Copy link
Contributor

mdaniel commented Apr 28, 2022

There's also luci-go but unknown if they're equivalent. Poking around in that infra tree surfaced what seems a lot more chromium-specific tool: https://chromium.googlesource.com/infra/goma/server/+/refs/tags/v0.0.25#goma-server This is the first I've heard of LUCI, but when I saw the architecture, I thought of this: https://blog.nelhage.com/post/building-llvm-in-90s/

I was actually wondering how one caches 33GB worth of chromium .git inside of GHA; I believe brave uses their own runners which are able to have the chromium repo mounted on EBS volumes or such. If you do go with some kind of lambda-esque approach, they now offer EFS mounts

❓ Have you considered contacting GitHub support to see if they have any suggestions?

Before I saw this ticket, I was actually going to propose creating a Vagrantfile, or a Dockerfile to at least allow contributors to be on the right versions of things

@buffrr
Copy link
Member Author

buffrr commented Apr 28, 2022

There's also luci-go but unknown if they're equivalent.

I saw that too maybe worth looking into. It appears that they used buildbot before but switched to luci.

I believe brave uses their own runners which are able to have the chromium repo mounted on EBS volumes or such. If you do go with some kind of lambda-esque approach, they now offer EFS mounts

Ah I see so brave uses jenkins for their CI. Maybe goma+gha combo could work that would make things simpler.

Poking around in that infra tree surfaced what seems a lot more chromium-specific tool: https://chromium.googlesource.com/infra/goma/server/+/refs/tags/v0.0.25#goma-server

So goma can be used to off load compilation into cloud servers that's pretty neat.

To set this up it seems that we need

  1. A service that implements a Remote Execution API apparently Brave uses EngFlow
  2. A goma server that connects to such service.
  3. and builds would use the goma client

I wonder if Github actions could use the goma client to off load builds so we can just setup a goma server/service to get something like CI with GHA hmm.

Have you considered contacting GitHub support to see if they have any suggestions?

I will send them an email maybe they know something.

@mdaniel
Copy link
Contributor

mdaniel commented Apr 29, 2022

I was curious how Arch did it, and it seems they don't use gclient et al, but rather curl https://commondatastorage.googleapis.com/chromium-browser-official/chromium-101.0.4951.41.tar.xz which is still hefty at 1.4G but more manageable (same for ubuntu)

Given that the GHA cache is limited to 10G, I was curious if there was a way of converting the tar to a "working" src that the build could use, but it seems to be missing at least llvm-build, possibly more

It did get pretty far, so it may be worth exploring how to sideload the llvm-build directory:

2022/04/29 02:54:07 [WARNING] environment variable `BEACON_CC_WRAPPER` is not set.
2022/04/29 02:54:07 [WARNING] environment variable `BEACON_GOOGLE_API_KEY` is not set.
2022/04/29 02:54:07 [WARNING] environment variable `BEACON_GOOGLE_DEFAULT_CLIENT_ID` is not set.
2022/04/29 02:54:07 [WARNING] environment variable `BEACON_GOOGLE_DEFAULT_CLIENT_SECRET` is not set.
Setting up the build
Done. Made 17269 targets from 3011 files in 13885ms
Building Beacon
ninja: Entering directory `out/Release'
[1/51787] COPY ../../beacon/components/welcome/beacon_welcome_proxy.ts gen/beacon/components/welcome/beacon_welcome_proxy.ts
[2/51787] CXX obj/buildtools/third_party/libc++/libc++/future.o
FAILED: obj/buildtools/third_party/libc++/libc++/future.o

where ☝️ is due to ccache: error: execv of ../../third_party/llvm-build/Release+Asserts/bin/clang++ failed: No such file or directory

@mdaniel
Copy link
Contributor

mdaniel commented Apr 30, 2022

I also just realized they have chromium GitHub mirror so if GitHub can work with you in some way, they don't need to pay for that clone traffic from GCP; that is, conceptually you just want a pipeline that does - uses: actions/checkout twice, once for chromium and once for Beacon. I can only presume gclient is bright enough to use the different origin, otherwise what'd be the point of having a GH mirror?

Related to that, I would be super curious if the Microsoft/git fork would work in CI. They allege it is for monorepos, which Beacon kind of is given its sibling dependency upon chromium, and that if a build doesn't touch every file the sparse checkout could be a huge win, too

@buffrr
Copy link
Member Author

buffrr commented May 11, 2022

Just an update: I had a call with EngFlow and their current offering is more suitable for larger teams. They seem to be flexible with their plans so this won't be completely ruled out for now. Buildbuddy said their remote execution API might work with goma so I will check that out. Even if we have a CI just for linux that would be good enough as a start.

I was curious how Arch did it, and it seems they don't use gclient et al, but rather curl .... where point_up is due to ccache: error: execv of ../../third_party/llvm-build/Release+Asserts/bin/clang++ failed: No such file or directory

I see my understanding is that gclient also pulls those dependencies so perhaps that was the issue. I found a PR that builds Ungoogled chromium using Github actions on macOS which is interesting ungoogled-software/ungoogled-chromium-macos#54 seems to take 11 to 13 hours :-D

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants