Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[infra] Update workflow with native_ packages #963

Merged
merged 4 commits into from
Feb 13, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 3 additions & 12 deletions .github/workflows/native.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,6 @@ jobs:
os: [ubuntu, macos, windows]
sdk: [stable, dev]
package: [native_assets_builder, native_assets_cli, native_toolchain_c]
dependencies: [published, path]
# Breaking changes temporarily break the example run on the Dart SDK until native_assets_builder is rolled into the Dart SDK dev build.
breaking-change: [false]
exclude:
Expand All @@ -42,9 +41,6 @@ jobs:
sdk: dev
- os: windows
sdk: dev
# Only run path deps on dev
- sdk: stable
dependencies: published

runs-on: ${{ matrix.os }}-latest

Expand All @@ -64,11 +60,6 @@ jobs:
ndk-version: r26b
if: ${{ matrix.sdk == 'stable' }}

- run: dart run ../../tools/check_pubspec_overrides.dart

- run: dart run ../../tools/delete_pubspec_overrides.dart
if: ${{ matrix.dependencies == 'published' }}

- run: dart pub get

- run: dart pub get -C test_data/dart_app/
Expand Down Expand Up @@ -133,19 +124,19 @@ jobs:

- name: Install coverage
run: dart pub global activate coverage
if: ${{ matrix.sdk == 'stable' && matrix.dependencies == 'published' }}
if: ${{ matrix.sdk == 'stable' }}

- name: Collect coverage
run: dart pub global run coverage:test_with_coverage
if: ${{ matrix.sdk == 'stable' && matrix.dependencies == 'published' }}
if: ${{ matrix.sdk == 'stable' }}

- name: Upload coverage
uses: coverallsapp/github-action@3dfc5567390f6fa9267c0ee9c251e4c8c3f18949
with:
flag-name: ${{ matrix.package }}_${{ matrix.os }}
github-token: ${{ secrets.GITHUB_TOKEN }}
parallel: true
if: ${{ matrix.sdk == 'stable' && matrix.dependencies == 'published' }}
if: ${{ matrix.sdk == 'stable' }}

coverage-finished:
needs: [build]
Expand Down
4 changes: 4 additions & 0 deletions pkgs/native_assets_builder/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,7 @@
## 0.3.3-wip

- Bump `package:native_assets_cli` to path dependency.

## 0.3.2

- Reintroduce `AssetRelativePath`, it's used in `dart build`.
Expand Down
42 changes: 24 additions & 18 deletions pkgs/native_assets_builder/CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,38 +9,44 @@ Both Dartdev and Flutter Tools are built in open source, and inside g3.
* Dartdev as part of the Dart SDK build on the [Dart CI].
* Dartdev as part of the g3 build (see CBuild comments on CLs on [Gerrit]).
* Flutter Tools as part of the workflows on GitHub.
* Flutter Tools as part of the g3 build (see the "Google testing" workflow on PRs on GitHub).

The versions used in these different places are as follows:

* The Dart SDK uses pinned dependencies in [DEPS], the current hash is in `native_rev`.
* The Flutter build on GitHub ussed published dependencies in [its `pubspec.yaml`].
* The g3 build uses the pinned dependencies that are rolled in from the Dart SDK. **Both for Dartdev _and_ Flutter Tools**.

## Rolling

The above means the following.

1. The DEPS in the Dart SDK can only be rolled forward as much as is still compatible with the published deps currently used in Flutter Tools.
2. Flutter Tools can only be rolled forward as much as is still compatible with with the deps pinned in the Dart SDK.

So, any breaking change must be done in the following way:

1. Introduce a new API.
2. Release a new version to pub (minor version).
3. Roll that version into Dart SDK, and migrate uses in the Dart SDK.
4. Wait for that Dart version to roll into g3.
5. Update the dependencies to Flutter Tools, and migrate uses.
6. Get the Flutter PR reviewed.
7. Wait for the Flutter PR to roll into g3.
8. Only then, remove the old API.
9. Release a new version to pub (major version, old API removed).
10. Roll both in to Dart/Flutter.
Breaking changes to `native_assets_cli`, `native_assets_builder`, and
`native_toolchain_c` have to be rolled into the Dart SDK with a CL that
runs all the `pkg` bots:
```
Cq-Include-Trybots: luci.dart.try:pkg-win-release-try,pkg-win-release-arm64-try,pkg-mac-release-try,pkg-mac-release-arm64-try,pkg-linux-release-try,pkg-linux-release-arm64-try,pkg-linux-debug-try
```
The CI on this repo runs with path dependencies between these packages
which corresponds to the path dependencies used in the DEPS file in the Dart
SDK.

Breaking changes to `native_assets_cli` and `native_assets_builder`
have to be rolled into the packages/flutter_tools in the flutter/flutter repo.
This package uses published dependencies.
So this requires:

1. Land a commit that sets the `native_assets_cli` to a stable version.
2. Publish `native_assets_cli` on pub.dev.
3. Land a commit that sets `native_assets_builder` to a stable version,
depends on the published version of `native_assets_cli`, and
removes the `publish_to: none`.
4. Publish `native_assets_builder` on pub.dev.
5. (Repeat these steps for `native_toolchain_c`.)
6. Finally, land a commit that sets all versions to WIP, and adds
`publish_to: none` back in.

The roll status can be seen on internal corp links only:

* Dart roll to g3: https://dart-in-g3-qa-prod.corp.google.com/dg3/Home#/cbuild
* Flutter roll to g3: https://frob.corp.google.com/

[`package:dartdev`]: https://github.com/dart-lang/sdk/tree/main/pkg/dartdev
[`package:flutter_tools`]: https://github.com/flutter/flutter/tree/master/packages/flutter_tools
Expand Down
8 changes: 6 additions & 2 deletions pkgs/native_assets_builder/pubspec.yaml
Original file line number Diff line number Diff line change
@@ -1,16 +1,20 @@
name: native_assets_builder
description: >-
This package is the backend that invokes top-level `build.dart` scripts.
version: 0.3.2
version: 0.3.3-wip
repository: https://github.com/dart-lang/native/tree/main/pkgs/native_assets_builder

environment:
sdk: '>=3.0.0 <4.0.0'

publish_to: none

dependencies:
graphs: ^2.3.1
logging: ^1.2.0
native_assets_cli: ^0.4.1
# native_assets_cli: ^0.4.1
native_assets_cli:
path: ../native_assets_cli/
package_config: ^2.1.0
yaml: ^3.1.2
yaml_edit: ^2.1.0
Expand Down
3 changes: 0 additions & 3 deletions pkgs/native_assets_builder/pubspec_overrides.yaml

This file was deleted.

Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,9 @@ dependencies:
cli_config: ^0.1.1
cyclic_package_2:
path: ../cyclic_package_2
native_assets_cli: ^0.4.1
# native_assets_cli: ^0.4.1
native_assets_cli:
path: ../../../native_assets_cli/
yaml: ^3.1.1
yaml_edit: ^2.1.0

Expand Down

This file was deleted.

Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,9 @@ dependencies:
cli_config: ^0.1.1
cyclic_package_1:
path: ../cyclic_package_1
native_assets_cli: ^0.4.1
# native_assets_cli: ^0.4.1
native_assets_cli:
path: ../../../native_assets_cli/
yaml: ^3.1.1
yaml_edit: ^2.1.0

Expand Down

This file was deleted.

This file was deleted.

11 changes: 0 additions & 11 deletions pkgs/native_assets_builder/test_data/manifest.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,20 +2,16 @@
# a completely clean setup.
- cyclic_package_1/build.dart
- cyclic_package_1/pubspec.yaml
- cyclic_package_1/pubspec_overrides.yaml
- cyclic_package_2/build.dart
- cyclic_package_2/pubspec.yaml
- cyclic_package_2/pubspec_overrides.yaml
- dart_app/bin/dart_app.dart
- dart_app/pubspec.yaml
- dart_app/pubspec_overrides.yaml
- native_add/build.dart
- native_add/ffigen.yaml
- native_add/lib/native_add.dart
- native_add/lib/src/native_add_bindings_generated.dart
- native_add/lib/src/native_add.dart
- native_add/pubspec.yaml
- native_add/pubspec_overrides.yaml
- native_add/src/native_add.c
- native_add/src/native_add.h
- native_add/test/native_add_test.dart
Expand All @@ -25,26 +21,19 @@
- native_subtract/lib/src/native_subtract_bindings_generated.dart
- native_subtract/lib/src/native_subtract.dart
- native_subtract/pubspec.yaml
- native_subtract/pubspec_overrides.yaml
- native_subtract/src/native_subtract.c
- native_subtract/src/native_subtract.h
- package_reading_metadata/build.dart
- package_reading_metadata/pubspec.yaml
- package_reading_metadata/pubspec_overrides.yaml
- package_with_metadata/build.dart
- package_with_metadata/pubspec.yaml
- package_with_metadata/pubspec_overrides.yaml
- some_dev_dep/bin/some_dev_dep.dart
- some_dev_dep/pubspec.yaml
- wrong_build_output/build.dart
- wrong_build_output/pubspec.yaml
- wrong_build_output/pubspec_overrides.yaml
- wrong_build_output_2/build.dart
- wrong_build_output_2/pubspec.yaml
- wrong_build_output_2/pubspec_overrides.yaml
- wrong_build_output_3/build.dart
- wrong_build_output_3/pubspec.yaml
- wrong_build_output_3/pubspec_overrides.yaml
- wrong_namespace_asset/build.dart
- wrong_namespace_asset/pubspec.yaml
- wrong_namespace_asset/pubspec_overrides.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,6 @@
- lib/src/native_add_bindings_generated.dart
- lib/src/native_add.dart
- pubspec.yaml
- pubspec_overrides.yaml
- src/native_add.c
- src/native_add.h
- test/native_add_test.dart
8 changes: 6 additions & 2 deletions pkgs/native_assets_builder/test_data/native_add/pubspec.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,12 @@ environment:
dependencies:
cli_config: ^0.1.1
logging: ^1.1.1
native_assets_cli: ^0.4.1
native_toolchain_c: ^0.3.4+1
# native_assets_cli: ^0.4.1
native_assets_cli:
path: ../../../native_assets_cli/
# native_toolchain_c: ^0.3.4+1
native_toolchain_c:
path: ../../../native_toolchain_c/

dev_dependencies:
ffigen: ^8.0.2
Expand Down

This file was deleted.

Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,12 @@ environment:
dependencies:
cli_config: ^0.1.1
logging: ^1.1.1
native_assets_cli: ^0.4.1
native_toolchain_c: ^0.3.4+1
# native_assets_cli: ^0.4.1
native_assets_cli:
path: ../../../native_assets_cli/
# native_toolchain_c: ^0.3.4+1
native_toolchain_c:
path: ../../../native_toolchain_c/

dev_dependencies:
ffigen: ^8.0.2
Expand Down

This file was deleted.

Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,12 @@ environment:
dependencies:
cli_config: ^0.1.1
logging: ^1.1.1
native_assets_cli: ^0.4.1
native_toolchain_c: ^0.3.4+1
# native_assets_cli: ^0.4.1
native_assets_cli:
path: ../../../native_assets_cli/
# native_toolchain_c: ^0.3.4+1
native_toolchain_c:
path: ../../../native_toolchain_c/

dev_dependencies:
ffigen: ^8.0.2
Expand Down

This file was deleted.

Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,9 @@ environment:

dependencies:
cli_config: ^0.1.1
native_assets_cli: ^0.4.1
# native_assets_cli: ^0.4.1
native_assets_cli:
path: ../../../native_assets_cli/
package_with_metadata:
path: ../package_with_metadata/
yaml: ^3.1.1
Expand Down

This file was deleted.

Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,9 @@ environment:

dependencies:
cli_config: ^0.1.1
native_assets_cli: ^0.4.1
# native_assets_cli: ^0.4.1
native_assets_cli:
path: ../../../native_assets_cli/
yaml: ^3.1.1
yaml_edit: ^2.1.0

Expand Down

This file was deleted.

This file was deleted.

Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,9 @@ environment:

dependencies:
cli_config: ^0.1.1
native_assets_cli: ^0.4.1
# native_assets_cli: ^0.4.1
native_assets_cli:
path: ../../../native_assets_cli/
yaml: ^3.1.1
yaml_edit: ^2.1.0

Expand Down

This file was deleted.

Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,9 @@ environment:

dependencies:
cli_config: ^0.1.1
native_assets_cli: ^0.4.1
# native_assets_cli: ^0.4.1
native_assets_cli:
path: ../../../native_assets_cli/
yaml: ^3.1.1
yaml_edit: ^2.1.0

Expand Down

This file was deleted.

Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,9 @@ environment:

dependencies:
cli_config: ^0.1.1
native_assets_cli: ^0.4.1
# native_assets_cli: ^0.4.1
native_assets_cli:
path: ../../../native_assets_cli/
yaml: ^3.1.1
yaml_edit: ^2.1.0

Expand Down

This file was deleted.

Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,9 @@ environment:

dependencies:
cli_config: ^0.1.1
native_assets_cli: ^0.4.1
# native_assets_cli: ^0.4.1
native_assets_cli:
path: ../../../native_assets_cli/
yaml: ^3.1.1
yaml_edit: ^2.1.0

Expand Down

This file was deleted.

Loading