Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[native_assets_builder] Report dart sources as dependencies of hooks #1780

Merged
merged 2 commits into from
Dec 6, 2024

Conversation

dcharkes
Copy link
Collaborator

@dcharkes dcharkes commented Dec 3, 2024

Closes: #1770

This PR adds the Dart sources to the dependencies in HookResult.

It also fixes the dep-file parsing w.r.t. to escapes.

Implementation details

We're passing around HookOutputs which is the deserialized output.json. We could add the Dart sources as dependencies to it after deserializing, but then the correspondence to the json is lost. So instead I've added an extra return value to the places where we pass HookOutput around.

@github-actions github-actions bot added package:native_assets_builder type-infra A repository infrastructure change or enhancement labels Dec 3, 2024
Base automatically changed from allow-list-env-vars to main December 4, 2024 10:15
@dcharkes dcharkes force-pushed the dependencies-dart-files branch from 51dd61c to fde2f0a Compare December 4, 2024 10:21
Copy link

github-actions bot commented Dec 4, 2024

PR Health

Breaking changes ✔️
Package Change Current Version New Version Needed Version Looking good?
Changelog Entry ✔️
Package Changed Files

Changes to files need to be accounted for in their respective changelogs.

API leaks ✔️

The following packages contain symbols visible in the public API, but not exported by the library. Export these symbols or remove them from your publicly visible API.

Package Leaked API symbols
License Headers ✔️
// Copyright (c) 2024, the Dart project authors. Please see the AUTHORS file
// for details. All rights reserved. Use of this source code is governed by a
// BSD-style license that can be found in the LICENSE file.
Files
no missing headers

All source files should start with a license header.

Unrelated files missing license headers
Files
pkgs/ffigen/example/libclang-example/generated_bindings.dart
pkgs/ffigen/example/shared_bindings/generate.dart
pkgs/ffigen/example/shared_bindings/lib/generated/a_gen.dart
pkgs/ffigen/example/shared_bindings/lib/generated/a_shared_b_gen.dart
pkgs/ffigen/example/shared_bindings/lib/generated/base_gen.dart
pkgs/ffigen/example/simple/generated_bindings.dart
pkgs/ffigen/lib/src/header_parser/clang_bindings/clang_bindings.dart
pkgs/ffigen/test/collision_tests/expected_bindings/_expected_decl_decl_collision_bindings.dart
pkgs/ffigen/test/collision_tests/expected_bindings/_expected_decl_symbol_address_collision_bindings.dart
pkgs/ffigen/test/collision_tests/expected_bindings/_expected_decl_type_name_collision_bindings.dart
pkgs/ffigen/test/collision_tests/expected_bindings/_expected_reserved_keyword_collision_bindings.dart
pkgs/ffigen/test/header_parser_tests/expected_bindings/_expected_comment_markup_bindings.dart
pkgs/ffigen/test/header_parser_tests/expected_bindings/_expected_dart_handle_bindings.dart
pkgs/ffigen/test/header_parser_tests/expected_bindings/_expected_enum_int_mimic_bindings.dart
pkgs/ffigen/test/header_parser_tests/expected_bindings/_expected_forward_decl_bindings.dart
pkgs/ffigen/test/header_parser_tests/expected_bindings/_expected_functions_bindings.dart
pkgs/ffigen/test/header_parser_tests/expected_bindings/_expected_imported_types_bindings.dart
pkgs/ffigen/test/header_parser_tests/expected_bindings/_expected_native_func_typedef_bindings.dart
pkgs/ffigen/test/header_parser_tests/expected_bindings/_expected_opaque_dependencies_bindings.dart
pkgs/ffigen/test/header_parser_tests/expected_bindings/_expected_packed_structs_bindings.dart
pkgs/ffigen/test/header_parser_tests/expected_bindings/_expected_regress_384_bindings.dart
pkgs/ffigen/test/header_parser_tests/expected_bindings/_expected_sort_bindings.dart
pkgs/ffigen/test/header_parser_tests/expected_bindings/_expected_struct_fptr_fields_bindings.dart
pkgs/ffigen/test/header_parser_tests/expected_bindings/_expected_typedef_bindings.dart
pkgs/ffigen/test/header_parser_tests/expected_bindings/_expected_unions_bindings.dart
pkgs/ffigen/test/header_parser_tests/expected_bindings/_expected_varargs_bindings.dart
pkgs/ffigen/test/large_integration_tests/_expected_cjson_bindings.dart
pkgs/ffigen/test/large_integration_tests/_expected_libclang_bindings.dart
pkgs/ffigen/test/large_integration_tests/_expected_sqlite_bindings.dart
pkgs/ffigen/test/native_test/_expected_native_test_bindings.dart
pkgs/jni/lib/src/third_party/generated_bindings.dart
pkgs/jni/lib/src/third_party/global_env_extensions.dart
pkgs/jni/lib/src/third_party/jni_bindings_generated.dart
pkgs/jnigen/android_test_runner/lib/main.dart
pkgs/jnigen/example/in_app_java/lib/android_utils.dart
pkgs/jnigen/example/kotlin_plugin/example/lib/main.dart
pkgs/jnigen/example/kotlin_plugin/lib/kotlin_bindings.dart
pkgs/jnigen/example/kotlin_plugin/lib/kotlin_plugin.dart
pkgs/jnigen/example/pdfbox_plugin/lib/pdfbox_plugin.dart
pkgs/jnigen/example/pdfbox_plugin/lib/src/third_party/org/apache/pdfbox/pdmodel/PDDocument.dart
pkgs/jnigen/example/pdfbox_plugin/lib/src/third_party/org/apache/pdfbox/pdmodel/PDDocumentInformation.dart
pkgs/jnigen/example/pdfbox_plugin/lib/src/third_party/org/apache/pdfbox/pdmodel/_package.dart
pkgs/jnigen/example/pdfbox_plugin/lib/src/third_party/org/apache/pdfbox/text/PDFTextStripper.dart
pkgs/jnigen/example/pdfbox_plugin/lib/src/third_party/org/apache/pdfbox/text/_package.dart
pkgs/jnigen/lib/src/bindings/descriptor.dart
pkgs/jnigen/lib/src/bindings/printer.dart
pkgs/jnigen/lib/src/elements/elements.g.dart
pkgs/jnigen/test/jackson_core_test/third_party/bindings/com/fasterxml/jackson/core/_package.dart
pkgs/jnigen/test/type_path_test.dart
pkgs/jnigen/tool/command_runner.dart
pkgs/native_assets_builder/test_data/native_dynamic_linking/bin/native_dynamic_linking.dart
pkgs/objective_c/lib/src/ns_input_stream.dart
pkgs/swift2objc/lib/src/config.dart
pkgs/swift2objc/lib/src/generate_wrapper.dart
pkgs/swift2objc/lib/src/generator/_core/utils.dart
pkgs/swift2objc/lib/src/generator/generator.dart
pkgs/swift2objc/lib/src/parser/parsers/declaration_parsers/parse_initializer_declaration.dart
pkgs/swift2objc/lib/src/transformer/transformers/transform_globals.dart
pkgs/swift2objc/lib/src/transformer/transformers/transform_variable.dart

@coveralls
Copy link

coveralls commented Dec 4, 2024

Coverage Status

coverage: 88.921% (-0.006%) from 88.927%
when pulling 59ba0b7 on dependencies-dart-files
into 7e9897b on main.

@dcharkes dcharkes force-pushed the dependencies-dart-files branch from fde2f0a to 90eee5b Compare December 4, 2024 12:12
@dcharkes dcharkes force-pushed the dependencies-dart-files branch from 90eee5b to 36da1b5 Compare December 4, 2024 12:18
@dcharkes dcharkes requested a review from mkustermann December 4, 2024 12:20
Copy link
Member

@mkustermann mkustermann left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm with comments

@@ -437,12 +437,12 @@ class NativeAssetsBuildRunner {
),
);
if (buildOutput == null) return null;
hookResult = hookResult.copyAdd(buildOutput);
hookResult = hookResult.copyAdd(buildOutput, [/*dry run is not cached*/]);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Didn't you say dry run was finally purged from existence?
Why is this code still here?

@@ -538,7 +538,7 @@ ${e.message}
[
...result.dependencies,
// Also depend on the hook source code.
hookHashesFile.uri,
hookHashes.file.uri,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's assume we did one build. Now I modify hook/build.dart by adding a few comments.

Next time we run a build, the system sees that hook sources changed and it will re-compile the hook, giving us a new dill file.

Now, since we only changed comments in the hook/build.dart, we will get an identical .dill file.

So the next step is running the hook. Though we can skip running the hook if none of the hook-reported deps changed and the hook.dill didn't change (which it didn't)
=> In this scenario we could avoid running the hook.

Though this code looks like we're going to re-run the hook, because we add the hooks.d file here (where hashes of sources changed, we we added some comments).

(One could argue this redundant run is ok because it's very uncommon, but from conceptual point we should only re-run things when needed)

pkgs/native_assets_builder/lib/src/model/hook_result.dart Outdated Show resolved Hide resolved
@dcharkes
Copy link
Collaborator Author

dcharkes commented Dec 6, 2024

Thanks @mkustermann!

I'll address the two remaining comments in separate PRs.

@auto-submit auto-submit bot merged commit 3aba894 into main Dec 6, 2024
37 checks passed
@auto-submit auto-submit bot deleted the dependencies-dart-files branch December 6, 2024 11:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
autosubmit package:native_assets_builder type-infra A repository infrastructure change or enhancement
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[native_assets_builder] Report all dependencies (or none and mandate to always be invoked)
3 participants