Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Write a new schema only if necessary #2877

Merged
merged 2 commits into from
Jan 30, 2025
Merged

Conversation

iwahbe
Copy link
Member

@iwahbe iwahbe commented Jan 28, 2025

As I explain in my change:

For the Schema language in particular, we only write files if the write
would cause a change. This allows build systems like Make to correctly
include bridge-metadata.json and schema.json as build dependencies
without always trying to rebuild.

We don't do this for all systems because reading every file before writing it is
expensive.

I've been playing around with using real file dependencies for Go and Make, and the way we always write bridge metadata (which is then a build input via go:embed) prevents Make from ever observing that a provider is already built.

@iwahbe iwahbe self-assigned this Jan 28, 2025
Copy link

codecov bot commented Jan 28, 2025

Codecov Report

Attention: Patch coverage is 62.06897% with 11 lines in your changes missing coverage. Please review.

Project coverage is 67.73%. Comparing base (a3a15d2) to head (cde62c6).
Report is 1 commits behind head on master.

Files with missing lines Patch % Lines
pkg/tfgen/generate.go 62.06% 7 Missing and 4 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #2877      +/-   ##
==========================================
- Coverage   67.74%   67.73%   -0.02%     
==========================================
  Files         328      328              
  Lines       42088    42116      +28     
==========================================
+ Hits        28513    28526      +13     
- Misses      11987    11999      +12     
- Partials     1588     1591       +3     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@iwahbe iwahbe force-pushed the iwahbe/emit-schema-only-if-changed branch 2 times, most recently from 4c57e26 to e18d7df Compare January 29, 2025 16:35
As I explain in my change:

```
For the Schema language in particular, we only write files if the write
would cause a change. This allows build systems like Make to correctly
include bridge-metadata.json and schema.json as build dependencies
without always trying to rebuild.

We don't do this for all systems because reading every file before writing it is
expensive.
```

I've been playing around with using real file dependencies for Go and Make, and the way we
always write bridge metadata (which is then a build input via go:embed) prevents Make from
ever observing that a provider is already built.
@iwahbe iwahbe force-pushed the iwahbe/emit-schema-only-if-changed branch from e18d7df to 7de12f4 Compare January 29, 2025 17:34
@iwahbe iwahbe marked this pull request as ready for review January 29, 2025 17:48
Copy link
Contributor

@VenelinMartinov VenelinMartinov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

Co-authored-by: VenelinMartinov <venelin.v.martinov@gmail.com>
@iwahbe iwahbe enabled auto-merge (squash) January 30, 2025 13:17
@iwahbe iwahbe merged commit 3e16a80 into master Jan 30, 2025
68 of 70 checks passed
@iwahbe iwahbe deleted the iwahbe/emit-schema-only-if-changed branch January 30, 2025 13:42
@danielrbradley
Copy link
Member

@iwahbe I think this might have the unintened concequence of leaving the schema as always being out of date. If the timestamp on the emitted file is still before that of the source files, then the next time the build is run, it will still have to generate the schema, only to compare it to what's on disk again. In the make world, timestamps of completed file targets always need to be updated even if the file hasn't changed.

I would suggest this might need to be reverted 🫤

@pulumi-bot
Copy link
Contributor

This PR has been shipped in release v3.103.0.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants