-
Notifications
You must be signed in to change notification settings - Fork 4.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add support for path mapping to CppArchive
#25081
Conversation
This mostly requires wiring up the existing machinery for structured variables and (most) link build variables. Utility methods on `PathMappers` are moved to `PathMapper` so that they can be dependend on by `AbstractCommandLine` without creating a cycle.
1f8abd6
to
580a076
Compare
@bazel-io fork 8.1.0 |
src/main/java/com/google/devtools/build/lib/actions/PathMapper.java
Outdated
Show resolved
Hide resolved
String execPathString = execPath.getPathString(); | ||
int startOfConfigSegment = execPathString.indexOf('/') + 1; | ||
return PathFragment.createAlreadyNormalized( | ||
execPathString.substring(0, startOfConfigSegment) | ||
+ "pm-" | ||
+ execPathString.substring(startOfConfigSegment)); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Even though this piece of code is just moved from PathMappers class, I'd like to start a bit of discussion on it - and perhaps opening a bug if there's a better way to do it.
I hope there's a way that avoids string manipulation. In current form it might get broken by other changes to output paths we're doing.
cc @gregestren @aranguyen
Also I wonder about edge cases - what if the path doesn't contain a '/'?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That's a good point, I am checking for /
now. Since, at least outside of C++ rules, user-specified paths don't end up as PathFragment
s, I think that a plain bazel-out
path can't happen, but it's better to be safe.
I could rewrite this to use PathFragment
methods, but those would equally depend on output paths being of the form (bazel|blaze)-out<separator><cfg><separator>everything_else
. This would also depend on the layout of output paths though, but I don't see a way around that for what path mapping needs to too. This snippet uses string manipulation mostly because this code path is assumed to be hot and it should't allocate unnecessarily.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@comius what other changes to output paths are you thinking of?
Are you concerned about resource efficiency, correctness, both?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I meant changes to output paths made in other locations of Blaze. For example replacing cpu with platform name. But my guess is, that you're not able to change the path much without breaking subtle/hard coded integrations.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Those efforts should be unaffected: Path mapping assumes that output paths of generated files start with bazel-out/$CFG/
for some value of $CFG
, but it treats that value as a blackbox. It may replace it, prepend to it, append to it, etc. but is specified to not branch on the particular value.
src/main/java/com/google/devtools/build/lib/rules/cpp/CcToolchainVariables.java
Show resolved
Hide resolved
src/main/java/com/google/devtools/build/lib/rules/cpp/LinkBuildVariables.java
Show resolved
Hide resolved
756d579
to
5f49286
Compare
5f49286
to
c0e0b4b
Compare
src/main/java/com/google/devtools/build/lib/actions/PathMapper.java
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A couple of linter sugesstions.
src/main/java/com/google/devtools/build/lib/actions/PathMapper.java
Outdated
Show resolved
Hide resolved
@@ -1122,7 +1123,8 @@ private CppLinkAction buildLinkAction( | |||
linkCommandLine, | |||
linkActionConstruction.getConfig().getActionEnvironment(), | |||
toolchainEnv, | |||
ImmutableMap.copyOf(executionInfo)); | |||
ImmutableMap.copyOf(executionInfo), | |||
PathMappers.getOutputPathsMode(linkActionConstruction.getConfig())); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is not reflected in src/main/starlark/builtins_bzl/common/cc/link/finalize_link_action.bzl
(it triggers IFTTT in the file)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That's expected, finalize_link_action.bzl
goes through ctx.actions.run
, which automatically retains the OutputPathsMode
in SpawnAction
. This doesn't require manual code.
src/main/java/com/google/devtools/build/lib/rules/cpp/CcToolchainVariables.java
Outdated
Show resolved
Hide resolved
This mostly requires wiring up the existing machinery for structured variables and (most) link build variables. Utility methods on `PathMappers` are moved to `PathMapper` so that they can be dependend on by `AbstractCommandLine` without creating a cycle. Work towards bazelbuild#22658 (reply in thread) Work towards bazelbuild#6526 Closes bazelbuild#25081. PiperOrigin-RevId: 721499678 Change-Id: I4551f268093c7b851791a41deb57292b2c23afb7
This mostly requires wiring up the existing machinery for structured variables and (most) link build variables. Utility methods on `PathMappers` are moved to `PathMapper` so that they can be dependend on by `AbstractCommandLine` without creating a cycle. Work towards #22658 (reply in thread) Work towards #6526 Closes #25081. PiperOrigin-RevId: 721499678 Change-Id: I4551f268093c7b851791a41deb57292b2c23afb7 Commit 7f6e649 Co-authored-by: Fabian Meumertzheim <fabian@meumertzhe.im>
This broke things internally, and is being reverted. I see it's already been merged into 8.1.0 so cc @meteorcloudy @Wyverald |
Please post any public details on the breakage when you have them. |
The Bazel relevant bits are likely:
Note this is just the first place it failed, I can't say for certain if there would have been more later. |
This mostly requires wiring up the existing machinery for structured variables and (most) link build variables.
Utility methods on
PathMappers
are moved toPathMapper
so that they can be dependend on byAbstractCommandLine
without creating a cycle.Work towards #22658 (reply in thread)
Work towards #6526