Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[release/2.3] Enable bf16 with fp32 weights for MIOpen batchnorm #1666

Merged
merged 6 commits into from
Dec 2, 2024
Merged

[release/2.3] Enable bf16 with fp32 weights for MIOpen batchnorm #1666

merged 6 commits into from
Dec 2, 2024

Conversation

jithunnair-amd
Copy link
Collaborator

@jithunnair-amd jithunnair-amd commented Nov 1, 2024 •
edited
Loading

This PR enables:

  • using MIOpen OCL_mix backend for bf16 batchnorm with fp32 weights (using torch autocast). This was required and tested for customer workload using NCHW (which is the only memory_layout enabled).
  • logging for MIOpen batchnorm using PYTORCH_MIOPEN_EXTRA_LOGGING env var.

TODO in separate PR: Need to implement PyTorch unit tests for this bf16/fp16 inputs + fp32 weights case.

@rocm-mici
Copy link

Jenkins build for 83df56886c30e1538fdab562af91696a6a115650 commit finished as FAILURE
Links: Blue Ocean view / Build artifacts

@rocm-mici
Copy link

Jenkins build for a4303a659a1807cb764bc96f1f31b39031d1cd15 commit finished as FAILURE
Links: Blue Ocean view / Build artifacts

@jithunnair-amd jithunnair-amd force-pushed the release/2.3_miopen_bf16_fp16_ocl_mix_enabled branch from 42dcf1d to 2ec5a3c Compare November 2, 2024 21:41
@rocm-mici
Copy link

Jenkins build for 2ec5a3cd6f2410c5e9109266935368c2e97bce32 commit finished as FAILURE
Links: Blue Ocean view / Build artifacts

@rocm-mici
Copy link

Jenkins build for 3a0bee39bb7c9cc46be2f7719b96910e77c6a994 commit finished as FAILURE
Links: Blue Ocean view / Build artifacts

pruthvistony
pruthvistony approved these changes Nov 4, 2024
View reviewed changes
Copy link
Collaborator

@pruthvistony pruthvistony left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Some cleanup is required, but good for initial implementation push.

@jithunnair-amd jithunnair-amd changed the title [release/2.3] Enable bf16 with fp32 weights for batchnorm [release/2.3] Enable bf16 with fp32 weights for MIOpen batchnorm Nov 5, 2024
@rocm-mici
Copy link

Jenkins build for 3a0bee39bb7c9cc46be2f7719b96910e77c6a994 commit finished as FAILURE
Links: Blue Ocean view / Build artifacts

@jithunnair-amd jithunnair-amd force-pushed the release/2.3_miopen_bf16_fp16_ocl_mix_enabled branch from 3a0bee3 to cbb5ecc Compare November 13, 2024 16:02
@jithunnair-amd jithunnair-amd force-pushed the release/2.3_miopen_bf16_fp16_ocl_mix_enabled branch 2 times, most recently from c189ac0 to 0448662 Compare December 2, 2024 19:50
@jithunnair-amd jithunnair-amd marked this pull request as ready for review December 2, 2024 19:56
@jithunnair-amd jithunnair-amd merged commit 7870ca9 into release/2.3 Dec 2, 2024
@jithunnair-amd jithunnair-amd deleted the release/2.3_miopen_bf16_fp16_ocl_mix_enabled branch December 2, 2024 19:56
@jithunnair-amd
Copy link
Collaborator Author

!cherry-pick --onto release/2.4

@rocm-mici rocm-mici mentioned this pull request Dec 12, 2024
Closed
@rocm-mici
Copy link

Created branch release/2.4_cherry-pick_pr-1666 and #1784. It contains a merge conflict. Please resolve it

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@pruthvistony pruthvistony pruthvistony approved these changes

@jeffdaily jeffdaily Awaiting requested review from jeffdaily jeffdaily is a code owner

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@jithunnair-amd @rocm-mici @pruthvistony