Fix: add event sync in fsdp tutorial

[zzb610]

Fixes #ISSUE_NUMBER

Description

The event should be synchronized before calling elapsed_time to ensure that the corresponding event has completed. This change is necessary to avoid runtime errors related to incomplete events.

For more details, refer to the relevant code in the PyTorch repository:

• CUDAEvent.h#L157

Example Code

import torch
def case():
    torch.cuda.set_device(0)

    init_start_event = torch.cuda.Event(enable_timing=True)
    init_end_event = torch.cuda.Event(enable_timing=True)

    init_start_event.record()
    for i in range(100):
        a = torch.randn(size=(1024, 1024), device="cuda")
        b = torch.randn(size=(1024, 1024), device="cuda")
        out = torch.add(a, b)
        out = torch.matmul(a, b)
        out = torch.matmul(a, b)
        out = torch.matmul(a, b)
        out = torch.matmul(a, b)
    init_end_event.record()

    # print("init_end_event.query(): ", init_end_event.query()) # False
    # # RuntimeError: CUDA error: device not ready
    # print(
    #     f"CUDA event elapsed time: {init_start_event.elapsed_time(init_end_event) / 1000}sec"
    # )

    # OK
    init_end_event.synchronize()
    print("init_end_event.query(): ", init_end_event.query())
    print(
        f"CUDA event elapsed time: {init_start_event.elapsed_time(init_end_event) / 1000}sec"
    )

cc @wconstab @osalpekar @H-Huang @kwen2501

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/tutorials/3166

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

• [DomainsOnly] Jobs fail with GLIBC version not found

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[zzb610]

@HamidShojanazeri Could you please take a look at this PR when you have a moment? Thanks a lot!