You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
My sharp version is 3.5
When I run NCCL+Sharp with TP(https://pytorch.org/docs/stable/distributed.tensor.parallel.html ),it turns out that with TP the model is divided into two and each requests its own set of resources, and a conflict occurs sharp + nccl.How to use SHARP with NCCL properly with tensor parallelism factor equal 2?
The text was updated successfully, but these errors were encountered:
My sharp version is 3.5
When I run NCCL+Sharp with TP(https://pytorch.org/docs/stable/distributed.tensor.parallel.html ),it turns out that with TP the model is divided into two and each requests its own set of resources, and a conflict occurs sharp + nccl.How to use SHARP with NCCL properly with tensor parallelism factor equal 2?
The text was updated successfully, but these errors were encountered: