Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Changes to simplify access to MCR-DL #16

Closed
wants to merge 6 commits into from
Closed

Conversation

RadhaGulhane13
Copy link
Contributor

Previously:

  1. Initialization for distribution requires the user to make application-level changes for different backends (nccl/mpi) in each distributed module (torch/mcr-dl), as shown here: https://github.com/OSU-Nowlab/MCR-DL/blob/main/benchmarks/utils.py#L61.
  2. The application code becomes messy when accessing the distributed module, and it also requires exporting args or knowing args.dist every time when accessing dist, as shown below:
if args.dist == 'torch':
    import torch.distributed as dist
elif args.dist == 'mcr_dl':
    import mcr_dl as dist

Now,

  1. Initialize dist using mcr_dl.init_processes() replacing torch.distributed.init_process_group() in application.
  2. Access dist by dist = mcr_dl.get_distributed_engine() instead of torch.distributed as dist in application.

Copy link

CLA Assistant Lite bot:
Thank you for your submission, we really appreciate it. Like many open-source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution. You can sign the CLA by just posting a Pull Request Comment same as the below format.


I have read the CLA Document and I hereby sign the CLA


0 out of 2 committers have signed the CLA.
@RadhaGulhane13
@Quentin-Anthony
You can retrigger this bot by commenting recheck in this Pull Request

@RadhaGulhane13
Copy link
Contributor Author

Refer : #17

@github-actions github-actions bot locked and limited conversation to collaborators Feb 26, 2024
@RadhaGulhane13 RadhaGulhane13 deleted the simplify-mcr-dl branch April 25, 2024 01:42
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants