diff --git a/.github/nowlab_cla.md b/.github/nowlab_cla.md new file mode 100644 index 00000000..d291de24 --- /dev/null +++ b/.github/nowlab_cla.md @@ -0,0 +1,66 @@ +### NOWLAB Individual Contributor License Agreement + +Thank you for your interest in contributing to open source software projects (“Projects”) made available by the Network-Based Computing Laboratory (NBCL) or its affiliates (“NBCL”). This Individual Contributor License Agreement (“Agreement”) sets out the terms governing any source code, object code, bug fixes, configuration changes, tools, specifications, documentation, data, materials, feedback, information or other works of authorship that you submit or have submitted, in any form and in any manner, to NBCL in respect of any of the Projects (collectively “Contributions”). If you have any questions respecting this Agreement, please contact panda@cse.ohio-state.edu. + + +You agree that the following terms apply to all of your past, present and future Contributions. Except for the licenses granted in this Agreement, you retain all of your right, title and interest in and to your Contributions. + + +**Copyright License.** You hereby grant, and agree to grant, to NBCL a non-exclusive, perpetual, irrevocable, worldwide, fully-paid, royalty-free, transferable copyright license to reproduce, prepare derivative works of, publicly display, publicly perform, and distribute your Contributions and such derivative works, with the right to sublicense the foregoing rights through multiple tiers of sublicensees. + + +**Patent License.** You hereby grant, and agree to grant, to NBCL a non-exclusive, perpetual, irrevocable, +worldwide, fully-paid, royalty-free, transferable patent license to make, have made, use, offer to sell, sell, +import, and otherwise transfer your Contributions, where such license applies only to those patent claims +licensable by you that are necessarily infringed by your Contributions alone or by combination of your +Contributions with the Project to which such Contributions were submitted, with the right to sublicense the +foregoing rights through multiple tiers of sublicensees. + + +**Moral Rights.** To the fullest extent permitted under applicable law, you hereby waive, and agree not to +assert, all of your “moral rights” in or relating to your Contributions for the benefit of NBCL, its assigns, and +their respective direct and indirect sublicensees. + + +**Third Party Content/Rights.** If your Contribution includes or is based on any source code, object code, bug +fixes, configuration changes, tools, specifications, documentation, data, materials, feedback, information or +other works of authorship that were not authored by you (“Third Party Content”) or if you are aware of any +third party intellectual property or proprietary rights associated with your Contribution (“Third Party Rights”), +then you agree to include with the submission of your Contribution full details respecting such Third Party +Content and Third Party Rights, including, without limitation, identification of which aspects of your +Contribution contain Third Party Content or are associated with Third Party Rights, the owner/author of the +Third Party Content and Third Party Rights, where you obtained the Third Party Content, and any applicable +third party license terms or restrictions respecting the Third Party Content and Third Party Rights. For greater +certainty, the foregoing obligations respecting the identification of Third Party Content and Third Party Rights +do not apply to any portion of a Project that is incorporated into your Contribution to that same Project. + + +**Representations.** You represent that, other than the Third Party Content and Third Party Rights identified by +you in accordance with this Agreement, you are the sole author of your Contributions and are legally entitled +to grant the foregoing licenses and waivers in respect of your Contributions. If your Contributions were +created in the course of your employment with your past or present employer(s), you represent that such +employer(s) has authorized you to make your Contributions on behalf of such employer(s) or such employer +(s) has waived all of their right, title or interest in or to your Contributions. + + +**Disclaimer.** To the fullest extent permitted under applicable law, your Contributions are provided on an "asis" +basis, without any warranties or conditions, express or implied, including, without limitation, any implied +warranties or conditions of non-infringement, merchantability or fitness for a particular purpose. You are not +required to provide support for your Contributions, except to the extent you desire to provide support. + + +**No Obligation.** You acknowledge that NBCL is under no obligation to use or incorporate your Contributions +into any of the Projects. The decision to use or incorporate your Contributions into any of the Projects will be +made at the sole discretion of NBCL or its authorized delegates .. + + +**Disputes.** This Agreement shall be governed by and construed in accordance with the laws of the State of +New York, United States of America, without giving effect to its principles or rules regarding conflicts of laws, +other than such principles directing application of New York law. The parties hereby submit to venue in, and +jurisdiction of the courts located in New York, New York for purposes relating to this Agreement. In the event +that any of the provisions of this Agreement shall be held by a court or other tribunal of competent jurisdiction +to be unenforceable, the remaining portions hereof shall remain in full force and effect. + + +**Assignment.** You agree that NBCL may assign this Agreement, and all of its rights, obligations and licenses +hereunder. \ No newline at end of file diff --git a/.github/workflows/cla.yml b/.github/workflows/cla.yml new file mode 100644 index 00000000..85b191c1 --- /dev/null +++ b/.github/workflows/cla.yml @@ -0,0 +1,43 @@ +name: "CLA Assistant" +on: + issue_comment: + types: [created] + pull_request_target: + types: [opened,closed,synchronize] + +# explicitly configure permissions, in case your GITHUB_TOKEN workflow permissions are set to read-only in repository settings +permissions: + actions: write + contents: write + pull-requests: write + statuses: write + +jobs: + CLAAssistant: + runs-on: ubuntu-latest + steps: + - name: "CLA Assistant" + if: (github.event.comment.body == 'recheck' || github.event.comment.body == 'I have read the CLA Document and I hereby sign the CLA') || github.event_name == 'pull_request_target' + uses: contributor-assistant/github-action@v2.3.0 + env: + GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} + # the below token should have repo scope and must be manually added by you in the repository's secret + # This token is required only if you have configured to store the signatures in a remote repository/organization + PERSONAL_ACCESS_TOKEN: ${{ secrets.PERSONAL_ACCESS_TOKEN }} + with: + path-to-signatures: '.github/signatures/version1/cla.json' + path-to-document: '.github/nowlab_cla.md' # e.g. a CLA or a DCO document + # branch should not be protected + branch: 'main' + allowlist: user1,bot* + + # the followings are the optional inputs - If the optional inputs are not given, then default values will be taken + #remote-organization-name: enter the remote organization name where the signatures should be stored (Default is storing the signatures in the same repository) + #remote-repository-name: enter the remote repository name where the signatures should be stored (Default is storing the signatures in the same repository) + #create-file-commit-message: 'For example: Creating file for storing CLA Signatures' + #signed-commit-message: 'For example: $contributorName has signed the CLA in $owner/$repo#$pullRequestNo' + #custom-notsigned-prcomment: 'pull request comment with Introductory message to ask new contributors to sign' + #custom-pr-sign-comment: 'The signature to be committed in order to sign the CLA' + #custom-allsigned-prcomment: 'pull request comment when all contributors has signed, defaults to **CLA Assistant Lite bot** All Contributors have signed the CLA.' + #lock-pullrequest-aftermerge: false - if you don't want this bot to automatically lock the pull request after merging (default - true) + #use-dco-flag: true - If you are using DCO instead of CLA \ No newline at end of file diff --git a/LICENSE b/LICENSE index 261eeb9e..6a6d3d03 100644 --- a/LICENSE +++ b/LICENSE @@ -1,6 +1,21 @@ Apache License Version 2.0, January 2004 http://www.apache.org/licenses/ +Copyright (c) 2023, The Ohio State University. All rights +reserved. + +The MCR-DL software package is developed by the team members of The +Ohio State University's Network-Based Computing Laboratory (NBCL), +headed by Professor Dhabaleswar K. (DK) Panda. + +Contact: +Prof. Dhabaleswar K. (DK) Panda +Dept. of Computer Science and Engineering +The Ohio State University +2015 Neil Avenue +Columbus, OH - 43210-1277 +Tel: (614)-292-5199; Fax: (614)-292-2911 +E-mail:panda@cse.ohio-state.edu TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION diff --git a/README.md b/README.md index bb7485a1..6dbebd59 100644 --- a/README.md +++ b/README.md @@ -17,7 +17,7 @@ The initial release of MCR-DL doesn't allow for mixed-backend optimizations. It - Any MPI library (we recommend MVAPICH2-GDR), NCCL, or both
Refer [MVAPICH2-GDR user guide](https://mvapich.cse.ohio-state.edu/userguide/gdr/) to install MVAPICH2-GDR. - PyTorch 1.12.1 or later
-Refer [PyTorch installation guide](/docs/installation/PYTORCH_INSTALLATION_GUIDE.md) to install PyTorch from source and configure MVAPICH2 support. +Refer [PyTorch installation guide](/docs/installation/PYTORCH_INSTALLATION_GUIDE.md) to install PyTorch from source and configure MVAPICH2-GDR support. *Note: We used the following versions during implementation and testing. diff --git a/benchmarks/__init__.py b/benchmarks/__init__.py index e69de29b..ce9d1475 100644 --- a/benchmarks/__init__.py +++ b/benchmarks/__init__.py @@ -0,0 +1,17 @@ +# Copyright 2023, The Ohio State University. All rights reserved. +# The MVAPICH software package is developed by the team members of +# The Ohio State University's Network-Based Computing Laboratory (NBCL), +# headed by Professor Dhabaleswar K. (DK) Panda. +# +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. \ No newline at end of file diff --git a/benchmarks/all_gather.py b/benchmarks/all_gather.py index f920f962..59ed3247 100644 --- a/benchmarks/all_gather.py +++ b/benchmarks/all_gather.py @@ -1,7 +1,20 @@ -# Copyright (c) Microsoft Corporation. -# SPDX-License-Identifier: Apache-2.0 - -# DeepSpeed Team +# Copyright 2023, The Ohio State University. All rights reserved. +# The MVAPICH software package is developed by the team members of +# The Ohio State University's Network-Based Computing Laboratory (NBCL), +# headed by Professor Dhabaleswar K. (DK) Panda. +# +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. import torch import sys, os, time diff --git a/benchmarks/all_reduce.py b/benchmarks/all_reduce.py index b5027934..041fc3a9 100644 --- a/benchmarks/all_reduce.py +++ b/benchmarks/all_reduce.py @@ -1,7 +1,20 @@ -# Copyright (c) Microsoft Corporation. -# SPDX-License-Identifier: Apache-2.0 - -# DeepSpeed Team +# Copyright 2023, The Ohio State University. All rights reserved. +# The MVAPICH software package is developed by the team members of +# The Ohio State University's Network-Based Computing Laboratory (NBCL), +# headed by Professor Dhabaleswar K. (DK) Panda. +# +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. import torch import sys, os, time diff --git a/benchmarks/all_to_all.py b/benchmarks/all_to_all.py index ebf472eb..3317d1e4 100644 --- a/benchmarks/all_to_all.py +++ b/benchmarks/all_to_all.py @@ -1,7 +1,20 @@ -# Copyright (c) Microsoft Corporation. -# SPDX-License-Identifier: Apache-2.0 - -# DeepSpeed Team +# Copyright 2023, The Ohio State University. All rights reserved. +# The MVAPICH software package is developed by the team members of +# The Ohio State University's Network-Based Computing Laboratory (NBCL), +# headed by Professor Dhabaleswar K. (DK) Panda. +# +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. import torch import sys, os, time diff --git a/benchmarks/broadcast.py b/benchmarks/broadcast.py index 4f848bea..f7ca6806 100644 --- a/benchmarks/broadcast.py +++ b/benchmarks/broadcast.py @@ -1,7 +1,20 @@ -# Copyright (c) Microsoft Corporation. -# SPDX-License-Identifier: Apache-2.0 - -# DeepSpeed Team +# Copyright 2023, The Ohio State University. All rights reserved. +# The MVAPICH software package is developed by the team members of +# The Ohio State University's Network-Based Computing Laboratory (NBCL), +# headed by Professor Dhabaleswar K. (DK) Panda. +# +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. import torch import sys, os, time diff --git a/benchmarks/constants.py b/benchmarks/constants.py index 7cded9b7..4f42e4a5 100644 --- a/benchmarks/constants.py +++ b/benchmarks/constants.py @@ -1,7 +1,20 @@ -# Copyright (c) Microsoft Corporation. -# SPDX-License-Identifier: Apache-2.0 - -# DeepSpeed Team +# Copyright 2023, The Ohio State University. All rights reserved. +# The MVAPICH software package is developed by the team members of +# The Ohio State University's Network-Based Computing Laboratory (NBCL), +# headed by Professor Dhabaleswar K. (DK) Panda. +# +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. from mcr_dl.cuda_accelerator import get_accelerator diff --git a/benchmarks/pt2pt.py b/benchmarks/pt2pt.py index 90ca8068..9b51a8b0 100644 --- a/benchmarks/pt2pt.py +++ b/benchmarks/pt2pt.py @@ -1,7 +1,20 @@ -# Copyright (c) Microsoft Corporation. -# SPDX-License-Identifier: Apache-2.0 - -# DeepSpeed Team +# Copyright 2023, The Ohio State University. All rights reserved. +# The MVAPICH software package is developed by the team members of +# The Ohio State University's Network-Based Computing Laboratory (NBCL), +# headed by Professor Dhabaleswar K. (DK) Panda. +# +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. import torch import sys, os, time diff --git a/benchmarks/run_all.py b/benchmarks/run_all.py index 66b88f04..f5695310 100644 --- a/benchmarks/run_all.py +++ b/benchmarks/run_all.py @@ -1,7 +1,20 @@ -# Copyright (c) Microsoft Corporation. -# SPDX-License-Identifier: Apache-2.0 - -# DeepSpeed Team +# Copyright 2023, The Ohio State University. All rights reserved. +# The MVAPICH software package is developed by the team members of +# The Ohio State University's Network-Based Computing Laboratory (NBCL), +# headed by Professor Dhabaleswar K. (DK) Panda. +# +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. import sys, os diff --git a/benchmarks/utils.py b/benchmarks/utils.py index 3b177e20..aca7bafe 100644 --- a/benchmarks/utils.py +++ b/benchmarks/utils.py @@ -1,7 +1,20 @@ -# Copyright (c) Microsoft Corporation. -# SPDX-License-Identifier: Apache-2.0 - -# DeepSpeed Team +# Copyright 2023, The Ohio State University. All rights reserved. +# The MVAPICH software package is developed by the team members of +# The Ohio State University's Network-Based Computing Laboratory (NBCL), +# headed by Professor Dhabaleswar K. (DK) Panda. +# +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. import torch import os, sys diff --git a/mcr_dl/__init__.py b/mcr_dl/__init__.py index 0fc64cc6..c276061f 100644 --- a/mcr_dl/__init__.py +++ b/mcr_dl/__init__.py @@ -1,7 +1,20 @@ -# Copyright (c) Microsoft Corporation. -# SPDX-License-Identifier: Apache-2.0 - -# DeepSpeed Team +# Copyright 2023, The Ohio State University. All rights reserved. +# The MVAPICH software package is developed by the team members of +# The Ohio State University's Network-Based Computing Laboratory (NBCL), +# headed by Professor Dhabaleswar K. (DK) Panda. +# +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. from .utils import * from .comm import * \ No newline at end of file diff --git a/mcr_dl/backend.py b/mcr_dl/backend.py index 2b9a3eae..13978967 100644 --- a/mcr_dl/backend.py +++ b/mcr_dl/backend.py @@ -1,7 +1,21 @@ -# Copyright (c) Microsoft Corporation. -# SPDX-License-Identifier: Apache-2.0 +# Copyright 2023, The Ohio State University. All rights reserved. +# The MVAPICH software package is developed by the team members of +# The Ohio State University's Network-Based Computing Laboratory (NBCL), +# headed by Professor Dhabaleswar K. (DK) Panda. +# +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. -# DeepSpeed Team """ MCR-DL Communication Backend. diff --git a/mcr_dl/comm.py b/mcr_dl/comm.py index 3160d77e..06718e1c 100644 --- a/mcr_dl/comm.py +++ b/mcr_dl/comm.py @@ -1,7 +1,21 @@ -# Copyright (c) Microsoft Corporation. -# SPDX-License-Identifier: Apache-2.0 +# Copyright 2023, The Ohio State University. All rights reserved. +# The MVAPICH software package is developed by the team members of +# The Ohio State University's Network-Based Computing Laboratory (NBCL), +# headed by Professor Dhabaleswar K. (DK) Panda. +# +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. -# DeepSpeed Team """ MCR-DL Communication Package: mcr_dl mcr_dl diff --git a/mcr_dl/config.yml b/mcr_dl/config.yml index 422ada38..5ed2ed5b 100644 --- a/mcr_dl/config.yml +++ b/mcr_dl/config.yml @@ -1,11 +1,11 @@ mpi: - path: '/path/to/mpi/installation' - include: '/path/to/mpi/installation/include' + path: '/home/gulhane.2/mvapich2-installation/nvidia/gdr2.3.7_cuda11.6_gcc10.3.0' + include: '/home/gulhane.2/mvapich2-installation/nvidia/gdr2.3.7_cuda11.6_gcc10.3.0/include' cuda: path: '/usr/local/cuda' include: '/usr/local/cuda/include' nccl: - path: '/path/to/nccl/installation/build' - include: '/path/to/nccl/installation/build/include' \ No newline at end of file + path: '/home/gulhane.2/MCR_DL/nccl/build' + include: '/home/gulhane.2/MCR_DL/nccl/build/include' diff --git a/mcr_dl/constants.py b/mcr_dl/constants.py index 0318ebe9..596f10c9 100644 --- a/mcr_dl/constants.py +++ b/mcr_dl/constants.py @@ -1,7 +1,20 @@ -# Copyright (c) Microsoft Corporation. -# SPDX-License-Identifier: Apache-2.0 - -# DeepSpeed Team +# Copyright 2023, The Ohio State University. All rights reserved. +# The MVAPICH software package is developed by the team members of +# The Ohio State University's Network-Based Computing Laboratory (NBCL), +# headed by Professor Dhabaleswar K. (DK) Panda. +# +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. import os from datetime import timedelta diff --git a/mcr_dl/cuda_accelerator.py b/mcr_dl/cuda_accelerator.py index f65a2487..c5948583 100644 --- a/mcr_dl/cuda_accelerator.py +++ b/mcr_dl/cuda_accelerator.py @@ -1,7 +1,20 @@ -# Copyright (c) Microsoft Corporation. -# SPDX-License-Identifier: Apache-2.0 - -# DeepSpeed Team +# Copyright 2023, The Ohio State University. All rights reserved. +# The MVAPICH software package is developed by the team members of +# The Ohio State University's Network-Based Computing Laboratory (NBCL), +# headed by Professor Dhabaleswar K. (DK) Panda. +# +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. import os import pkgutil diff --git a/mcr_dl/git_version_info.py b/mcr_dl/git_version_info.py index 0fc99cdd..badbfa51 100644 --- a/mcr_dl/git_version_info.py +++ b/mcr_dl/git_version_info.py @@ -1,7 +1,21 @@ -# Copyright (c) Microsoft Corporation. -# SPDX-License-Identifier: Apache-2.0 +# Copyright 2023, The Ohio State University. All rights reserved. +# The MVAPICH software package is developed by the team members of +# The Ohio State University's Network-Based Computing Laboratory (NBCL), +# headed by Professor Dhabaleswar K. (DK) Panda. +# +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. -# DeepSpeed Team try: # This is populated by setup.py from .git_version_info_installed import * # noqa: F401 # type: ignore diff --git a/mcr_dl/mpi.py b/mcr_dl/mpi.py index aee05eed..8bbd14c6 100644 --- a/mcr_dl/mpi.py +++ b/mcr_dl/mpi.py @@ -1,6 +1,20 @@ -''' -Copyright 2021 The Microsoft DeepSpeed Team -''' +# Copyright 2023, The Ohio State University. All rights reserved. +# The MVAPICH software package is developed by the team members of +# The Ohio State University's Network-Based Computing Laboratory (NBCL), +# headed by Professor Dhabaleswar K. (DK) Panda. +# +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. import torch import torch.distributed as dist diff --git a/mcr_dl/nccl.py b/mcr_dl/nccl.py index 08f5cfdf..b5892631 100644 --- a/mcr_dl/nccl.py +++ b/mcr_dl/nccl.py @@ -1,6 +1,20 @@ -''' -Copyright 2021 The Microsoft DeepSpeed Team -''' +# Copyright 2023, The Ohio State University. All rights reserved. +# The MVAPICH software package is developed by the team members of +# The Ohio State University's Network-Based Computing Laboratory (NBCL), +# headed by Professor Dhabaleswar K. (DK) Panda. +# +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. from mcr_dl.ops.comm.nccl import build_nccl_op from mcr_dl.ops.comm.mpi import build_mpi_op diff --git a/mcr_dl/ops/comm/__init__.py b/mcr_dl/ops/comm/__init__.py index 22c41e62..817f369a 100644 --- a/mcr_dl/ops/comm/__init__.py +++ b/mcr_dl/ops/comm/__init__.py @@ -1,3 +1,21 @@ +# Copyright 2023, The Ohio State University. All rights reserved. +# The MVAPICH software package is developed by the team members of +# The Ohio State University's Network-Based Computing Laboratory (NBCL), +# headed by Professor Dhabaleswar K. (DK) Panda. +# +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + from .nccl import * from .mpi import * # from .common import * \ No newline at end of file diff --git a/mcr_dl/ops/comm/common.py b/mcr_dl/ops/comm/common.py index af855087..beafafa1 100644 --- a/mcr_dl/ops/comm/common.py +++ b/mcr_dl/ops/comm/common.py @@ -1,3 +1,21 @@ +# Copyright 2023, The Ohio State University. All rights reserved. +# The MVAPICH software package is developed by the team members of +# The Ohio State University's Network-Based Computing Laboratory (NBCL), +# headed by Professor Dhabaleswar K. (DK) Panda. +# +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + from ..op_builder import CommBuilder common_cpp_module = None diff --git a/mcr_dl/ops/comm/mpi.py b/mcr_dl/ops/comm/mpi.py index adf586b8..a16dd3f4 100644 --- a/mcr_dl/ops/comm/mpi.py +++ b/mcr_dl/ops/comm/mpi.py @@ -1,3 +1,21 @@ +# Copyright 2023, The Ohio State University. All rights reserved. +# The MVAPICH software package is developed by the team members of +# The Ohio State University's Network-Based Computing Laboratory (NBCL), +# headed by Professor Dhabaleswar K. (DK) Panda. +# +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + from mcr_dl.ops import op_builder mpi_cpp_module = None diff --git a/mcr_dl/ops/comm/nccl.py b/mcr_dl/ops/comm/nccl.py index d0c4cbf7..aebf8158 100644 --- a/mcr_dl/ops/comm/nccl.py +++ b/mcr_dl/ops/comm/nccl.py @@ -1,3 +1,21 @@ +# Copyright 2023, The Ohio State University. All rights reserved. +# The MVAPICH software package is developed by the team members of +# The Ohio State University's Network-Based Computing Laboratory (NBCL), +# headed by Professor Dhabaleswar K. (DK) Panda. + + +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at + +# http://www.apache.org/licenses/LICENSE-2.0 + +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + from mcr_dl.ops import op_builder nccl_cpp_module = None diff --git a/mcr_dl/ops/csrc/comm/common.cpp b/mcr_dl/ops/csrc/comm/common.cpp index 7fc81399..03cfd95a 100644 --- a/mcr_dl/ops/csrc/comm/common.cpp +++ b/mcr_dl/ops/csrc/comm/common.cpp @@ -1,3 +1,22 @@ +/* +* The MVAPICH software package is developed by the team members of +* The Ohio State University's Network-Based Computing Laboratory (NBCL), +* headed by Professor Dhabaleswar K. (DK) Panda. +* +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at + +* http://www.apache.org/licenses/LICENSE-2.0 + +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + #include #include diff --git a/mcr_dl/ops/csrc/comm/mpi.cpp b/mcr_dl/ops/csrc/comm/mpi.cpp index 5cf37c21..f65b94d9 100644 --- a/mcr_dl/ops/csrc/comm/mpi.cpp +++ b/mcr_dl/ops/csrc/comm/mpi.cpp @@ -1,3 +1,22 @@ +/* +* The MVAPICH software package is developed by the team members of +* The Ohio State University's Network-Based Computing Laboratory (NBCL), +* headed by Professor Dhabaleswar K. (DK) Panda. +* +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at + +* http://www.apache.org/licenses/LICENSE-2.0 + +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + #include #include #include diff --git a/mcr_dl/ops/csrc/comm/nccl.cpp b/mcr_dl/ops/csrc/comm/nccl.cpp index 5414e0e6..ba63af14 100644 --- a/mcr_dl/ops/csrc/comm/nccl.cpp +++ b/mcr_dl/ops/csrc/comm/nccl.cpp @@ -1,3 +1,22 @@ +/* +* The MVAPICH software package is developed by the team members of +* The Ohio State University's Network-Based Computing Laboratory (NBCL), +* headed by Professor Dhabaleswar K. (DK) Panda. +* +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at + +* http://www.apache.org/licenses/LICENSE-2.0 + +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + #include #include #include diff --git a/mcr_dl/ops/csrc/comm/pg.cpp b/mcr_dl/ops/csrc/comm/pg.cpp index f373009f..0a141065 100644 --- a/mcr_dl/ops/csrc/comm/pg.cpp +++ b/mcr_dl/ops/csrc/comm/pg.cpp @@ -1,3 +1,22 @@ +/* +* The MVAPICH software package is developed by the team members of +* The Ohio State University's Network-Based Computing Laboratory (NBCL), +* headed by Professor Dhabaleswar K. (DK) Panda. +* +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at + +* http://www.apache.org/licenses/LICENSE-2.0 + +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + #include #include diff --git a/mcr_dl/ops/csrc/includes/comm.h b/mcr_dl/ops/csrc/includes/comm.h index 266b7182..5d7d794c 100644 --- a/mcr_dl/ops/csrc/includes/comm.h +++ b/mcr_dl/ops/csrc/includes/comm.h @@ -1,3 +1,22 @@ +/* +* The MVAPICH software package is developed by the team members of +* The Ohio State University's Network-Based Computing Laboratory (NBCL), +* headed by Professor Dhabaleswar K. (DK) Panda. +* +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at + +* http://www.apache.org/licenses/LICENSE-2.0 + +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + // enum class ReduceOp { // SUM = 0, // AVG, diff --git a/mcr_dl/ops/csrc/includes/pg.hpp b/mcr_dl/ops/csrc/includes/pg.hpp index ae3a43b6..bd91a474 100644 --- a/mcr_dl/ops/csrc/includes/pg.hpp +++ b/mcr_dl/ops/csrc/includes/pg.hpp @@ -1,3 +1,22 @@ +/* +* The MVAPICH software package is developed by the team members of +* The Ohio State University's Network-Based Computing Laboratory (NBCL), +* headed by Professor Dhabaleswar K. (DK) Panda. +* +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at + +* http://www.apache.org/licenses/LICENSE-2.0 + +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + #pragma once #include diff --git a/mcr_dl/ops/op_builder/__init__.py b/mcr_dl/ops/op_builder/__init__.py index 733d973e..4dd4de1f 100644 --- a/mcr_dl/ops/op_builder/__init__.py +++ b/mcr_dl/ops/op_builder/__init__.py @@ -1,3 +1,21 @@ +# Copyright 2023, The Ohio State University. All rights reserved. +# The MVAPICH software package is developed by the team members of +# The Ohio State University's Network-Based Computing Laboratory (NBCL), +# headed by Professor Dhabaleswar K. (DK) Panda. +# +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + from .comm import NCCLCommBuilder from .comm import MPICommBuilder from .builder import get_default_compute_capabilities, OpBuilder diff --git a/mcr_dl/ops/op_builder/all_ops.py b/mcr_dl/ops/op_builder/all_ops.py index e2e336a6..3ec77d6a 100644 --- a/mcr_dl/ops/op_builder/all_ops.py +++ b/mcr_dl/ops/op_builder/all_ops.py @@ -1,7 +1,20 @@ -# Copyright (c) Microsoft Corporation. -# SPDX-License-Identifier: Apache-2.0 - -# DeepSpeed Team +# Copyright 2023, The Ohio State University. All rights reserved. +# The MVAPICH software package is developed by the team members of +# The Ohio State University's Network-Based Computing Laboratory (NBCL), +# headed by Professor Dhabaleswar K. (DK) Panda. +# +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. import os import pkgutil diff --git a/mcr_dl/ops/op_builder/builder.py b/mcr_dl/ops/op_builder/builder.py index ec1c0109..9561ac2c 100644 --- a/mcr_dl/ops/op_builder/builder.py +++ b/mcr_dl/ops/op_builder/builder.py @@ -1,7 +1,20 @@ -# Copyright (c) Microsoft Corporation. -# SPDX-License-Identifier: Apache-2.0 - -# DeepSpeed Team +# Copyright 2023, The Ohio State University. All rights reserved. +# The MVAPICH software package is developed by the team members of +# The Ohio State University's Network-Based Computing Laboratory (NBCL), +# headed by Professor Dhabaleswar K. (DK) Panda. +# +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. import os import sys diff --git a/mcr_dl/ops/op_builder/comm.py b/mcr_dl/ops/op_builder/comm.py index af66d3fd..1a597d9f 100644 --- a/mcr_dl/ops/op_builder/comm.py +++ b/mcr_dl/ops/op_builder/comm.py @@ -1,3 +1,21 @@ +# Copyright 2023, The Ohio State University. All rights reserved. +# The MVAPICH software package is developed by the team members of +# The Ohio State University's Network-Based Computing Laboratory (NBCL), +# headed by Professor Dhabaleswar K. (DK) Panda. +# +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + from .builder import CUDAOpBuilder from .config import ConfigPath diff --git a/mcr_dl/ops/op_builder/config.py b/mcr_dl/ops/op_builder/config.py index d4a7ccb3..1c88979c 100644 --- a/mcr_dl/ops/op_builder/config.py +++ b/mcr_dl/ops/op_builder/config.py @@ -1,3 +1,21 @@ +# Copyright 2023, The Ohio State University. All rights reserved. +# The MVAPICH software package is developed by the team members of +# The Ohio State University's Network-Based Computing Laboratory (NBCL), +# headed by Professor Dhabaleswar K. (DK) Panda. +# +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + import os import yaml import mcr_dl diff --git a/mcr_dl/pydantic_v1.py b/mcr_dl/pydantic_v1.py index 7302bd6d..7738c3e1 100644 --- a/mcr_dl/pydantic_v1.py +++ b/mcr_dl/pydantic_v1.py @@ -1,7 +1,21 @@ -# Copyright (c) Microsoft Corporation. -# SPDX-License-Identifier: Apache-2.0 +# Copyright 2023, The Ohio State University. All rights reserved. +# The MVAPICH software package is developed by the team members of +# The Ohio State University's Network-Based Computing Laboratory (NBCL), +# headed by Professor Dhabaleswar K. (DK) Panda. +# +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. -# DeepSpeed Team """Pydantic v1 compatibility module. Pydantic v2 introduced breaking changes that hinder its adoption: diff --git a/mcr_dl/reduce_op.py b/mcr_dl/reduce_op.py index 49393f0c..f04a03d2 100644 --- a/mcr_dl/reduce_op.py +++ b/mcr_dl/reduce_op.py @@ -1,7 +1,20 @@ -# Copyright (c) Microsoft Corporation. -# SPDX-License-Identifier: Apache-2.0 - -# DeepSpeed Team +# Copyright 2023, The Ohio State University. All rights reserved. +# The MVAPICH software package is developed by the team members of +# The Ohio State University's Network-Based Computing Laboratory (NBCL), +# headed by Professor Dhabaleswar K. (DK) Panda. +# +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. from enum import Enum diff --git a/mcr_dl/torch.py b/mcr_dl/torch.py index 5a8a61d5..f3ab8c09 100644 --- a/mcr_dl/torch.py +++ b/mcr_dl/torch.py @@ -1,7 +1,20 @@ -# Copyright (c) Microsoft Corporation. -# SPDX-License-Identifier: Apache-2.0 - -# DeepSpeed Team +# Copyright 2023, The Ohio State University. All rights reserved. +# The MVAPICH software package is developed by the team members of +# The Ohio State University's Network-Based Computing Laboratory (NBCL), +# headed by Professor Dhabaleswar K. (DK) Panda. +# +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. import os diff --git a/mcr_dl/utils/__init__.py b/mcr_dl/utils/__init__.py index 89fabf8b..de4134d5 100644 --- a/mcr_dl/utils/__init__.py +++ b/mcr_dl/utils/__init__.py @@ -1,7 +1,20 @@ -# Copyright (c) Microsoft Corporation. -# SPDX-License-Identifier: Apache-2.0 - -# DeepSpeed Team +# Copyright 2023, The Ohio State University. All rights reserved. +# The MVAPICH software package is developed by the team members of +# The Ohio State University's Network-Based Computing Laboratory (NBCL), +# headed by Professor Dhabaleswar K. (DK) Panda. +# +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. from .logging import logger, log_dist from .comms_logging import get_caller_func diff --git a/mcr_dl/utils/comms_logging.py b/mcr_dl/utils/comms_logging.py index a61371f0..f973c131 100644 --- a/mcr_dl/utils/comms_logging.py +++ b/mcr_dl/utils/comms_logging.py @@ -1,7 +1,20 @@ -# Copyright (c) Microsoft Corporation. -# SPDX-License-Identifier: Apache-2.0 - -# DeepSpeed Team +# Copyright 2023, The Ohio State University. All rights reserved. +# The MVAPICH software package is developed by the team members of +# The Ohio State University's Network-Based Computing Laboratory (NBCL), +# headed by Professor Dhabaleswar K. (DK) Panda. +# +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. import math from mcr_dl.utils import log_dist diff --git a/mcr_dl/utils/dist.py b/mcr_dl/utils/dist.py index 16a7e027..a9ef7884 100644 --- a/mcr_dl/utils/dist.py +++ b/mcr_dl/utils/dist.py @@ -1,7 +1,20 @@ -# Copyright (c) Microsoft Corporation. -# SPDX-License-Identifier: Apache-2.0 - -# DeepSpeed Team +# Copyright 2023, The Ohio State University. All rights reserved. +# The MVAPICH software package is developed by the team members of +# The Ohio State University's Network-Based Computing Laboratory (NBCL), +# headed by Professor Dhabaleswar K. (DK) Panda. +# +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. import os import inspect diff --git a/mcr_dl/utils/logging.py b/mcr_dl/utils/logging.py index f7e90e0c..5bec1895 100644 --- a/mcr_dl/utils/logging.py +++ b/mcr_dl/utils/logging.py @@ -1,7 +1,20 @@ -# Copyright (c) Microsoft Corporation. -# SPDX-License-Identifier: Apache-2.0 - -# DeepSpeed Team +# Copyright 2023, The Ohio State University. All rights reserved. +# The MVAPICH software package is developed by the team members of +# The Ohio State University's Network-Based Computing Laboratory (NBCL), +# headed by Professor Dhabaleswar K. (DK) Panda. +# +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. import functools import logging diff --git a/mcr_dl/utils/timer.py b/mcr_dl/utils/timer.py index 319684fe..a1dc0aeb 100755 --- a/mcr_dl/utils/timer.py +++ b/mcr_dl/utils/timer.py @@ -1,7 +1,20 @@ -# Copyright (c) Microsoft Corporation. -# SPDX-License-Identifier: Apache-2.0 - -# DeepSpeed Team +# Copyright 2023, The Ohio State University. All rights reserved. +# The MVAPICH software package is developed by the team members of +# The Ohio State University's Network-Based Computing Laboratory (NBCL), +# headed by Professor Dhabaleswar K. (DK) Panda. +# +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. import time from numpy import mean diff --git a/setup.py b/setup.py index 4eb8ab2e..7decd860 100644 --- a/setup.py +++ b/setup.py @@ -1,8 +1,21 @@ +# Copyright 2023, The Ohio State University. All rights reserved. +# The MVAPICH software package is developed by the team members of +# The Ohio State University's Network-Based Computing Laboratory (NBCL), +# headed by Professor Dhabaleswar K. (DK) Panda. +# +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. -# Copyright (c) Microsoft Corporation. -# SPDX-License-Identifier: Apache-2.0 - -# DeepSpeed Team """ MCR-DL Setup """ diff --git a/tests/__init__.py b/tests/__init__.py index e69de29b..ce9d1475 100644 --- a/tests/__init__.py +++ b/tests/__init__.py @@ -0,0 +1,17 @@ +# Copyright 2023, The Ohio State University. All rights reserved. +# The MVAPICH software package is developed by the team members of +# The Ohio State University's Network-Based Computing Laboratory (NBCL), +# headed by Professor Dhabaleswar K. (DK) Panda. +# +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. \ No newline at end of file diff --git a/tests/common.py b/tests/common.py index 64069a55..27f9f820 100644 --- a/tests/common.py +++ b/tests/common.py @@ -1,7 +1,20 @@ -# Copyright (c) Microsoft Corporation. -# SPDX-License-Identifier: Apache-2.0 - -# DeepSpeed Team +# Copyright 2023, The Ohio State University. All rights reserved. +# The MVAPICH software package is developed by the team members of +# The Ohio State University's Network-Based Computing Laboratory (NBCL), +# headed by Professor Dhabaleswar K. (DK) Panda. +# +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. import os import re diff --git a/tests/conftest.py b/tests/conftest.py index be130a54..0069fbe3 100644 --- a/tests/conftest.py +++ b/tests/conftest.py @@ -1,3 +1,21 @@ +# Copyright 2023, The Ohio State University. All rights reserved. +# The MVAPICH software package is developed by the team members of +# The Ohio State University's Network-Based Computing Laboratory (NBCL), +# headed by Professor Dhabaleswar K. (DK) Panda. +# +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + from pytest import fixture diff --git a/tests/main.py b/tests/main.py index e1a06a70..e22e72ca 100644 --- a/tests/main.py +++ b/tests/main.py @@ -1,3 +1,21 @@ +# Copyright 2023, The Ohio State University. All rights reserved. +# The MVAPICH software package is developed by the team members of +# The Ohio State University's Network-Based Computing Laboratory (NBCL), +# headed by Professor Dhabaleswar K. (DK) Panda. +# +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + import os import time import argparse diff --git a/tests/simple_model.py b/tests/simple_model.py index 76331f77..240d0f2a 100644 --- a/tests/simple_model.py +++ b/tests/simple_model.py @@ -1,3 +1,20 @@ +# Copyright 2023, The Ohio State University. All rights reserved. +# The MVAPICH software package is developed by the team members of +# The Ohio State University's Network-Based Computing Laboratory (NBCL), +# headed by Professor Dhabaleswar K. (DK) Panda. +# +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. import torch diff --git a/tests/test_dist.py b/tests/test_dist.py index 902fe4c2..53951eb7 100644 --- a/tests/test_dist.py +++ b/tests/test_dist.py @@ -1,7 +1,20 @@ -# Copyright (c) Microsoft Corporation. -# SPDX-License-Identifier: Apache-2.0 - -# DeepSpeed Team +# Copyright 2023, The Ohio State University. All rights reserved. +# The MVAPICH software package is developed by the team members of +# The Ohio State University's Network-Based Computing Laboratory (NBCL), +# headed by Professor Dhabaleswar K. (DK) Panda. +# +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. import os import pytest