This repository has been archived by the owner on Jan 21, 2025. It is now read-only.

[Wrong Code Comments] In moe.py, there are two wrong code comments #305

Open

xiaodathereal opened this issue Mar 25, 2021 · 0 comments

xiaodathereal commented Mar 25, 2021

The following two comments seem wrong, they need to be switched.

mesh/mesh_tensorflow/transformer/moe.py

Line 423 in 4e07d5e

# Extra reshape reduces communication cost for model-parallel versions.

mesh/mesh_tensorflow/transformer/moe.py

Line 434 in 4e07d5e

# Split over batch -> split over experts

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.