You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Instead of letting the GC collect the managed tensors, pre-allocate them and reuse. Especially important for logits generation, where tensors are 50k+ floating point values. Unfortunately, the tensors are re-allocated each step
The text was updated successfully, but these errors were encountered:
Instead of letting the GC collect the managed tensors, pre-allocate them and reuse. Especially important for logits generation, where tensors are 50k+ floating point values. Unfortunately, the tensors are re-allocated each step
The text was updated successfully, but these errors were encountered: