Fix model kwargs #35875
Fix model kwargs #35875
Conversation
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
test_modeling_names.txt
Outdated
| loss = self.loss_function( | ||
| shift_logits.view(batch_size * seq_length, vocab_size), | ||
| shift_labels.view(batch_size * seq_length), | ||
| vocab_size=vocab_size, | ||
| **kwargs, | ||
| ) |
There was a problem hiding this comment.
bit weird, the refactor here should make you only have to pass inputs and the shifts will happen inside
|
Is it normal that some checks were not successful? |
| if hasattr(self, "_loss_function"): | ||
| return self._loss_function | ||
|
|
There was a problem hiding this comment.
@ArthurZucker this was needed to be added for a few models that don't need everything the loss func was up to. Case was xglm
muellerzr
force-pushed
the
muellerzr-fix-model-kwargs
branch
from
cbddbc9 to
6b380f4
Compare
|
Finally ready to go, sorry it took me a bit, lots of models to triple check 😓 |
There was a problem hiding this comment.
Taking this comment into account: #34191 (comment)
cc @bauwenst the getter and setter for self._loss_function should be of help!
I need to review but I think it does help to be able to set self._loss_function for sure. Now the questions is whether or not we want to explicitly do it in our of our models or not!
muellerzr
force-pushed
the
muellerzr-fix-model-kwargs
branch
from
d93121a to
038dc55
Compare
* Save state * Make a failing test * Better test * mpt -> done, many more to go * Rm extranious * Bamba * Bert * big_bird * biogpt * bloom * codegen * ctrl * data2vec * dbrx * Through up to Dbrx * electra * ernie * falcon * Fuyu/persimmon * Include noop kwargs to base models * Rebase * Skip musigen * Refactor/skip mllama * Revert makefile * Rm file * Fix PT failing, need to modify rest of loss funcs to not resize * Propagate some * Continue * More * More options * Mostly fixed * Proved that it's the same * Bloom is good * Make ability to override loss func possible * Fixup * Clean * Fix xglm * Quality tests * Skip OCR2 * Make specific loss for xglm * Make order the same/line up 1:1 * xglm * Skip fx output loss bloom model * Didn't pass in pad_token_id * Fix quality
What does this PR do?
Adds unused
**kwargsto particular models so thatnum_items_in_batchcan work as intendedFixes #35838
Before submitting
Pull Request section?
to it if that's the case.
documentation guidelines, and
here are tips on formatting docstrings.
Who can review?
Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.
@ArthurZucker