Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chatGLM2 好像目前不支持ptuning训练,有计划什么时候支持么 #136

Open
BlueSkyyyyyy opened this issue Nov 15, 2023 · 2 comments
Labels
bug Something isn't working

Comments

@BlueSkyyyyyy
Copy link

No description provided.

@KaiLv69
Copy link
Collaborator

KaiLv69 commented Nov 16, 2023

你好,是计划支持ptuning的,请问你遇到了什么报错?

@BlueSkyyyyyy
Copy link
Author

你好,是计划支持ptuning的,请问你遇到了什么报错?

目前遇到的问题:
https://github.com/OpenLMLab/collie/blob/dev/collie/utils/pipeline_engine.py 412行,assert _grad_key is None, "More than one tensors requires grad." 。和chatglm2中的model写法矛盾,里面吧prefix embedding也写到这里了,不止一个需要grad的tensor。

之前也遇到了一些问题,主要是一些和past_key_values形状相关的问题,没有记录下来。

@KaiLv69 KaiLv69 added the bug Something isn't working label Nov 21, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants