Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

单机多卡跑gpt2_pretrain.py遇到如下问题 #534

Open
treestreamymw opened this issue Mar 6, 2024 · 0 comments
Open

单机多卡跑gpt2_pretrain.py遇到如下问题 #534

treestreamymw opened this issue Mar 6, 2024 · 0 comments

Comments

@treestreamymw
Copy link

F20240306 12:52:30.421669 11024 ctrl_client.cpp:54] Check failed: rpc_client_.GetStubAt(i)->CallMethodCtrlMethod::kLoadServer( &client_ctx, request, &response).error_code() == grpc::StatusCode::OK (14 vs. 0) Machine 0 lost
*** Check failure stack trace: ***
@ 0x7fa53f8039ca google::LogMessage::Fail()
@ 0x7fa53f803cb2 google::LogMessage::SendToLog()
@ 0x7fa53f803537 google::LogMessage::Flush()
@ 0x7fa53f8060a9 google::LogMessageFatal::~LogMessageFatal()
@ 0x7fa535118195 _ZZN7oneflow14GrpcCtrlClientC4ERKNS_10ProcessCtxEENKUlvE_clEv
@ 0x7fa53f81840f execute_native_thread_routine
@ 0x7fa6292476db start_thread
@ 0x7fa62882861f clone

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant