Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[OPTIMIZATION] 任务日志友好提示 #446

Open
ZhuoZhuoCrayon opened this issue Jan 17, 2022 · 9 comments
Open

[OPTIMIZATION] 任务日志友好提示 #446

ZhuoZhuoCrayon opened this issue Jan 17, 2022 · 9 comments
Assignees
Labels
kind/optimization 优化 module/backend 后台 priority/high 优先级高(High priority) version/V2.2.X V2.2.X

Comments

@ZhuoZhuoCrayon
Copy link
Member

你想要什么功能

为什么需要

建议如何实现

@ZhuoZhuoCrayon
Copy link
Member Author

File "/data/bkee/bknodeman/nodeman/script_tools/wmiexec.py", line 118, in run
iInterface = dcom.CoCreateInstanceEx(wmi.CLSID_WbemLevel1Login, wmi.IID_IWbemLevel1Login)
File "/data/bkee/.envs/bknodeman-nodeman/lib/python3.6/site-packages/impacket/dcerpc/v5/dcomrt.py", line 1061, in CoCreateInstanceEx
iInterface = scm.RemoteCreateInstance(clsid, iid)
File "/data/bkee/.envs/bknodeman-nodeman/lib/python3.6/site-packages/impacket/dcerpc/v5/dcomrt.py", line 1840, in RemoteCreateInstance
resp = self.__portmap.request(request)
File "/data/bkee/.envs/bknodeman-nodeman/lib/python3.6/site-packages/impacket/dcerpc/v5/rpcrt.py", line 857, in request
answer = self.recv()
File "/data/bkee/.envs/bknodeman-nodeman/lib/python3.6/site-packages/impacket/dcerpc/v5/rpcrt.py", line 1308, in recv
response_data = self._transport.recv(forceRecv, count=MSRPCRespHeader._SIZE)
File "/data/bkee/.envs/bknodeman-nodeman/lib/python3.6/site-packages/impacket/dcerpc/v5/transport.py", line 376, in recv
buffer += self.__socket.recv(count-len(buffer))
socket.timeout: timed out
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/data/bkee/bknodeman/nodeman/apps/backend/components/collections/agent.py", line 103, in execute
result = self._execute(data, parent_data)
File "/data/bkee/bknodeman/nodeman/apps/backend/components/collections/agent.py", line 660, in _execute
host, [f'if not exist "{installation_tool.dest_dir}" mkdir {installation_tool.dest_dir}']
File "/data/bkee/bknodeman/nodeman/apps/backend/components/collections/agent.py", line 704, in execute_windows_commands
no_output=True,
File "/data/bkee/bknodeman/nodeman/apps/backend/utils/wmi.py", line 28, in execute_cmd
result_data = executor.run(ipaddr)
File "/data/bkee/bknodeman/nodeman/script_tools/wmiexec.py", line 141, in run
raise Exception(str(e))
Exception: timed out
[2022-01-17 20:17:30 INFO] 下发脚本命令失败: timed out, 请先尝试查看日志并处理,若无法解决,请联系管理员处理

@ZhuoZhuoCrayon
Copy link
Member Author

[2022-01-18 20:15:57 ERROR] [proxy] Traceback (most recent call last):/n File '/opt/py36/lib/python3.6/site-packages/impacket/smbconnection.py', line 271, in login/n return self._SMBConnection.login(user, password, domain, lmhash, nthash)/n File '/opt/py36/lib/python3.6/site-packages/impacket/smb3.py', line 935, in login/n if packet.isValidAnswer(STATUS_SUCCESS):/n File '/opt/py36/lib/python3.6/site-packages/impacket/smb3structs.py', line 454, in isValidAnswer/n raise smb3.SessionError(self['Status'], self)/nimpacket.smb3.SessionError: SMB SessionError: STATUS_LOGON_FAILURE(The attempted logon is invalid. This is either due to a bad username or authentication information.)/n/nDuring handling of the above exception, another exception occurred:/n/nTraceback (most recent call last):/n File '/tmp/bkjob/root/bk_gse_script_20002162629_20002658710.py', line 894, in /n main()/n File '/tmp/bkjob/root/bk_gse_script_20002162629_20002658710.py', line 482, in main/n tmp_dir,/n File '/tmp/bkjob/root/bk_gse_script_20002162629_20002658710.py', line 179, in windows_cmd/n execute_win_commands(['mkdir {}'.format(tmp_dir)], login_ip, user, identity, download_url)/n File '/tmp/bkjob/root/bk_gse_script_20002162629_20002658710.py', line 197, in execute_win_commands/n res = execute_cmd(cmd, login_ip, user, identity, download_url, noOutput=noOutput)/n File '/tmp/bkjob/root/bk_gse_script_20002162629_20002658710.py', line 325, in execute_cmd/n result_data = executor.run(ipaddr)/n File '/tmp/bkjob/root/wmiexec.py', line 81, in run/n smbConnection.login(self.__username, self.__password, self.__domain, self.__lmhash, self.__nthash)/n File '/opt/py36/lib/python3.6/site-packages/impacket/smbconnection.py', line 273, in login/n raise SessionError(e.get_error_code(), e.get_error_packet())/nimpacket.smbconnection.SessionError: SMB SessionError: STATUS_LOGON_FAILURE(The attempted logon is invalid. This is either due to a bad username or authentication information.)/n

@ZhuoZhuoCrayon
Copy link
Member Author

登录失败提示生成

  • 根据用户输入的认证信息,生成相应的排查方案

@ZhuoZhuoCrayon
Copy link
Member Author

[2022-03-01 20:26:39 ERROR] [3803011] IO 异常:Failure
[2022-03-01 20:26:39 DEBUG] ******* Begin of collected logs: *******
Traceback (most recent call last):
File "/data/bkee/.envs/bknodeman-nodeman/lib/python3.6/site-packages/asgiref/sync.py", line 482, in thread_handler
raise exc_info[1]
File "/data/bkee/bknodeman/nodeman/apps/utils/exc.py", line 54, in wrapped_async_executor
return await wrapped(*args, **kwargs)
File "/data/bkee/bknodeman/nodeman/apps/core/remote/clients/file.py", line 85, in put
await self._client.put(localpaths, remotepath)
File "/data/bkee/.envs/bknodeman-nodeman/lib/python3.6/site-packages/asyncssh/sftp.py", line 2371, in put
error_handler)
File "/data/bkee/.envs/bknodeman-nodeman/lib/python3.6/site-packages/asyncssh/sftp.py", line 2169, in _begin_copy
max_requests, progress_handler, error_handler)
File "/data/bkee/.envs/bknodeman-nodeman/lib/python3.6/site-packages/asyncssh/sftp.py", line 2109, in _copy
dstpath, progress_handler).run()
File "/data/bkee/.envs/bknodeman-nodeman/lib/python3.6/site-packages/asyncssh/sftp.py", line 423, in run
await self.start()
File "/data/bkee/.envs/bknodeman-nodeman/lib/python3.6/site-packages/asyncssh/sftp.py", line 535, in start
self._dst = await self._dstfs.open(self._dstpath, 'wb', block_size=None)
File "/data/bkee/.envs/bknodeman-nodeman/lib/python3.6/site-packages/asyncssh/sftp.py", line 2838, in open
handle = await self._handler.open(path, pflags, attrs)
File "/data/bkee/.envs/bknodeman-nodeman/lib/python3.6/site-packages/asyncssh/sftp.py", line 1363, in open
UInt32(pflags), attrs.encode())
File "/data/bkee/.envs/bknodeman-nodeman/lib/python3.6/site-packages/asyncssh/sftp.py", line 1192, in _make_request
result = self._packet_handlers[resptype](self, resp)
File "/data/bkee/.envs/bknodeman-nodeman/lib/python3.6/site-packages/asyncssh/sftp.py", line 1224, in _process_status
raise _construct_sftp_error(code, reason, lang)
asyncssh.sftp.SFTPFailure: Failure

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/data/bkee/.envs/bknodeman-nodeman/lib/python3.6/site-packages/asgiref/sync.py", line 482, in thread_handler
raise exc_info[1]
File "/data/bkee/bknodeman/nodeman/apps/utils/exc.py", line 54, in wrapped_async_executor
return await wrapped(*args, **kwargs)
File "/data/bkee/bknodeman/nodeman/apps/backend/components/collections/agent_new/install.py", line 592, in execute_cygwin_commands_async
await file_client.put(localpaths=localpaths, remotepath=install_sub_inst_obj.installation_tool.dest_dir)
File "/data/bkee/bknodeman/nodeman/apps/utils/exc.py", line 59, in wrapped_async_executor
return await sync_to_async(self.exc_handler)(wrapped, instance, args, kwargs, exc)
File "/data/bkee/.envs/bknodeman-nodeman/lib/python3.6/site-packages/asgiref/sync.py", line 444, in call
ret = await asyncio.wait_for(future, timeout=None)
File "/opt/py36/lib/python3.6/asyncio/tasks.py", line 339, in wait_for
return (yield from fut)
File "/opt/py36/lib/python3.6/concurrent/futures/thread.py", line 56, in run
result = self.fn(*self.args, **self.kwargs)
File "/data/bkee/.envs/bknodeman-nodeman/lib/python3.6/site-packages/asgiref/sync.py", line 484, in thread_handler
return func(*args, **kwargs)
File "/data/bkee/bknodeman/nodeman/apps/core/remote/clients/file.py", line 32, in exc_handler
raise exceptions.RemoteIOError({"err_msg": caught_exc}) from caught_exc
apps.core.remote.exceptions.RemoteIOError: [3803011] IO 异常:Failure

******** End of collected logs *********

Windows:占用文件目录导致的

@ZhuoZhuoCrayon
Copy link
Member Author

[2022-03-04 10:35:32 ERROR] [3803009] 命令返回非零值:exit_status -> 23, stdout -> , stderr -> stty: standard input: Inappropriate ioctl for device
stty: standard input: Inappropriate ioctl for device
curl: (23) Failed writing body (0 != 13138)


删除对应下载的文件

@ZhuoZhuoCrayon
Copy link
Member Author

[2022-03-09 09:13:27 ERROR] [3803007] 远程连接失败:[Errno 110] Connect call failed ('xxxx', 3600)
[2022-03-09 09:13:27 DEBUG] ******* Begin of collected logs: *******
Traceback (most recent call last):
File "/data/bkee/bknodeman/nodeman/apps/core/remote/conns/asyncssh_impl.py", line 63, in connect
**self.options
File "/data/bkee/.envs/bknodeman-nodeman/lib/python3.6/site-packages/asyncssh/connection.py", line 6895, in connect
timeout=options.connect_timeout)
File "/opt/py36/lib/python3.6/asyncio/tasks.py", line 358, in wait_for
return fut.result()
File "/data/bkee/.envs/bknodeman-nodeman/lib/python3.6/site-packages/asyncssh/connection.py", line 300, in _connect
local_addr=local_addr)
File "/opt/py36/lib/python3.6/asyncio/base_events.py", line 783, in create_connection
raise exceptions[0]
File "/opt/py36/lib/python3.6/asyncio/base_events.py", line 770, in create_connection
yield from self.sock_connect(sock, address)
File "/opt/py36/lib/python3.6/asyncio/selector_events.py", line 450, in sock_connect
return (yield from fut)
File "/opt/py36/lib/python3.6/asyncio/selector_events.py", line 480, in _sock_connect_cb
raise OSError(err, 'Connect call failed %s' % (address,))
TimeoutError: [Errno 110] Connect call failed ('xxxxxx', 3600)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/data/bkee/.envs/bknodeman-nodeman/lib/python3.6/site-packages/asgiref/sync.py", line 482, in thread_handler
raise exc_info[1]
File "/data/bkee/bknodeman/nodeman/apps/utils/exc.py", line 54, in wrapped_async_executor
return await wrapped(*args, **kwargs)
File "/data/bkee/bknodeman/nodeman/apps/backend/components/collections/agent_new/install.py", line 561, in execute_linux_commands_async
async with conns.AsyncsshConn(**install_sub_inst_obj.conns_init_params) as conn:
File "/data/bkee/bknodeman/nodeman/apps/core/remote/conns/asyncssh_impl.py", line 122, in aenter
await self.connect()
File "/data/bkee/bknodeman/nodeman/apps/core/remote/conns/asyncssh_impl.py", line 74, in connect
raise exceptions.DisconnectError({"err_msg": e}) from e
apps.core.remote.exceptions.DisconnectError: [3803007] 远程连接失败:[Errno 110] Connect call failed ('xxx', 3600)

******** End of collected logs *********


端口填错

@ZhuoZhuoCrayon
Copy link
Member Author

[2022-03-09 22:24:20 ERROR] [3803005] 认证失败,请检查认证信息是否有误:Permission denied
[2022-03-09 22:24:20 DEBUG] ******* Begin of collected logs: *******
Traceback (most recent call last):
File "/data/bkee/bknodeman/nodeman/apps/core/remote/conns/asyncssh_impl.py", line 63, in connect
**self.options
File "/data/bkee/.envs/bknodeman-nodeman/lib/python3.6/site-packages/asyncssh/connection.py", line 6895, in connect
timeout=options.connect_timeout)
File "/opt/py36/lib/python3.6/asyncio/tasks.py", line 358, in wait_for
return fut.result()
File "/data/bkee/.envs/bknodeman-nodeman/lib/python3.6/site-packages/asyncssh/connection.py", line 303, in _connect
await conn.wait_established()
File "/data/bkee/.envs/bknodeman-nodeman/lib/python3.6/site-packages/asyncssh/connection.py", line 2262, in wait_established
await self._waiter
asyncssh.misc.PermissionDenied: Permission denied

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/data/bkee/.envs/bknodeman-nodeman/lib/python3.6/site-packages/asgiref/sync.py", line 482, in thread_handler
raise exc_info[1]
File "/data/bkee/bknodeman/nodeman/apps/utils/exc.py", line 54, in wrapped_async_executor
return await wrapped(*args, **kwargs)
File "/data/bkee/bknodeman/nodeman/apps/backend/components/collections/agent_new/choose_access_point.py", line 163, in detect_host_to_aps_network__ssh
async with conns.AsyncsshConn(**remote_conn_helper.conns_init_params) as conn:
File "/data/bkee/bknodeman/nodeman/apps/core/remote/conns/asyncssh_impl.py", line 122, in aenter
await self.connect()
File "/data/bkee/bknodeman/nodeman/apps/core/remote/conns/asyncssh_impl.py", line 68, in connect
raise exceptions.PermissionDeniedError({"err_msg": e}) from e
apps.core.remote.exceptions.PermissionDeniedError: [3803005] 认证失败,请检查认证信息是否有误:Permission denied

******** End of collected logs *********

@ZhuoZhuoCrayon
Copy link
Member Author

💡 枚举各种场景情况的堆栈,给出具体的解决方案

@ZhuoZhuoCrayon
Copy link
Member Author

ZhuoZhuoCrayon commented Aug 8, 2023

概要: [3803005] 认证失败,请检查认证信息是否有误:Permission denied
出现次数:1

Traceback (most recent call last):
File "/data/bkee/bknodeman/nodeman/apps/core/remote/conns/asyncssh_impl.py", line 63, in connect
**self.options
File "/data/bkee/.envs/bknodeman-nodeman/lib/python3.6/site-packages/asyncssh/connection.py", line 6895, in connect
timeout=options.connect_timeout)
File "/opt/py36/lib/python3.6/asyncio/tasks.py", line 358, in wait_for
return fut.result()
File "/data/bkee/.envs/bknodeman-nodeman/lib/python3.6/site-packages/asyncssh/connection.py", line 303, in _connect
await conn.wait_established()
File "/data/bkee/.envs/bknodeman-nodeman/lib/python3.6/site-packages/asyncssh/connection.py", line 2262, in wait_established
await self._waiter
asyncssh.misc.PermissionDenied: Permission denied

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/data/bkee/.envs/bknodeman-nodeman/lib/python3.6/site-packages/asgiref/sync.py", line 482, in thread_handler
raise exc_info[1]
File "/data/bkee/bknodeman/nodeman/apps/utils/exc.py", line 55, in wrapped_async_executor
return await wrapped(*args, **kwargs)
File "/data/bkee/bknodeman/nodeman/apps/backend/components/collections/agent_new/install.py", line 502, in execute_shell_solution_async
async with conns.AsyncsshConn(**install_sub_inst_obj.conns_init_params) as conn:
File "/data/bkee/bknodeman/nodeman/apps/core/remote/conns/asyncssh_impl.py", line 122, in aenter
await self.connect()
File "/data/bkee/bknodeman/nodeman/apps/core/remote/conns/asyncssh_impl.py", line 68, in connect
raise exceptions.PermissionDeniedError({"err_msg": e}) from e
apps.core.remote.exceptions.PermissionDeniedError: [3803005] 认证失败,请检查认证信息是否有误:Permission denied

******** End of collected logs *********

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/optimization 优化 module/backend 后台 priority/high 优先级高(High priority) version/V2.2.X V2.2.X
Projects
None yet
Development

No branches or pull requests

2 participants