request not retried when callback possibly not obtained?

[volodymyrss]

I think we had a feature to retry request if its stuck for a while, right, @burnout87 ?

@andriineronov noticed stuck requests.

[burnout87]

the frontend regularly re-issues the same request, by setting a timeout

mmoda-frontend-module/js/mmoda.instrument.js

Line 254 in 9e5af32

requestTimer = setTimeout(AJAX_call, 5000);

@andriineronov noticed stuck requests.

were the requests stuck on a "submitted" state?

[burnout87]

otherwise, perhaps you refer to this:

https://github.com/oda-hub/dispatcher-app/blob/a45dccd9421eb971df6dd03c23a3efc29efc903b/cdci_data_analysis/flask_app/dispatcher_query.py#L2196-L2208

and such logic is implemented in the dispatcher

[volodymyrss]

the frontend regularly re-issues the same request, by setting a timeout

mmoda-frontend-module/js/mmoda.instrument.js

Line 254 in 9e5af32

requestTimer = setTimeout(AJAX_call, 5000);

@andriineronov noticed stuck requests.

were the requests stuck on a "submitted" state?

No, in "progress". Next time if I am there we check if the requests are re-issued every 5000.

[burnout87]

No, in "progress". Next time if I am there we check if the requests are re-issued every 5000.

ok, but it can be easily verified via the dev tool of the browser, in the network tab. which instrument is causing problems?

[burnout87]

which one is the workflow causing the issue? I could try it locally

[volodymyrss]

which one is the workflow causing the issue? I could try it locally

I think it was LST. But it does not seem specific to workflow.

[burnout87]

I am trying the workflow locally and my execution is also stuck, but in the backend as I can see the fronend is re-sending the request. This is the error I see in the backend:

2025-01-10 11:35:13,628 - nb2workflow.nbadapter - WARNING - git command error: Cmd('git') failed due to: exit code(128)
  cmdline: git clone -v --recurse-submodules -- /home/gabriele/Workspace/lst/.git /tmp/nb2w-bdca669_
  stderr: 'Cloning into '/tmp/nb2w-bdca669_'...
done.
Downloading IRF/dec_4822/irf_node_corsika_theta_19.456_az_0.0_.fits (101 KB)
Error downloading object: IRF/dec_4822/irf_node_corsika_theta_19.456_az_0.0_.fits (6e5e5a0): Smudge error: Error downloading IRF/dec_4822/irf_node_corsika_theta_19.456_az_0.0_.fits (6e5e5a0937f8df5aa741c481700d4df8735e60af70ac9c5790a97b82754429fb): EOF

Errors logged to /tmp/nb2w-bdca669_/.git/lfs/logs/20250110T113513.625719085.log
Use `git lfs logs last` to view the log.
error: external filter 'git-lfs filter-process' failed
fatal: IRF/dec_4822/irf_node_corsika_theta_19.456_az_0.0_.fits: smudge filter lfs failed
warning: Clone succeeded, but checkout failed.
You can inspect what was checked out with 'git status'
and retry with 'git restore --source=HEAD :/'

'

which something @dsavchenko and I actually noticed yesterday

[dsavchenko]

@burnout87 you definitely have a problem with git-lfs locally. I am never able to reproduce this issue on my laptop.
But with this backend exception, like with any other, the failed status should be propagated to frontend, I wonder why is it stuck.

[dsavchenko]

There is possibly (not easy to consistently reproduce) a similar issue but there the status is stuck at "submitted"

[burnout87]

I managed to reproduce this issue, locally. request is stuck in "submitted" state

[volodymyrss]

Dispatcher needs to retry when it did not receive a callback after a while.

Frontend should also do that.

Is it already done?

[burnout87]

Dispatcher needs to retry when it did not receive a callback after a while.

I am checking this

Frontend should also do that.

Frontend is constantly re-requesting