add timeout to e2e tests

[mattseddon]

From #405 (comment)

[cloudflare-workers-and-pages]

Deploying datachain-documentation with    Cloudflare Pages

Latest commit:	70645ab
Status:	✅  Deploy successful!
Preview URL:	https://18da05eb.datachain-documentation.pages.dev
Branch Preview URL:	https://add-e2e-timeout.datachain-documentation.pages.dev

View logs

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 86.88%. Comparing base (d36d06f) to head (70645ab).
Report is 5 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #406      +/-   ##
==========================================
- Coverage   87.31%   86.88%   -0.43%     
==========================================
  Files          92       92              
  Lines        9981     9982       +1     
  Branches     2041     1900     -141     
==========================================
- Hits         8715     8673      -42     
- Misses        911      947      +36     
- Partials      355      362       +7     

Flag	Coverage Δ
datachain	86.88% <100.00%> (-0.38%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[skshetry]

I'm investigating here: https://github.com/iterative/datachain/actions/runs/10770282498.

I will merge this if E2E is the problem.

[skshetry]

It looks like the issue is with parallel udf tests. Although I am not sure, but adding --timeout to the Windows tests started crashing xdist workers and it says it was running test_parallel (or, similar other tests).

Need to check if the crashes happen on main or not.

[mattseddon]

It looks like the issue is with parallel udf tests. Although I am not sure, but adding --timeout to the Windows tests started crashing xdist workers and it says it was running test_parallel (or, similar other tests).

From the docs: "For each test item the pytest-timeout plugin starts a timer thread which will terminate the whole process after the specified timeout. When a test item finishes this timer thread is cancelled and the test run continues." so these tests are timing out. Adding timeout_func_only = true to the config seems to improve things but there are still random timeouts in the suite.

I am still seeing runs > 35 minutes long (here) even with tests getting killed after the timeout. Given this and the fact that the original run in question from #405 was at 92% completed with 4 workers, I do not think it is unfeasible that the entire suite is just running slowly and needs > 40 minutes to complete.

[skshetry]

I do not think it is unfeasible that the entire suite is just running slowly and needs > 40 minutes to complete.

Check the timestamp on https://github.com/iterative/datachain/actions/runs/10750425347/job/29816550855?pr=403.

The second-to-last line is at 10-minute mark and the last line appears 30 minutes later when the job times out.